Implementation of a Framework-Based Information System for Evaluating the Quality of Scientific Publications in Indonesian Higher Education Institutions Using Promethee

Abstract

Introduction: Scientific publications play a crucial role in assessing the academic performance of higher education institutions, especially in Indonesia. The current evaluation system still prioritizes the quantity of publications over their quality, with citation indicators and journal quartile rankings not yet being the main focus. Based on this, there is a need for an evaluation system that can provide a comprehensive assessment, ensuring that the quality of scientific publications in Indonesian universities can be measured transparently and objectively.

Objectives: aims of this study is to develop a framework-based information system using the promethee method to objectively and data-driven evaluate the quality of scientific publications in Indonesian universities. The system is designed to assess publications based on indicators such as citations, and journal quartile rankings.

Methods: This study employs an information system approach and multi-criteria analysis to evaluate the quality of higher education scientific publications in Indonesian. The primary data is obtained from Scopus and Scimago Journal Rank (SJR), which provide information on the number of publications, journal indexing, citation counts, and journal quartile rankings (Q1 – Q4, Non-Q) as a journal metrics. The data from Scopus is then ranked using the promethee method, with Min-Max normalization applied to standardize the evaluation process.

Results: we analysis of 83,601 data scientific publications during the 2020-2024 period, the distribution is as follows: 19,921 publications in Q1, 18,745 in Q2, 25,573 in Q3, 11,125 in Q4, and 8,237 publications classified as Non-Q (without a quartile ranking). Based on the ranking using the promethee method, Universitas Airlangga secured the first position with the highest Net Flow (1.415652), followed by Universitas Indonesia  in second place (1.202783) and Universitas Gadjah Mada in third place (0.719385).

Conclusions: The application of the promethee method in scientific publication evaluation offers a more comprehensive multi-criteria approach. promethee can be applied in research to generate a more objective ranking, making it a valuable tool for stakeholders in Indonesia to effectively monitor scientific publications.