Intelligent Reviewer Matching and Research Ethics Screening based on Rule-based and Multilabel Classification Algorithms (ReMatch+)

Abstract

Introduction: This paper proposed the combination of rule-based algorithms and machine learning techniques to address the issues of accuracy and efficiency of reviewer matching and ethical issue detection in research ethics approval processes.

Objectives: The model addresses these by optimizing two key processes: reviewer matching and ethical issue prediction. Three core experiments were conducted. First, various rule-based algorithms, including Keyword Matching, TF-IDF, BM25, LSA, and Bag-of-Words (BoW), were used to align reviewer expertise with research fields, with effectiveness evaluated based on matching scores and thresholds. Second, the system's performance in reviewer matching was validated using precision, recall, and F1 scores against ground truth data. Third, a multi-label classification approach was employed to train machine learning models to detect ethical issues such as Privacy and Confidentiality, Informed Consent, and Conflict of Interest.

Methods: Various classification techniques that combining TF-IDF and BoW with models like Support Vector Machines (SVM), Random Forests (RF), and Decision Trees (DT), were compared using metrics like subset accuracy and per-label accuracy.

Results: The results demonstrate the positive outcomes of using rule-based and machine learning approaches with TFIDF-SVM performs the best overall, achieving average per-label accuracy (0.89) and subset accuracy (0.38)

Conclusions: Future work could explore the inclusion of semantic-rich models, such as transformers, to further enhance the performance of both reviewer assignment and ethical issue detection.