Enhancement of Random Forest Applied to Program-Recommendation for Waitlisted Applicants

Abstract

This study introduces an Enhanced Random Forest model for a program recommendation system aimed at supporting waitlisted applicants. The study integrates feature engineering, feature selection, and a hybrid hyperparameter tuning approach to improve the model’s classification accuracy, reliability, and interpretability. Feature engineering was used to capture interrelationship between program choices and associated college departments, allowing the model to recognize nuance relationships. Recursive Feature Elimination systematically eliminated the least significant features, ensuring that the model focuses on features that have high predictive attributes. To mitigate the limitations of traditional hyperparameter tuning methods, a hybrid approach of RandomizedSearchCV and GridSearchCV was utilized to effectively find optimal parameters while minimizing computation costs. A dataset of 1,505 applicants who underwent the Pamantasan ng Lungsod ng Maynila Admission Test for school year 2024 was utilized to train models. The Enhanced Random Forest was compared against Traditional Random Forest, AdaBoost, Gradient Boosting, and Extra Trees Classifier using an 8-fold cross-validation with different scoring metrics such as accuracy, F1-Score, and ROC-AUC. Results show a significant improvement, where the enhanced model achieved an accuracy of 75.10% with an F1-Score of 74.23% and an ROC-AUC score of 96.57%, outperforming other models across all metrics. The Enhanced Random Forest also achieved a lower standard deviation across all metrics resulting in a stable model compared to the other models. A web application using the enhanced model was developed to support the program recommendation process. The findings demonstrate the effectiveness of the proposed enhancements in addressing challenges in an admission-based recommendation system by offering a robust framework for academic decision-making.