Novel Applications of Statistical and Machine Learning Methods to Analyze Trial Level Data from Congnitive Measures

Abstract

Traditional analytical methods use aggregate metrics, which fail to show the fine-grained patterns that emerge from trial-level fluctuations of cognitive performance due to dynamic internal states like attention, fatigue, and learning processes. This research examines how statistical and machine learning (ML) methods can analyse trial-level behavioural and physiological data to improve understanding of cognitive dynamics. A total of 9,276 trials were obtained from 112 participants who completed Stroop, N-back, and Go/No-Go tasks. The annotation of each trial included reaction time measurements alongside accuracy data, task condition information, and EEG-derived alpha power measurements. Our analysis incorporated Bayesian hierarchical models, generalized linear mixed models, state-space models, Random Forest, XGBoost, deep neural networks, and Long Short-Term Memory (LSTM) networks to forecast both reaction times and task accuracy. The LSTM model demonstrated the best predictive power by achieving R² = 0.862 for RT prediction and AUC-ROC = 0.925 for accuracy classification. The AUC-ROC score reached 0.925 for classification, while R² reached 0.862 for reaction time prediction, which proved superior to all other techniques. The predictive features of trial number, task congruency, and EEG alpha power emerged through Shapley Additive Explanations (SHAP) and LSTM saliency maps. The research demonstrates how combining statistical transparency with ML flexibility helps reveal personalized and time-dependent cognitive patterns. The proposed method provides a powerful structure for modelling trials while creating potential applications for individualized cognitive assessment systems in educational and mental health settings.