Transformer-Based Approaches for Blood Clot Detection in Clinical Text: A Comparative Study of BERT, RoBERTa, T5, and RNN Architectures

Abstract

Introduction: Identifying blood clots is essential for averting life-threatening disorders including pulmonary embolism and deep vein thrombosis (DVT). Conventional diagnostic methods, dependent on medical imaging and specialist evaluation, have constraints in scalability. Recent breakthroughs in deep learning have facilitated the utilization of unstructured clinical text data for automated detection, presenting a potential alternative.

Objectives: This paper examines the utilization of Transformer-based architectures, including BERT, RoBERTa, and T5, for extracting contextual insights from medical records, such as electronic health records (EHRs), discharge summaries, and laboratory reports.

Methods: The efficacy of these models is assessed in comparison to recurrent neural networks (RNNs), particularly LSTM and GRU, which frequently encounter difficulties with long-range relationships in textual data.

Results: The findings demonstrate that RoBERTa surpasses all other models, with an accuracy of 95.1%, a precision of 93.7%, a recall of 96.4%, and an F1-score of 95.0%. BERT exhibited impressive results with an accuracy of 94.2%, and T5 achieved a performance of 93.5%. In contrast, the RNN-based models LSTM and GRU demonstrated inferior performance, with LSTM achieving 89.3% accuracy and GRU 88.7%. Furthermore, RoBERTa got the highest ROC-AUC score of 0.978, highlighting its exceptional capacity to differentiate between blood clot-positive and negative instances.

Conclusions: These findings underscore the capability of Transformer-based models to improve the precision and dependability of blood clot detection systems, indicating a notable progression in AI-driven healthcare solutions for clinical decision-making.