Evaluating Deep Learning Algorithms for Log-Based Anomaly Detection: Insights from Public and Private Datasets

Abstract

Anomaly detection in network logs is crucial for maintaining the security and efficiency of modern IT systems. This paper evaluates several deep learning algorithms, including Autoencoders, Variational Autoencoders (VAE), Recurrent Neural Networks (RNN), Long Short-Term Memory networks (LSTM), Convolutional Neural Networks (CNN), and Generative Adversarial Networks (GAN), for log-based anomaly detection using public datasets such as UNSW, KDD99, and Kyoto, as well as a private dataset consisting of 300,000 log entries. Each model is benchmarked using key performance metrics such as accuracy, precision, recall, F1-score, anomaly detection rate, false alarm rate, and memory consumption. To address the limitations of existing models, this paper proposes a novel hybrid framework—Adaptive Dual-Attention Temporal Convolutional Network (ADATCN)—which integrates temporal and spatial attention mechanisms with Temporal Convolutional Networks (TCNs). Experimental evaluations show that ADATCN achieves an anomaly detection accuracy of 95.5% on the UNSW dataset, outperforming LSTM (90.82%) and GAN (67.53%). It also reduces the false alarm rate to 3.0%, compared to 4.27% for LSTM and 9.86% for GAN. On the private dataset, ADATCN achieves a precision of 0.99, recall of 0.95, F1-score of 0.97, and FAR of 1.0%, confirming its capability to detect threats with minimal false positives. Additionally, ADATCN demonstrates improved memory efficiency, requiring significantly less computational overhead than RNN and LSTM models, making it suitable for real-time deployment in resource-constrained environments