AI-Driven Anomaly Detection in Encrypted Network Traffic Without Decryption

Abstract

Internet communication has been encrypted to the standard of better privacy but at the same time has created black holes in security monitoring. Crime actors are becoming more and more abusers of encryption as a tablier to cover attacks, making the use of deep packet inspection useless. The approach to anomaly detection described in this paper is based on AI and does not need to decrypt traffic to work. We use machine learning and deep neural networks to rely on side-channel characteristics (packet timing, sizes, metadata) and differentiate between malicious and benign flows. The suggested approach is tested on sample encrypted traffic data, and it is shown that with the high detection accuracy (98%), the false positive rate (approximately 1%), it is possible to provide robust threat detection without decrypting the traffic. Detection accuracy and false positive rate are two important performance measures that are discussed in detail, and the precision, recall, and F1-score are also used to confirm the effectiveness of the method. The results are also presented in the paper with tables and figures that depict the performance of the model as opposed to the base approaches. The results indicate that AI methods have the potential to address the visibility gap in encryption and can provide high security levels with privacy..