Hybrid Insider Threat Detection Using Adversary Emulation, Endpoint Forensics, and Unsupervised Machine Learning

Abstract

Internal threats are a big security problem. When internal users perform malicious activities, they use their legitimate privileges to perform legitimate-looking activities. There is not much classified data to analyze. Most detection methods use synthetic data or supervised learning, which makes it difficult to detect the actual behavior of the attacker.

This study proposes a hybrid framework for detecting insider threats that combines attack simulation, endpoint forensic analysis, and unsupervised machine learning. Attacks are simulated using the MITRE CALDERA platform, forensic data is collected via Velociraptor, and user behavior is analyzed using Isolation Forest and AutoEncoder models to detect abnormal activity without labeled data.

The analysis results are integrated into a unified risk index ranging from 0 to 100 and classified according to CERT criteria to facilitate interpretation of the results. Experiments demonstrate that the proposed framework can detect many types of internal attacks, including abuse of authentication, depletion of system resources, covert execution, and log tampering, making it a practical and applicable solution in the environments of real- world security.