ML based anomaly detection framework for Insurance Policy Data

Main Article Content

Shreekant Malviya
Abhishek Jaiswal
Vivek Koli

Abstract

This research presents an insurance-policy machine-learning anomalydetection framework and also evaluates it on a portfolio level. The framework incorporates, by use of a production-faithful synthetic corpus of 500,000 policy and claims records with a mix of numeric, categorical, and narrative fields, batch and streaming ingestion, aggressive preprocessing, and multimodal feature engineering. The quality of data at the baseline was quite good (98% field completeness, 95% cross-table consistency, 2.5% noise), which made it possible to model. In a 70:30 temporal split, Selected (Random Forest, XGBoost) and unsupervised (Isolation Forest, deep autoencoder) detectors were also trained with cross-validation (5-fold), as well as with the mentioned Bayesian hyperparameter optimization. The assessment has focused on ROC-AUC, PR-AUC, Precision@K, and cost curves along the lines of reviewer capacity. The autoencoder reached ROCAUC of up to 0.96 (Isolation Forest 0.94; Random Forest 0.91) and produced false positives being nearly one-fourth that of rule-based baselines, and was able to operate in real-time (≤200 ms end-to-end). The interpretable outputs, global rankings, and case-level reason codes through SHAP, path-length, and reconstruction-error attributions increased the acceptance of the investigator and reduced disposition periods. Using a pilot integration ensured operational readiness, maintaining a lift at the level of production throughput and service objectives. The contribution is a practical, scalable blueprint that integrates accuracy, latency, and control to limit the fraud-loss reduction in insurers to shape a measurable reduction and clinical review lines. The study identifies data governance, interpretability, and ongoing retraining as important factors that contribute to the long-term performance and responsible implementation. Findings are transferred across business lines in the country.

Article Details

Section
Articles