Quantifying the Operational Impact of Missed Defective Units in ML-Based Quality Inspection

Abstract

Industrial manufacturing uses a variety of Key Performance Indicators (KPIs) to measure and manage the quality of their products. These indicators evaluate different aspects and help to identify losses related to quality issues, and thus to pinpoint areas for improvement. For instance, Overall Equipment Effectiveness (OEE) and first pass yield are widely used metrics in industry, they determine the ability of a process to produce good products by measuring the proportion of detected defects. With the growing utilization of machine learning in defect detection, additional indicators have emerged, this time connected to the behaviour of the used algorithms. Among these, Recall plays an operational role, it indicates how effectively defective units were identified and removed from the production flow. With a low Recall, there is a risk that some defective items go unnoticed, which leads to various losses because those units continue to consume time and resources. This case referred to as late defect detection; because, instead of being detected early, defective units go down the production system and are discovered later at stages where the lost is significantly higher. Hence, in this work, we point out the importance to consider the impact of late defect detection on multi-stage production system by investigating how variations in Recall shape process performances. We introduce two key performance indicators, the first one helps to quantify value-added time wasted due to late defect detection, while the second one allows to determine the impact of this loss on each production process. We have implemented a simulation using Arena software and we have declared our proposed metrics via Record module. The results showed that the impact of late defect detection on processes varies depending on whether a process have a reserve capacity or not.