Industrial surface defect triage benchmark · NEU-CLS
Which 6% should a human look at?
A steel-defect classifier is right 94% of the time. The interesting question is what to do with the rest: which parts to route to an inspector, in what order, and which mistakes are expensive.
Six ways steel strip goes wrong
1,800 grayscale images, 300 per class, evaluated on a fixed 80/20 holdout (seed 42).
The model race
A dummy baseline anchors the score: on a balanced six-class problem, guessing gets 16.7%.
Where it gets confused
Rows are the true class, columns the prediction. Hover a cell. Off-diagonal cells are the 22 mistakes out of 360.
Selected cell
Pareto of misses by true class
Spend the inspector's time where it counts
Predictions are ranked by entropy (least confident first). Pick a review budget and see how many of the classifier's 22 misclassifications land in front of a human.
Not all misses cost the same
Per-class false-negative costs are illustrative, chosen to show cost-weighted reasoning rather than taken from a plant.