Evaluation reportGenerated by the project's CI evaluation on held-out data.About the projectCode on GitHub

Finding defects it was never shown

Three surfaces that vary from part to part, five kinds of small, subtle defects. Each detector learns from 120 good parts per surface only and is graded on 140 test parts with pixel-exact masks.

0.974image AUROC, mean over surfaces (pixel statistics 0.847, PCA 0.764)
77%of defects caught with 3.9% of good parts flagged
0.950pixel AUROC: the map points at the defect, not just the part
43 msper image on one CPU core, no GPU and no pretrained network

Image AUROC by surface

methodweavetileswoodmean
pixel statistics0.9140.8500.7780.847
PCA reconstruction0.7780.7920.7220.764
patch NN, random bank0.9570.9250.9100.931
patch nearest neighbour0.9900.9810.9500.974

Image AUROC by defect

methodscratchspotholecrackstain
pixel statistics0.7970.9551.0000.6680.817
PCA reconstruction0.6970.8850.9960.5930.649
patch NN, random bank0.9740.9571.0000.9640.759
patch nearest neighbour0.9980.9901.0000.9970.883

At the operating threshold (set so 2% of separate good parts are flagged)

methoddefects caughtgood parts flaggedms per image
pixel statistics44.2%0.0%0.5
PCA reconstruction35.0%2.8%1.1
patch NN, random bank60.0%3.3%35.9
patch nearest neighbour77.1%3.9%42.6

Examples (top: the part with the defect outlined; bottom: the anomaly map)

weave, scratch
tiles, crack
wood, spot
weave, stain
tiles, hole