Capstone 3 of 8
Visual defect inspection
Business requirement
A factory wants to flag scratched or dented parts from camera images so inspectors only check suspicious items.
Lessons to finish first
Architecture
- Generated images of good and defective parts
- Train/validation/test split
- Small CNN or a pretrained model with transfer learning
- Evaluation focused on missed defects
- Export to ONNX
- Inference service used by the line software
Build steps
- Generate 2,000 small grayscale images: plain parts, plus parts with drawn scratches or dents (about 10% defective).
- Train a small convolutional network from scratch on the laptop CPU; record accuracy and recall for defects.
- Try transfer learning from a small pretrained image model and compare.
- Pick a threshold so that very few defects are missed, and report how many good parts are sent to inspectors.
- Show the 10 worst mistakes and describe what they have in common.
- Export the model to ONNX and score images from a .NET or Python service.
Python route and .NET route
- Python
- NumPy to generate images, PyTorch and torchvision (small CNN, a pretrained ResNet-18 for transfer learning), torch.onnx.export.
- .NET (C#)
- Microsoft.ML.OnnxRuntime to run the exported model; ML.NET image classification (Microsoft.ML.Vision) is an alternative for training directly in .NET.
Example input and output
- Input
part_00421.png (64 × 64 grayscale)- Output
{"defect_probability": 0.91, "decision": "send to inspector"}
Data
Generated images only. Real factory images need permission and careful labelling.
Starter file structure
vision/
data/generate_images.py
train.py
export_onnx.py
Inspect/Program.cs
README.mdTests
- Defect recall on the test set meets the agreed target
- ONNX model gives the same decision as the training framework on 50 sample images
- Service rejects images of the wrong size
How it will be judged
- Class imbalance handled (weights or sampling)
- Threshold chosen from the cost of a missed defect
- Error analysis with examples
Security and privacy
Validate image size and format before decoding; limit upload size.
Definition of done
- Recall, precision and confusion matrix reported
- ONNX export verified
- Inspector workload estimate written down
Stretch goals
- Highlight the defect location with a saliency map
- Add an anomaly-detection approach trained only on good parts
- Measure latency per image on CPU
What to show an employer
Confusion matrix, a grid of worst errors and an ONNX inference demo.
Completion
Saved in this browser only.