Part 7: Production
Run AI and ML reliably for real users: serving models as APIs, MLOps, monitoring, cost and scale, and reference architectures that combine .NET and Python.
Key terms in this part (22)
- API (application programming interface)
- Batch serving
- Champion and challenger
- Concept drift
- Data drift
- Deploying to cloud
- Distillation
- Feature store
- Latency and cost
- Latency budget
- MLOps
- Model routing
- Observability
- ONNX
- PSI (population stability index)
- Rate limit
- Response caching
- Retries and fallbacks
- Secrets management
- Shadow mode
- Token cost tracking
- Tracing
From prototype to production
- Serving a model as an API
Turn a saved model into a web API that loads the model once, validates input, returns a versioned answer and reports its health, in Python and in ASP.NET Core.
- "MLOps: versioning, pipelines and safe releases"
Make model training repeatable and releases safe by versioning data, code and models, recording metrics in a registry, and promoting a new model only when it passes a gate.
- Monitoring models and AI apps in production
Watch a live model or LLM app for data drift, falling quality, latency and cost, and decide when to alert, retrain or roll back.
- Speed, cost and scaling
Make models and LLM apps faster and cheaper with batching, caching, smaller models, quantisation and ONNX export, and know when to add more machines or GPUs.
- Production architecture for AI systems
Design a complete AI system (a .NET application calling a Python model service and an LLM), with containers, secrets, fallbacks, security and a clear checklist, so you can plan any industry case.
The reference architecture: .NET application with a Python model service
Many companies keep their business system in .NET and run ML models in Python. The two talk through a small, versioned contract. Details and code are in Production architecture for AI systems.
- ClientWeb or mobile app
- ASP.NET Core APISign-in, permissions, business rules, audit
- Python service /v1Model inference only; validated input and output
- Modelscikit-learn, PyTorch, ONNX or an LLM