AI & ML Quick Path

Capstone 7 of 8

Private assistant running fully on a laptop

Language modelsAdvancedEffort 20–30 h (estimate)

Business requirement

A law office wants a document assistant that never sends text outside the building: a local model summarises and answers questions over local files.

Lessons to finish first

Architecture

  1. Local model runtime (for example Ollama with a small quantised model)
  2. Local embeddings and vector index
  3. Desktop or local web UI
  4. Structured output for summaries
  5. Speed and quality measurements

Build steps

  1. Install a local model runtime and download two small open models of different sizes.
  2. Measure tokens per second and memory use for each on your laptop.
  3. Build summarisation with structured output (title, key points, deadlines) and validate the JSON.
  4. Reuse capstone 4’s pipeline with local embeddings for question answering over local files.
  5. Compare quality on 20 test questions between the two models; choose one and explain why.
  6. Confirm with a network monitor or by disconnecting that nothing leaves the machine.

Python route and .NET route

Python
Ollama Python client or an OpenAI-compatible local endpoint, Pydantic for output validation, the local summariser example as a starting point.
.NET (C#)
OllamaSharp behind Microsoft.Extensions.AI IChatClient and IEmbeddingGenerator; GetResponseAsync<T> for structured summaries.

Example input and output

Input
contract_2026_04.txt
Output
{"title": "Service agreement", "key_points": [...], "deadlines": ["2026-12-01"]}

Data

Documents you write yourself; no real client files.

Starter file structure

Structure (Python file | .NET project)
local-assistant/
  summarise.py | Summarise/Program.cs
  ask.py
  bench.py
  README.md

Tests

  • Works with the network disconnected
  • Invalid JSON from the model is caught and retried
  • Quality and speed table for both models

How it will be judged

  • Model choice justified with measurements
  • Validation of every model output
  • Clear statement of the hardware it was tested on

Security and privacy

All processing local; encrypt the index at rest; no telemetry from the app.

Definition of done

  • Offline demo
  • Measured tokens per second and memory
  • Validated structured summaries

Stretch goals

  • Fine-tune a LoRA adapter for the office’s summary style
  • Add a GPU and measure the change
  • Package as a desktop app

What to show an employer

Speed and quality table plus an offline demo recording.

Completion

Saved in this browser only.