AI & ML Quick Path

Hardware: laptop to large-scale

What each class of machine can reasonably do, how to estimate GPU memory, and when to move from your laptop to bigger hardware. Specifications checked 2026-11-02.

No prices. Prices change by region and month and were not verified for this guide, so machines are grouped by relative cost. Nothing here is a purchasing recommendation.

GPU memory: the simple arithmetic

For running a model (weights only): number of parameters × bytes per weight. An 8-billion-parameter model at 16 bits (2 bytes) needs about 8 × 2 = 16 GB. That is the minimum; real use needs more.

What8B model estimateWhy it changes
Weights at 32 / 16 / 8 / 4 bits32 / 16 / 8 / 4 GBBytes per weight (see quantisation)
KV cache, 16-bit, 8,192 tokens≈ 1.07 GB per conversationGrows with context length and number of users (assumes 32 layers, 8 KV heads, head size 128)
Full fine-tuning (AdamW, mixed precision)≈ 128 GB before activationsWeights, gradients and optimiser state ≈ 16 bytes per parameter
LoRA or QLoRA fine-tuningFar less than full fine-tuningBase model frozen (and quantised for QLoRA); only small adapters train

These numbers come from examples/memory-estimate/memory_estimate.py. The method is explained in From laptop to big hardware: scaling training and Running language models on your own machine.

Six hardware scenarios

  • Cost: No new spend

    Existing laptop or desktop, no dedicated GPU

    CPU
    Any recent 4+ core CPU
    System RAM
    16 GB minimum, 32 GB comfortable
    GPU
    Integrated graphics
    VRAM
    Shared with system RAM
    SSD
    512 GB+
    Cooling / PSU
    As shipped
    Upgrades
    Add RAM if the slots allow
    Portability
    As existing
    Platform notes
    Windows, macOS or Linux all work for Python, .NET and API-based AI

    Reasonable workloads

    • Python and .NET development: Suitable
    • Notebooks and data analysis: Suitable
    • Classical ML (scikit-learn): Suitable
    • Small local model inference (≤ 3B): Limited
    • 7B/8B quantized inference: Limited
    • LoRA / QLoRA experiments: Not practical
    • Larger fine-tunes: Not practical
    • Distributed training: Not practical

    Enough for the foundations, the ML track, model APIs, RAG and agents. Use free or paid notebook GPUs or cloud GPUs for occasional deep-learning work.

  • Cost: Low

    Budget laptop with an entry GPU

    CPU
    6–8 cores
    System RAM
    16–32 GB
    GPU
    Laptop GPU with 8 GB VRAM
    VRAM
    8 GB
    SSD
    1 TB
    Cooling / PSU
    Laptop GPUs throttle under long training runs
    Upgrades
    Usually RAM and SSD only
    Portability
    High
    Platform notes
    Laptop GPUs often have less VRAM and power than desktop cards with similar names; check the exact model and wattage

    Reasonable workloads

    • Python and .NET development: Suitable
    • Notebooks and data analysis: Suitable
    • Classical ML (scikit-learn): Suitable
    • Small local model inference (≤ 3B): Suitable
    • 7B/8B quantized inference: Limited
    • LoRA / QLoRA experiments: Not practical
    • Larger fine-tunes: Not practical
    • Distributed training: Not practical

    4-bit 7B/8B models may load with short contexts; fine-tuning is impractical. Buy for portability, not for training.

  • Cost: Low

    Low-cost desktop

    CPU
    6–8 cores
    System RAM
    32 GB
    GPU
    Desktop GPU with 16 GB VRAM (for example GeForce RTX 5060 Ti 16 GB)
    VRAM
    16 GB
    SSD
    1–2 TB NVMe
    Cooling / PSU
    Quality PSU sized for the GPU vendor recommendation
    Upgrades
    GPU, RAM and storage replaceable
    Portability
    None
    Platform notes
    NVIDIA CUDA has the widest support in PyTorch and LLM tooling; check support for other vendors per tool

    Reasonable workloads

    • Python and .NET development: Suitable
    • Notebooks and data analysis: Suitable
    • Classical ML (scikit-learn): Suitable
    • Small local model inference (≤ 3B): Suitable
    • 7B/8B quantized inference: Suitable
    • LoRA / QLoRA experiments: Limited
    • Larger fine-tunes: Not practical
    • Distributed training: Not practical

    A practical entry point for local LLM labs: quantized 7B/8B inference and small QLoRA experiments with short sequences.

  • Cost: Medium

    Medium desktop

    CPU
    8–16 cores
    System RAM
    64 GB
    GPU
    16 GB current card (for example RTX 5070 Ti / 5080) or a used 24 GB card (condition-dependent)
    VRAM
    16–24 GB
    SSD
    2 TB NVMe
    Cooling / PSU
    Good airflow; PSU per GPU vendor guidance
    Upgrades
    Room for a larger GPU
    Portability
    None
    Platform notes
    Used GPUs need checks for thermals, warranty and stability

    Reasonable workloads

    • Python and .NET development: Suitable
    • Notebooks and data analysis: Suitable
    • Classical ML (scikit-learn): Suitable
    • Small local model inference (≤ 3B): Suitable
    • 7B/8B quantized inference: Suitable
    • LoRA / QLoRA experiments: Suitable
    • Larger fine-tunes: Limited
    • Distributed training: Not practical

    Comfortable for most portfolio projects, LoRA/QLoRA on 7B/8B models with modest sequence lengths, and serving benchmarks.

  • Cost: High

    High-end workstation

    CPU
    High-core-count desktop or workstation CPU
    System RAM
    96–128 GB
    GPU
    GeForce RTX 5090 (32 GB) or professional card such as RTX PRO 6000 Blackwell (96 GB)
    VRAM
    32–96 GB
    SSD
    2–4 TB NVMe
    Cooling / PSU
    Check card length, slot width, power connectors and sustained cooling
    Upgrades
    Second GPU only with a suitable board, PSU and case
    Portability
    None
    Platform notes
    Professional cards add ECC memory and certified drivers

    Reasonable workloads

    • Python and .NET development: Suitable
    • Notebooks and data analysis: Suitable
    • Classical ML (scikit-learn): Suitable
    • Small local model inference (≤ 3B): Suitable
    • 7B/8B quantized inference: Suitable
    • LoRA / QLoRA experiments: Suitable
    • Larger fine-tunes: Suitable
    • Distributed training: Limited

    Worth it only if you use it most days. Still far from what pretraining a modern foundation model requires.

  • Cost: Pay per use

    Professional / cloud GPU access

    CPU
    Provided by instance
    System RAM
    Provided by instance
    GPU
    Data-centre GPUs (single or multi-GPU instances)
    VRAM
    Typically 24–80+ GB per GPU
    SSD
    Instance disk + persistent volume
    Cooling / PSU
    Managed by provider
    Upgrades
    Choose a larger instance
    Portability
    Accessible from any machine
    Platform notes
    Notebook services have changing quotas; never assume free access is guaranteed

    Reasonable workloads

    • Python and .NET development: Suitable
    • Notebooks and data analysis: Suitable
    • Classical ML (scikit-learn): Suitable
    • Small local model inference (≤ 3B): Suitable
    • 7B/8B quantized inference: Suitable
    • LoRA / QLoRA experiments: Suitable
    • Larger fine-tunes: Suitable
    • Distributed training: Suitable

    Best for occasional heavy jobs and multi-GPU practice. Budget GPU-hours, storage and data transfer, and shut instances down when idle.

Ratings are planning guidance. Whether a model fits depends on precision, context length, batch size and the software used; always measure.

From laptop to large-scale: what changes

StageHardwareWhat changes in your code
Learn and prototypeYour laptop (CPU, maybe a small GPU)Nothing special: small data, small models, short runs.
Serious experimentsOne desktop or cloud GPU (16–24 GB)Move model and data to the GPU (.to("cuda")), use mixed precision, larger batches.
Bigger modelsOne large GPU (48–96 GB) or severalGradient accumulation, gradient checkpointing, quantised loading for fine-tuning (QLoRA).
Many GPUsA multi-GPU server or clusterData parallel training (DDP), then sharded training (FSDP or DeepSpeed) when the model no longer fits one GPU.
Production servingGPU servers behind a load balancerA dedicated LLM server (for example vLLM), batching, autoscaling and monitoring.

The laptop-to-GPU path is shown with real code in Train a small language model on your laptop and From laptop to big hardware: scaling training.

Model API, own machine or cloud GPU?

Use a model API when

You are building products, RAG or agents and do not need to change model weights. Check how the provider handles and stores your data.

Use your own machine when

You do classical ML, small deep-learning and language-model experiments, local models, or data must not leave the device.

Rent cloud GPUs when

A job does not fit your machine, needs several GPUs, or runs rarely. Estimate GPU-hours × hourly rate + storage, set a budget alert, and shut machines down when idle.

Check your own machine

Commands
python --version                 # Python 3.10 or newer
python -c "import sklearn; print(sklearn.__version__)"
nvidia-smi                       # NVIDIA GPU name, memory and driver (if you have one)
python -c "import torch; print(torch.cuda.is_available())"   # True if PyTorch can use the GPU
dotnet --version                 # .NET SDK version

On Apple silicon Macs, PyTorch uses the GPU through torch.backends.mps.is_available(). Set up your laptop for AI and ML in Python and .NET walks through the full setup.

Hardware terms

VRAM
Memory on the GPU. Model weights, KV cache and training data in use must fit here.
System RAM
Main memory; holds datasets and can hold model parts that do not fit the GPU (much slower).
CUDA
NVIDIA's software platform for GPU computing; PyTorch builds target specific CUDA versions.
Mixed precision
Training with 16-bit numbers where safe and 32-bit where needed; faster and uses less memory.
NVMe SSD
Fast storage; models and datasets load faster.
Board power
How much power the GPU can draw; size the power supply and cooling to match.