Hardware: laptop to large-scale
What each class of machine can reasonably do, how to estimate GPU memory, and when to move from your laptop to bigger hardware. Specifications checked 2026-11-02.
No prices. Prices change by region and month and were not verified for this guide, so machines are grouped by relative cost. Nothing here is a purchasing recommendation.
GPU memory: the simple arithmetic
For running a model (weights only): number of parameters × bytes per weight. An 8-billion-parameter model at 16 bits (2 bytes) needs about 8 × 2 = 16 GB. That is the minimum; real use needs more.
| What | 8B model estimate | Why it changes |
|---|---|---|
| Weights at 32 / 16 / 8 / 4 bits | 32 / 16 / 8 / 4 GB | Bytes per weight (see quantisation) |
| KV cache, 16-bit, 8,192 tokens | ≈ 1.07 GB per conversation | Grows with context length and number of users (assumes 32 layers, 8 KV heads, head size 128) |
| Full fine-tuning (AdamW, mixed precision) | ≈ 128 GB before activations | Weights, gradients and optimiser state ≈ 16 bytes per parameter |
| LoRA or QLoRA fine-tuning | Far less than full fine-tuning | Base model frozen (and quantised for QLoRA); only small adapters train |
These numbers come from examples/memory-estimate/memory_estimate.py. The method is explained in From laptop to big hardware: scaling training and Running language models on your own machine.
Six hardware scenarios
-
Cost: No new spend
Existing laptop or desktop, no dedicated GPU
- CPU
- Any recent 4+ core CPU
- System RAM
- 16 GB minimum, 32 GB comfortable
- GPU
- Integrated graphics
- VRAM
- Shared with system RAM
- SSD
- 512 GB+
- Cooling / PSU
- As shipped
- Upgrades
- Add RAM if the slots allow
- Portability
- As existing
- Platform notes
- Windows, macOS or Linux all work for Python, .NET and API-based AI
Reasonable workloads
- Python and .NET development: Suitable
- Notebooks and data analysis: Suitable
- Classical ML (scikit-learn): Suitable
- Small local model inference (≤ 3B): Limited
- 7B/8B quantized inference: Limited
- LoRA / QLoRA experiments: Not practical
- Larger fine-tunes: Not practical
- Distributed training: Not practical
Enough for the foundations, the ML track, model APIs, RAG and agents. Use free or paid notebook GPUs or cloud GPUs for occasional deep-learning work.
-
Cost: Low
Budget laptop with an entry GPU
- CPU
- 6–8 cores
- System RAM
- 16–32 GB
- GPU
- Laptop GPU with 8 GB VRAM
- VRAM
- 8 GB
- SSD
- 1 TB
- Cooling / PSU
- Laptop GPUs throttle under long training runs
- Upgrades
- Usually RAM and SSD only
- Portability
- High
- Platform notes
- Laptop GPUs often have less VRAM and power than desktop cards with similar names; check the exact model and wattage
Reasonable workloads
- Python and .NET development: Suitable
- Notebooks and data analysis: Suitable
- Classical ML (scikit-learn): Suitable
- Small local model inference (≤ 3B): Suitable
- 7B/8B quantized inference: Limited
- LoRA / QLoRA experiments: Not practical
- Larger fine-tunes: Not practical
- Distributed training: Not practical
4-bit 7B/8B models may load with short contexts; fine-tuning is impractical. Buy for portability, not for training.
-
Cost: Low
Low-cost desktop
- CPU
- 6–8 cores
- System RAM
- 32 GB
- GPU
- Desktop GPU with 16 GB VRAM (for example GeForce RTX 5060 Ti 16 GB)
- VRAM
- 16 GB
- SSD
- 1–2 TB NVMe
- Cooling / PSU
- Quality PSU sized for the GPU vendor recommendation
- Upgrades
- GPU, RAM and storage replaceable
- Portability
- None
- Platform notes
- NVIDIA CUDA has the widest support in PyTorch and LLM tooling; check support for other vendors per tool
Reasonable workloads
- Python and .NET development: Suitable
- Notebooks and data analysis: Suitable
- Classical ML (scikit-learn): Suitable
- Small local model inference (≤ 3B): Suitable
- 7B/8B quantized inference: Suitable
- LoRA / QLoRA experiments: Limited
- Larger fine-tunes: Not practical
- Distributed training: Not practical
A practical entry point for local LLM labs: quantized 7B/8B inference and small QLoRA experiments with short sequences.
-
Cost: Medium
Medium desktop
- CPU
- 8–16 cores
- System RAM
- 64 GB
- GPU
- 16 GB current card (for example RTX 5070 Ti / 5080) or a used 24 GB card (condition-dependent)
- VRAM
- 16–24 GB
- SSD
- 2 TB NVMe
- Cooling / PSU
- Good airflow; PSU per GPU vendor guidance
- Upgrades
- Room for a larger GPU
- Portability
- None
- Platform notes
- Used GPUs need checks for thermals, warranty and stability
Reasonable workloads
- Python and .NET development: Suitable
- Notebooks and data analysis: Suitable
- Classical ML (scikit-learn): Suitable
- Small local model inference (≤ 3B): Suitable
- 7B/8B quantized inference: Suitable
- LoRA / QLoRA experiments: Suitable
- Larger fine-tunes: Limited
- Distributed training: Not practical
Comfortable for most portfolio projects, LoRA/QLoRA on 7B/8B models with modest sequence lengths, and serving benchmarks.
-
Cost: High
High-end workstation
- CPU
- High-core-count desktop or workstation CPU
- System RAM
- 96–128 GB
- GPU
- GeForce RTX 5090 (32 GB) or professional card such as RTX PRO 6000 Blackwell (96 GB)
- VRAM
- 32–96 GB
- SSD
- 2–4 TB NVMe
- Cooling / PSU
- Check card length, slot width, power connectors and sustained cooling
- Upgrades
- Second GPU only with a suitable board, PSU and case
- Portability
- None
- Platform notes
- Professional cards add ECC memory and certified drivers
Reasonable workloads
- Python and .NET development: Suitable
- Notebooks and data analysis: Suitable
- Classical ML (scikit-learn): Suitable
- Small local model inference (≤ 3B): Suitable
- 7B/8B quantized inference: Suitable
- LoRA / QLoRA experiments: Suitable
- Larger fine-tunes: Suitable
- Distributed training: Limited
Worth it only if you use it most days. Still far from what pretraining a modern foundation model requires.
-
Cost: Pay per use
Professional / cloud GPU access
- CPU
- Provided by instance
- System RAM
- Provided by instance
- GPU
- Data-centre GPUs (single or multi-GPU instances)
- VRAM
- Typically 24–80+ GB per GPU
- SSD
- Instance disk + persistent volume
- Cooling / PSU
- Managed by provider
- Upgrades
- Choose a larger instance
- Portability
- Accessible from any machine
- Platform notes
- Notebook services have changing quotas; never assume free access is guaranteed
Reasonable workloads
- Python and .NET development: Suitable
- Notebooks and data analysis: Suitable
- Classical ML (scikit-learn): Suitable
- Small local model inference (≤ 3B): Suitable
- 7B/8B quantized inference: Suitable
- LoRA / QLoRA experiments: Suitable
- Larger fine-tunes: Suitable
- Distributed training: Suitable
Best for occasional heavy jobs and multi-GPU practice. Budget GPU-hours, storage and data transfer, and shut instances down when idle.
No hardware scenarios match these filters.
Ratings are planning guidance. Whether a model fits depends on precision, context length, batch size and the software used; always measure.
From laptop to large-scale: what changes
| Stage | Hardware | What changes in your code |
|---|---|---|
| Learn and prototype | Your laptop (CPU, maybe a small GPU) | Nothing special: small data, small models, short runs. |
| Serious experiments | One desktop or cloud GPU (16–24 GB) | Move model and data to the GPU (.to("cuda")), use mixed precision, larger batches. |
| Bigger models | One large GPU (48–96 GB) or several | Gradient accumulation, gradient checkpointing, quantised loading for fine-tuning (QLoRA). |
| Many GPUs | A multi-GPU server or cluster | Data parallel training (DDP), then sharded training (FSDP or DeepSpeed) when the model no longer fits one GPU. |
| Production serving | GPU servers behind a load balancer | A dedicated LLM server (for example vLLM), batching, autoscaling and monitoring. |
The laptop-to-GPU path is shown with real code in Train a small language model on your laptop and From laptop to big hardware: scaling training.
Model API, own machine or cloud GPU?
Use a model API when
You are building products, RAG or agents and do not need to change model weights. Check how the provider handles and stores your data.
Use your own machine when
You do classical ML, small deep-learning and language-model experiments, local models, or data must not leave the device.
Rent cloud GPUs when
A job does not fit your machine, needs several GPUs, or runs rarely. Estimate GPU-hours × hourly rate + storage, set a budget alert, and shut machines down when idle.
Check your own machine
python --version # Python 3.10 or newer
python -c "import sklearn; print(sklearn.__version__)"
nvidia-smi # NVIDIA GPU name, memory and driver (if you have one)
python -c "import torch; print(torch.cuda.is_available())" # True if PyTorch can use the GPU
dotnet --version # .NET SDK versionOn Apple silicon Macs, PyTorch uses the GPU through torch.backends.mps.is_available(). Set up your laptop for AI and ML in Python and .NET walks through the full setup.
Hardware terms
- VRAM
- Memory on the GPU. Model weights, KV cache and training data in use must fit here.
- System RAM
- Main memory; holds datasets and can hold model parts that do not fit the GPU (much slower).
- CUDA
- NVIDIA's software platform for GPU computing; PyTorch builds target specific CUDA versions.
- Mixed precision
- Training with 16-bit numbers where safe and 32-bit where needed; faster and uses less memory.
- NVMe SSD
- Fast storage; models and datasets load faster.
- Board power
- How much power the GPU can draw; size the power supply and cooling to match.