- Memory decides the card, speed comes second
- Reading the VRAM table by model size and method
- Why those VRAM numbers are a floor, not a target
- Five steps from model size to a rented card
- Where to rent: six GPU clouds and the memory they list
- My verdict on picking hardware for a first run
- Frequently asked questions
Choose a GPU by memory first. Work out your model's parameter count and whether you'll use LoRA, QLoRA or full fine-tuning, because the method sets the VRAM you need. Pick the smallest card with that memory plus headroom for batch size and context length, then rent it by the hour from a GPU cloud rather than buying.
GPU spec sheets are full of numbers, and nearly all of them are a distraction when you're fine-tuning. The one that decides whether your job runs at all is video memory. Get that right and the rest is a matter of how long you're happy to wait.
Memory decides the card, speed comes second
Two things drive how much VRAM a fine-tuning job needs: the size of the model, counted in parameters, and the method. QLoRA loads the model in 4-bit, LoRA in 16-bit, and full fine-tuning updates every weight, which needs far more. Change either of those and the card you need changes with it.
If you're still weighing whether a GPU is the right hardware at all, our comparison of CPUs and GPUs for AI work covers the basics.
Reading the VRAM table by model size and method
Unsloth publishes a requirements table that lists the VRAM needed for each model size, from 3B up to 405B parameters, under QLoRA and LoRA. Two rows show how much the method matters:
| Model size | QLoRA (4-bit) | LoRA (16-bit) |
|---|---|---|
| 7B | 5 GB | 19 GB |
| 70B | 41 GB | 164 GB |
The same model needs several times the memory under LoRA as it does under QLoRA. That gap is often the difference between a single mid-range card and a multi-GPU machine. Hugging Face's PEFT docs make the same point from the other side, saying QLoRA lets you fine-tune a 65B parameter model on a single 48GB GPU.
Why those VRAM numbers are a floor, not a target
Here's where people come unstuck. Unsloth's own requirements page calls its table the absolute minimum, says some models need more, and notes that running out of memory is commonly caused by setting the batch size too high. Treat the table as the floor you can't go below, not a size that'll run comfortably.
So plan headroom above the listed number for batch size and context length. If a run still runs out of memory, the same page suggests dropping the batch size to somewhere between 1 and 3 before you reach for a bigger card.
Five steps from model size to a rented card
- Write down the parameter count and your method (QLoRA, LoRA or full). The sizing table above is sorted by exactly these two things.
- Look up the VRAM for that pair and add headroom. The listed number is a minimum.
- Match the memory to a card. Compare the VRAM column on each provider, for example 16, 48, 80, 96 or 141 GB, and pick the smallest that fits with room to spare. Lambda, RunPod, Hyperstack, TensorDock and Vast.ai all show VRAM per GPU.
- Rent it by the hour and run a short trial with a small batch. Check memory use before you launch the long run, so you can step up a card if the trial runs out.
- Go multi-GPU or managed if one card isn't enough. Large models and full fine-tuning can exceed a single card. Multi-GPU instances help, or a managed fine-tuning service means you don't size hardware at all.
Where to rent: six GPU clouds and the memory they list
Lambda
On-demand instances with 1 to 8 NVIDIA GPUs for training, fine-tuning and serving, with options including B200 (180 GB), H100 (80 GB), A100 (80 and 40 GB) and V100 (16 GB). Instances come with Lambda Stack preinstalled, so there are no drivers to install.
RunPod
Pods with on-demand GPUs across many regions. Its listing shows VRAM, RAM and vCPUs side by side for each card, such as B200 at 180 GB, RTX Pro 6000 at 96 GB and H100 at 80 GB, and it has a page for fine-tuning use cases.
Vast.ai
You search GPUs by model, VRAM and availability through the console or API, then deploy through the CLI, SDK or API. Being able to filter by VRAM directly is exactly what step three needs.
TensorDock
Virtual machines with dedicated GPUs and root access. It says it offers 45 GPU models, from the RTX 4090 up to the H100 SXM5 80GB, with KVM virtualisation and full OS control.
Want AI doing the heavy lifting in your marketing?
I build the systems that handle the boring 80 percent, so you get your week back. Done properly, with the human kept in.
Hyperstack
Its table lists VRAM for every model, including RTX Pro 6000 SE at 96 GB, H200 SXM at 141 GB, H100 SXM at 80 GB, A6000 at 48 GB and A4000 at 16 GB. It also bills in per-minute cycles, which suits short trial runs.
CoreWeave
An AI cloud with larger training clusters and a Serverless SFT product that fine-tunes open models with nothing to provision. That's the managed route from step five, where you skip choosing a card entirely.
For a broader comparison of providers, see our roundup of cloud GPU providers and the list of GPU rental platforms.
My verdict on picking hardware for a first run
Size by memory, never by the name on the card. Start with QLoRA if your task allows it, look up the minimum in the sizing table, add headroom, and rent the smallest card that clears it for a short trial. You'll learn more from one short test run than from an afternoon of spec sheets, and renting means a wrong guess only means stepping up a size.
Want your tool on this list? Featured spots and reviews here are paid partnerships. Here's how to partner with me.
Frequently asked questions
How much VRAM do I need to fine-tune a 7B model?
Unsloth's table lists 5 GB for QLoRA (4-bit) and 19 GB for LoRA (16-bit), as absolute minimums. Plan headroom above that.
What's the difference between QLoRA and LoRA memory needs?
QLoRA loads the model in 4-bit and LoRA in 16-bit, so QLoRA needs far less VRAM for the same model size, as the sizing table above shows.
Can I fine-tune a large model on a single GPU?
Hugging Face's PEFT docs say QLoRA lets you fine-tune a 65B model on a single 48GB GPU.
Do I need to buy a GPU?
No. GPU clouds rent cards on demand, so you can try a card on a short run before committing to a long one.
What if I run out of memory mid-run?
Lower the batch size to between 1 and 3, or move to a card with more VRAM.
