Hardware for Qwen — and for the models that come after it

The baseline is a 24 GB-class consumer card. It runs the 4-bit Qwen3.8-27B at roughly 45–70 tokens per second with the model taking ~18 of the 24 GB — which is exactly the problem. A purchase made to run a 27 B model is already full on day one. The real target is three things at once:

The workload behind the question: many-hours, unattended, sequential analytical reads over structured financial data — batch throughput and headroom matter more than latency. Nothing here needs a model to answer in under a second.

Marketing sells cores and TFLOPS. For running language models locally, those are a distant third. The order is:

1. Memory capacity decides what you can run. A model's weights have to sit in fast memory in their entirety, every token. If it doesn't fit, it either doesn't run or it spills to system RAM and slows 5–10×. At 4-bit quantisation the rule of thumb is 0.5–0.6 GB per billion parameters, plus room for context.

2. Memory bandwidth decides how fast. Generating each token means reading every weight from memory once. So the ceiling on tokens per second is simply bandwidth ÷ model size in memory. A 17 GB model on a 1,792 GB/s card cannot exceed ~105 tok/s single-stream no matter what; the same model on a 256 GB/s box cannot exceed ~15. Software (speculative decoding, multi-token prediction) beats that ceiling by guessing several tokens per read — that is where the "tuned" figures below come from.

3. Compute decides prompt processing and batching. Reading a long document in (prefill) and serving many requests at once are compute-bound. This is where a discrete GPU is 3–5× a Mac at the same bandwidth, and it is the number that matters most for fine-tuning.

Ceiling = bandwidth ÷ 17 GB (the 4-bit Qwen3.8-27B in memory). Both numbers appear per machine in the candidates table below, next to what was actually measured.

2026 is a memory-shortage year. Hyperscalers are spending close to $700 billion on AI infrastructure and have bought the entire 2026 output of high-bandwidth memory; every gigabyte of HBM consumes 3–4× the wafer capacity of ordinary DRAM, so ordinary DRAM, GDDR7 and NAND are starved. Memory is more than 80% of the bill of materials on a high-end graphics card. When memory doubled in early 2026, cards followed — and NVIDIA is on its third GeForce price hike of the year. Industry consensus is that the squeeze lasts through 2026 and into 2027; the HBM gap does not close until 2028–29.

Two consequences for a buyer. First, waiting is not obviously rewarded — the usual "prices fall after launch" curve has inverted, and the one refresh that would help (a 24 GB RTX 5080 SUPER) is finished but held back because its memory got too expensive to ship at the planned price. Second, the answer to "isn't a workstation only a few thousand more?" has changed underneath the question, which is the next section.

The intuition comes from 2025, when it was true. A 96 GB RTX PRO 6000 Blackwell pre-ordered below $8,000 while a 32 GB RTX 5090 listed at $2,000 — the pro card cost about $6,000 more and bought three times the memory. That gap is now $9,000–12,000 on the card alone ($13,250–16,000 vs $4,200–5,100), and a workstation is the card plus a platform built to feed it.

What the extra money actually buys, in order of how much it matters for this use:

So the honest answer: a workstation is the only NVIDIA path to the 100 B+ class on one card, and it costs about three times a consumer box, not "a few thousand more." What the price shock has done is open a lane that did not exist in 2025 — Apple's 512 GB Mac Studio at roughly $11,000 now holds more model memory than a $35,000 dual-PRO-6000 workstation, at two-thirds of the bandwidth and with far less compute. That trade is the crux of the candidate table.

The pattern of 2026 is unambiguous: every lab ships one small dense model that fits a desk card (Qwen's 27 B, Gemma's 27 B) and one or more large mixture-of-experts models where only a fraction of the parameters fire per token. MoE is what makes big models viable on unified-memory boxes — a 120 B model with 5 B active reads only ~3 GB per token, so even 256 GB/s feels fine — but all of the weights still have to sit in memory. Capacity, not bandwidth, is what gates the next generation.

Read as tiers: 24–32 GB runs the dense 27–32 B class and nothing above it. 96–128 GB is the 100 B–120 B MoE class — the models that are, right now, the strongest thing you can run on one machine. 256–512 GB reaches 235 B–400 B. The trillion-parameter flagships (Qwen3.8-Max, DeepSeek V4 Pro, Kimi K3) are out of reach of any single machine anyone can buy; those are API models regardless of the licence.

Memory figures use 0.55 GB per billion parameters at 4-bit plus context headroom; vendor model sizes from the Wavect open-weight comparison (reviewed Aug 7, 2026), Unsloth docs and the Qwen3.8 release notes. "Runs it" assumes a single machine and 4-bit weights.

The question that decides how high to aim: as models get bigger, how much smarter do they get? Capability here is the Artificial Analysis Intelligence Index — one number from nine independent evaluations, on the same scale for every model — plotted against the memory each model needs at 4-bit. The best closed model (not purchasable at any price) is the dashed reference.

Where this leaves the "how high to aim" question: the evidence says aim for the shelf the next generation of small-and-mid models will land on, not for today's giants. The 100–180 GB tier already holds two models at 56–57.5 (Qwen3.8-Flash-Next 180B-A6B, GLM-5.3-Flash 320B) — within five points of the closed frontier — and both fit a 128 GB box or a 512 GB Mac, neither of which needs a workstation. The 96 GB card is the fast way to run that tier; the 512 GB Mac is the roomy way. Nothing above 512 GB buys measurable capability today. Index: Artificial Analysis v4.1.1 via artificialanalysis.ai and BenchLM, Snapshot August 28, 2026; memory at 0.55 GB per billion parameters plus headroom. Vendor-reported scores are excluded.

One table, one set of machines. Everything is buyable today or on announced pre-order, US pricing, quoted as card / complete system where both exist — "graphics card only" rows need a PC around them; "complete machine" rows are the whole box. "Ceiling" is bandwidth ÷ 17 GB, the hard single-stream limit for the 4-bit Qwen3.8-27B; the figure under it is what people actually measured. "Class" is the largest model family the machine runs at a usable speed.

Price sources on the verification date: videocardprices.com and Newegg trackers (RTX 5090, PRO 6000), NVIDIA US marketplace (PRO 6000 MSRP), Apple pre-order pricing via TechRepublic (M5 Ultra), IntuitionLabs (DGX Spark), Layer3Labs / vendor listings (Strix Halo mini PCs), Compute Exchange and JarvisLabs (used A100 / H100), ServeTheHome and Tom's Hardware (DGX Station), RunPod / Spheron (rental). Benchmarks: Context Studios (Aug 19), Hardware Corner (Mar 11), ComputingForGeeks (Aug 6), IntuitionLabs (Aug 12), RunAIHome (Jun 3), all 2026.

Strip away the also-rans and two machines survive goal #2. They are opposites.

The way to choose is to decide which sentence is true for you:

"I will spend most of the machine's life running one model as hard as possible, and I want to train it." → the PRO 6000. Speed, CUDA and batch throughput compound over a year of unattended runs, and fine-tuning is a first-class citizen. It caps at the 120 B class, which is the current frontier and likely stays a serious tier through 2027.

"I want to be able to try whatever gets released, including the 235 B–400 B models, and I mostly run, not train." → the M5 Ultra 512 GB. Nothing else under $30,000 holds those models. Accept slower tokens and a thinner training story.

Fine-tuning does not mean retraining. The practical methods freeze the model and train small adapter layers (LoRA), optionally on a 4-bit base (QLoRA). Memory need is driven by whether the base is held at 4-bit or 16-bit, and the quality gap between them is smaller than the memory gap.

Two things worth knowing before the purchase is justified by "we'll fine-tune it". First, most of the improvement people want from a local model — domain vocabulary, output format, house style, using a private corpus — comes cheaper from retrieval and prompting than from training, and needs no extra hardware. Second, when training genuinely is the answer, renting eight H100s for an afternoon is often cheaper than owning the capacity for it; the local machine's job is then to run the result. A 96 GB card is the sweet spot precisely because it does both for the 27 B–70 B range without a rental.

A single RTX 5090 is not an upgrade for this purpose. It is the same 32 B ceiling as the 24 GB class, faster. Buying it in 2026 at 2.5× MSRP to gain speed on a model that already runs is the one clearly wrong move.

The 100 B-class MoE tier is the minimum "next year" target, and 96 GB is the smallest memory that serves it fast. Every serious open release of 2026 has landed there or above; a machine that cannot hold ~70 GB of weights will be running last year's models by next summer.

If the answer must be one machine: the RTX PRO 6000 workstation for a training-and-throughput operation, the M5 Ultra 512 GB for a run-everything operation. The price gap between them (~$9,000) is the price of CUDA and speed; the memory gap (416 GB) is the price of reach. Neither is a bad buy; they are different bets on what the next year's models look like — and the evidence of 2026 is that they get bigger faster than cards get cheaper.

A defensible third path: a $1,500–2,000 Strix Halo box today to learn what the 120 B MoE class is actually worth on real work, before committing $11,000–20,000. It is slow, but it answers the only question that matters — does a bigger model change the output enough to pay for — for the price of a graphics card's sales tax.

Not investment advice, and not a hardware endorsement: prices in this market change weekly and every figure above carries its verification date. Re-check the trackers cited before spending money.

RTX 5090 pricing — videocardprices.com tracker, Tech Insider GPU prices 2026. RTX PRO 6000 pricing — Tom's Hardware, VideoCardz, Thunder Compute. RTX 5090 vs PRO 6000 benchmarks — ComputingForGeeks, Spheron. RTX PRO 5000 72 GB — Spheron. Qwen3.8-27B per-hardware ladder — Context Studios, Unsloth GGUF report. Mac Studio M5 Ultra — TechRepublic, ModelFit; M5 Max benchmarks — Hardware Corner. DGX Spark — IntuitionLabs. DGX Station — ServeTheHome. Strix Halo — RunAIHome, Layer3Labs. Dual-5090 workstations — Puget Systems, Petronella build guide. Used A100 / H100 — Compute Exchange, JarvisLabs. Memory shortage — Barrack AI, TrendForce, Tom's Guide (RTX 50 SUPER). Open-model sizes — Wavect comparison, the Qwen3.8-Max memory math. Fine-tuning memory — Spheron VRAM guide, VRLA Tech. Bandwidth rule of thumb — Runyard, InventiveHQ. Quarterly practitioner guide — Julien Simon, April 2026.