A Trillion Parameters, 49 Awake: Running DeepSeek V4 Class Models Locally

The headline number made every local-runner wince: 1.6 trillion parameters. Your machine runs 7B models and feels proud about it. Case closed, surely — a model that size belongs on a data-center cluster, not a desk.

Then comes the second number: 49 billion active per token. And that number changes the conversation completely, because modern open-weight giants like the DeepSeek V4 family are Mixture-of-Experts models — and MoE doesn't play by the arithmetic you learned on dense models. The catch is that it doesn't play by any simple arithmetic, and the local-operator story is genuinely new: models with frontier-class ability whose running cost is small, but whose sitting-on-your-disk cost is enormous.

Here's what the open-weight MoE generation actually means for a local stack — the hardware math nobody spells out, the context-window trap inside the "1 million tokens" spec, and where these models fit in the tiers of a local-first setup.

MoE in Plain English: The Hospital Model

Picture a hospital. A dense model is a hospital where every patient is seen by every doctor on staff — brilliant, thorough, absurdly expensive. A Mixture-of-Experts model is a hospital with a triage desk: your symptom page summons the two or three specialists who actually know your problem, and the other 120 doctors stay in the break room.

• Total parameters = the full staff roster. It determines how big the building is — your RAM and disk bill.
• Active parameters = who actually walks into your room. It determines how long the visit takes — your tokens-per-second and the per-token compute.
• Routing = the triage nurse, a small learned component that decides which experts each token needs.

That's why the two numbers in the spec sheet live such separate lives. Flash's "284B total, 13B active" means it thinks like a 13B model per token — fast, cheap — while knowing like something far larger, because across a whole document many different experts get consulted.

The Local Math Nobody Spells Out

Here's the part that trips everyone up, usually after the download finishes and the machine swaps: you pay memory for the whole roster, even though only a few doctors work at a time. Different tokens route to different experts, so all experts must be resident.

Read the table twice and the strategy falls out:

• The Flash tier is the local prize. With quantization, high-RAM machines (a 128 GB laptop, a 192 GB workstation) hold the full expert roster and get large-model insight at small-model speed. This is the first time that trade has existed at a desk.
• The Pro tier is not a desk model. ~1.6T weights, even aggressively quantized, is a multi-GPU server or a rented-GPU job. Open weights mean you can host it privately — on hardware you rent by the hour, with your data on it and nothing else.

Quantization and MoE

Quantization (running weights at 4-bit-ish precision instead of 16) shrinks the building for dense and MoE models alike — the roster costs a quarter of the office space. MoE adds one nuance: because each token only consults a few experts, some runners quantize the rarely-routed experts more aggressively than the hot ones, squeezing extra room out of the cold specialists. Practical takeaway: Q4-style MoE builds are the ones targeted at consumer hardware; start there, and only reach for higher precision if you can name the task where it hurts.

The 1M-Token Window Has a Meter Running

The spec sheet says one million tokens of context. The spec sheet is telling the truth — and hiding the bill. As covered in the context-window piece, the KV cache (the model's working notes on your conversation) grows with context, and it grows on your RAM. On a hosted API, a giant window is a pricing line; locally, it's memory you physically own.

So the local practice stays the same, just with bigger numbers: set num_ctx to what the task needs, not what the model advertises.

# the million-token window, sized honestly
ollama run v4-flash
>>> /set parameter num_ctx 32768    # real task, real memory
# 1M only for the whole-codebase day, on the machine that can hold it

Hybrid-attention tricks reduce how fast that cache grows — that's the engineering these releases brag about, and it's real — but "grows slower" is not "free." A million live tokens still costs gigabytes. The rolling-summary habit and knowledge-folder retrieval from earlier posts remain exactly the right companions to a big-window model: window for the task, files for the library.

Where the Tiers Land in a Local Stack

The MoE generation slots cleanly into the tier structure of a local-first setup:

The interesting shift: the workhorse tier just got an upgrade that used to require the deep bench. That's the whole significance of open-weight MoE for local operators — the boundary between "what I run" and "what I rent" moved.

Wiring It In: Nothing New Required

The quiet benefit of having built the stack the model-agnostic way: a new open-weight giant is a Tuesday, not a replatform.

# release day, local stack edition
ollama pull v4-flash-q4
npx promptfoo@latest eval -c evals/core-tasks.yaml \
  --provider openai:gpt-4o-mini:http://localhost:11434/v1:v4-flash-q4
# green -> flip MODEL_TAG behind the one door; red -> stay, with the failing list in hand

One door, one tag, one eval suite — the same ten-minute ceremony as any model swap, which is precisely the point of having built it that way. The eval step matters more with MoE newcomers, not less: routing behavior can make a model oddly strong on some tasks and oddly mediocre on neighbors, and your suite — not the leaderboard — tells you which side of that line your workloads sit on.

The Residency Bonus

One more thing the open weights buy, worth stating plainly for anyone in regulated work: when you host the weights, the compliance story collapses to your own walls. Hosted APIs route your prompts to someone else's jurisdiction; self-hosted weights keep the data and the model in the same controlled environment, whether that's a hospital cluster, a government rack, or the desk machine in a Bangkok apartment. For healthcare, legal, and finance workloads, "the model never phones home" isn't a feature — it's the procurement argument.

Make It Practical This Month

• Check your RAM against the Flash-tier roster size before downloading anything — the swap-fest of an oversized model is the one failure mode nobody warns you about.
• Pull a Q4 MoE build, set num_ctx honestly, and run your eval suite against your current daily driver. Let the numbers place it in a tier.
• Keep the retrieval-and-summaries habit even with a giant window; big windows forgive more, but they never forgive for free.
• If a Pro-tier job is real (whole codebase, book-length analysis), rent the GPUs for the afternoon — with open weights, the model travels to the metal, not your data to a vendor.

The trillion-parameter headline was about the industry. The 49-billion-awake detail is about your desk — and it's the first time the interesting number was the small one.

Frequently Asked Questions

If only 13–49B parameters are active, why can't my 32 GB machine run it? Because the inactive experts aren't deleted — they're on call. Different tokens route to different specialists, so every expert must be resident in memory for when the router calls them. Active parameters decide speed; total parameters decide the memory bill. A swap-happy machine is the usual first symptom of skipping that arithmetic.

Do quantized MoE builds lose the intelligence that made the model famous? Some, and mostly in the far corners. Q4-style builds keep the bulk of benchmark performance for everyday work; degradation tends to show up first in edge-of-ability tasks like formal math and nuanced long-context reasoning. The practical answer is the eval suite: run your real workloads against the quantized build, and let your own pass rate decide whether the memory savings are worth it.

Can I actually use a 1M-token context window locally? Mechanically yes, economically rarely. The KV cache for a million live tokens costs substantial RAM on top of the weights, even with hybrid-attention improvements slowing its growth. Reserve the full window for the genuine whole-codebase or whole-book days on hardware that can hold it; use retrieval plus rolling summaries for daily work, exactly as with small-window models.

Is self-hosting open weights compliant for regulated data? It's the strongest starting position, not an automatic pass. Weights on your hardware mean no prompts leave your environment — but you still own the operational controls: access, logging, backups, and the physical security of the machine. Regulated shops should treat self-hosted models like any other on-prem system: audit the box, not just the brochure.