Evergreen Tools
|
← Back to AI Tools

POCKET-Darwin-180B: A 180B GGUF Model That Runs on a Laptop

VIDRAFT released POCKET-Darwin-180B on 2 October 2026: a 4-bit GGUF quantisation of its 180B-class Darwin-180B-RSI model that the publisher says shrinks a roughly 360GB BF16 model to about 111GB across four files, using sparse MoE activation and llama.cpp SSD streaming so it can run on consumer hardware without a GPU

Tool Overview

Features, steps and FAQ below

Features

  • ✓ Compression: the publisher says the model went from a 360GB BF16 build to an approximately 111GB 4-bit GGUF split into four files, with MMLU-Pro unchanged at 87.65 versus the full-precision model
  • ✓ Sparse activation: as a mixture-of-experts model it activates only about 3 billion parameters per token, so CPU inference stays practical; the publisher says this cuts hardware requirements by roughly 250x compared with the full-precision server setup
  • ✓ Throughput: in the publisher's own testing it reaches up to around 21 tokens per second on a single server CPU and about 4.17 tokens per second on a laptop
  • ✓ How it runs: it is llama.cpp compatible and runs on consumer hardware such as a gaming laptop with 8GB VRAM and 32GB RAM, with no enterprise GPU cluster required; weights ship on both Hugging Face and ModelScope
  • ✓ Where it fits: on-premise, air-gapped or local inference without a cloud account or dedicated GPU server, following VIDRAFT's naming strategy where RSI stands for capability and POCKET for deployability

How to Use

  1. Download FINAL-Bench/POCKET-Darwin-180B-GGUF from Hugging Face (or fetch it from ModelScope)
  2. Load the Q4_K_XL GGUF with llama.cpp to run inference (CPU is enough; a GPU helps but is not required)
  3. Run it on a fixed test set and compare against the full-precision baseline to confirm quality after quantisation
  4. Deploy it locally or in an air-gapped environment for inference that does not need a network connection

FAQ

What is POCKET-Darwin-180B?

A 4-bit GGUF quantised model released by VIDRAFT on 2 October 2026, a compressed, locally executable build of the 180B-class Darwin-180B-RSI, published on both Hugging Face and ModelScope.

Does it need a GPU?

The publisher says no: sparse MoE activation plus llama.cpp SSD streaming lets it run on consumer hardware such as a laptop with 8GB VRAM and 32GB RAM, with a GPU helpful but not required.

Does quantisation hurt quality?

The publisher says no: this 4-bit GGUF scores 87.65 on MMLU-Pro, the same as the full-precision model's 87.65. Multiple write-ups nonetheless advise always measuring a quant against its baseline on a fixed test rather than judging it by eye.

How fast is it?

In the publisher's own figures, up to around 21 tokens per second on a single server CPU and about 4.17 tokens per second on a laptop.

Why does it matter?

It lowers the bar for running a 180B-class model from an enterprise GPU cluster to consumer hardware, which suits developers who need large models locally, offline or in air-gapped settings.