Measured .gguf file sizes from the Unsloth release, with KV cache derived
from the model's own config. Qwen 3.8 is a hybrid: only 16 of its 64 layers use full
attention, so the cache is a quarter of what a naive layer count suggests.
| Quantization | Weights | KV | Total | Headroom | Fits |
|---|
Weight sizes are real .gguf byte sizes returned by the Hugging Face API for
unsloth/Qwen3.8-27B-GGUF.
Measured, not derived from a bits-per-weight formula — imatrix quants don't follow one.
KV cache comes from config.json,
counting only the full-attention layers (see note 02).
Corrected 2026-08-20: an earlier version of this page counted all 64 layers and overstated KV cache by 4×. Thanks to u/RISCArchitect on r/LocalLLM for catching it.
2 × full_attn_layers × kv_heads × head_dim × context × bytes
Qwen 3.8 is a hybrid architecture. Its layer_types array shows
16 full_attention layers and 48 linear_attention (DeltaNet) layers, with
full_attention_interval: 4 — every fourth layer. Only the 16 full-attention
layers hold a KV cache that grows with context.
So at F16 that is 16 × 4 × 256 × ctx × (bytes_K + bytes_V) — about
0.0625 GB per 1K tokens, plus a fixed ~0.07 GB of DeltaNet recurrent state that
does not scale with context at all.
K and V are separate tensors and llama.cpp quantizes them independently
(--cache-type-k / --cache-type-v). V tolerates lower precision
than K, so Q8/Q5 and Q8/Q4 splits are common at high context — at 160K on
24GB, Q8/Q4 costs 3.8 GB of cache against 10.1 GB for F16/F16.
Counting all 64 layers — as most calculators do, this one included until u/RISCArchitect pointed it out — overstates the cache by exactly 4×. At 200K that is the difference between a computed 48.8 GiB and the real 12.2 GiB.
Reserved for CUDA or Metal context, activations, and compute buffers. A quant that exactly equals your VRAM will not load. Tight means it fits with under 1.5 GB spare — expect trouble if anything else touches the GPU.
Multi-GPU splits, speculative decoding drafts, vision-tower memory during image input, and batch sizes above one. All of those add. Read the result as a ceiling for single-GPU, single-stream text inference.