Qwen3.6 VRAM requirements: 27B vs 35B-A3B
Short answer: use a 24GB GPU for Qwen3.6-27B Q4, or 32GB for a comfortable Qwen3.6-35B-A3B Q4 setup. A 16GB GPU can run either with a smaller quant or CPU offload, but it is not a clean full-GPU Q4 tier.
Estimates, not benchmarks · 4K text context · working set includes loaded weights, architecture-aware KV cache and runtime · recommended tiers include 1.5GB allocation headroom
Which Qwen3.6 should you download?
Qwen 3.6 27B
You want dense-model consistency, vision/coding quality, and have a 24GB GPU or 32GB+ unified memory.
Qwen 3.6 35B-A3B
You value faster generation, coding agents, or need expert offload to system RAM. Prefer 32GB VRAM or 48GB+ unified memory.
Qwen3.6 download and memory by quant
Download size is not VRAM required. The working-set estimate adds loaded-weight overhead, the 4K text cache, and runtime memory; the safer tier also leaves 1.5GB for allocations. Vision inputs and longer context need more.
Qwen 3.6 27B
| Quant | Download | Est. working set @4K | Safer hardware tier |
|---|---|---|---|
| Q3_K_M | 13.6 GB | ~15.1 GB | 24 GB |
| IQ4_XS | 15.4 GB | ~17 GB | 24 GB |
| Q4_0 | 15.8 GB | ~17.4 GB | 24 GB |
| Q4_K_S | 15.9 GB | ~17.5 GB | 24 GB |
| Q4_K_M ★ | 16.8 GB | ~18.5 GB | 24 GB |
| Q5_K_M | 19.5 GB | ~21.3 GB | 24 GB |
| Q6_K | 22.5 GB | ~24.5 GB | 32 GB |
| Q8_0 | 28.6 GB | ~30.9 GB | 48 GB |
Qwen 3.6 35B-A3B
| Quant | Download | Est. working set @4K | Safer hardware tier |
|---|---|---|---|
| Q3_K_M | 16.6 GB | ~18.1 GB | 24 GB |
| IQ4_XS | 17.7 GB | ~19.3 GB | 24 GB |
| Q4_K_S | 20.9 GB | ~22.6 GB | 32 GB |
| MXFP4 | 21.7 GB | ~23.5 GB | 32 GB |
| Q4_K_M ★ | 22.1 GB | ~23.9 GB | 32 GB |
| Q5_K_M | 26.5 GB | ~28.5 GB | 32 GB |
| Q6_K | 29.3 GB | ~31.4 GB | 48 GB |
| Q8_0 | 36.9 GB | ~39.4 GB | 48 GB |
Best Qwen3.6 quant for your GPU
Largest tracked quant that fits at 4K text context while preserving 1.5GB headroom. “Offload” means the model can still run with enough system RAM, but not entirely in VRAM.
| GPU VRAM | Qwen3.6-27B | Qwen3.6-35B-A3B | System RAM guidance |
|---|---|---|---|
| 8 GB | CPU offload | Expert/CPU offload | 32–64GB RAM |
| 12 GB | CPU offload | Expert/CPU offload | 32–64GB RAM |
| 16 GB | CPU offload | Expert/CPU offload | 32–64GB RAM |
| 24 GB | Q5_K_M | IQ4_XS | 32GB minimum; 64GB useful |
| 32 GB | Q6_K | Q5_K_M | 32GB+ RAM |
| 48 GB | Q8_0 | Q8_0 | 32GB+ RAM |
| 64 GB | Q8_0 | Q8_0 | 32GB+ RAM |
| 80 GB | Q8_0 | Q8_0 | 32GB+ RAM |
How context length changes memory
Qwen3.6 is not a conventional all-attention Transformer: three linear-attention layers are followed by one full-attention layer. The estimates below count the token-growing KV cache only on those full-attention layers, based on the official model configs. Runtime, cache precision, vision input and implementation still change real usage.
| Context | 27B cache | 27B total (Q4_K_M) | 35B-A3B cache | 35B-A3B total (Q4_K_M) |
|---|---|---|---|---|
| 4K | ~0.3 GB | ~18.5 GB | ~0.1 GB | ~23.9 GB |
| 32K | ~2 GB | ~20.2 GB | ~0.6 GB | ~24.4 GB |
| 128K | ~8 GB | ~26.2 GB | ~2.5 GB | ~26.3 GB |
| 256K | ~16 GB | ~34.2 GB | ~5 GB | ~28.8 GB |
Sources: official Qwen3.6 model configs and weights. These remain planning estimates—not measured peak allocation.
How to run Qwen3.6 locally
Pick the runtime that matches your hardware. Start at 4K or 8K context, confirm the model is stable, then increase context—advertised maximum context is not a sensible starting allocation.
| Runtime | Best for | Recommended path |
|---|---|---|
| llama.cpp | NVIDIA, AMD, CPU offload | Download a trusted GGUF, start with -c 4096, and tune GPU layers to your VRAM. |
| LM Studio | Windows/macOS GUI | Choose the quant from the tables above; verify estimated memory before loading. |
| MLX | Apple Silicon | Use an MLX conversion and leave macOS several GB of unified-memory headroom. |
| Lily | M5+ Apple Silicon, narrow fast path | Only supports Qwen3.6-35B-A3B in one MLX affine 4-bit/group-64 layout on macOS 26+; use it only if its strict API fits your workload. |
| Transformers / vLLM | Servers and full precision | Use the official checkpoint; FP8/BF16 typically requires multi-GPU or data-center memory. |
Official resources: Qwen3.6 repository, 27B weights, and 35B-A3B weights.
Run Qwen3.6 faster on Apple Silicon with Perplexity Lily
Lily is an Apache-2.0 Rust and Metal inference server specialized for exactly one model: Qwen3.6-35B-A3B converted to MLX affine 4-bit weights with group size 64. PyTorch and MLX are not in its execution path, but the specialization comes with hard compatibility limits: Apple GPU family 10 (M5 or newer), macOS 26+, and no GGUF, AWQ, GPTQ, dense Qwen, other quant layouts, streaming, tools, multimodal input, or sampling controls.
git clone https://github.com/perplexityai/pplx-garden.git
cd pplx-garden/lily
hf download mlx-community/Qwen3.6-35B-A3B-4bit \
--revision 38740b847e4cb78f352aba30aa41c76e08e6eb46 \
--local-dir /path/to/Qwen3.6-35B-A3B-4bit
cargo build --release --locked
./target/release/lily --model /path/to/Qwen3.6-35B-A3B-4bit \
--bind 127.0.0.1:8000 --max-seq 4096| Perplexity M5 Max benchmark | Lily | MLX 0.32.2 | Lily advantage |
|---|---|---|---|
| Decode at 256-token context | 194.7 tok/s | 148.5 tok/s | 1.31× |
| Decode at 4K context | 182.8 tok/s | 145.6 tok/s | 1.26× |
| Decode at 131K context | 92.7 tok/s | 75.0 tok/s | 1.24× |
✓ Verified publisher benchmark · M5 Max 40-core GPU · 128GB unified memory · macOS 27.0 · median of two rounds · 64-token greedy decode. This compares production-engine throughput, not identical model graphs or HTTP latency; MLX computes full-vocabulary logprobs while Lily does not.
Use Lily when you have a compatible M5-class Mac and want fast, deterministic, text-only local chat behind a minimal OpenAI-compatible endpoint. Use MLX or llama.cpp when you need broader Mac support, another quant, vision, streaming, tool calling, sampling, or a general-purpose runtime. Perplexity publishes no tested 24GB configuration, so do not treat 24GB as an official Lily minimum; the published speed numbers come from a 128GB machine.
Can a phone run Qwen3.6?
Not usefully at the recommended Q4 quant. Three 24GB Android configurations in our database can memory-fit some smaller 27B quants, but estimated generation is below 3 tokens/s and the recommended Q4_K_M working set exceeds the usable-memory budget. Treat Qwen3.6 as a desktop or high-memory Mac model.
See the per-phone edge-case calculations →Qwen3.6 FAQ
How much VRAM does Qwen3.6-27B need?
The Q4_K_M file is 16.8GB and its estimated 4K text working set is ~18.5GB. A 24GB GPU is the practical recommendation for full GPU offload with useful headroom. A 16GB GPU needs a smaller quant or partial CPU offload.
How much VRAM does Qwen3.6-35B-A3B need?
The Q4_K_M file is 22.1GB and its estimated 4K text working set is ~23.9GB. It can technically sit near a 24GB limit, but 32GB is safer. On smaller GPUs, llama.cpp can keep experts in system RAM; the full model still needs to exist across VRAM and RAM.
Can Qwen3.6 run on a 16GB GPU?
Yes, with compromises. Qwen3.6-27B Q3_K_M has a 13.6GB file but our estimated working set plus safety margin exceeds 16GB. Use a shorter context and partial CPU offload with 32GB+ system RAM. The MoE 35B-A3B is often the better offload candidate because only about 3B parameters are active per token.
Which is better: Qwen3.6 27B or 35B-A3B?
Choose 27B for predictable dense-model behavior and quality per stored GB. Choose 35B-A3B for faster local decoding, coding agents, and better tolerance of expert offload. On 24GB, 27B Q4_K_M is the simpler fit; on 32GB+, both are strong options.
How much system RAM do I need when the model does not fit in VRAM?
Use at least 32GB system RAM for a 16GB GPU setup and preferably 64GB when running Qwen3.6-35B-A3B or long context. CPU offload needs enough RAM for the portion outside VRAM plus the OS, runtime, cache, and file mapping.
Can a phone run Qwen3.6 locally?
A few 24GB Android configurations can technically load low-bit 27B quants, but estimated speed is below a comfortable chat threshold. No phone we track fits the recommended 27B Q4_K_M working set. Qwen3.6 is a desktop, workstation, or high-memory Mac model in practice.
What Macs can run Qwen3.6 with Perplexity Lily?
Lily currently requires an M5-or-newer Mac, macOS 26 or later, and one exact Qwen3.6-35B-A3B MLX affine 4-bit checkpoint. Perplexity's published benchmark used an M5 Max with 128GB unified memory, so its speed results do not establish a 24GB minimum or equivalent performance on smaller Macs.