Qwen · 36B parameters · 3B active
Qwen3.6 35B-A3B: GPU memory requirements and which GPU to rent
A 2026 Qwen hybrid: 30 of its 40 layers use linear attention, so only 10 layers keep a growing KV cache while the others hold a fixed state per sequence. 36 billion parameters, 3 billion active.
- Weights at FP836 GB72 GB at 16-bit · 20 GB at 4-bit
- FP8, 8k context, 1 sequence39 GBweights + KV cache + runtime
- Suggested here1× L40S$429 a month · dedicated
Inference
How much GPU memory Qwen3.6 35B-A3B needs to serve
Total memory for one sequence at each context length, and for 16 concurrent sequences at 8k — weights, KV cache at 16-bit and runtime overhead included.
| Weights precision | Weights | 4k context | 32k context | 128k context | 8k × 16 sequences | Suggested here, 8k |
|---|---|---|---|---|---|---|
| 16-bit (BF16 / FP16) | 72 GB | 77 GB | 77 GB | 79 GB | 80 GB | 1× RTX PRO 6000 · $749/mo |
| 8-bit (FP8 / INT8) | 36 GB | 39 GB | 40 GB | 42 GB | 43 GB | 1× L40S · $429/mo |
| 4-bit (GPTQ, AWQ, Q4) | 20 GB | 22 GB | 23 GB | 25 GB | 26 GB | 1× L4 · $169/mo |
GB are 109 bytes. Context lengths are shown for scale where the model supports them; an FP8 KV cache halves the cache part of each figure. As a mixture of experts, every expert stays resident, so memory follows the total parameter count, not the active one. Thirty of its forty layers use linear attention: a fixed 60 MiB state per sequence instead of a growing cache.
Hardware
GPU servers that run Qwen3.6 35B-A3B
For FP8 inference with an 8k context and one sequence: the configurations we would suggest, in order — one large card ahead of several small ones.
Fine-tuning Qwen3.6 35B-A3B
Memory for a 2k-token sequence and a micro-batch of 4, with gradient checkpointing. Full fine-tuning keeps gradients and Adam states for every parameter; LoRA and QLoRA train small adapters on a frozen base.
| Method | Memory | Suggested here |
|---|---|---|
| QLoRA, 4-bit base | 27 GB | 1× RTX 5090 · $279/mo |
| LoRA, 16-bit base | 81 GB | 1× RTX PRO 6000 · $749/mo |
| Full fine-tune, mixed precision | 583 GB | 8× A100 · $5,192/mo |
Architecture at a glance
Thirty of its forty layers use linear attention: a fixed 60 MiB state per sequence instead of a growing cache. Figures from the model's published configuration; method on the calculator page.
Qwen3.6 35B-A3B hardware: questions answered
How much VRAM does Qwen3.6 35B-A3B need?
For the weights alone: about 72 GB at 16-bit, 36 GB at FP8 and 20 GB at 4-bit. Serving it at FP8 with an 8k-token context and one sequence takes about 39 GB in total once the KV cache and the runtime are counted; long contexts and many concurrent users add the KV cache on top — about 0.02 GB per 1,000 tokens at 16-bit.
Can Qwen3.6 35B-A3B run on a single GPU?
Yes, at FP8 with a moderate context: one NVIDIA L40S (48 GB GDDR6) holds it, for $429 a month here. Longer contexts or more concurrent users may need a larger card or a second GPU.
What is the cheapest GPU setup to run Qwen3.6 35B-A3B?
For FP8 inference with an 8k context and one sequence, the configuration we would suggest is 1× L40S · $429/mo; at 4-bit, 1× L4 · $169/mo. Suggestions favour one large card over several small ones: splitting a model across PCIe cards costs speed.
How much GPU memory do I need to fine-tune Qwen3.6 35B-A3B?
With a 2k-token sequence and a micro-batch of 4: about 27 GB for QLoRA on a 4-bit base, 81 GB for LoRA on a 16-bit base, and 583 GB for a full fine-tune with Adam in mixed precision. Gradient checkpointing is assumed; sharding the optimizer across GPUs spreads the full fine-tune over several cards.
Run Qwen3.6 35B-A3B on 1× L40S for $429 a month
Dedicated, single-tenant and live in under 10 minutes — with vLLM, the Hugging Face stack or Ollama preinstalled if you want. Paid in BTC, ETH or USDT, with no KYC.