Store › model › LLM
Qwen3.8-35B-A3B-Distill-GGUF
by empero-ai · source Hugging Face · updated 2026-09-16
apache-2.020 GB~24 GB RAMsource aliveunlabeled
GGUF quantizations of empero-ai/Qwen3.8-35B-A3B-Distill — a distillation of the Qwen3.8 frontier models into the Qwen3.6-35B-A3B Mixture-of-Experts architecture — for llama.cpp, Ollama, LM Studio, Ja…
Add to LogiShell Open in the web IDE
The button opens LogiShell with this card; nothing installs from a link by itself. Inside the app the install goes through lsh models install hf:empero-ai/Qwen3.8-35B-A3B-Distill-GGUF and its progress lives in the Resource Center.
Source and license
- Source: https://huggingface.co/empero-ai/Qwen3.8-35B-A3B-Distill-GGUF
- License: apache-2.0
- Requirements: about 24 GB of RAM, 20 GB on disk (estimate: suggested file size × 1.15 + 0.5 GB; a real measurement comes with lsh models). Runs with llama.cpp, ollama.
- Tags:
ggufllama.cppquantizedempero-aiqwen3.6qwen3.8distillationreasoningmoegated-deltanettext-generationenendpoints_compatibleconversational
Numbers
- 479,410 downloads on Hugging Face
- 169 likes
- license apache-2.0
- 20 GB for Qwen3.8-35B-A3B-Q4_K_M.gguf
- 327,838 npm downloads a week for node-llama-cpp
- latest node-llama-cpp@3.22.1
Numbers as of 2026-10-02 21:00 UTC, from the source API.
Summary
Summary not ready yet: the numbers are here, the text is not. It is written by the collector through the LogiShell model facade when a provider key is present.
Reviews
No reviews yet. Reviews are written inside LogiShell: open this card in the app.
Files
| file | quant | size |
|---|---|---|
Qwen3.8-35B-A3B-BF16.gguf | BF16 | 66 GB |
Qwen3.8-35B-A3B-IQ2_M.gguf | IQ2_M | 12 GB |
Qwen3.8-35B-A3B-IQ3_M.gguf | IQ3_M | 15 GB |
Qwen3.8-35B-A3B-IQ4_XS.gguf | IQ4_XS | 18 GB |
Qwen3.8-35B-A3B-Q2_K.gguf | Q2_K | 13 GB |
Qwen3.8-35B-A3B-Q3_K_M.gguf | Q3_K_M | 16 GB |
Qwen3.8-35B-A3B-Q4_K_M.gguf | Q4_K_M | 20 GB |
Qwen3.8-35B-A3B-Q5_K_M.gguf | Q5_K_M | 24 GB |
Qwen3.8-35B-A3B-Q6_K.gguf | Q6_K | 27 GB |
Qwen3.8-35B-A3B-Q8_0.gguf | Q8_0 | 35 GB |
mmproj-Qwen3.8-35B-A3B-F16.gguf | F16 | 858 MB |
From the source README
Qwen3.8-35B-A3B — GGUF
Developed by Empero
GGUF quantizations of empero-ai/Qwen3.8-35B-A3B-Distill — a distillation of the Qwen3.8 frontier models into the Qwen3.6-35B-A3B Mixture-of-Experts architecture — for llama.cpp, Ollama, LM Studio, Jan, KoboldCpp, and other stock GGUF runtimes.
This card is about choosing a file and running it. The capability writeup, benchmark results, and best practices live on the main model card.
35B total parameters with ~3B active per token — the MoE sparsity means it runs considerably faster than a dense 35B at the same quant, but the whole weight file still has to fit in RAM or VRAM.
> [!Note]
> Qwen3.6-class models are hybrids: 30 Gated DeltaNet layers and 10 full-attention layers, with 256 experts routed 8-per-token. A recent llama.cpp build with Qwen3.6 / Gated DeltaNet MoE support is required — older builds will fail to load the architecture.
Files
| File | Quant | Size | Notes |
|---|---|---:|---|
| `Qwen3.8-35B-A3B-IQ2_M.gguf` | IQ2_M | 12.558 GB | Smallest usable. Fits a 16 GB card. |
| `Qwen3.8-35B-A3B-Q2_K.gguf` | Q2_K | 13.839 GB | 2-bit K-quant; widest runtime support at this size. |
| `Qwen3.8-35B-A3B-IQ3_M.gguf` | IQ3_M | 16.340 GB | Strong quality per byte at 3-bit. |
| `Qwen3.8-35B-A3B-Q3_K_M.gguf` | Q3_K_M | 17.664 GB | Conventional 3-bit K-quant. |
| `Qwen3.8-35B-A3B-IQ4_XS.gguf` | IQ4_XS | 19.628 GB | Near Q4_K_M quality, ~2 GB smaller. |
| `Qwen3.8-35B-A3B-Q4_K_M.gguf` | Q4_K_M | 21.713 GB | Recommended. Best quality/size balance for most users. |
| `Qwen3.8-35B-A3B-Q5_K_M.gguf` | Q5_K_M | 25.348 GB | Higher quality, modest size increase. |
| `Qwen3.8-35B-A3B-Q6_K.gguf` | Q6_K | 29.209 GB | Near-lossless. |
| `Qwen3.8-35B-A3B-Q8_…
Source: https://huggingface.co/empero-ai/Qwen3.8-35B-A3B-Distill-GGUF
Card id model:hf:empero-ai/Qwen3.8-35B-A3B-Distill-GGUF · collected 2026-10-02 21:00 UTC · JSON