{"v":1,"id":"model:hf:Qwen/Qwen3-4B-GGUF","slug":"model-qwen-qwen3-4b-gguf","kind":"model","category":"llm","title":"Qwen3-4B-GGUF","summary":"Qwen3 is the latest generation of large language models in Qwen series, offering a comprehensive suite of dense and mixture-of-experts (MoE) models. Built upon extensive training, Qwen3 delivers grou…","source":{"provider":"hf","ref":"Qwen/Qwen3-4B-GGUF","url":"https://huggingface.co/Qwen/Qwen3-4B-GGUF","rev":"bc640142c66e1fdd12af0bd68f40445458f3869b","fetchedAt":"2026-10-02T20:53:24.235Z","etag":"W/\"1c98-5RXOkb6iiEoGrQfan1xdsQOlH7Q\""},"author":{"name":"Qwen","url":"https://huggingface.co/Qwen"},"license":{"spdx":"apache-2.0","raw":"apache-2.0","url":"https://huggingface.co/Qwen/Qwen3-4B-GGUF/blob/main/LICENSE","open":true},"metrics":{"downloads":472740,"downloadsWeek":327838,"likes":173,"stars":27665,"openIssues":68,"pushedAt":"2026-01-09T03:05:47Z","takenAt":"2026-10-02T20:53:24.235Z"},"tags":["gguf","text-generation","endpoints_compatible","conversational"],"pipeline":"text-generation","links":{"github":"QwenLM/Qwen3","npm":"node-llama-cpp"},"updatedAt":"2025-05-21T06:32:23.000Z","collectedAt":"2026-10-02T20:53:24.235Z","review":{"numbers":["472,740 downloads on Hugging Face","173 likes","license apache-2.0","2.3 GB for Qwen3-4B-Q4_K_M.gguf","27,665 stars on QwenLM/Qwen3","68 open issues and PRs","327,838 npm downloads a week for node-llama-cpp","latest node-llama-cpp@3.22.1"],"log":null},"trust":"unlabeled","health":{"status":"alive","checkedAt":"2026-10-02T20:53:24.235Z","http":200},"description":"# Qwen3-4B-GGUF\n\n    \n\n## Qwen3 Highlights\n\nQwen3 is the latest generation of large language models in Qwen series, offering a comprehensive suite of dense and mixture-of-experts (MoE) models. Built upon extensive training, Qwen3 delivers groundbreaking advancements in reasoning, instruction-following, agent capabilities, and multilingual support, with the following key features:\n\n- **Uniquely support of seamless switching between thinking mode** (for complex logical reasoning, math, and coding) and **non-thinking mode** (for efficient, general-purpose dialogue) **within single model**, ensuring optimal performance across various scenarios.\n- **Significantly enhancement in its reasoning capabilities**, surpassing previous QwQ (in thinking mode) and Qwen2.5 instruct models (in non-thinking mode) on mathematics, code generation, and commonsense logical reasoning.\n- **Superior human preference alignment**, excelling in creative writing, role-playing, multi-turn dialogues, and instruction following, to deliver a more natural, engaging, and immersive conversational experience.\n- **Expertise in agent capabilities**, enabling precise integration with external tools in both thinking and unthinking modes and achieving leading performance among open-source models in complex agent-based tasks.\n- **Support of 100+ languages and dialects** with strong capabilities for **multilingual instruction following** and **translation**.\n\n## Model Overview\n\n**Qwen3-4B** has the following features:\n- Type: Causal Language Models\n- Training Stage: Pretraining & Post-training\n- Number of Parameters: 4.0B\n- Number of Paramaters (Non-Embedding): 3.6B\n- Number of Layers: 36\n- Number of Attention Heads (GQA): 32 for Q and 8 for KV\n- Context Length: 32,768 natively and [131,072 tokens with YaRN](#processing-long-texts).\n\n- Quantization: q4_K_M, q5_0, q5_K_M, q6_K, q8_0\n\nFor more details, including benchmark evaluation, hardware requirements, and…\n\nSource: https://huggingface.co/Qwen/Qwen3-4B-GGUF","install":{"kind":"model","hfId":"Qwen/Qwen3-4B-GGUF","gated":false,"format":"gguf","files":[{"name":"Qwen3-4B-Q4_K_M.gguf","size":2497280256,"quant":"Q4_K_M","sha256":"7485fe6f11af29433bc51cab58009521f205840f5b4ae3a32fa7f92e8534fdf5"},{"name":"Qwen3-4B-Q5_0.gguf","size":2823710976,"quant":"Q5_0","sha256":"7f99c1aeefbcf991f04f67104e3d6f7b899e95170359ce081b0618d4f11878f5"},{"name":"Qwen3-4B-Q5_K_M.gguf","size":2889513184,"quant":"Q5_K_M","sha256":"aca596860e8cb40af6539e3f2ea40df305f42515deac56d49c08d39a02e6533f"},{"name":"Qwen3-4B-Q6_K.gguf","size":3306260704,"quant":"Q6_K","sha256":"8a08533841623c2a689763b9318d8e27b3e052ad5b38015d4be4c04fa96f68a3"},{"name":"Qwen3-4B-Q8_0.gguf","size":4280404704,"quant":"Q8_0","sha256":"8c2f07f26af9747e41988551106f149b03eb9b5cb6df636027b6bf6278473300"}],"totalBytes":15797169824,"suggestedFile":"Qwen3-4B-Q4_K_M.gguf","requirements":{"ramGb":4,"diskBytes":2497280256,"note":"estimate: suggested file size × 1.15 + 0.5 GB; a real measurement comes with lsh models"},"runWith":["llama.cpp","ollama"],"command":"lsh models install hf:Qwen/Qwen3-4B-GGUF"}}