Store › model › LLM
Qwen2.5-32B-Instruct-GGUF
by bartowski · source Hugging Face · updated 2024-09-19
apache-2.018 GB~22 GB RAMsource aliveunlabeled
Llamacpp imatrix Quantizations of Qwen2.5-32B-Instruct
Add to LogiShell Open in the web IDE
The button opens LogiShell with this card; nothing installs from a link by itself. Inside the app the install goes through lsh models install hf:bartowski/Qwen2.5-32B-Instruct-GGUF and its progress lives in the Resource Center.
Source and license
- Source: https://huggingface.co/bartowski/Qwen2.5-32B-Instruct-GGUF
- License: apache-2.0 · text
- Requirements: about 22 GB of RAM, 18 GB on disk (estimate: suggested file size × 1.15 + 0.5 GB; a real measurement comes with lsh models). Runs with llama.cpp, ollama.
- Tags:
ggufchattext-generationenendpoints_compatibleimatrixconversational
Numbers
- 379,473 downloads on Hugging Face
- 76 likes
- license apache-2.0
- 18 GB for Qwen2.5-32B-Instruct-Q4_K_M.gguf
- 327,838 npm downloads a week for node-llama-cpp
- latest node-llama-cpp@3.22.1
Numbers as of 2026-10-02 21:00 UTC, from the source API.
Summary
Summary not ready yet: the numbers are here, the text is not. It is written by the collector through the LogiShell model facade when a provider key is present.
Reviews
No reviews yet. Reviews are written inside LogiShell: open this card in the app.
Files
| file | quant | size |
|---|---|---|
Qwen2.5-32B-Instruct-IQ2_M.gguf | IQ2_M | 10 GB |
Qwen2.5-32B-Instruct-IQ2_S.gguf | IQ2_S | 9.7 GB |
Qwen2.5-32B-Instruct-IQ2_XS.gguf | IQ2_XS | 9.3 GB |
Qwen2.5-32B-Instruct-IQ2_XXS.gguf | IQ2_XXS | 8.4 GB |
Qwen2.5-32B-Instruct-IQ3_M.gguf | IQ3_M | 14 GB |
Qwen2.5-32B-Instruct-IQ3_XS.gguf | IQ3_XS | 13 GB |
Qwen2.5-32B-Instruct-IQ4_XS.gguf | IQ4_XS | 16 GB |
Qwen2.5-32B-Instruct-Q2_K.gguf | Q2_K | 11 GB |
Qwen2.5-32B-Instruct-Q2_K_L.gguf | Q2_K_L | 12 GB |
Qwen2.5-32B-Instruct-Q3_K_L.gguf | Q3_K_L | 16 GB |
Qwen2.5-32B-Instruct-Q3_K_M.gguf | Q3_K_M | 15 GB |
Qwen2.5-32B-Instruct-Q3_K_S.gguf | Q3_K_S | 13 GB |
Qwen2.5-32B-Instruct-Q3_K_XL.gguf | Q3_K_XL | 17 GB |
Qwen2.5-32B-Instruct-Q4_0.gguf | Q4_0 | 17 GB |
Qwen2.5-32B-Instruct-Q4_0_4_4.gguf | Q4_0 | 17 GB |
Qwen2.5-32B-Instruct-Q4_0_4_8.gguf | Q4_0 | 17 GB |
Qwen2.5-32B-Instruct-Q4_0_8_8.gguf | Q4_0 | 17 GB |
Qwen2.5-32B-Instruct-Q4_K_L.gguf | Q4_K_L | 19 GB |
Qwen2.5-32B-Instruct-Q4_K_M.gguf | Q4_K_M | 18 GB |
Qwen2.5-32B-Instruct-Q4_K_S.gguf | Q4_K_S | 17 GB |
Qwen2.5-32B-Instruct-Q5_K_L.gguf | Q5_K_L | 22 GB |
Qwen2.5-32B-Instruct-Q5_K_M.gguf | Q5_K_M | 22 GB |
Qwen2.5-32B-Instruct-Q5_K_S.gguf | Q5_K_S | 21 GB |
Qwen2.5-32B-Instruct-Q6_K.gguf | Q6_K | 25 GB |
Qwen2.5-32B-Instruct-Q6_K_L.gguf | Q6_K_L | 25 GB |
Qwen2.5-32B-Instruct-Q8_0.gguf | Q8_0 | 32 GB |
Qwen2.5-32B-Instruct-f16/Qwen2.5-32B-Instruct-f16-00001-of-00002.gguf | F16 | 37 GB |
Qwen2.5-32B-Instruct-f16/Qwen2.5-32B-Instruct-f16-00002-of-00002.gguf | F16 | 24 GB |
From the source README
Llamacpp imatrix Quantizations of Qwen2.5-32B-Instruct
Using llama.cpp release b3772 for quantization.
Original model: https://huggingface.co/Qwen/Qwen2.5-32B-Instruct
All quants made using imatrix option with dataset from here
Run them in LM Studio
Prompt format
system
{system_prompt}
user
{prompt}
assistant
What's new:
Update context length settings and tokenizer
Download a file (not the whole branch) from below:
| Filename | Quant type | File Size | Split | Description |
| -------- | ---------- | --------- | ----- | ----------- |
| Qwen2.5-32B-Instruct-f16.gguf | f16 | 65.54GB | true | Full F16 weights. |
| Qwen2.5-32B-Instruct-Q8_0.gguf | Q8_0 | 34.82GB | false | Extremely high quality, generally unneeded but max available quant. |
| Qwen2.5-32B-Instruct-Q6_K_L.gguf | Q6_K_L | 27.26GB | false | Uses Q8_0 for embed and output weights. Very high quality, near perfect, *recommended*. |
| Qwen2.5-32B-Instruct-Q6_K.gguf | Q6_K | 26.89GB | false | Very high quality, near perfect, *recommended*. |
| Qwen2.5-32B-Instruct-Q5_K_L.gguf | Q5_K_L | 23.74GB | false | Uses Q8_0 for embed and output weights. High quality, *recommended*. |
| Qwen2.5-32B-Instruct-Q5_K_M.gguf | Q5_…
Source: https://huggingface.co/bartowski/Qwen2.5-32B-Instruct-GGUF
Card id model:hf:bartowski/Qwen2.5-32B-Instruct-GGUF · collected 2026-10-02 21:00 UTC · JSON