{"v":1,"id":"model:hf:bartowski/Qwen2.5-32B-Instruct-GGUF","slug":"model-bartowski-qwen2-5-32b-instruct-gguf","kind":"model","category":"llm","title":"Qwen2.5-32B-Instruct-GGUF","summary":"Llamacpp imatrix Quantizations of Qwen2.5-32B-Instruct","source":{"provider":"hf","ref":"bartowski/Qwen2.5-32B-Instruct-GGUF","url":"https://huggingface.co/bartowski/Qwen2.5-32B-Instruct-GGUF","rev":"2116cbb385b8ce3a4d28cf3bf1cd2039a55821a6","fetchedAt":"2026-10-02T21:00:49.443Z","etag":"W/\"2c26-wZmm0zIFQxPX3KwxsZEWLSPGhB0\""},"author":{"name":"bartowski","url":"https://huggingface.co/bartowski"},"license":{"spdx":"apache-2.0","raw":"apache-2.0","url":"https://huggingface.co/Qwen/Qwen2.5-32B-Instruct/blob/main/LICENSE","open":true},"metrics":{"downloads":379473,"downloadsWeek":327838,"likes":76,"takenAt":"2026-10-02T21:00:49.443Z"},"tags":["gguf","chat","text-generation","en","endpoints_compatible","imatrix","conversational"],"pipeline":"text-generation","links":{"github":"QwenLM/Qwen3","npm":"node-llama-cpp"},"updatedAt":"2024-09-19T11:15:19.000Z","collectedAt":"2026-10-02T21:00:49.443Z","review":{"numbers":["379,473 downloads on Hugging Face","76 likes","license apache-2.0","18 GB for Qwen2.5-32B-Instruct-Q4_K_M.gguf","327,838 npm downloads a week for node-llama-cpp","latest node-llama-cpp@3.22.1"],"log":null},"trust":"unlabeled","health":{"status":"alive","checkedAt":"2026-10-02T21:00:49.443Z","http":200},"description":"## Llamacpp imatrix Quantizations of Qwen2.5-32B-Instruct\n\nUsing llama.cpp release b3772 for quantization.\n\nOriginal model: https://huggingface.co/Qwen/Qwen2.5-32B-Instruct\n\nAll quants made using imatrix option with dataset from [here](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8)\n\nRun them in [LM Studio](https://lmstudio.ai/)\n\n## Prompt format\n\n```\nsystem\n{system_prompt}\nuser\n{prompt}\nassistant\n```\n\n## What's new:\n\nUpdate context length settings and tokenizer\n\n## Download a file (not the whole branch) from below:\n\n| Filename | Quant type | File Size | Split | Description |\n| -------- | ---------- | --------- | ----- | ----------- |\n| [Qwen2.5-32B-Instruct-f16.gguf](https://huggingface.co/bartowski/Qwen2.5-32B-Instruct-GGUF/tree/main/Qwen2.5-32B-Instruct-f16) | f16 | 65.54GB | true | Full F16 weights. |\n| [Qwen2.5-32B-Instruct-Q8_0.gguf](https://huggingface.co/bartowski/Qwen2.5-32B-Instruct-GGUF/blob/main/Qwen2.5-32B-Instruct-Q8_0.gguf) | Q8_0 | 34.82GB | false | Extremely high quality, generally unneeded but max available quant. |\n| [Qwen2.5-32B-Instruct-Q6_K_L.gguf](https://huggingface.co/bartowski/Qwen2.5-32B-Instruct-GGUF/blob/main/Qwen2.5-32B-Instruct-Q6_K_L.gguf) | Q6_K_L | 27.26GB | false | Uses Q8_0 for embed and output weights. Very high quality, near perfect, *recommended*. |\n| [Qwen2.5-32B-Instruct-Q6_K.gguf](https://huggingface.co/bartowski/Qwen2.5-32B-Instruct-GGUF/blob/main/Qwen2.5-32B-Instruct-Q6_K.gguf) | Q6_K | 26.89GB | false | Very high quality, near perfect, *recommended*. |\n| [Qwen2.5-32B-Instruct-Q5_K_L.gguf](https://huggingface.co/bartowski/Qwen2.5-32B-Instruct-GGUF/blob/main/Qwen2.5-32B-Instruct-Q5_K_L.gguf) | Q5_K_L | 23.74GB | false | Uses Q8_0 for embed and output weights. High quality, *recommended*. |\n| [Qwen2.5-32B-Instruct-Q5_K_M.gguf](https://huggingface.co/bartowski/Qwen2.5-32B-Instruct-GGUF/blob/main/Qwen2.5-32B-Instruct-Q5_K_M.gguf) | Q5_…\n\nSource: https://huggingface.co/bartowski/Qwen2.5-32B-Instruct-GGUF","install":{"kind":"model","hfId":"bartowski/Qwen2.5-32B-Instruct-GGUF","gated":false,"format":"gguf","files":[{"name":"Qwen2.5-32B-Instruct-IQ2_M.gguf","size":11264441216,"quant":"IQ2_M","sha256":"38ff614186bf6cafff3ea7fd140d54c251cb8c28b5b1ac7cb1e2c50c13b22c32"},{"name":"Qwen2.5-32B-Instruct-IQ2_S.gguf","size":10387569536,"quant":"IQ2_S","sha256":"2ae3d9b9d4ce6bdd8f99ccf149ca495d227ecaf74dbbb0ca35d6662caf3ffdd8"},{"name":"Qwen2.5-32B-Instruct-IQ2_XS.gguf","size":9957550976,"quant":"IQ2_XS","sha256":"2823e23d67f78a70566b619ded48ece723b1f7d5d9ef5f1a61aeed100aaba364"},{"name":"Qwen2.5-32B-Instruct-IQ2_XXS.gguf","size":9028250496,"quant":"IQ2_XXS","sha256":"48f89b4e11bb844ac98062d32bbde6dec6cc4b7aca06afdcf22b807c6c0de4dd"},{"name":"Qwen2.5-32B-Instruct-IQ3_M.gguf","size":14810123136,"quant":"IQ3_M","sha256":"f7acfc3c2a65bd42c4930f62ee7fdb07c42ea926f2cf53b56581b97eb1767709"},{"name":"Qwen2.5-32B-Instruct-IQ3_XS.gguf","size":13705513856,"quant":"IQ3_XS","sha256":"bbbd396f393ce6b821b8bfe94b745f3bbcc10dd29e784e35c6d347c272ed7920"},{"name":"Qwen2.5-32B-Instruct-IQ4_XS.gguf","size":17693154176,"quant":"IQ4_XS","sha256":"898d78ffc180a5f61ce5cd1f4c006c0cefd7e7f7310699db51b8944dcbf3250e"},{"name":"Qwen2.5-32B-Instruct-Q2_K.gguf","size":12313099136,"quant":"Q2_K","sha256":"523ccba36e7ef9669b15cb504fde034630082a74c62e0224409a62044704d4e6"},{"name":"Qwen2.5-32B-Instruct-Q2_K_L.gguf","size":13073419136,"quant":"Q2_K_L","sha256":"048b2fa2c50281b486c2023b4a0be525a1af6fce7cc479f4f1ce316b56c86ca1"},{"name":"Qwen2.5-32B-Instruct-Q3_K_L.gguf","size":17247079296,"quant":"Q3_K_L","sha256":"c0acf3f75cc6e4574054f8ccb9e2b3c2c9a61666bfd84de4de4cc5b2f7b5c0d3"},{"name":"Qwen2.5-32B-Instruct-Q3_K_M.gguf","size":15935048576,"quant":"Q3_K_M","sha256":"c87acbeeba9fca2c92952fc94fb1691b07a0dc6b43cacf61e7f798f31414dd54"},{"name":"Qwen2.5-32B-Instruct-Q3_K_S.gguf","size":14392331136,"quant":"Q3_K_S","sha256":"a1aefda97dc4641598db576edf61b7b5f70f40abab33fbea39fb84fcaa9be44e"},{"name":"Qwen2.5-32B-Instruct-Q3_K_XL.gguf","size":17928326016,"quant":"Q3_K_XL","sha256":"a6784f80506c4c77b3516428b805a94f3702c895c63b08b3227da43c247c43f8"},{"name":"Qwen2.5-32B-Instruct-Q4_0.gguf","size":18711010176,"quant":"Q4_0","sha256":"edc1aa9db2d886329413ca3db40852376736a220c660e7a1b2a166aacf9d9094"},{"name":"Qwen2.5-32B-Instruct-Q4_0_4_4.gguf","size":18640231296,"quant":"Q4_0","sha256":"c3dd6ecdd959fe36d5bacbf07590e95d0243bb1c8761a7d4b82781f619bde7ff"},{"name":"Qwen2.5-32B-Instruct-Q4_0_4_8.gguf","size":18640231296,"quant":"Q4_0","sha256":"4c4e46f13cae1a5a75e8eab92ce1f490344574efefc335a35934de74d49cceb5"},{"name":"Qwen2.5-32B-Instruct-Q4_0_8_8.gguf","size":18640231296,"quant":"Q4_0","sha256":"7bea18a4d1899dee3380b2eea2bd8c655f042209979ffe3808937ee8528f81b0"},{"name":"Qwen2.5-32B-Instruct-Q4_K_L.gguf","size":20429179776,"quant":"Q4_K_L","sha256":"1ebb873990111e765f51d5146bea583b345aa7de513157e57ed471918f10f32c"},{"name":"Qwen2.5-32B-Instruct-Q4_K_M.gguf","size":19851336576,"quant":"Q4_K_M","sha256":"2e5f6daea180dbc59f65a40641e94d3973b5dbaa32b3c0acf54647fa874e519e"},{"name":"Qwen2.5-32B-Instruct-Q4_K_S.gguf","size":18784410496,"quant":"Q4_K_S","sha256":"97b51ba6fd524e09b15d36f726725445710aec0b10fb2584604b5f6803e6cd08"},{"name":"Qwen2.5-32B-Instruct-Q5_K_L.gguf","size":23742679936,"quant":"Q5_K_L","sha256":"488fefae2704ebbab8bb5e197ddd244a782dad316118df8e23e87e0cf6e98ae5"},{"name":"Qwen2.5-32B-Instruct-Q5_K_M.gguf","size":23262157696,"quant":"Q5_K_M","sha256":"371c9d50b4c2db96c7b6695b81a23c03c10a0ee780a8c3fe9193aac148febdaf"},{"name":"Qwen2.5-32B-Instruct-Q5_K_S.gguf","size":22638254976,"quant":"Q5_K_S","sha256":"aac01239f8f3d1fc05ef019d211ff10aceb42cc5699793da13facbd2e3c2570b"},{"name":"Qwen2.5-32B-Instruct-Q6_K.gguf","size":26886155136,"quant":"Q6_K","sha256":"3393e05b12396e0e6c786f5fb99837eb77ef9eb0b30d940e30d94c2f5506054a"},{"name":"Qwen2.5-32B-Instruct-Q6_K_L.gguf","size":27263273856,"quant":"Q6_K_L","sha256":"61b659e7083ee71f3e45018a64eb9c05e491673011ff536ca6e93e374ec98a2b"},{"name":"Qwen2.5-32B-Instruct-Q8_0.gguf","size":34820885376,"quant":"Q8_0","sha256":"fc5103335689b33d576ebe8bf50b35a5e04ca6f7cf0753c2f2df1079dabbb78f"},{"name":"Qwen2.5-32B-Instruct-f16/Qwen2.5-32B-Instruct-f16-00001-of-00002.gguf","size":39880800160,"quant":"F16","sha256":"f5c2e4041be1187bc474e94344bef8aeaa60f36d9f8889f36496989f248cf7ef"},{"name":"Qwen2.5-32B-Instruct-f16/Qwen2.5-32B-Instruct-f16-00002-of-00002.gguf","size":25655169952,"quant":"F16","sha256":"68e497ae4e14fcd7193b0d6d8d71cdcf652761af30c97266921e9b4677ee79bd"}],"totalBytes":535581914688,"suggestedFile":"Qwen2.5-32B-Instruct-Q4_K_M.gguf","requirements":{"ramGb":22,"diskBytes":19851336576,"note":"estimate: suggested file size × 1.15 + 0.5 GB; a real measurement comes with lsh models"},"runWith":["llama.cpp","ollama"],"command":"lsh models install hf:bartowski/Qwen2.5-32B-Instruct-GGUF"}}