{"v":1,"id":"model:hf:bartowski/DeepSeek-Coder-V2-Lite-Instruct-GGUF","slug":"model-bartowski-deepseek-coder-v2-lite-instruct-gguf","kind":"model","category":"llm","title":"DeepSeek-Coder-V2-Lite-Instruct-GGUF","summary":"Llamacpp imatrix Quantizations of DeepSeek-Coder-V2-Lite-Instruct","source":{"provider":"hf","ref":"bartowski/DeepSeek-Coder-V2-Lite-Instruct-GGUF","url":"https://huggingface.co/bartowski/DeepSeek-Coder-V2-Lite-Instruct-GGUF","rev":"8f248fa2072348f77a8bc37754e470de1f61866e","fetchedAt":"2026-10-02T21:00:47.419Z","etag":"W/\"1fb8-K19le1O/zj1ryIYx0O53C6ROxCo\""},"author":{"name":"bartowski","url":"https://huggingface.co/bartowski"},"license":{"spdx":null,"raw":"other","open":null,"note":"custom license: read it at the source before installing"},"metrics":{"downloads":384832,"downloadsWeek":327838,"likes":195,"takenAt":"2026-10-02T21:00:47.419Z"},"tags":["gguf","text-generation","endpoints_compatible","imatrix","conversational"],"pipeline":"text-generation","links":{"github":"deepseek-ai/DeepSeek-V3","npm":"node-llama-cpp"},"updatedAt":"2024-06-20T17:12:43.000Z","collectedAt":"2026-10-02T21:00:47.419Z","review":{"numbers":["384,832 downloads on Hugging Face","195 likes","license other","9.7 GB for DeepSeek-Coder-V2-Lite-Instruct-Q4_K_M.gguf","327,838 npm downloads a week for node-llama-cpp","latest node-llama-cpp@3.22.1"],"log":null},"trust":"unlabeled","health":{"status":"alive","checkedAt":"2026-10-02T21:00:47.419Z","http":200},"description":"## Llamacpp imatrix Quantizations of DeepSeek-Coder-V2-Lite-Instruct\n\nUsing llama.cpp release b3166 for quantization.\n\nOriginal model: https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct\n\nAll quants made using imatrix option with dataset from [here](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8)\n\n## Prompt format\n\n```\n{system_prompt}\n\nUser: {prompt}\n\nAssistant: Assistant:\n```\n\n## Download a file (not the whole branch) from below:\n\n| Filename | Quant type | File Size | Description |\n| -------- | ---------- | --------- | ----------- |\n| [DeepSeek-Coder-V2-Lite-Instruct-Q8_0_L.gguf](https://huggingface.co/bartowski/DeepSeek-Coder-V2-Lite-Instruct-GGUF/blob/main/DeepSeek-Coder-V2-Lite-Instruct-Q8_0_L.gguf) | Q8_0_L | 17.09GB | *Experimental*, uses f16 for embed and output weights. Please provide any feedback of differences. Extremely high quality, generally unneeded but max available quant. |\n| [DeepSeek-Coder-V2-Lite-Instruct-Q8_0.gguf](https://huggingface.co/bartowski/DeepSeek-Coder-V2-Lite-Instruct-GGUF/blob/main/DeepSeek-Coder-V2-Lite-Instruct-Q8_0.gguf) | Q8_0 | 16.70GB | Extremely high quality, generally unneeded but max available quant. |\n| [DeepSeek-Coder-V2-Lite-Instruct-Q6_K_L.gguf](https://huggingface.co/bartowski/DeepSeek-Coder-V2-Lite-Instruct-GGUF/blob/main/DeepSeek-Coder-V2-Lite-Instruct-Q6_K_L.gguf) | Q6_K_L | 14.56GB | *Experimental*, uses f16 for embed and output weights. Please provide any feedback of differences. Very high quality, near perfect, *recommended*. |\n| [DeepSeek-Coder-V2-Lite-Instruct-Q6_K.gguf](https://huggingface.co/bartowski/DeepSeek-Coder-V2-Lite-Instruct-GGUF/blob/main/DeepSeek-Coder-V2-Lite-Instruct-Q6_K.gguf) | Q6_K | 14.06GB | Very high quality, near perfect, *recommended*. |\n| [DeepSeek-Coder-V2-Lite-Instruct-Q5_K_L.gguf](https://huggingface.co/bartowski/DeepSeek-Coder-V2-Lite-Instruct-GGUF/blob/main/DeepSeek-Co…\n\nSource: https://huggingface.co/bartowski/DeepSeek-Coder-V2-Lite-Instruct-GGUF","install":{"kind":"model","hfId":"bartowski/DeepSeek-Coder-V2-Lite-Instruct-GGUF","gated":false,"format":"gguf","files":[{"name":"DeepSeek-Coder-V2-Lite-Instruct-IQ2_M.gguf","size":6328456960,"quant":"IQ2_M","sha256":"c09685b4ee1480320382a2aedb9bed19c4af5033ec466f7ff9ce5d596c3b78bf"},{"name":"DeepSeek-Coder-V2-Lite-Instruct-IQ2_S.gguf","size":6005212928,"quant":"IQ2_S","sha256":"1c5a5efd2adad07c1161918668554640af31db7e8adbfc0e55ee63aedd0774bc"},{"name":"DeepSeek-Coder-V2-Lite-Instruct-IQ2_XS.gguf","size":5967402752,"quant":"IQ2_XS","sha256":"13106ce47dee9380b03ff27e88c87076604a01c85fea9fab191f827c234b46d3"},{"name":"DeepSeek-Coder-V2-Lite-Instruct-IQ3_M.gguf","size":7553175296,"quant":"IQ3_M","sha256":"08db93121a9e6fa3cb4978c4b4e9c37e9407160ceec77f0894c45f43bf2d91d1"},{"name":"DeepSeek-Coder-V2-Lite-Instruct-IQ3_XS.gguf","size":7122857728,"quant":"IQ3_XS","sha256":"f459cedf6b8f9186414aa817cbc85ba1d8b81465073855744efbf52fc7c8188e"},{"name":"DeepSeek-Coder-V2-Lite-Instruct-IQ3_XXS.gguf","size":6964057856,"quant":"IQ3_XXS","sha256":"d3fd8f7ca9c97f3e424dac63c5c719226e9811f567081e92d747288b83cc5c9f"},{"name":"DeepSeek-Coder-V2-Lite-Instruct-IQ4_XS.gguf","size":8571593472,"quant":"IQ4_XS","sha256":"ac0a996714d4e8ed06b4398096bae88a32c349ceab42ffe629c2ddf4c4e0706c"},{"name":"DeepSeek-Coder-V2-Lite-Instruct-Q2_K.gguf","size":6430464768,"quant":"Q2_K","sha256":"5e2ac4b3477aa11ff460739ec326040ad07a3fc1c42da8e15b0465054879b5d3"},{"name":"DeepSeek-Coder-V2-Lite-Instruct-Q3_K_L.gguf","size":8459398912,"quant":"Q3_K_L","sha256":"be5fd70b1ffe99d0fb2d511250d9d02a08b894b04cb5bb2c2c8113ff5a07c62c"},{"name":"DeepSeek-Coder-V2-Lite-Instruct-Q3_K_M.gguf","size":8126607104,"quant":"Q3_K_M","sha256":"30f78fce33d19c4bf68c410cd6504d424346dfbe846b4d3df5558e0be078a2a1"},{"name":"DeepSeek-Coder-V2-Lite-Instruct-Q3_K_S.gguf","size":7487663872,"quant":"Q3_K_S","sha256":"508e1ead6515d50a68d3d0f1e0d7f0c29f3ca351404703266793af6708ea89f5"},{"name":"DeepSeek-Coder-V2-Lite-Instruct-Q4_K_L.gguf","size":10913280768,"quant":"Q4_K_L","sha256":"ce11aa91f0f0daa9add3ee1c5496a5097a32793f0541b130e9910eba0cf13ea1"},{"name":"DeepSeek-Coder-V2-Lite-Instruct-Q4_K_M.gguf","size":10364416768,"quant":"Q4_K_M","sha256":"603bd3f8a0281d16571da7c08bd661ee17ff0d1be6fcbd1b42242da257ef0bb8"},{"name":"DeepSeek-Coder-V2-Lite-Instruct-Q4_K_S.gguf","size":9533608704,"quant":"Q4_K_S","sha256":"2a0c965bf42e1ef3bca38d7503c3a071bec3d9ba46128146a074ff3fc7b762d2"},{"name":"DeepSeek-Coder-V2-Lite-Instruct-Q5_K_L.gguf","size":12373963520,"quant":"Q5_K_L","sha256":"1f8706d261cd44e50219ef0c669e18d4b88adeac2917fc8dcf887472d189c23a"},{"name":"DeepSeek-Coder-V2-Lite-Instruct-Q5_K_M.gguf","size":11851313920,"quant":"Q5_K_M","sha256":"3de21719a8ffb4f6acc4b636d4ca38d882e0d0aa9a5d417106f985e0e0a4a735"},{"name":"DeepSeek-Coder-V2-Lite-Instruct-Q5_K_S.gguf","size":11143058176,"quant":"Q5_K_S","sha256":"fe9b55d85300769825f0d9d69ce4d259ad0438ab42222d48ee3fdf99a728dcf8"},{"name":"DeepSeek-Coder-V2-Lite-Instruct-Q6_K.gguf","size":14066972416,"quant":"Q6_K","sha256":"1ff79f43ad5728d3179bf8fa7ee2993652f4306d6aeca9c35055f4f5b7b864cd"},{"name":"DeepSeek-Coder-V2-Lite-Instruct-Q6_K_L.gguf","size":14561769216,"quant":"Q6_K_L","sha256":"9d9244931cd48f36764e7c0707c100603e7d4c95cd588d7b18e01db13bd44471"},{"name":"DeepSeek-Coder-V2-Lite-Instruct-Q8_0.gguf","size":16702518016,"quant":"Q8_0","sha256":"af12f49d16dc54f2148af3f6dc3ee5de9461e8586a1d13e279b7e7623bf796fb"},{"name":"DeepSeek-Coder-V2-Lite-Instruct-Q8_0_L.gguf","size":17095734016,"quant":"Q8_0","sha256":"c41b0dbf1599296ba76343b942c39691a96a20ff382dad8cfc674889360d773e"},{"name":"DeepSeek-Coder-V2-Lite-Instruct-f32.gguf/DeepSeek-Coder-V2-Lite-Instruct-f32-00001-of-00002.gguf","size":39436017344,"quant":"F32","sha256":"84094402373964af422045f01639f9bdb4180387ecc12db406f3feb6365a2d66"},{"name":"DeepSeek-Coder-V2-Lite-Instruct-f32.gguf/DeepSeek-Coder-V2-Lite-Instruct-f32-00002-of-00002.gguf","size":23393915904,"quant":"F32","sha256":"a3325357f0edf29bf67f8f687f578f0e280fd6bda04a710144e64da075393c92"}],"totalBytes":270453460416,"suggestedFile":"DeepSeek-Coder-V2-Lite-Instruct-Q4_K_M.gguf","requirements":{"ramGb":12,"diskBytes":10364416768,"note":"estimate: suggested file size × 1.15 + 0.5 GB; a real measurement comes with lsh models"},"runWith":["llama.cpp","ollama"],"command":"lsh models install hf:bartowski/DeepSeek-Coder-V2-Lite-Instruct-GGUF"}}