{"v":1,"id":"model:hf:bloomer010/Ling-3.0-tiny-GGUF","slug":"model-bloomer010-ling-3-0-tiny-gguf","kind":"model","category":"llm","title":"Ling-3.0-tiny-GGUF","summary":"GGUF conversions of inclusionAI/Ling-3.0-tiny, converted directly from the released BF16 safetensors.","source":{"provider":"hf","ref":"bloomer010/Ling-3.0-tiny-GGUF","url":"https://huggingface.co/bloomer010/Ling-3.0-tiny-GGUF","rev":"9abc60203407b59a4f22eaeeb09b789717643fb0","fetchedAt":"2026-10-11T15:33:38.599Z","etag":"W/\"547e-6xkPXk9VvH97R37PvFXznNRbhoE\""},"author":{"name":"bloomer010","url":"https://huggingface.co/bloomer010"},"license":{"spdx":"mit","raw":"mit","open":true},"metrics":{"downloads":265045,"downloadsWeek":261158,"likes":112,"stars":130822,"openIssues":2517,"lastRelease":{"tag":"v0.6.0","at":"2026-10-05T16:56:22Z"},"pushedAt":"2026-10-11T14:46:19Z","takenAt":"2026-10-11T15:33:38.599Z"},"tags":["llama.cpp","gguf","bailing_hybrid","bailingmoe3","mixture-of-experts","conversational","text-generation","custom_code","endpoints_compatible"],"pipeline":"text-generation","links":{"github":"ggml-org/llama.cpp","npm":"node-llama-cpp"},"updatedAt":"2026-09-20T16:42:18.000Z","collectedAt":"2026-10-11T15:33:38.599Z","review":{"numbers":["265,045 downloads on Hugging Face","112 likes","license mit","4.5 GB for Ling-3.0-tiny-Q4_K_M.gguf","130,822 stars on ggml-org/llama.cpp","2,517 open issues and PRs","last release v0.6.0 on 2026-10-05","261,158 npm downloads a week for node-llama-cpp","latest node-llama-cpp@3.22.1"],"log":null},"trust":"unlabeled","health":{"status":"alive","checkedAt":"2026-10-11T15:33:38.599Z","http":200},"description":"# Ling-3.0-tiny GGUF\n\nGGUF conversions of [inclusionAI/Ling-3.0-tiny](https://huggingface.co/inclusionAI/Ling-3.0-tiny),\nconverted directly from the released BF16 safetensors.\n\n## 🦙🚨 llama.cpp 🦙🚨\nConsistent agentic use (tool calling, reasoning split) currently requires two unmerged llama.cpp PRs:\n- Dedicated Ling parser: [#28682](https://github.com/ggml-org/llama.cpp/pull/28682) (✅ Merged as of 9/19)\n- Invalid UTF-8 handling in the PEG parser: [#29161](https://github.com/ggml-org/llama.cpp/pull/29161) (✅ Merged as of 9/20)\n\nWithout both, tool calls inside an unclosed think block are dropped and some turns fail with a 500.\nWill update this note as they merge.\n\nThe model does occasionally terminate its response, mid-think, without any sort of closing.\nThis is inherent in the weights, even at full precision.\n\nTo run with `llama-server`:\n```bash\nllama-server -hf bloomer010/Ling-3.0-tiny-GGUF:Q4_K_M\n```\n\n## Quant Sizing\n\nFor tiny models, precision is especially crucial.\n\nGenerally...\nLarger files = More precision.\nSmaller files = More compression = More slop and misbehavin'.\n\nUse UD-Q8_K_XL for near-full precision performance.\n\n| Quant | Size | your memory |\n| --- | ---: | --- |\n| BF16 | 15.8 GB | 16 GB+ |\n| UD-Q8_K_XL | 11.19 GB | 12 GB+ |\n| Q8_0 | 8.41 GB | 10 GB+ |\n| UD-Q6_K_XL | 7.27 GB | 8 GB+ |\n| Q6_K | 6.50 GB | 8 GB+ |\n| Q5_K_M | 5.64 GB | 7 GB+ |\n| Q5_K_S | 5.48 GB | 6 GB+ |\n| Q5_0 | 5.48 GB | 6 GB+ |\n| Q4_K_M | 4.82 GB | 6 GB+ |\n| Q4_K_S | 4.55 GB | 6 GB+ |\n| Q4_0 | 4.53 GB | 6 GB+ |\n| MXFP4_MOE | 4.72 GB | 6 GB+ ¹ |\n| IQ4_XS | 4.29 GB | 5 GB+ |\n| Q3_K_M | 3.84 GB | 5 GB+ |\n| Q3_K_S | 3.51 GB | 5 GB+ |\n| IQ3_S | 3.51 GB | 4 GB+ |\n| IQ3_XXS | 3.13 GB | 4 GB+ |\n| Q2_K | 2.99 GB | 4 GB+ |\n| IQ2_M | 2.70 GB | 3 GB+ |\n| IQ2_S | 2.48 GB | 3 GB+ |\n| IQ2_XS | 2.43 GB | 3 GB+ |\n| IQ2_XXS | 2.21 GB | 3 GB+ |\n| IQ1_M | 1.93 GB | 3 GB+ |\n| IQ1_S | 1.76 GB | 2 GB+ |\n| Q1_0 | 1.30 GB | 2 GB+ |\n\n¹ `MXFP4_MOE`…\n\nSource: https://huggingface.co/bloomer010/Ling-3.0-tiny-GGUF","install":{"kind":"model","hfId":"bloomer010/Ling-3.0-tiny-GGUF","gated":false,"format":"gguf","files":[{"name":"Ling-3.0-tiny-BF16.gguf","size":15803475456,"quant":"BF16","sha256":"4606ccd7772f3fa6860da30729e2c4a94819a5e4c2913bfca6a8a451fd977ded"},{"name":"Ling-3.0-tiny-IQ1_M.gguf","size":1928455520,"quant":"IQ1_M","sha256":"2cbf44462ce109ec4b83c62b8c038e8b897854eacd698ba6f5cb72d938fb4222"},{"name":"Ling-3.0-tiny-IQ1_S.gguf","size":1760678240,"quant":"IQ1_S","sha256":"705204b5e98cb333740d8005fa39410a9c4132ba4a0ceb5a64ea2c299f39a181"},{"name":"Ling-3.0-tiny-IQ2_M.gguf","size":2704009568,"quant":"IQ2_M","sha256":"d7175990cb296bb942bacfa21b87fdc9f2ca25b4a92191305918fb02735e5343"},{"name":"Ling-3.0-tiny-IQ2_S.gguf","size":2480306528,"quant":"IQ2_S","sha256":"a98967f50bd4efde69a294646a3fd6371ce5374e0b9678ff530e2a08f7cc0e78"},{"name":"Ling-3.0-tiny-IQ2_XS.gguf","size":2434146656,"quant":"IQ2_XS","sha256":"760133f72e2c4633a03bb35c585b071d55dee964cde0248980f809f999a6f69e"},{"name":"Ling-3.0-tiny-IQ2_XXS.gguf","size":2208084320,"quant":"IQ2_XXS","sha256":"7d064056a3989a8ee7a6e39520c53d65ca14b207efe32452245442616048a3c5"},{"name":"Ling-3.0-tiny-IQ3_S.gguf","size":3511515488,"quant":"IQ3_S","sha256":"a0d92fa4adfdcf5e9dd363777d1762f9afce3cf35daaccbf781834036f3b8d72"},{"name":"Ling-3.0-tiny-IQ3_XXS.gguf","size":3129076064,"quant":"IQ3_XXS","sha256":"ef1c986bf0b5797c5750080985c5a9bc2013ab71430690b8e3c501cfd75d66ce"},{"name":"Ling-3.0-tiny-IQ4_XS.gguf","size":4287511904,"quant":"IQ4_XS","sha256":"c49b4e2bfe7dab6939f1fc3dfe9e2288ee68b4f683c65bf976575ff6504eeb14"},{"name":"Ling-3.0-tiny-MXFP4_MOE.gguf","size":4718249024,"sha256":"133868cd55329c3dbf1b9b8b47e75489f012d6319b1c05a04a4fc8abb1c71f6f"},{"name":"Ling-3.0-tiny-Q1_0.gguf","size":1300255808,"quant":"Q1_0","sha256":"f06aab7dd2b5feb2e60e52f01ba4f9c83f932a7fd1680a7dcd7244f5b00ba606"},{"name":"Ling-3.0-tiny-Q2_K.gguf","size":2986636352,"quant":"Q2_K","sha256":"165c3b4a4bbd18bbfb80ba469527354e25504f6c3ac54d9622a1cb70bad81e31"},{"name":"Ling-3.0-tiny-Q3_K_M.gguf","size":3841570880,"quant":"Q3_K_M","sha256":"f2e88f405618654a077882216c4f65f56a5af33997203a10da24e2f946601972"},{"name":"Ling-3.0-tiny-Q3_K_S.gguf","size":3511515200,"quant":"Q3_K_S","sha256":"5eb62ea8b1807cedb87cae725c6983c53bf3f6047893760042aced70479ec4df"},{"name":"Ling-3.0-tiny-Q4_0.gguf","size":4526279744,"quant":"Q4_0","sha256":"f4d993793e4afb3706e0572801bc7a51a5656f2146e33fec2cbc57abcdc8d0b1"},{"name":"Ling-3.0-tiny-Q4_K_M.gguf","size":4823895104,"quant":"Q4_K_M","sha256":"a21f717779203d86b53996239c4903941858b938c50e85b03e6a981132b5621a"},{"name":"Ling-3.0-tiny-Q4_K_S.gguf","size":4554886208,"quant":"Q4_K_S","sha256":"8566b871f280f3d3c0b950391b873581a8ea596a35e6c9157a5e949a26e71e14"},{"name":"Ling-3.0-tiny-Q5_0.gguf","size":5482138688,"quant":"Q5_0","sha256":"5e6b1d493a93ff36fecce3d58bb82bce3d9573d8e550d77c398a160a3c5b0e2d"},{"name":"Ling-3.0-tiny-Q5_K_M.gguf","size":5635443776,"quant":"Q5_K_M","sha256":"42e20684cb7fd928f0d777c21edbe825102cf8f917e4c32566a2a126eed96fed"},{"name":"Ling-3.0-tiny-Q5_K_S.gguf","size":5482531904,"quant":"Q5_K_S","sha256":"04ddf2e4e434c074f9ea8b8b1851abdd8b0f66394c3ee4f02f23553185f93c8d"},{"name":"Ling-3.0-tiny-Q6_K.gguf","size":6499262528,"quant":"Q6_K","sha256":"f9698dcc5801274ac7b33fd287081811525e9fac326700d08c2b40c761b92e63"},{"name":"Ling-3.0-tiny-Q8_0.gguf","size":8408187968,"quant":"Q8_0","sha256":"ee5ecf3241602f4f8ba8486a2cb4645ab35349a616869a4fcd1bb72a4a5037e6"},{"name":"Ling-3.0-tiny-UD-Q4_K_XL.gguf","size":5340611552,"quant":"Q4_K_XL","sha256":"7faee4091379ce1c644e09dd88848a8f0bc3348a216d2ee1c90a5388de6a5b13"},{"name":"Ling-3.0-tiny-UD-Q6_K_XL.gguf","size":7274546528,"quant":"Q6_K_XL","sha256":"a31376fc9c56309046cff6c43888adf180c5c40fa48228ef644937ee9bd5d64b"},{"name":"Ling-3.0-tiny-UD-Q8_K_XL.gguf","size":11188839488,"quant":"Q8_K_XL","sha256":"ab041da9664d927db6594d051be6c17cffa386cf91f545eba0479fcadc9cdc9f"},{"name":"Ling-3.0-tiny-f16.gguf","size":15803475456,"quant":"F16","sha256":"511c2baafa80e58b7e037c2d0a6dd3a3c951e3165545d456c1c66ec430913263"},{"name":"Ling-3.0-tiny-imatrix.gguf","size":44016704,"sha256":"566f65b22c6f45f4b46c23982a35fd04a0b093fa3577647d476bb9c7472c37c9"}],"totalBytes":141669602656,"suggestedFile":"Ling-3.0-tiny-Q4_K_M.gguf","requirements":{"ramGb":6,"diskBytes":4823895104,"note":"estimate: suggested file size × 1.15 + 0.5 GB; a real measurement comes with lsh models"},"runWith":["llama.cpp","ollama"],"command":"lsh models install hf:bloomer010/Ling-3.0-tiny-GGUF"}}