{"v":1,"id":"model:hf:second-state/All-MiniLM-L6-v2-Embedding-GGUF","slug":"model-second-state-all-minilm-l6-v2-embedding-gguf","kind":"model","category":"embedding","title":"All-MiniLM-L6-v2-Embedding-GGUF","summary":"embedding model by second-state, gguf files on Hugging Face.","source":{"provider":"hf","ref":"second-state/All-MiniLM-L6-v2-Embedding-GGUF","url":"https://huggingface.co/second-state/All-MiniLM-L6-v2-Embedding-GGUF","rev":"544f204f2eaa2d71361ffc74d6df7170285b286a","fetchedAt":"2026-10-02T20:51:52.693Z","etag":"W/\"12a6-4uqs4SHPgQctoEzVCcxrFpjnkj4\""},"author":{"name":"second-state","url":"https://huggingface.co/second-state"},"license":{"spdx":"apache-2.0","raw":"apache-2.0","open":true},"metrics":{"downloads":124802,"downloadsWeek":4402710,"likes":25,"stars":19144,"openIssues":1365,"lastRelease":{"tag":"v6.1.0","at":"2026-09-18T10:44:39Z"},"pushedAt":"2026-10-01T20:19:25Z","takenAt":"2026-10-02T20:51:52.693Z"},"tags":["sentence-transformers","gguf","bert","feature-extraction","sentence-similarity","transformers","en","endpoints_compatible","deploy:azure"],"pipeline":"feature-extraction","links":{"github":"huggingface/sentence-transformers","npm":"@huggingface/transformers"},"updatedAt":"2024-05-01T04:35:19.000Z","collectedAt":"2026-10-02T20:51:52.693Z","review":{"numbers":["124,802 downloads on Hugging Face","25 likes","license apache-2.0","0.0 GB for all-MiniLM-L6-v2-Q4_K_M.gguf","19,144 stars on huggingface/sentence-transformers","1,365 open issues and PRs","last release v6.1.0 on 2026-09-18","4,402,710 npm downloads a week for @huggingface/transformers","latest @huggingface/transformers@4.3.0"],"log":null},"trust":"unlabeled","health":{"status":"alive","checkedAt":"2026-10-02T20:51:52.693Z","http":200},"description":"# All-MiniLM-L6-v2-GGUF\n\n## Original Model\n\n[sentence-transformers/all-MiniLM-L6-v2](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2)\n\n## Run with LlamaEdge\n\n- LlamaEdge version: [v0.8.2](https://github.com/LlamaEdge/LlamaEdge/releases/tag/0.8.2) and above\n\n- Context size: `384`\n\n- Vector size: `256`\n\n- Run as LlamaEdge service\n\n  ```bash\n  wasmedge --dir .:. --nn-preload default:GGML:AUTO:all-MiniLM-L6-v2-ggml-model-f16.gguf \\\n    llama-api-server.wasm \\\n    --prompt-template llama-2-chat \\\n    --ctx-size 256 \\\n    --model-name all-MiniLM-L6-v2\n  ```\n\n## Quantized GGUF Models\n\n| Name | Quant method | Bits | Size | Use case |\n| ---- | ---- | ---- | ---- | ----- |\n| [all-MiniLM-L6-v2-Q2_K.gguf](https://huggingface.co/second-state/All-MiniLM-L6-v2-Embedding-GGUF/blob/main/all-MiniLM-L6-v2-Q2_K.gguf)     | Q2_K   | 2 | 19.2 MB| smallest, significant quality loss - not recommended for most purposes |\n| [all-MiniLM-L6-v2-Q3_K_L.gguf](https://huggingface.co/second-state/All-MiniLM-L6-v2-Embedding-GGUF/blob/main/all-MiniLM-L6-v2-Q3_K_L.gguf) | Q3_K_L | 3 | 20.5 MB| small, substantial quality loss |\n| [all-MiniLM-L6-v2-Q3_K_M.gguf](https://huggingface.co/second-state/All-MiniLM-L6-v2-Embedding-GGUF/blob/main/all-MiniLM-L6-v2-Q3_K_M.gguf) | Q3_K_M | 3 | 19.9 MB| very small, high quality loss |\n| [all-MiniLM-L6-v2-Q3_K_S.gguf](https://huggingface.co/second-state/All-MiniLM-L6-v2-Embedding-GGUF/blob/main/all-MiniLM-L6-v2-Q3_K_S.gguf) | Q3_K_S | 3 | 19.2 MB| very small, high quality loss |\n| [all-MiniLM-L6-v2-Q4_0.gguf](https://huggingface.co/second-state/All-MiniLM-L6-v2-Embedding-GGUF/blob/main/all-MiniLM-L6-v2-Q4_0.gguf)     | Q4_0   | 4 | 19.7 MB| legacy; small, very high quality loss - prefer using Q3_K_M |\n| [all-MiniLM-L6-v2-Q4_K_M.gguf](https://huggingface.co/second-state/All-MiniLM-L6-v2-Embedding-GGUF/blob/main/all-MiniLM-L6-v2-Q4_K_M.gguf) | Q4_K_M | 4 | 21 MB| medium, balance…\n\nSource: https://huggingface.co/second-state/All-MiniLM-L6-v2-Embedding-GGUF","install":{"kind":"model","hfId":"second-state/All-MiniLM-L6-v2-Embedding-GGUF","gated":false,"format":"gguf","files":[{"name":"all-MiniLM-L6-v2-Q2_K.gguf","size":19229632,"quant":"Q2_K","sha256":"05276e016346f1f72d7581237745758b24df9f7770c4de90d1393494a4485acd"},{"name":"all-MiniLM-L6-v2-Q3_K_L.gguf","size":20473792,"quant":"Q3_K_L","sha256":"873c032bb68e5a0937aaf5ab1dcf593c988d33f3508d9658dddf93c71e85eb06"},{"name":"all-MiniLM-L6-v2-Q3_K_M.gguf","size":19939264,"quant":"Q3_K_M","sha256":"8924abba60768d1fad4b2f24b0b1d9852fa92b98e23384b4bf0c22aeff960ecc"},{"name":"all-MiniLM-L6-v2-Q3_K_S.gguf","size":19229632,"quant":"Q3_K_S","sha256":"3f262eeca245fb741513ef71d81584843cfb1e0e623f785af9284ea993922454"},{"name":"all-MiniLM-L6-v2-Q4_0.gguf","size":19699648,"quant":"Q4_0","sha256":"8cb01a79a55e00a936ca17a07886302bbb005680afd4255d570787ba4b8c8c1d"},{"name":"all-MiniLM-L6-v2-Q4_K_M.gguf","size":20999104,"quant":"Q4_K_M","sha256":"2ec4cee28a27a9c973d5f5230930d6ef6e52694bd2bc71be26a9bef5b1d755e6"},{"name":"all-MiniLM-L6-v2-Q4_K_S.gguf","size":20694976,"quant":"Q4_K_S","sha256":"f578fb3eb4627880e86528f765c1024b73f1f41fc06a07a3dd1f449d10843ffe"},{"name":"all-MiniLM-L6-v2-Q5_0.gguf","size":21026752,"quant":"Q5_0","sha256":"374ae99e7e3aeb042d3ed39a0c97ae24ff583eb46bb2b5e21194d2b74b8bbfd8"},{"name":"all-MiniLM-L6-v2-Q5_K_M.gguf","size":21717952,"quant":"Q5_K_M","sha256":"60c7e141495321c7d303ec5ccc79296cfeb044263af840c583fed695d423aee8"},{"name":"all-MiniLM-L6-v2-Q5_K_S.gguf","size":21469120,"quant":"Q5_K_S","sha256":"0de5c3d1343904debb305a2f6c55f57ccdc68786968b5d4726dd404f6c4c4244"},{"name":"all-MiniLM-L6-v2-Q6_K.gguf","size":24150976,"quant":"Q6_K","sha256":"e94635fd3ddb878b633f3cac7841a6ea734c6bedea57b5e26f0ff5eb3713382c"},{"name":"all-MiniLM-L6-v2-Q8_0.gguf","size":25008064,"quant":"Q8_0","sha256":"263215c3cadd6e16740741a7624ab4cbb6c8e777688bd5331ecfbf5681c2f8ed"},{"name":"all-MiniLM-L6-v2-ggml-model-f16.gguf","size":45949216,"quant":"F16","sha256":"797b70c4edf85907fe0a49eb85811256f65fa0f7bf52166b147fd16be2be4662"}],"totalBytes":299588128,"suggestedFile":"all-MiniLM-L6-v2-Q4_K_M.gguf","requirements":{"ramGb":1,"diskBytes":20999104,"note":"estimate: suggested file size × 1.15 + 0.5 GB; a real measurement comes with lsh models"},"runWith":["llama.cpp"],"command":"lsh models install hf:second-state/All-MiniLM-L6-v2-Embedding-GGUF"}}