{"v":1,"id":"model:hf:jinaai/jina-embeddings-v5-text-nano-retrieval-GGUF","slug":"model-jinaai-jina-embeddings-v5-text-nano-retrieval-gguf","kind":"model","category":"embedding","title":"jina-embeddings-v5-text-nano-retrieval-GGUF","summary":"jina-embeddings-v5-text-nano-retrieval-GGUF","source":{"provider":"hf","ref":"jinaai/jina-embeddings-v5-text-nano-retrieval-GGUF","url":"https://huggingface.co/jinaai/jina-embeddings-v5-text-nano-retrieval-GGUF","rev":"59cfaceeeb7d738c404659435af4c0da74d06c96","fetchedAt":"2026-10-02T21:00:55.537Z","etag":"W/\"1295-UsB/OlkPrghWGV4q/JBEp/e7mMA\""},"author":{"name":"jinaai","url":"https://huggingface.co/jinaai"},"license":{"spdx":"cc-by-nc-4.0","raw":"cc-by-nc-4.0","open":false,"note":"restricted license: terms at the source apply"},"metrics":{"downloads":4575,"downloadsWeek":327838,"likes":4,"takenAt":"2026-10-02T21:00:55.537Z"},"tags":["llama.cpp","gguf","embedding","eurobert","llama-cpp","jina-embeddings-v5","sentence-similarity","multilingual","feature-extraction"],"pipeline":"sentence-similarity","links":{"github":"ggml-org/llama.cpp","npm":"node-llama-cpp"},"updatedAt":"2026-02-27T12:23:08.000Z","collectedAt":"2026-10-02T21:00:55.537Z","review":{"numbers":["4,575 downloads on Hugging Face","4 likes","license cc-by-nc-4.0","0.1 GB for v5-nano-retrieval-Q4_K_M.gguf","327,838 npm downloads a week for node-llama-cpp","latest node-llama-cpp@3.22.1"],"log":null},"trust":"unlabeled","health":{"status":"alive","checkedAt":"2026-10-02T21:00:55.537Z","http":200},"description":"# jina-embeddings-v5-text-nano-retrieval-GGUF\n\nGGUF quantizations of [jina-embeddings-v5-text-nano-retrieval](https://huggingface.co/jinaai/jina-embeddings-v5-text-nano) using llama.cpp. A 239M parameter multilingual embedding model quantized for efficient inference.\n\n[Elastic Inference Service](https://www.elastic.co/docs/explore-analyze/elastic-inference/eis) | [ArXiv](https://arxiv.org/abs/2602.15547) | [Blog](https://jina.ai/news/jina-embeddings-v5-text-distilling-4b-quality-into-sub-1b-multilingual-embeddings)\n\n> [!IMPORTANT]\n> We highly recommend to first read [this blog post for more technical details and customized llama.cpp build](https://jina.ai/news/optimizing-ggufs-for-decoder-only-embedding-models).\n\n## Overview\n\n`jina-embeddings-v5-text-nano-retrieval` is a task-specific embedding model for **retrieval**, part of the [jina-embeddings-v5-text](https://huggingface.co/jinaai/jina-embeddings-v5-text-nano) model family.\n| Feature | Value |\n| --- | --- |\n| Parameters | 239M |\n| Task | `retrieval` |\n| Embedding Dimension | 768 |\n| Matryoshka Dimensions | 32, 64, 128, 256, 512, 768 |\n| Pooling Strategy | Last-token pooling |\n| Base Model | [jina-embeddings-v5-text-nano](https://huggingface.co/jinaai/jina-embeddings-v5-text-nano) |\n\n## Usage with llama.cpp\n\n  via Elastic Inference Service\n\nThe fastest way to use v5-text in production. Elastic Inference Service (EIS) provides managed embedding inference with built-in scaling, so you can generate embeddings directly within your Elastic deployment.\n\n```bash\nPUT _inference/text_embedding/jina-v5\n{\n  \"service\": \"elastic\",\n  \"service_settings\": {\n    \"model_id\": \"jina-embeddings-v5-text-nano\"\n  }\n}\n```\n\nSee the [Elastic Inference Service documentation](https://www.elastic.co/docs/explore-analyze/elastic-inference/eis) for setup details.\n\n```bash\n# Build llama.cpp (upstream)\ngit clone https://github.com/ggml-org/llama.cpp\ncd llama.cpp…\n\nSource: https://huggingface.co/jinaai/jina-embeddings-v5-text-nano-retrieval-GGUF","install":{"kind":"model","hfId":"jinaai/jina-embeddings-v5-text-nano-retrieval-GGUF","gated":false,"format":"gguf","files":[{"name":"v5-nano-retrieval-F16.gguf","size":431396128,"quant":"F16","sha256":"0c09e71f0bbcfceb06c44d56da2fdcee8b14b2e0d72b33e321ee2a7d7b350826"},{"name":"v5-nano-retrieval-IQ1_M.gguf","size":101703232,"quant":"IQ1_M","sha256":"0990f1569f4317bdd4a7b7c4e8a88da579f1a360d86655ad967f2dd5a2e75d33"},{"name":"v5-nano-retrieval-IQ1_S.gguf","size":99436096,"quant":"IQ1_S","sha256":"65115271dc63d78a2172386821574816e8b46cfdb7a51259bec5185fe5897f55"},{"name":"v5-nano-retrieval-IQ2_M.gguf","size":113702464,"quant":"IQ2_M","sha256":"24a6d6ecb1e23b28b0230c049a1a0186166b5da2caedbef7447b2950cea3ec0e"},{"name":"v5-nano-retrieval-IQ2_XXS.gguf","size":105481792,"quant":"IQ2_XXS","sha256":"18e63fbd8694a98ff365212fcf63f50de60d49414500da8fa12b70d0324f759e"},{"name":"v5-nano-retrieval-IQ4_NL.gguf","size":152405056,"quant":"IQ4_NL","sha256":"f243c19458bcd8162632b89bfeaf02b077b24c2d812e5ece1b73bd380732ba29"},{"name":"v5-nano-retrieval-IQ4_XS.gguf","size":148866112,"quant":"IQ4_XS","sha256":"6c53e3f56df187833a88a6b07953469653903e7ddc7d2bc5b926f16820e46199"},{"name":"v5-nano-retrieval-Q2_K.gguf","size":130176064,"quant":"Q2_K","sha256":"79dba23470b6c7b5a283a13ddff5d8a7c7c3b269d259bbb8202f9087509d7381"},{"name":"v5-nano-retrieval-Q3_K_M.gguf","size":143152192,"quant":"Q3_K_M","sha256":"191937158b1b0369ccdfb6214f24537c7ff955d2d5849712338ac085d8bb113b"},{"name":"v5-nano-retrieval-Q4_K_M.gguf","size":156966976,"quant":"Q4_K_M","sha256":"f50822244ba0c7a348c5455b99bb8a0afd182511e8a816888c5dc65d972e51d5"},{"name":"v5-nano-retrieval-Q5_K_M.gguf","size":168910912,"quant":"Q5_K_M","sha256":"46fbc0423862cb6a5d4ff776d885f349d2a87c36d821dd5630f9fa184c9b4b92"},{"name":"v5-nano-retrieval-Q5_K_S.gguf","size":166560832,"quant":"Q5_K_S","sha256":"6aaf456de892e5244b924bd8533a57fd509d27dfbdb0a74c250a5d92bdfb09c3"},{"name":"v5-nano-retrieval-Q6_K.gguf","size":181601344,"quant":"Q6_K","sha256":"cbbad17eeb1189c4f4706e6df89b5fbab17c72b7878a8a71a74fdb2a46f8d995"},{"name":"v5-nano-retrieval-Q8_0.gguf","size":232883776,"quant":"Q8_0","sha256":"86b6e6279e9b9e71389f02a082764a2ac2b15a50e37482c26f98d69092f12442"}],"totalBytes":2333242976,"suggestedFile":"v5-nano-retrieval-Q4_K_M.gguf","requirements":{"ramGb":1,"diskBytes":156966976,"note":"estimate: suggested file size × 1.15 + 0.5 GB; a real measurement comes with lsh models"},"runWith":["llama.cpp"],"command":"lsh models install hf:jinaai/jina-embeddings-v5-text-nano-retrieval-GGUF"}}