{"v":1,"id":"model:hf:PeterAM4/Qwen3-Embedding-0.6B-GGUF","slug":"model-peteram4-qwen3-embedding-0-6b-gguf","kind":"model","category":"embedding","title":"Qwen3-Embedding-0.6B-GGUF","summary":"All-in-one GGUF quantizations of Qwen/Qwen3-Embedding-0.6B, from 8-bit down to 1-bit, with importance-matrix calibration optimized for financial and technical text retrieval.","source":{"provider":"hf","ref":"PeterAM4/Qwen3-Embedding-0.6B-GGUF","url":"https://huggingface.co/PeterAM4/Qwen3-Embedding-0.6B-GGUF","rev":"bb661aeeeafa4ff7303b5f5e1e80255efd6a513c","fetchedAt":"2026-10-02T20:53:56.067Z","etag":"W/\"3583-yOWbtEMN6uv6hAWJPpbNDFMfYHk\""},"author":{"name":"PeterAM4","url":"https://huggingface.co/PeterAM4"},"license":{"spdx":"apache-2.0","raw":"apache-2.0","open":true},"metrics":{"downloads":2591,"downloadsWeek":327838,"likes":3,"stars":27665,"openIssues":68,"pushedAt":"2026-01-09T03:05:47Z","takenAt":"2026-10-02T20:53:56.067Z"},"tags":["gguf","quantized","embedding","sentence-transformers","Qwen3","imatrix","llama-cpp","sentence-similarity","en","zh","ja","ko","fr","de","es","pt","ru","ar","dataset:explodinggradients/fiqa","dataset:PatronusAI/financebench","dataset:zeroshot/twitter-financial-news-sentiment","dataset:philschmid/finanical-rag-embedding-dataset","dataset:openai/gsm8k","dataset:DigitalLearningGmbH/MATH-lighteval","endpoints_compatible","conversational"],"pipeline":"sentence-similarity","links":{"github":"QwenLM/Qwen3","npm":"node-llama-cpp"},"updatedAt":"2026-02-07T10:43:06.000Z","collectedAt":"2026-10-02T20:53:56.067Z","review":{"numbers":["2,591 downloads on Hugging Face","3 likes","license apache-2.0","0.4 GB for Qwen3-Embedding-0.6B-Q4_K_M-imat.gguf","27,665 stars on QwenLM/Qwen3","68 open issues and PRs","327,838 npm downloads a week for node-llama-cpp","latest node-llama-cpp@3.22.1"],"log":null},"trust":"unlabeled","health":{"status":"alive","checkedAt":"2026-10-02T20:53:56.067Z","http":200},"description":"# Qwen3-Embedding-0.6B -- GGUF\n\nAll-in-one GGUF quantizations of [Qwen/Qwen3-Embedding-0.6B](https://huggingface.co/Qwen/Qwen3-Embedding-0.6B), from 8-bit down to 1-bit, with importance-matrix calibration optimized for financial and technical text retrieval.\n\nQwen3-Embedding-0.6B is a compact, multilingual embedding model well suited for RAG pipelines, semantic search, and document retrieval. These quantizations make it practical to run on edge devices, laptops, and resource-constrained servers -- particularly for financial NLP workloads where low latency and small memory footprint matter.\n\nThe importance matrix was calibrated on a mixed corpus weighted toward financial data (financial Q&A from FiQA, SEC 10-K filings from FinanceBench, financial sentiment from Twitter, RAG pairs) alongside math reasoning and general text, so the quantized models preserve the weights most relevant to financial domain embeddings.\n\n| Property | Value |\n|----------|-------|\n| Base model | [Qwen/Qwen3-Embedding-0.6B](https://huggingface.co/Qwen/Qwen3-Embedding-0.6B) |\n| Parameters | 595,776,512 |\n| Max context | 32,768 tokens |\n| Pooling | Last token |\n| Embedding dim | 1024 |\n| License | Apache 2.0 |\n| Quantized with | [llama.cpp](https://github.com/ggerganov/llama.cpp) |\n\n---\n\n## Why quantize?\n\nDespite their size, neural networks are remarkably sparse in information density. Most of the 16 bits allocated per weight during training exist to make gradient descent work -- not to store knowledge. Current estimates put the actual information content at roughly 2 bits per parameter. The remaining 14 bits are redundancy.\n\nThis explains why aggressive quantization works: compressing from 16-bit to 4-bit (75% reduction) discards almost exclusively noise. Our benchmark data confirms this -- Q3_K_M-imat at 4.66 BPW scores within +0.62 PPL of the full BF16 baseline while being 70% smaller than BF16 (331 MB vs 1.1 GB). For comparis…\n\nSource: https://huggingface.co/PeterAM4/Qwen3-Embedding-0.6B-GGUF","install":{"kind":"model","hfId":"PeterAM4/Qwen3-Embedding-0.6B-GGUF","gated":false,"format":"gguf","files":[{"name":"Qwen3-Embedding-0.6B-BF16.gguf","size":1197629184,"quant":"BF16","sha256":"2157971f1029eecda7dd47045602a0e6ea566aa75ee35d3514715afc4ff7f272"},{"name":"Qwen3-Embedding-0.6B-IQ1_M-imat.gguf","size":215857568,"quant":"IQ1_M","sha256":"96e4067f12435a88ec3b11c1ec2b2aac7bc26105380c6f2258b6ba816ef4ea9a"},{"name":"Qwen3-Embedding-0.6B-IQ1_S-imat.gguf","size":207821216,"quant":"IQ1_S","sha256":"35ff51b6e3dc617e2172594ec94b2d58dfb3b302039841335988416e94dd2cd1"},{"name":"Qwen3-Embedding-0.6B-IQ2_M-imat.gguf","size":264714656,"quant":"IQ2_M","sha256":"6bd605a6284767b981de0806c69f66552a8a14a53dc9d728a41dd0e64f7e22e8"},{"name":"Qwen3-Embedding-0.6B-IQ2_S-imat.gguf","size":253999520,"quant":"IQ2_S","sha256":"31dbd295920e57a86974e699a51e86ec9ac73b75e9f3de645dd9c493de3fd54f"},{"name":"Qwen3-Embedding-0.6B-IQ2_XS-imat.gguf","size":241801632,"quant":"IQ2_XS","sha256":"eff4c307f59e6d0850e6d1c4d28ee5e572549aa2a5f299d9fc8dc9a27976ecb3"},{"name":"Qwen3-Embedding-0.6B-IQ2_XXS-imat.gguf","size":229251488,"quant":"IQ2_XXS","sha256":"0ead842ffd5982d1ed81f22f250ff52b93d04d9ca71bb62bfd127210c4eb0354"},{"name":"Qwen3-Embedding-0.6B-IQ3_M-imat.gguf","size":335796192,"quant":"IQ3_M","sha256":"bedb41dd695a42ebbc313f255762ec9f6a9b07b318d6f38fdd32bf4a1cc708a3"},{"name":"Qwen3-Embedding-0.6B-IQ3_S-imat.gguf","size":322844640,"quant":"IQ3_S","sha256":"a66ded5a4d45efd762a8cb591a845767e5b8ad4fa1255541ff4bfdbbed701666"},{"name":"Qwen3-Embedding-0.6B-IQ3_XS-imat.gguf","size":312522720,"quant":"IQ3_XS","sha256":"b0b1a5e38d4932ad397d0c4bdb0e3cee1ba6eae98db2a47203754c830ac166d0"},{"name":"Qwen3-Embedding-0.6B-IQ3_XXS-imat.gguf","size":278821280,"quant":"IQ3_XXS","sha256":"d26061f24fa5769200221c032ee07ea6858c12d020a4a57bbd50b624d7c9688a"},{"name":"Qwen3-Embedding-0.6B-IQ4_NL-imat.gguf","size":381335520,"quant":"IQ4_NL","sha256":"c42db09657d046b154baa294463e58fa3b241551d262825d5aee50d9a92ce22f"},{"name":"Qwen3-Embedding-0.6B-IQ4_XS-imat.gguf","size":367572960,"quant":"IQ4_XS","sha256":"233223e29389e6a3487f2df7e26a1847a95dde2ad62696fe5528d066c77f4d2f"},{"name":"Qwen3-Embedding-0.6B-Q2_K-imat.gguf","size":296007648,"quant":"Q2_K","sha256":"8e19ca79f95824b05bfd628b52b5867b6977144b1cafef326ea7cfe854432e29"},{"name":"Qwen3-Embedding-0.6B-Q2_K_S-imat.gguf","size":280328160,"quant":"Q2_K_S","sha256":"3714f660d4cbf13355ff93d52b7191eef21518d57299a7a452d204d333ab3e68"},{"name":"Qwen3-Embedding-0.6B-Q3_K_L-imat.gguf","size":368261088,"quant":"Q3_K_L","sha256":"da42556e05ce2e0f316e4db2b0344ca7970d76d62be7d746bd4b9eeccafdd803"},{"name":"Qwen3-Embedding-0.6B-Q3_K_M-imat.gguf","size":346896352,"quant":"Q3_K_M","sha256":"7056ab1ce1ed50093cda91b27cc21025b0aacd9addc2442ffe056df2a0a83e41"},{"name":"Qwen3-Embedding-0.6B-Q3_K_S-imat.gguf","size":322844640,"quant":"Q3_K_S","sha256":"90f087e68e0f8d4e5ecc3c9ed36cb9dbbdadc36ee142a67542015dfe2670005d"},{"name":"Qwen3-Embedding-0.6B-Q4_0-imat.gguf","size":381925344,"quant":"Q4_0","sha256":"d64c751c09425ab44bcf4a2f224c98ebfb789a6caada1e7a5e19a325268ca899"},{"name":"Qwen3-Embedding-0.6B-Q4_1-imat.gguf","size":408860640,"quant":"Q4_1","sha256":"4ba25a82158a722aa286611164a7d76522c9a0ede421448c2d91e1f66bd65120"},{"name":"Qwen3-Embedding-0.6B-Q4_K_M-imat.gguf","size":396474336,"quant":"Q4_K_M","sha256":"4d0f2e6e4132e96a0a0eac04cebfd11ff08cee89d3128aba6b276ae06f808c30"},{"name":"Qwen3-Embedding-0.6B-Q4_K_S-imat.gguf","size":383039456,"quant":"Q4_K_S","sha256":"e279822fc42c7c97631abae63efd9da8335f8299d25ccb585b794c33ba6a934b"},{"name":"Qwen3-Embedding-0.6B-Q5_0.gguf","size":436385536,"quant":"Q5_0","sha256":"272313b22c2b87cde7dc3a4d5726bbdd218b75fc512446df1facb55ea29cfb46"},{"name":"Qwen3-Embedding-0.6B-Q5_1.gguf","size":463910656,"quant":"Q5_1","sha256":"1617d2878f2177ad3d4bdf3d22b85f117d721b327abd31d32ccbde086dedcd3b"},{"name":"Qwen3-Embedding-0.6B-Q5_K_M.gguf","size":444184320,"quant":"Q5_K_M","sha256":"57719bdda6ae4de8aadebfa8a8a29629f81a6fcab328ff1d38bb223b0759e1b2"},{"name":"Qwen3-Embedding-0.6B-Q5_K_S.gguf","size":436385536,"quant":"Q5_K_S","sha256":"9bd8011bc78abc0e41217fb701c547e69cbe6031b7f295c51f7f4e83fc3772dd"},{"name":"Qwen3-Embedding-0.6B-Q6_K.gguf","size":494876416,"quant":"Q6_K","sha256":"603197d349cbf1d284d8277f5e8326b8b0500d4522908e9d0d46185f06e82551"},{"name":"Qwen3-Embedding-0.6B-Q8_0.gguf","size":639150144,"quant":"Q8_0","sha256":"3e42f708341ab5ff73e9d4e95cfc10b68ed98f0e62b2eea352a37a3b8f3c4f9e"},{"name":"Qwen3-Embedding-0.6B-TQ1_0-imat.gguf","size":226506720,"quant":"TQ1_0","sha256":"a239a55d399a7b00d4566d104d0184310a877bc1f930b410e0e73988d0523f82"},{"name":"Qwen3-Embedding-0.6B-TQ2_0-imat.gguf","size":247150560,"quant":"TQ2_0","sha256":"71b7a9546ea9a668d1af73e4c764bf9537c3258b1ce11b23ab745b3460068c5f"}],"totalBytes":11183156128,"suggestedFile":"Qwen3-Embedding-0.6B-Q4_K_M-imat.gguf","requirements":{"ramGb":1,"diskBytes":396474336,"note":"estimate: suggested file size × 1.15 + 0.5 GB; a real measurement comes with lsh models"},"runWith":["llama.cpp"],"command":"lsh models install hf:PeterAM4/Qwen3-Embedding-0.6B-GGUF"}}