{"v":1,"id":"model:hf:cstr/bge-m3-GGUF","slug":"model-cstr-bge-m3-gguf","kind":"model","category":"embedding","title":"bge-m3-GGUF","summary":"GGUF format of BAAI/bge-m3 for use with CrispEmbed.","source":{"provider":"hf","ref":"cstr/bge-m3-GGUF","url":"https://huggingface.co/cstr/bge-m3-GGUF","rev":"12557c1e33245a4f4a54e0f2f8fc40dd3fb98109","fetchedAt":"2026-10-02T21:00:10.661Z","etag":"W/\"a18-nTaSF0vllRZuDoyfzjwm+gCd3eI\""},"author":{"name":"cstr","url":"https://huggingface.co/cstr"},"license":{"spdx":"mit","raw":"mit","open":true},"metrics":{"downloads":2123,"downloadsWeek":4402710,"likes":1,"takenAt":"2026-10-02T21:00:10.661Z"},"tags":["gguf","embeddings","ggml","text-embeddings","xlm-r","crispembed","feature-extraction","multilingual","endpoints_compatible"],"pipeline":"feature-extraction","links":{"github":"FlagOpen/FlagEmbedding","npm":"@huggingface/transformers"},"updatedAt":"2026-08-02T15:19:22.000Z","collectedAt":"2026-10-02T21:00:10.661Z","review":{"numbers":["2,123 downloads on Hugging Face","1 likes","license mit","0.6 GB for bge-m3-q8_0.gguf","4,402,710 npm downloads a week for @huggingface/transformers","latest @huggingface/transformers@4.3.0"],"log":null},"trust":"unlabeled","health":{"status":"alive","checkedAt":"2026-10-02T21:00:10.661Z","http":200},"description":"# bge-m3 GGUF\n\nGGUF format of [BAAI/bge-m3](https://huggingface.co/BAAI/bge-m3) for use with [CrispEmbed](https://github.com/CrispStrobe/CrispEmbed).\n\nBGE-M3. Dense + sparse + ColBERT multi-vector retrieval in one model. 100+ languages, 8192 context.\n\n## Files\n\n| File | Quantization | Size |\n|------|-------------|------|\n| [bge-m3-q4_k.gguf](https://huggingface.co/cstr/bge-m3-GGUF/resolve/main/bge-m3-q4_k.gguf) | Q4_K | 438 MB |\n| [bge-m3-q8_0.gguf](https://huggingface.co/cstr/bge-m3-GGUF/resolve/main/bge-m3-q8_0.gguf) | Q8_0 | 583 MB |\n| [bge-m3.gguf](https://huggingface.co/cstr/bge-m3-GGUF/resolve/main/bge-m3.gguf) | F32 | 2175 MB |\n\n## Quick Start\n\n```bash\n# Download\nhuggingface-cli download cstr/bge-m3-GGUF bge-m3-q4_k.gguf --local-dir .\n\n# Run with CrispEmbed\n./crispembed -m bge-m3-q4_k.gguf \"Hello world\"\n\n# Or with auto-download\n./crispembed -m bge-m3 \"Hello world\"\n```\n\n## Model Details\n\n| Property | Value |\n|----------|-------|\n| Architecture | XLM-R |\n| Parameters | 568M |\n| Embedding Dimension | 1024 |\n| Layers | 24 |\n| Pooling | mean |\n| Tokenizer | SentencePiece |\n| Base Model | [BAAI/bge-m3](https://huggingface.co/BAAI/bge-m3) |\n\n## Verification\n\nVerified bit-identical to HuggingFace sentence-transformers (cosine similarity >= 0.999 on test texts).\n\n## Usage with CrispEmbed\n\nCrispEmbed is a lightweight C/C++ text embedding inference engine using ggml.\nNo Python runtime, no ONNX. Supports BERT, XLM-R, Qwen3, and Gemma3 architectures.\n\n```bash\n# Build CrispEmbed\ngit clone https://github.com/CrispStrobe/CrispEmbed\ncd CrispEmbed\ncmake -S . -B build && cmake --build build -j\n\n# Encode\n./build/crispembed -m bge-m3-q4_k.gguf \"query text\"\n\n# Server mode\n./build/crispembed-server -m bge-m3-q4_k.gguf --port 8080\ncurl -X POST http://localhost:8080/v1/embeddings \\\n    -d '{\"input\": [\"Hello world\"], \"model\": \"bge-m3\"}'\n```\n\n## Credits\n\n- Original model: [BAAI/bge-m3](https://huggingface.co/BAAI/bge-m3)\n- Inference eng…\n\nSource: https://huggingface.co/cstr/bge-m3-GGUF","install":{"kind":"model","hfId":"cstr/bge-m3-GGUF","gated":false,"format":"gguf","files":[{"name":"bge-m3-iq4_xs.gguf","size":449244128,"quant":"IQ4_XS","sha256":"d52ab762314946dfc8351913356156ffa58da595b7925805e32834cb5af726c6"},{"name":"bge-m3-q4_k-imatrix.gguf","size":458681312,"quant":"Q4_K","sha256":"942c339565cec14b975364cd26bc5d8a1b1ae8968313276a84dc417a26a94b73"},{"name":"bge-m3-q4_k.gguf","size":458681312,"quant":"Q4_K","sha256":"600f8b576c8eced20d47263511ed7a7d85b2cc6ba8c26edcd7ae1f89181d2e86"},{"name":"bge-m3-q8_0.gguf","size":609676256,"quant":"Q8_0","sha256":"777b9a12bd606063d0af16decb6857681871a7ec4a8126fab6cec7ed4fc426a8"},{"name":"bge-m3.gguf","size":2273425056,"sha256":"357fc469a9025ded46c3edba9a834a7d30d53d44850e120606bb43c92f7f1ebe"}],"totalBytes":4249708064,"suggestedFile":"bge-m3-q8_0.gguf","requirements":{"ramGb":2,"diskBytes":609676256,"note":"estimate: suggested file size × 1.15 + 0.5 GB; a real measurement comes with lsh models"},"runWith":["llama.cpp"],"command":"lsh models install hf:cstr/bge-m3-GGUF"}}