{"v":1,"id":"model:hf:second-state/E5-Mistral-7B-Instruct-Embedding-GGUF","slug":"model-second-state-e5-mistral-7b-instruct-embedding-gguf","kind":"model","category":"embedding","title":"E5-Mistral-7B-Instruct-Embedding-GGUF","summary":"embedding model by second-state, gguf files on Hugging Face.","source":{"provider":"hf","ref":"second-state/E5-Mistral-7B-Instruct-Embedding-GGUF","url":"https://huggingface.co/second-state/E5-Mistral-7B-Instruct-Embedding-GGUF","rev":"3e8b3777ee103fc655e835bd96ab72fe57af4695","fetchedAt":"2026-10-02T20:52:36.111Z","etag":"W/\"140c-aFnDRHIr4MGzI4UJ9gJpnFEgu8k\""},"author":{"name":"second-state","url":"https://huggingface.co/second-state"},"license":{"spdx":"mit","raw":"mit","open":true},"metrics":{"downloads":3819,"likes":15,"takenAt":"2026-10-02T20:52:36.111Z"},"tags":["transformers","gguf","mistral","feature-extraction","en","endpoints_compatible","deploy:azure","conversational"],"pipeline":"feature-extraction","links":{"github":"mistralai/mistral-inference"},"updatedAt":"2024-07-01T03:20:13.000Z","collectedAt":"2026-10-02T20:52:36.111Z","review":{"numbers":["3,819 downloads on Hugging Face","15 likes","license mit","4.1 GB for e5-mistral-7b-instruct-Q4_K_M.gguf"],"log":null},"trust":"unlabeled","health":{"status":"alive","checkedAt":"2026-10-02T20:52:36.111Z","http":200},"description":"# E5-Mistral-7B-Instruct-Embedding-GGUF\n\n## Original Model\n\n[intfloat/e5-mistral-7b-instruct](https://huggingface.co/intfloat/e5-mistral-7b-instruct)\n\n## Run with LlamaEdge\n\n- LlamaEdge version: [v0.8.2](https://github.com/LlamaEdge/LlamaEdge/releases/tag/0.8.2) and above\n\n- Prompt template\n\n  - Prompt type: `embedding`\n\n- Context size: `4096`\n\n- Run as LlamaEdge service\n\n  ```bash\n  wasmedge --dir .:. --nn-preload default:GGML:AUTO:e5-mistral-7b-instruct-Q5_K_M.gguf \\\n    llama-api-server.wasm \\\n    --prompt-template embedding \\\n    --ctx-size 4096 \\\n    --model-name e5-mistral-7b-instruct\n  ```\n\n## Quantized GGUF Models\n\n| Name | Quant method | Bits | Size | Use case |\n| ---- | ---- | ---- | ---- | ----- |\n| [e5-mistral-7b-instruct-Q2_K.gguf](https://huggingface.co/second-state/E5-Mistral-7B-Instruct-Embedding-GGUF/blob/main/e5-mistral-7b-instruct-Q2_K.gguf)     | Q2_K   | 2 | 2.72 GB| smallest, significant quality loss - not recommended for most purposes |\n| [e5-mistral-7b-instruct-Q3_K_L.gguf](https://huggingface.co/second-state/E5-Mistral-7B-Instruct-Embedding-GGUF/blob/main/e5-mistral-7b-instruct-Q3_K_L.gguf) | Q3_K_L | 3 | 3.82 GB| small, substantial quality loss |\n| [e5-mistral-7b-instruct-Q3_K_M.gguf](https://huggingface.co/second-state/E5-Mistral-7B-Instruct-Embedding-GGUF/blob/main/e5-mistral-7b-instruct-Q3_K_M.gguf) | Q3_K_M | 3 | 3.52 GB| very small, high quality loss |\n| [e5-mistral-7b-instruct-Q3_K_S.gguf](https://huggingface.co/second-state/E5-Mistral-7B-Instruct-Embedding-GGUF/blob/main/e5-mistral-7b-instruct-Q3_K_S.gguf) | Q3_K_S | 3 | 3.16 GB| very small, high quality loss |\n| [e5-mistral-7b-instruct-Q4_0.gguf](https://huggingface.co/second-state/E5-Mistral-7B-Instruct-Embedding-GGUF/blob/main/e5-mistral-7b-instruct-Q4_0.gguf)     | Q4_0   | 4 | 4.11 GB| legacy; small, very high quality loss - prefer using Q3_K_M |\n| [e5-mistral-7b-instruct-Q4_K_M.gguf](https://hug…\n\nSource: https://huggingface.co/second-state/E5-Mistral-7B-Instruct-Embedding-GGUF","install":{"kind":"model","hfId":"second-state/E5-Mistral-7B-Instruct-Embedding-GGUF","gated":false,"format":"gguf","files":[{"name":"e5-mistral-7b-instruct-Q2_K.gguf","size":2719242464,"quant":"Q2_K","sha256":"f3de8028cf73de436e3b7469d326e082b8f4bbfaa3a78482cce7f50c81623bdd"},{"name":"e5-mistral-7b-instruct-Q3_K_L.gguf","size":3822024928,"quant":"Q3_K_L","sha256":"d885afa61abd5ba076420caf37773ce858803ad8cdd83e7ef908c7eda70157df"},{"name":"e5-mistral-7b-instruct-Q3_K_M.gguf","size":3518986464,"quant":"Q3_K_M","sha256":"b3f601d9fbfe171b30d91662cbdd12afce505594c2a2b3ac2dff1af95e04c4d6"},{"name":"e5-mistral-7b-instruct-Q3_K_S.gguf","size":3164567776,"quant":"Q3_K_S","sha256":"fc0a283d2c6a2f4da84faea44b54923f1870f9c072f8449bbd9e4d5f14dad7bc"},{"name":"e5-mistral-7b-instruct-Q4_0.gguf","size":4108916960,"quant":"Q4_0","sha256":"f75c0e30696fa7f1656320905943fb42e489e7c4156c37f37e9a17060b5e3e89"},{"name":"e5-mistral-7b-instruct-Q4_K_M.gguf","size":4368439520,"quant":"Q4_K_M","sha256":"96f5b8305d2091b48dc1191c082e61cca1f0fc7181650106bed3dd5a495ab5e4"},{"name":"e5-mistral-7b-instruct-Q4_K_S.gguf","size":4140374240,"quant":"Q4_K_S","sha256":"936c68fccd86093be69cfc22c0a7bbee50b1ec9f6712fc2ab62cfcbd11355629"},{"name":"e5-mistral-7b-instruct-Q5_0.gguf","size":4997716192,"quant":"Q5_0","sha256":"390dd1811b1555fdf790a14a26d73db7b4b703b2d3104b7e40eb6c783f3ed218"},{"name":"e5-mistral-7b-instruct-Q5_K_M.gguf","size":5131409632,"quant":"Q5_K_M","sha256":"606d222135473106ac45139cc4f2582ad3065a3344a81e18e113e70043409faf"},{"name":"e5-mistral-7b-instruct-Q5_K_S.gguf","size":4997716192,"quant":"Q5_K_S","sha256":"de884c70ce387a6886ef33b83a163e5451d05c058c846eda4e7982e1a90282ce"},{"name":"e5-mistral-7b-instruct-Q6_K.gguf","size":5942065376,"quant":"Q6_K","sha256":"a93b1c51cfe65192858dd4fc02eab38ab9616f4843f5a38c1db9371cca625946"},{"name":"e5-mistral-7b-instruct-Q8_0.gguf","size":7695857888,"quant":"Q8_0","sha256":"165660156014e0e4779b194d200d24ff7e47805eea27b0198283f7892469bc23"},{"name":"e5-mistral-7b-instruct-f16.gguf","size":14484732064,"quant":"F16","sha256":"b21f7dcb5718c536b04d4463f4c1907d45dafc35308505e587f71019a5f63ba3"}],"totalBytes":69092049696,"suggestedFile":"e5-mistral-7b-instruct-Q4_K_M.gguf","requirements":{"ramGb":6,"diskBytes":4368439520,"note":"estimate: suggested file size × 1.15 + 0.5 GB; a real measurement comes with lsh models"},"runWith":["llama.cpp"],"command":"lsh models install hf:second-state/E5-Mistral-7B-Instruct-Embedding-GGUF"}}