{"v":1,"id":"model:hf:handy-computer/Voxtral-Mini-4B-Realtime-2602-gguf","slug":"model-handy-computer-voxtral-mini-4b-realtime-2602-gguf","kind":"model","category":"speech","title":"Voxtral-Mini-4B-Realtime-2602-gguf","summary":"Voxtral-Mini-4B-Realtime-2602: transcribe.cpp GGUF","source":{"provider":"hf","ref":"handy-computer/Voxtral-Mini-4B-Realtime-2602-gguf","url":"https://huggingface.co/handy-computer/Voxtral-Mini-4B-Realtime-2602-gguf","rev":"65d1c9408859a0ca0f1c11b025ee951486af21d6","fetchedAt":"2026-10-02T20:59:17.806Z","etag":"W/\"eca-ruB8r+Ss+GZq5lQXz1F40sIPkBQ\""},"author":{"name":"handy-computer","url":"https://huggingface.co/handy-computer"},"license":{"spdx":"apache-2.0","raw":"apache-2.0","open":true},"metrics":{"downloads":400835,"likes":6,"takenAt":"2026-10-02T20:59:17.806Z"},"tags":["transcribe.cpp","gguf","asr","speech-to-text","voxtral","audio-llm","streaming","multilingual","automatic-speech-recognition","en","fr","es","de","ru","zh","ja","it","pt","nl","ar","hi","ko"],"pipeline":"automatic-speech-recognition","links":{"github":"mistralai/mistral-inference"},"updatedAt":"2026-09-15T07:05:44.000Z","collectedAt":"2026-10-02T20:59:17.806Z","review":{"numbers":["400,835 downloads on Hugging Face","6 likes","license apache-2.0","2.6 GB for Voxtral-Mini-4B-Realtime-2602-Q4_K_M.gguf"],"log":null},"trust":"unlabeled","health":{"status":"alive","checkedAt":"2026-10-02T20:59:17.806Z","http":200},"description":"# Voxtral-Mini-4B-Realtime-2602: transcribe.cpp GGUF\n\nGGUF conversions of [mistralai/Voxtral-Mini-4B-Realtime-2602](https://huggingface.co/mistralai/Voxtral-Mini-4B-Realtime-2602) for use\nwith [transcribe.cpp](https://github.com/handy-computer/transcribe.cpp).\n\nPorted from upstream commit\n[2769294](https://huggingface.co/mistralai/Voxtral-Mini-4B-Realtime-2602/commit/2769294),\npinned 2026-06-06.\nValidated against the Transformers reference at transcribe.cpp commit\n[483c122](https://github.com/handy-computer/transcribe.cpp/tree/483c122)\non 2026-06-06.\n\nStreaming audio-LLM speech-to-text. A ~970M causal audio encoder (left-pad\ncausal conv stem + 32-layer sliding-window RoPE transformer) feeds a\n4-frame-group projector whose audio embeddings are added onto a ~3.4B\nMinistral decoder (26 layers, GQA 32/8, NEOX RoPE) with delay-token latency\nconditioning, emitting one text token per 80 ms audio slot (12.5 Hz). Takes a\n16 kHz mono WAV and supports both incremental streaming (configurable\nlatency/quality via --stream-chunk-ms and --stream-voxtral-delay) and offline\ntranscription using an accuracy-first 2400 ms delay. Architecturally distinct\nfrom the offline Voxtral 2507 family — own arch, streaming frontend, causal\nencoder, additive audio fusion.\n\n## Downloads\n\n| Quantization | Download | Size | WER (LibriSpeech test-clean) |\n| --- | --- | ---: | ---: |\n| BF16 | [Voxtral-Mini-4B-Realtime-2602-BF16.gguf](https://huggingface.co/handy-computer/Voxtral-Mini-4B-Realtime-2602-gguf/resolve/main/Voxtral-Mini-4B-Realtime-2602-BF16.gguf) | 8.87 GB | 2.08% |\n| F16 | [Voxtral-Mini-4B-Realtime-2602-F16.gguf](https://huggingface.co/handy-computer/Voxtral-Mini-4B-Realtime-2602-gguf/resolve/main/Voxtral-Mini-4B-Realtime-2602-F16.gguf) | 8.88 GB | 2.09% |\n| Q8_0 | [Voxtral-Mini-4B-Realtime-2602-Q8_0.gguf](https://huggingface.co/handy-computer/Voxtral-Mini-4B-Realtime-2602-gguf/resolve/main/Voxtral-Mini-4B-Re…\n\nSource: https://huggingface.co/handy-computer/Voxtral-Mini-4B-Realtime-2602-gguf","install":{"kind":"model","hfId":"handy-computer/Voxtral-Mini-4B-Realtime-2602-gguf","gated":false,"format":"gguf","files":[{"name":"Voxtral-Mini-4B-Realtime-2602-BF16.gguf","size":8868301088,"quant":"BF16","sha256":"dca6a7bece55858472e1b2df1f928642bdc1e964b1a91f3378c3a4e72fc5bc04"},{"name":"Voxtral-Mini-4B-Realtime-2602-F16.gguf","size":8879114528,"quant":"F16","sha256":"243258ae0ad80f748b18cfd9c8141998157d74eb32fbb131ba8f69e0c01564d7"},{"name":"Voxtral-Mini-4B-Realtime-2602-Q4_K_M.gguf","size":2830493984,"quant":"Q4_K_M","sha256":"39dc1f65539373a406edea7490505822d77c12edff521744678717eef4da4723"},{"name":"Voxtral-Mini-4B-Realtime-2602-Q5_K_M.gguf","size":3281439008,"quant":"Q5_K_M","sha256":"e20a7582c5cf8159454c909b7f57b184b287cb3d7c3ec85744727a2cec51b08f"},{"name":"Voxtral-Mini-4B-Realtime-2602-Q6_K.gguf","size":3661018912,"quant":"Q6_K","sha256":"92038784cffa20ea8ee925f10fd28c110285124d94730eacce48bab6c214fa48"},{"name":"Voxtral-Mini-4B-Realtime-2602-Q8_0.gguf","size":4731791648,"quant":"Q8_0","sha256":"6fb22249463c1e7a600d920ad732e2dd0a231510e9db59fb8061d3d1d48d0c67"}],"totalBytes":32252159168,"suggestedFile":"Voxtral-Mini-4B-Realtime-2602-Q4_K_M.gguf","requirements":{"ramGb":4,"diskBytes":2830493984,"note":"estimate: suggested file size × 1.15 + 0.5 GB; a real measurement comes with lsh models"},"runWith":["whisper.cpp"],"command":"lsh models install hf:handy-computer/Voxtral-Mini-4B-Realtime-2602-gguf"}}