Nemotron-3-Embed-1B-Q4_K_M-GGUF

Nemotron-3-Embed-1B Q4_K_M GGUF

An independently converted and quantized GGUF of NVIDIA's
`nvidia/Nemotron-3-Embed-1B-BF16`,
prepared for local embedding inference in LM Studio and llama.cpp-compatible runtimes.

This repository is not an official NVIDIA release and is not affiliated with or endorsed by NVIDIA.

File

| File | Quantization | Size | SHA-256 |
|---|---:|---:|---|
| `nemotron-3-embed-1b-q4_k_m.gguf` | Q4_K_M | 749,352,096 bytes (714.6 MiB) | `9a74166f51dbc280073748fa199bea49283bd21f7f9280f2dec2b4d975ddfd1d` |

The model produces 2,048-dimensional, L2-normalized embeddings. Its GGUF metadata declares a
262,144-token maximum context. The release was functionally tested at a 4,096-token context; very
large contexts were not validated and require substantially more memory.

Use with LM Studio

Download the GGUF from Files and versions, then drag it into LM Studio or place it in LM Studio's
models directory. LM Studio should classify it as an embedding model.

Load it with a 4,096-token context and full GPU offload, start the local server, and call the
OpenAI-compatible embeddings endpoint:

curl http://127.0.0.1:1234/v1/embeddings \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "nemotron-3-embed-1b-q4",
    "input": [
      "query: What is retrieval-augmented generation?",
      "passage: Retrieval-augmented generation adds retrieved documents to a model prompt."
    ]
  }'

The exact model identifier can differ if LM Studio assigns another load name; check
`http://127.0.0.1:1234/v1/models` when needed.

Retrieval format