{"v":1,"id":"model:hf:bartowski/Meta-Llama-3.1-8B-Instruct-GGUF","slug":"model-bartowski-meta-llama-3-1-8b-instruct-gguf","kind":"model","category":"llm","title":"Meta-Llama-3.1-8B-Instruct-GGUF","summary":"Llamacpp imatrix Quantizations of Meta-Llama-3.1-8B-Instruct","source":{"provider":"hf","ref":"bartowski/Meta-Llama-3.1-8B-Instruct-GGUF","url":"https://huggingface.co/bartowski/Meta-Llama-3.1-8B-Instruct-GGUF","rev":"bf5b95e96dac0462e2a09145ec66cae9a3f12067","fetchedAt":"2026-10-02T21:00:27.548Z","etag":"W/\"6740-ZIzdRyIAC6BDFuoqZGxPfzfreP0\""},"author":{"name":"bartowski","url":"https://huggingface.co/bartowski"},"license":{"spdx":"llama3.1","raw":"llama3.1","open":false,"note":"restricted license: terms at the source apply"},"metrics":{"downloads":485003,"downloadsWeek":327838,"likes":408,"takenAt":"2026-10-02T21:00:27.548Z"},"tags":["gguf","facebook","meta","pytorch","llama","llama-3","text-generation","en","de","fr","it","pt","hi","es","th","endpoints_compatible","conversational"],"pipeline":"text-generation","links":{"github":"meta-llama/llama-models","npm":"node-llama-cpp"},"updatedAt":"2024-12-01T04:12:27.000Z","collectedAt":"2026-10-02T21:00:27.548Z","review":{"numbers":["485,003 downloads on Hugging Face","408 likes","license llama3.1","4.6 GB for Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf","327,838 npm downloads a week for node-llama-cpp","latest node-llama-cpp@3.22.1"],"log":null},"trust":"unlabeled","health":{"status":"alive","checkedAt":"2026-10-02T21:00:27.548Z","http":200},"description":"## Llamacpp imatrix Quantizations of Meta-Llama-3.1-8B-Instruct\n\nUsing llama.cpp release b3472 for quantization.\n\nOriginal model: https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct\n\nAll quants made using imatrix option with dataset from [here](https://gist.github.com/bartowski1182/eb213dccb3571f863da82e99418f81e8)\n\nRun them in [LM Studio](https://lmstudio.ai/)\n\n## Torrent files\nhttps://aitorrent.zerroug.de/bartowski-meta-llama-3-1-8b-instruct-gguf-torrent/\n\n## Prompt format\n\n```\nsystem\n\nCutting Knowledge Date: December 2023\nToday Date: 26 Jul 2024\n\n{system_prompt}user\n\n{prompt}assistant\n```\n\n## What's new\n\n- 30-07-2024: Updated chat template to fix small bug with tool usage being undefined, if you don't use the built-in chat template it shouldn't change anything\n\n## Download a file (not the whole branch) from below:\n\n| Filename | Quant type | File Size | Split | Description |\n| -------- | ---------- | --------- | ----- | ----------- |\n| [Meta-Llama-3.1-8B-Instruct-f32.gguf](https://huggingface.co/bartowski/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-f32.gguf) | f32 | 32.13GB | false | Full F32 weights. |\n| [Meta-Llama-3.1-8B-Instruct-Q8_0.gguf](https://huggingface.co/bartowski/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q8_0.gguf) | Q8_0 | 8.54GB | false | Extremely high quality, generally unneeded but max available quant. |\n| [Meta-Llama-3.1-8B-Instruct-Q6_K_L.gguf](https://huggingface.co/bartowski/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q6_K_L.gguf) | Q6_K_L | 6.85GB | false | Uses Q8_0 for embed and output weights. Very high quality, near perfect, *recommended*. |\n| [Meta-Llama-3.1-8B-Instruct-Q6_K.gguf](https://huggingface.co/bartowski/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q6_K.gguf) | Q6_K | 6.60GB | false | Very high quality, near perfect, *recommended*. |\n| [Meta-Llam…\n\nSource: https://huggingface.co/bartowski/Meta-Llama-3.1-8B-Instruct-GGUF","install":{"kind":"model","hfId":"bartowski/Meta-Llama-3.1-8B-Instruct-GGUF","gated":false,"format":"gguf","files":[{"name":"Meta-Llama-3.1-8B-Instruct-IQ2_M.gguf","size":2948285856,"quant":"IQ2_M","sha256":"849856e1e7eff8ea7425a2a4cee50f3d547165194f50ea3112c9fc07cb08daad"},{"name":"Meta-Llama-3.1-8B-Instruct-IQ3_M.gguf","size":3784828320,"quant":"IQ3_M","sha256":"e15a3e54436cb7de4b201c084ae59755627908341faa15bfc1f2259b8ae63e96"},{"name":"Meta-Llama-3.1-8B-Instruct-IQ3_XS.gguf","size":3518752160,"quant":"IQ3_XS","sha256":"c57b55244c1bb8e754c7d2d07bfb37cf3c65838d56d602269c33096adb5707b1"},{"name":"Meta-Llama-3.1-8B-Instruct-IQ4_NL.gguf","size":4677993888,"quant":"IQ4_NL","sha256":"45040d3661c30312547cdea52549b25847cea77305a3fc911699ff676c006486"},{"name":"Meta-Llama-3.1-8B-Instruct-IQ4_XS.gguf","size":4447667616,"quant":"IQ4_XS","sha256":"ca364e65589a519b537a2b4eff6b4f5f9c2c0ea0b62d6070c73c366087ba00a3"},{"name":"Meta-Llama-3.1-8B-Instruct-Q2_K.gguf","size":3179136416,"quant":"Q2_K","sha256":"3f7f9265554ac91f1fb31c1b63b9cada4362533dfaa3f6e8dfc33c48d36909ce"},{"name":"Meta-Llama-3.1-8B-Instruct-Q2_K_L.gguf","size":3692160416,"quant":"Q2_K_L","sha256":"e793fcb27ff29334877bb9a51491f5fd9d32013394c181f1714a0702e5cc7822"},{"name":"Meta-Llama-3.1-8B-Instruct-Q3_K_L.gguf","size":4321961376,"quant":"Q3_K_L","sha256":"81ec07b5cee309925d9720dddc8376be14df5b82bbec2ff359f2adc017fa0efc"},{"name":"Meta-Llama-3.1-8B-Instruct-Q3_K_M.gguf","size":4018922912,"quant":"Q3_K_M","sha256":"6be122b5c8f2a33974953e58e0ffd2be505661acc6f4caf733e4bca130e89fea"},{"name":"Meta-Llama-3.1-8B-Instruct-Q3_K_S.gguf","size":3664504224,"quant":"Q3_K_S","sha256":"756c317e12ce5a57ab095df69b1ae8dd2f51d5f8660ba500999912d0638689c6"},{"name":"Meta-Llama-3.1-8B-Instruct-Q3_K_XL.gguf","size":4781630880,"quant":"Q3_K_XL","sha256":"5d4752a684325bbc9dfb18713fd78bafad30d74798d6e422775d9d292472f32e"},{"name":"Meta-Llama-3.1-8B-Instruct-Q4_0_4_4.gguf","size":4661216672,"quant":"Q4_0","sha256":"89f6957664441a4dd927e8f72fc4accb5008fd310f4af35f8f4e9d25de3e307d"},{"name":"Meta-Llama-3.1-8B-Instruct-Q4_0_4_8.gguf","size":4661216672,"quant":"Q4_0","sha256":"b9384fb0c903c9a4e2533133a5ba9b3cf7ca8894d2496a1746978443c6eefa47"},{"name":"Meta-Llama-3.1-8B-Instruct-Q4_0_8_8.gguf","size":4661216672,"quant":"Q4_0","sha256":"0efffa698e2c92958e4a4351519ab9b83609666a0fba819336158cdc9cf9cd5b"},{"name":"Meta-Llama-3.1-8B-Instruct-Q4_K_L.gguf","size":5310637472,"quant":"Q4_K_L","sha256":"51f7096352114e61ebf9673edb1ad5c040b2945d9d23e0463c84dfbd174cc285"},{"name":"Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf","size":4920739232,"quant":"Q4_K_M","sha256":"7b064f5842bf9532c91456deda288a1b672397a54fa729aa665952863033557c"},{"name":"Meta-Llama-3.1-8B-Instruct-Q4_K_S.gguf","size":4692673952,"quant":"Q4_K_S","sha256":"7b04439ae150b98bb52c219bcbbd6838d0e8d796d5295560d7d9c1f3fa836578"},{"name":"Meta-Llama-3.1-8B-Instruct-Q5_K_L.gguf","size":6057223584,"quant":"Q5_K_L","sha256":"a46abe91db41bba3b76ce3e3b4e7dffe5a414ae229d51c6be15b12cac70665ac"},{"name":"Meta-Llama-3.1-8B-Instruct-Q5_K_M.gguf","size":5732992416,"quant":"Q5_K_M","sha256":"14e10feba0c82a55da198dcd69d137206ad22d116a809926d27fa5f2398c69c7"},{"name":"Meta-Llama-3.1-8B-Instruct-Q5_K_S.gguf","size":5599298976,"quant":"Q5_K_S","sha256":"99deaedac85547c2560a415eedd3019fa5a17b43b809ae526788852061d05614"},{"name":"Meta-Llama-3.1-8B-Instruct-Q6_K.gguf","size":6596011424,"quant":"Q6_K","sha256":"33981adf6bae52c503fb5c24f72539010632f7ed290a56c1315a8cd50adca587"},{"name":"Meta-Llama-3.1-8B-Instruct-Q6_K_L.gguf","size":6850471328,"quant":"Q6_K_L","sha256":"9bf5598b3cc6c5804c520aa6349266d2e2c9a22402e157bd9b187dc34806dad6"},{"name":"Meta-Llama-3.1-8B-Instruct-Q8_0.gguf","size":8540775840,"quant":"Q8_0","sha256":"9da71c45c90a821809821244d4971e5e5dfad7eb091f0b8ff0546392393b6283"},{"name":"Meta-Llama-3.1-8B-Instruct-f32.gguf","size":32128885888,"quant":"F32","sha256":"d6455a43be1f09218bddaf143cb05d5b8b4c2fe73289004238ab81a29a554ed3"}],"totalBytes":143449204192,"suggestedFile":"Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf","requirements":{"ramGb":6,"diskBytes":4920739232,"note":"estimate: suggested file size × 1.15 + 0.5 GB; a real measurement comes with lsh models"},"runWith":["llama.cpp","ollama"],"command":"lsh models install hf:bartowski/Meta-Llama-3.1-8B-Instruct-GGUF"}}