Store › model › LLM

Meta-Llama-3.1-8B-Instruct-GGUF

Name: Meta-Llama-3.1-8B-Instruct-GGUF
Author: bartowski

by bartowski · source Hugging Face · updated 2024-12-01

llama3.14.6 GB~6 GB RAMsource aliveunlabeled

Llamacpp imatrix Quantizations of Meta-Llama-3.1-8B-Instruct

Add to LogiShell Open in the web IDE

The button opens LogiShell with this card; nothing installs from a link by itself. Inside the app the install goes through lsh models install hf:bartowski/Meta-Llama-3.1-8B-Instruct-GGUF and its progress lives in the Resource Center.

Source and license

Source: https://huggingface.co/bartowski/Meta-Llama-3.1-8B-Instruct-GGUF
License: llama3.1 (restricted license: terms at the source apply)
Requirements: about 6 GB of RAM, 4.6 GB on disk (estimate: suggested file size × 1.15 + 0.5 GB; a real measurement comes with lsh models). Runs with llama.cpp, ollama.
Tags: gguf facebook meta pytorch llama llama-3 text-generation en de fr it pt hi es th endpoints_compatible

Numbers

485,003 downloads on Hugging Face
408 likes
license llama3.1
4.6 GB for Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf
327,838 npm downloads a week for node-llama-cpp
latest node-llama-cpp@3.22.1

Numbers as of 2026-10-02 21:00 UTC, from the source API.

Summary

Summary not ready yet: the numbers are here, the text is not. It is written by the collector through the LogiShell model facade when a provider key is present.

Reviews

No reviews yet. Reviews are written inside LogiShell: open this card in the app.

Files

file	quant	size
`Meta-Llama-3.1-8B-Instruct-IQ2_M.gguf`	IQ2_M	2.7 GB
`Meta-Llama-3.1-8B-Instruct-IQ3_M.gguf`	IQ3_M	3.5 GB
`Meta-Llama-3.1-8B-Instruct-IQ3_XS.gguf`	IQ3_XS	3.3 GB
`Meta-Llama-3.1-8B-Instruct-IQ4_NL.gguf`	IQ4_NL	4.4 GB
`Meta-Llama-3.1-8B-Instruct-IQ4_XS.gguf`	IQ4_XS	4.1 GB
`Meta-Llama-3.1-8B-Instruct-Q2_K.gguf`	Q2_K	3.0 GB
`Meta-Llama-3.1-8B-Instruct-Q2_K_L.gguf`	Q2_K_L	3.4 GB
`Meta-Llama-3.1-8B-Instruct-Q3_K_L.gguf`	Q3_K_L	4.0 GB
`Meta-Llama-3.1-8B-Instruct-Q3_K_M.gguf`	Q3_K_M	3.7 GB
`Meta-Llama-3.1-8B-Instruct-Q3_K_S.gguf`	Q3_K_S	3.4 GB
`Meta-Llama-3.1-8B-Instruct-Q3_K_XL.gguf`	Q3_K_XL	4.5 GB
`Meta-Llama-3.1-8B-Instruct-Q4_0_4_4.gguf`	Q4_0	4.3 GB
`Meta-Llama-3.1-8B-Instruct-Q4_0_4_8.gguf`	Q4_0	4.3 GB
`Meta-Llama-3.1-8B-Instruct-Q4_0_8_8.gguf`	Q4_0	4.3 GB
`Meta-Llama-3.1-8B-Instruct-Q4_K_L.gguf`	Q4_K_L	4.9 GB
`Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf`	Q4_K_M	4.6 GB
`Meta-Llama-3.1-8B-Instruct-Q4_K_S.gguf`	Q4_K_S	4.4 GB
`Meta-Llama-3.1-8B-Instruct-Q5_K_L.gguf`	Q5_K_L	5.6 GB
`Meta-Llama-3.1-8B-Instruct-Q5_K_M.gguf`	Q5_K_M	5.3 GB
`Meta-Llama-3.1-8B-Instruct-Q5_K_S.gguf`	Q5_K_S	5.2 GB
`Meta-Llama-3.1-8B-Instruct-Q6_K.gguf`	Q6_K	6.1 GB
`Meta-Llama-3.1-8B-Instruct-Q6_K_L.gguf`	Q6_K_L	6.4 GB
`Meta-Llama-3.1-8B-Instruct-Q8_0.gguf`	Q8_0	8.0 GB
`Meta-Llama-3.1-8B-Instruct-f32.gguf`	F32	30 GB

From the source README

Llamacpp imatrix Quantizations of Meta-Llama-3.1-8B-Instruct

Using llama.cpp release b3472 for quantization.

Original model: https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct

All quants made using imatrix option with dataset from here

Run them in LM Studio

## Torrent files
https://aitorrent.zerroug.de/bartowski-meta-llama-3-1-8b-instruct-gguf-torrent/

Prompt format

system

Cutting Knowledge Date: December 2023
Today Date: 26 Jul 2024

{system_prompt}user

{prompt}assistant
```

What's new

Card id model:hf:bartowski/Meta-Llama-3.1-8B-Instruct-GGUF · collected 2026-10-02 21:00 UTC · JSON