meta-llama/Llama-3.1-8B-Instruct¶
Model Information¶
meta-llama/Llama-3.1-8B-Instruct is part of Meta's LLaMA 3.1 collection — a family of multilingual large language models (LLMs) available in 8B, 70B, and 405B sizes. These instruction-tuned, text-only models are optimized for multilingual dialogue tasks and consistently outperform many open-source and commercial chat models on common industry benchmarks.
- Model Developer: Meta
- Model Release Date: July 23, 2024
- Supported Languages: English, German, French, Italian, Portuguese, Hindi, Spanish, Thai
Model Architecture¶
meta-llama/Llama-3.1-8B-Instruct is an auto-regressive language model based on an enhanced transformer architecture.
The instruction-tuned versions leverage:
- Supervised Fine-Tuning (SFT)
- Reinforcement Learning with Human Feedback (RLHF)
These techniques align the model with human preferences around helpfulness, relevance, and safety.
Benchmark Scores¶
| Category | Benchmark | Shots | Metric | LLaMA 3.1 8B Instruct |
|---|---|---|---|---|
| General | MMLU (CoT) | 0 | Acc. (avg) | 73.0 |
| MMLU Pro (CoT) | 5 | Acc. (avg) | 48.3 | |
| Steerability | IFEval | – | – | 80.4 |
| Reasoning | GPQA Diamond (CoT) | 0 | Accuracy | 31.8 |
| Code | HumanEval | 0 | Pass@1 | 72.6 |
| MBPP EvalPlus (base) | 0 | Pass@1 | 72.8 | |
| Math | MATH (CoT) | 0 | Sympy Score | 51.9 |
| Tool Use | BFCL v2 | 0 | AST Macro Avg. | 65.4 |
| Multilingual | MGSM | 0 | EM (exact match) | 68.9 |