Key Specifications

Vendor	anthropic
Version	3.5-haiku
Release Date	2024-11-04
Context Window	200000 tokens
Input Modalities	text, image
Output Modalities	text
License	Proprietary
Documentation	https://docs.anthropic.com/claude/docs

Benchmark Performance

Benchmark	Score	Unit	Evaluated At	Notes	Source
MMLU	77.6	%	2024-11-04	5-shot	view
HUMANEVAL	73.8	pass@1	2024-11-04	—	view
GSM8K	75.5	%	2024-11-04	0-shot CoT	view
MATH	53	%	2024-11-04	0-shot CoT	view
BBH	70.2	%	2024-11-04	3-shot CoT	view
GPQA	31.1	%	2024-11-04	0-shot	view
IFEVAL	73.6	%	2024-11-04	prompt_strict	view
ARC	91.8	%	2024-11-04	challenge	view
MUSR	52.3	%	2024-11-04	0-shot	view
WINOGRANDE	83.8	%	2024-11-04	0-shot	view

Name: Claude 3.5 Haiku
Brand: anthropic

Anthropic Claude 3.5 Haiku 经济型模型, 200K 上下文, 速度比 Sonnet 快 2 倍, 性能超越 Claude 3 Opus。

Vendor	Version	Release Date	Context Window	Input Modalities	Output Modalities	License
Anthropic	3.5-haiku	2024-11-04	200K	text, image	text	Proprietary

Benchmark	Score	Unit	Notes
MMLU (Massive Multitask Language Understanding)	77.6	%	5-shot
HumanEval	73.8	pass@1	—
GSM8K (Grade School Math 8K)	75.5	%	0-shot CoT
MATH	53.0	%	0-shot CoT
BBH (BIG-Bench Hard)	70.2	%	3-shot CoT
GPQA	31.1	%	0-shot
IFEval	73.6	%	prompt_strict
ARC	91.8	%	challenge
MUSR	52.3	%	0-shot
WinoGrande	83.8	%	0-shot

Input	Output	Cache Read	Cache Write
—	—	—	—

per million tokens