High Throughput Inference Recommendation: DeepSeek Coder V2
High Throughput Inference → DeepSeek Coder V2
Use Case Overview
DeepSeek Coder V2 by Deepseek is a recommended option for the High Throughput Inference use case.
Recommended Models
| Rank |
Vendor |
Context Window |
Score |
| #5 |
Deepseek |
128K |
34/100 |
| Benchmark |
Score |
| MMLU |
72.3 |
| HUMANEVAL |
88.9 |
| GSM8K |
62.9 |
| MATH |
38.2 |
| BBH |
72.3 |
| GPQA |
25.4 |
Strengths
- HumanEval 88.9, excellent code generation.
- Mixture-of-Experts architecture.
Requirements
- API key from deepseek
- Input length within 128K context window