PyPI page
Home page
Author:
Dongfu Jiang
Summary:
A unified inference engine for large language models (LLMs) including open-source models (VLLM, SGLang, Together) and commercial models (OpenAI, Mistral, Claude).
Latest version:
0.0.25
Required dependencies:
accelerate
|
anthropic
|
fastapi
|
fire
|
fireworks-ai
|
google-generativeai
|
icecream
|
mistralai
|
numpy
|
openai
|
pillow
|
prettytable
|
requests
|
sentencepiece
|
together
|
torch
|
tqdm
|
transformers
|
vllm
Optional dependencies:
flash-attn
|
sglang
Downloads last day:
3
Downloads last week:
174
Downloads last month:
318