PyPI page
Author:
Bob Huang
Summary:
Flexible and modular LLM inference for mini-batch
Latest version:
0.1.1
Required dependencies:
aiohttp
|
chz
|
docker
|
fastapi
|
html2text
|
lxml
|
modelscope
|
openai-harmony
|
pydantic
|
requests
|
safetensors
|
structlog
|
tenacity
|
termcolor
|
tiktoken
|
uvicorn
Optional dependencies:
pytest
|
triton-runner
Downloads last day:
0
Downloads last week:
7
Downloads last month:
11
7-day smoothing