PyPI page
Home page
Author:
None
Summary:
Flexible and modular LLM inference for mini-batch
Latest version:
0.1.1
Required dependencies:
aiohttp
|
chz
|
docker
|
fastapi
|
html2text
|
lxml
|
modelscope
|
openai-harmony
|
pydantic
|
requests
|
safetensors
|
structlog
|
tenacity
|
termcolor
|
tiktoken
|
uvicorn
Optional dependencies:
pytest
|
triton-runner
Downloads last day:
2
Downloads last week:
51
Downloads last month:
124