PyPI page
Home page
Author:
None
License:
MIT
Summary:
Tenant-fair LLM inference orchestration on a single GPU. No Kubernetes.
Latest version:
0.1.2
Required dependencies:
aiohttp
|
httpx
|
numpy
|
pandas
|
prometheus-client
|
pyyaml
|
rich
Optional dependencies:
black
|
datasets
|
ipykernel
|
jupyter
|
matplotlib
|
mypy
|
numba
|
numpy
|
nvidia-ml-py
|
py-spy
|
pytest
|
pytest-aiohttp
|
pytest-asyncio
|
ruff
|
seaborn
|
sglang
|
transformers
|
vllm
|
yappi
Downloads last day:
6
Downloads last week:
30
Downloads last month:
321