PyPI page
Home page
Author:
Shrey Patel
License:
MIT
Summary:
Tenant-fair LLM inference orchestration on a single GPU. No Kubernetes.
Latest version:
0.1.6
Required dependencies:
aiohttp
|
certifi
|
prometheus-client
|
pyyaml
|
rich
Optional dependencies:
datasets
|
httpx
|
ipykernel
|
jupyter
|
kvwarden
|
matplotlib
|
mypy
|
numpy
|
nvidia-ml-py
|
pandas
|
py-spy
|
pytest
|
pytest-aiohttp
|
pytest-asyncio
|
ruff
|
seaborn
|
sglang
|
transformers
|
vllm
|
yappi
Downloads last day:
1
Downloads last week:
9
Downloads last month:
21
7-day smoothing