PyPI page
Author:
Byron Hsu
Summary:
High-performance Rust-based load balancer for VLLM with multiple routing algorithms and prefill-decode disaggregation support
Latest version:
0.1.16
Required dependencies:
aiohttp
|
fastapi
|
orjson
|
requests
|
setproctitle
|
uvicorn
Optional dependencies:
pytest
|
pytest-asyncio
|
pytest-cov
Downloads last day:
1,692
Downloads last week:
14,096
Downloads last month:
79,992
7-day smoothing