PyPI page
Home page
Author:
Yasmin Moslem
Summary:
Cluster, Route, Escalate: a cascaded framework for cost-aware LLM serving
Latest version:
0.6.2
Required dependencies:
anyio
|
numpy
|
pyyaml
|
scikit-learn
|
sentence-transformers
Optional dependencies:
accelerate
|
datasets
|
fastapi
|
httpx
|
litellm
|
pyarrow
|
pytest
|
torch
|
transformers
|
uvicorn
|
vllm
Downloads last day:
34
Downloads last week:
223
Downloads last month:
578
7-day smoothing