PyPI page
Author:
Marshall Wang
Summary:
Efficient LLM inference on Slurm clusters using vLLM.
Latest version:
0.9.0
Required dependencies:
click
|
pydantic
|
pyyaml
|
requests
|
rich
Optional dependencies:
orjson
|
ray
|
sgl-kernel
|
sglang
|
torchao
|
torchcodec
|
vllm
Downloads last day:
2
Downloads last week:
11
Downloads last month:
129
7-day smoothing