PyPI page
Home page
Author:
None
Summary:
Efficient LLM inference on Slurm clusters using vLLM.
Latest version:
0.7.3
Required dependencies:
click
|
pydantic
|
pyyaml
|
requests
|
rich
Optional dependencies:
cupy-cuda12x
|
flashinfer-python
|
ray
|
sglang
|
torch
|
vllm
|
xgrammar
Downloads last day:
7
Downloads last week:
50
Downloads last month:
195