PyPI page
Home page
Author:
None
Summary:
A readable LLM inference server implementing paged attention and continuous batching
Latest version:
0.5.0
Required dependencies:
aiohttp
|
appinfra
|
filelock
|
httpx
|
urllib3
|
werkzeug
Optional dependencies:
anthropic
|
appinfra
|
coverage
|
flashinfer-python
|
llm-infer
|
llm-saia
|
mypy
|
pynvml
|
pytest
|
pytest-asyncio
|
pytest-cov
|
pytest-xdist
|
ruff
|
safetensors
|
tiktoken
|
torch
|
transformers
|
types-pyyaml
|
vllm
Downloads last day:
6
Downloads last week:
27
Downloads last month:
263