PyPI page
Author:
LLM Works LLC
Summary:
Multi-backend LLM client library, a from-scratch PagedAttention inference engine, and an OpenAI-compatible serve wrapper.
Latest version:
0.7.1
Required dependencies:
aiohttp
|
appinfra
|
filelock
|
httpx
|
pydantic
|
urllib3
|
werkzeug
Optional dependencies:
anthropic
|
appinfra
|
coverage
|
flashinfer-python
|
google-auth
|
llm-infer
|
llm-saia
|
mypy
|
pynvml
|
pytest
|
pytest-asyncio
|
pytest-cov
|
pytest-xdist
|
ruff
|
safetensors
|
tiktoken
|
torch
|
transformers
|
types-pyyaml
|
vllm
Downloads last day:
17
Downloads last week:
64
Downloads last month:
660
7-day smoothing