PyPI page
Home page
Author:
USF Team
License:
Apache-2.0
Summary:
vLLM-USF: A high-throughput and memory-efficient inference engine for LLMs (USF Custom Build)
Latest version:
0.0.2
Required dependencies:
aiohttp
|
blake3
|
cachetools
|
cbor2
|
cloudpickle
|
compressed-tensors
|
depyf
|
diskcache
|
einops
|
fastapi
|
filelock
|
gguf
|
lark
|
lm-format-enforcer
|
mistral_common
|
msgspec
|
ninja
|
numpy
|
openai
|
openai-harmony
|
opencv-python-headless
|
partial-json-parser
|
pillow
|
prometheus-fastapi-instrumentator
|
prometheus_client
|
protobuf
|
psutil
|
py-cpuinfo
|
pybase64
|
pydantic
|
python-json-logger
|
pyyaml
|
pyzmq
|
regex
|
requests
|
scipy
|
sentencepiece
|
setproctitle
|
tiktoken
|
tokenizers
|
tqdm
|
transformers-usf
|
typing_extensions
|
watchfiles
Optional dependencies:
mypy
|
pytest
|
pytest-asyncio
|
pytest-cov
|
ruff
Downloads last day:
1
Downloads last week:
71
Downloads last month:
80