PyPI page
Home page
Author:
vLLM Team
License:
Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for ...
Summary:
A high-throughput and memory-efficient inference and serving engine for LLMs
Latest version:
0.8.4.post4
Required dependencies:
aiohttp
|
blake3
|
cachetools
|
cloudpickle
|
compressed-tensors
|
depyf
|
einops
|
fastapi
|
filelock
|
gguf
|
huggingface-hub
|
importlib_metadata
|
lark
|
llguidance
|
lm-format-enforcer
|
mistral_common
|
msgspec
|
ninja
|
numba
|
numpy
|
openai
|
opencv-python-headless
|
opentelemetry-api
|
opentelemetry-exporter-otlp
|
opentelemetry-sdk
|
opentelemetry-semantic-conventions-ai
|
outlines
|
partial-json-parser
|
pillow
|
prometheus-fastapi-instrumentator
|
prometheus_client
|
protobuf
|
psutil
|
py-cpuinfo
|
pydantic
|
python-json-logger
|
pyyaml
|
pyzmq
|
ray
|
requests
|
scipy
|
sentencepiece
|
setuptools
|
six
|
tiktoken
|
tokenizers
|
torch
|
torchaudio
|
torchvision
|
tqdm
|
transformers
|
typing_extensions
|
watchfiles
|
xformers
|
xgrammar
Optional dependencies:
boto3
|
fastsafetensors
|
librosa
|
runai-model-streamer
|
runai-model-streamer-s3
|
soundfile
|
tensorizer
Downloads last day:
3
Downloads last week:
160
Downloads last month:
197