PyPI page
Home page
Author:
vLLM Team, Neural Magic
License:
Neural Magic Enterprise License
Summary:
A high-throughput and memory-efficient inference and serving engine for LLMs
Latest version:
0.6.3.0
Required dependencies:
aiohttp
|
einops
|
fastapi
|
filelock
|
gguf
|
importlib-metadata
|
lm-format-enforcer
|
mistral-common
|
msgspec
|
numpy
|
nvidia-ml-py
|
openai
|
outlines
|
partial-json-parser
|
pillow
|
prometheus-client
|
prometheus-fastapi-instrumentator
|
protobuf
|
psutil
|
py-cpuinfo
|
pydantic
|
pyyaml
|
pyzmq
|
ray
|
requests
|
sentencepiece
|
setuptools
|
six
|
tiktoken
|
tokenizers
|
torch
|
torchvision
|
tqdm
|
transformers
|
typing-extensions
|
uvicorn
|
xformers
Optional dependencies:
librosa
|
nm-magic-wand
|
soundfile
|
tensorizer
Downloads last day:
64
Downloads last week:
129
Downloads last month:
165