vllm

PyPI page
Home page
Author: vLLM Team
License: Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTI...
Summary: A high-throughput and memory-efficient inference and serving engine for LLMs
Latest version: 0.8.2
Required dependencies: aiohttp | blake3 | cachetools | cloudpickle | compressed-tensors | depyf | einops | fastapi | filelock | gguf | importlib_metadata | lark | llguidance | lm-format-enforcer | mistral_common | msgspec | ninja | numba | numpy | openai | outlines | partial-json-parser | pillow | prometheus-fastapi-instrumentator | prometheus_client | protobuf | psutil | py-cpuinfo | pydantic | python-json-logger | pyyaml | pyzmq | ray | requests | scipy | sentencepiece | setuptools | six | tiktoken | tokenizers | torch | torchaudio | torchvision | tqdm | transformers | typing_extensions | watchfiles | xformers | xgrammar
Optional dependencies: boto3 | decord | fastsafetensors | librosa | runai-model-streamer | runai-model-streamer-s3 | soundfile | tensorizer

Downloads last day: 48,275
Downloads last week: 771,277
Downloads last month: 2,521,736