PyPI page
Home page
Author:
OpenMMLab
Summary:
A toolset for compressing, deploying and serving LLM
Latest version:
0.10.2.dev0
Required dependencies:
accelerate
|
aiohttp
|
einops
|
fastapi
|
fire
|
mmengine-lite
|
numpy
|
nvidia-cublas-cu12
|
nvidia-cuda-runtime-cu12
|
nvidia-curand-cu12
|
nvidia-nccl-cu12
|
openai
|
openai_harmony
|
partial_json_parser
|
peft
|
pillow
|
prometheus_client
|
protobuf
|
pydantic
|
pynvml
|
pyzmq
|
ray
|
safetensors
|
sentencepiece
|
shortuuid
|
tiktoken
|
torch
|
torchvision
|
transformers
|
triton
|
uvicorn
|
xgrammar
Optional dependencies:
accelerate
|
aiohttp
|
cmake_build_extension
|
datasets
|
einops
|
fastapi
|
fire
|
mmengine-lite
|
numpy
|
openai
|
openai_harmony
|
partial_json_parser
|
peft
|
pillow
|
prometheus_client
|
protobuf
|
pybind11
|
pydantic
|
pynvml
|
pyzmq
|
ray
|
safetensors
|
sentencepiece
|
setuptools
|
shortuuid
|
tiktoken
|
torch
|
torchvision
|
transformers
|
transformers_stream_generator
|
uvicorn
|
xgrammar
Downloads last day:
3
Downloads last week:
26
Downloads last month:
185