PyPI page
Author:
OpenMMLab
Summary:
A toolset for compressing, deploying and serving LLM
Latest version:
0.10.2.dev0
Required dependencies:
accelerate
|
aiohttp
|
einops
|
fastapi
|
fire
|
mmengine-lite
|
numpy
|
nvidia-cublas-cu12
|
nvidia-cuda-runtime-cu12
|
nvidia-curand-cu12
|
nvidia-nccl-cu12
|
openai
|
openai-harmony
|
partial-json-parser
|
peft
|
pillow
|
prometheus-client
|
protobuf
|
pydantic
|
pynvml
|
pyzmq
|
ray
|
safetensors
|
sentencepiece
|
shortuuid
|
tiktoken
|
torch
|
torchvision
|
transformers
|
triton
|
uvicorn
|
xgrammar
Optional dependencies:
accelerate
|
aiohttp
|
cmake-build-extension
|
datasets
|
einops
|
fastapi
|
fire
|
mmengine-lite
|
numpy
|
openai
|
openai-harmony
|
partial-json-parser
|
peft
|
pillow
|
prometheus-client
|
protobuf
|
pybind11
|
pydantic
|
pynvml
|
pyzmq
|
ray
|
safetensors
|
sentencepiece
|
setuptools
|
shortuuid
|
tiktoken
|
torch
|
torchvision
|
transformers
|
transformers-stream-generator
|
triton
|
uvicorn
|
xgrammar
Downloads last day:
1
Downloads last week:
35
Downloads last month:
135
7-day smoothing