PyPI page
Home page
Author:
Huawei
License:
Apache 2.0
Summary:
A high-throughput and memory-efficient inference and serving engine for LLMs
Latest version:
0.4.2.post3
Required dependencies:
absl-py
|
accelerate
|
attrs
|
cloudpickle
|
decorator
|
numpy
|
pandas
|
psutil
|
scipy
|
torch
|
torch-npu
|
tornado
|
transformers
Downloads last day:
0
Downloads last week:
113
Downloads last month:
130