PyPI page
Home page
Author:
None
Summary:
A high-performance inference framework for large language models, focusing on efficiency, flexibility, and availability.
Latest version:
0.1.2
Required dependencies:
accelerate
|
bitsandbytes
|
blobfile
|
cmake
|
cython
|
einops
|
faker
|
fastapi
|
fire
|
flashinfer-python
|
hydra-core
|
ninja
|
packaging
|
protobuf
|
sentencepiece
|
setuptools
|
tiktoken
|
tqdm
|
transformers
|
uvicorn
|
wheel
Downloads last day:
9
Downloads last week:
15
Downloads last month:
26