PyPI page
Home page
Author:
huangyuyang
Summary:
High-performance C++ inference engine for dense and MoE language models
Latest version:
0.1.8.2
Required dependencies:
aria2
|
blobfile
|
fastapi
|
imageio
|
imageio-ffmpeg
|
jinja2
|
modelscope
|
nvidia-cublas-cu12
|
nvidia-cuda-runtime-cu12
|
nvidia-nccl-cu12
|
openai
|
openpyxl
|
pandas
|
partial-json-parser
|
pydantic
|
pypdf
|
pyreadline3
|
python-pptx
|
shortuuid
|
tiktoken
|
transformers
|
triton
|
uvicorn
|
xlsxwriter
Optional dependencies:
aria2
|
blobfile
|
fastapi
|
imageio
|
imageio-ffmpeg
|
modelscope
|
openai
|
openpyxl
|
pandas
|
partial-json-parser
|
pydantic
|
pypdf
|
python-pptx
|
shortuuid
|
tiktoken
|
uvicorn
|
xlsxwriter
Downloads last day:
22
Downloads last week:
111
Downloads last month:
1,948
7-day smoothing