PyPI page
Home page
Author:
None
Summary:
A minimal API server for local HuggingFace LLMs or VLLM LLMs
Latest version:
0.4.3
Required dependencies:
accelerate
|
flask
|
kernels
|
nvidia-ml-py
|
sentencepiece
|
torch
|
transformers
Optional dependencies:
vllm
Downloads last day:
2
Downloads last week:
134
Downloads last month:
732