PyPI page
Home page
Author:
DamitDev
Summary:
Process manager for model inference backends (llama.cpp, HuggingFace)
Latest version:
0.1.10
Required dependencies:
docker
|
fastapi
|
harbor-oci-client
|
huggingface-hub
|
pebble
|
psutil
|
pydantic
|
pydantic-settings
|
python-multipart
|
python-socketio
|
uvicorn
Optional dependencies:
accelerate
|
nvidia-ml-py
|
pytest
|
torch
|
transformers
Downloads last day:
2
Downloads last week:
111
Downloads last month:
166