PyPI page
Home page
Author:
amirrouh
License:
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTI...
Summary:
Self-hosted, multi-modal AI server for your own GPU: chat/vision LLMs (Qwen, Llama, Gemma, DeepSeek), text-to-speech (Kokoro, OuteTTS), and image generation (SDXL, Flux, Z-Image, Qwen-Image) behind one OpenAI-compatible endpoint. Wraps llama.cpp and stable-diffusion.cpp, auto-downloads GGUF models from Hugging Face, hot-swaps VRAM, speculative decoding — no compiling. Type `inferhost` and you're done.
Latest version:
0.14.2
Required dependencies:
fastapi
|
httpx
|
huggingface-hub
|
kokoro-onnx
|
litellm
|
psutil
|
pydantic-settings
|
pyyaml
|
rich
|
textual
|
tomli-w
Optional dependencies:
mypy
|
pytest
|
pytest-asyncio
|
ruff
Downloads last day:
6
Downloads last week:
131
Downloads last month:
701
7-day smoothing