PyPI page
Home page
Author:
None
License:
MIT
Summary:
An OpenAI-compatible API LLM engine with smart prompt caching, batch processing, structured output with guided decoding, and function calling for all models using MLX.
Latest version:
0.2.1
Required dependencies:
fastapi
|
mlx-lm
|
mlx-vlm
|
outlines
|
uvicorn
Downloads last day:
3
Downloads last week:
50
Downloads last month:
115