PyPI page
Home page
Author:
None
License:
BUSL-1.1
Summary:
Local LLM inference server for Apple Silicon. Block-level paged KV cache for long-context workloads. 5.4× faster end-to-end on 4K-token prompts vs Ollama, less RAM, INT3 support for Qwen3. OpenAI-compatible API.
Latest version:
9.34.14
Required dependencies:
fastapi
|
huggingface-hub
|
mlx
|
mlx-lm
|
mlx-vlm
|
numpy
|
orjson
|
rich
|
safetensors
|
sse-starlette
|
transformers
|
uvicorn
|
zstandard
Optional dependencies:
accelerate
|
cryptography
|
datasets
|
hnswlib
|
httpx
|
llmlingua
|
lm-eval
|
maturin
|
mkdocs-exclude
|
mkdocs-material
|
mypy
|
nltk
|
pytest
|
pytest-asyncio
|
pytest-cov
|
pytest-timeout
|
rouge_score
|
ruff
|
sacrebleu
|
sqlite-vec
|
torch
|
xgrammar
Downloads last day:
18
Downloads last week:
723
Downloads last month:
2,346