PyPI page
Home page
Author:
FlashML Team
Summary:
Local MoE-offload LLM inference runtime with OpenAI- and Anthropic-compatible APIs
Latest version:
0.1.3
Required dependencies:
apache-tvm-ffi
|
einops
|
fastapi
|
flashlib
|
gguf
|
huggingface-hub
|
modelscope
|
msgpack
|
numpy
|
openai
|
partial-json-parser
|
pillow
|
prompt-toolkit
|
pydantic
|
pyzmq
|
safetensors
|
torch
|
torchvision
|
tqdm
|
transformers
|
triton
|
uvicorn
Optional dependencies:
flashinfer-python
|
freetoken
|
pytest
|
sglang-kernel
Downloads last day:
72
Downloads last week:
552
Downloads last month:
6,114
7-day smoothing