PyPI page
Home page
Author:
sage6
Summary:
Expert-Aware Multi-Batch Pipeline for MoE + Speculative Decoding inference optimization (CPU-PCIe-GPU).
Latest version:
0.1.0
Required dependencies:
accelerate
|
huggingface-hub
|
numpy
|
psutil
|
safetensors
|
scipy
|
speculators
|
torch
|
tqdm
|
transformers
Optional dependencies:
datasets
|
guidellm
|
mypy
|
pytest
|
pytest-cov
|
ruff
|
vllm
Downloads last day:
2
Downloads last week:
8
Downloads last month:
12