PyPI page
Home page
Author:
Jordan Anderson
Summary:
Fine-tune and serve large Mixture-of-Experts models on one GPU, with 4-bit experts in GPU memory, RAM or SSD.
Latest version:
0.54.0
Required dependencies:
bitsandbytes
|
torch
Optional dependencies:
accelerate
|
compressed-tensors
|
datasets
|
fastapi
|
grouped-nf4-gemm
|
httpx
|
huggingface-hub
|
jsonschema
|
pydantic
|
pytest
|
safetensors
|
transformers
|
uvicorn
Downloads last day:
51
Downloads last week:
1,039
Downloads last month:
4,011
7-day smoothing