PyPI page
Home page
Author:
None
Summary:
A CLI benchmarking framework for LLM inference across FP16/INT8/INT4/HQQ/Quanto/AWQ/GPTQ quantization
Latest version:
0.3.5
Required dependencies:
accelerate
|
bitsandbytes
|
datasets
|
hqq
|
optimum-quanto
|
psutil
|
rich
|
torch
|
transformers
|
typer
Optional dependencies:
gptqmodel
|
groq
|
optimum
|
vllm
Downloads last day:
2
Downloads last week:
930
Downloads last month:
1,987