PyPI page
Home page
Author:
Joao Marques
License:
Apache-2.0
Summary:
Near-lossless 3-bit/2-bit KV cache quantization via E8 lattice VQ. Training-free, calibration-free, 9 architecture families validated. Optional token eviction for higher compression.
Latest version:
0.6.2
Required dependencies:
numpy
|
torch
|
zstandard
Optional dependencies:
accelerate
|
kvpress
|
torch
|
transformers
Downloads last day:
19
Downloads last week:
383
Downloads last month:
956