PyPI page
Home page
Author:
Dmitri Evseev (Arbi City)
Summary:
Optimized CUDAgraph-enabled kernels and attention backend for vLLM, SGLang and more based on TurboQuant near-lossless KV cache compression.
Latest version:
0.68.1
Required dependencies:
apache-tvm-ffi
|
nvidia-cutlass-dsl
|
quack-kernels
|
torch
Optional dependencies:
absl-py
|
cryptography
|
datasets
|
flash-attn
|
flash-attn-4
|
immutabledict
|
import-linter
|
langdetect
|
lm-eval
|
nltk
|
pyright
|
pytest
|
pytest-timeout
|
pytest-xdist
|
ray
|
sacrebleu
|
scipy
|
transformers
|
triton
|
turbo-attn
|
vllm
Downloads last day:
349
Downloads last week:
2,122
Downloads last month:
7,008
7-day smoothing