PyPI page
Home page
Author:
KVCache Auto-Tuner Contributors
Summary:
Automatic KV-cache optimization for HuggingFace Transformers - Find the optimal cache strategy, attention backend, and configuration for your model and hardware.
Latest version:
0.1.4
Required dependencies:
pydantic
|
rich
|
typer
Optional dependencies:
accelerate
|
mkdocs
|
mkdocs-material
|
mkdocstrings
|
mypy
|
pre-commit
|
psutil
|
pynvml
|
pytest
|
pytest-asyncio
|
pytest-cov
|
ruff
|
torch
|
transformers
Downloads last day:
14
Downloads last week:
14
Downloads last month:
26