PyPI page
Author:
Srihari Unnikrishnan
Summary:
Chunk-level KV cache reuse for faster HuggingFace inference
Latest version:
0.9.0
Required dependencies:
accelerate
|
sentencepiece
|
torch
|
transformers
Optional dependencies:
autoawq-kernels
|
fastapi
|
furo
|
httpx
|
huggingface-hub
|
myst-parser
|
ninja
|
pydantic
|
pytest
|
pytest-asyncio
|
ruff
|
safetensors
|
sphinx
|
sphinx-copybutton
|
uvicorn
Downloads last day:
0
Downloads last week:
5
Downloads last month:
44
7-day smoothing