PyPI page
Home page
Author:
WorldFlow AI
License:
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTI...
Summary:
Semantic KV cache reuse for LLM inference engines (vLLM, SGLang, TRT-LLM)
Latest version:
0.3.13
Required dependencies:
numpy
|
rapidfuzz
Optional dependencies:
aiohttp
|
datasets
|
onnxruntime
|
onnxruntime-gpu
|
onnxscript
|
pytest
|
pytest-asyncio
|
rich
|
rouge-score
|
ruff
|
sentence-transformers
|
sglang
|
tensorrt_llm
|
torch
|
tqdm
|
transformers
|
triton
|
vllm
Downloads last day:
1
Downloads last week:
14
Downloads last month:
252