PyPI page
Home page
Author:
Yinsicheng Jiang, Yeqi Huang, Cheng Deng, Luo Mai
License:
Apache-2.0
Summary:
Fast Long-Context Inference via Context Reuse
Latest version:
0.4.1
Required dependencies:
aiohttp
|
datasets
|
elasticsearch
|
fastapi
|
httpx
|
numpy
|
openai
|
pytest
|
scipy
|
tqdm
|
transformers
|
ujson
|
uvicorn
Optional dependencies:
black
|
bumpver
|
cupy-cuda12x
|
ipython
|
isort
|
pip-tools
|
pytest
|
pytest-cov
|
sglang
Downloads last day:
26
Downloads last week:
89
Downloads last month:
332