PyPI page
Home page
Author:
The LLMLingua team
License:
MIT License
Summary:
To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.
Latest version:
0.2.2
Required dependencies:
accelerate
|
nltk
|
numpy
|
tiktoken
|
torch
|
transformers
Optional dependencies:
accelerate
|
black
|
flake8
|
isort
|
nltk
|
numpy
|
pre-commit
|
pytest
|
pytest-xdist
|
tiktoken
|
torch
|
transformers
Downloads last day:
869
Downloads last week:
4,501
Downloads last month:
22,291