PyPI page
Author:
LMCache Team
Summary:
A LLM serving engine extension to reduce TTFT and increase throughput, especially under long-context scenarios.
Latest version:
0.5.7.dev0
Required dependencies:
matplotlib
|
openai
|
prometheus-client
|
sortedcontainers
Downloads last day:
62
Downloads last week:
252
Downloads last month:
579
7-day smoothing