PyPI page
Home page
Author:
None
Summary:
A LLM serving engine extension to reduce TTFT and increase throughput, especially under long-context scenarios.
Latest version:
0.4.5.dev0
Required dependencies:
matplotlib
|
openai
Downloads last day:
7
Downloads last week:
27
Downloads last month:
116