PyPI page
Home page
Author:
None
Summary:
Adaptive inference budget controller for self-hosted LLMs. Controls thinking tokens, tracks GPU cost per query.
Latest version:
0.1.0
Required dependencies:
fastapi
|
httpx
|
numpy
|
pydantic
|
rich
|
tiktoken
|
uvicorn
Optional dependencies:
matplotlib
|
openai
|
pandas
|
pynvml
Downloads last day:
2
Downloads last week:
6
Downloads last month:
52