PyPI page
Home page
Author:
putkoff
Summary:
hugpy's router: one OpenAI-compatible address in front of every box's hugpy-wrapper front door. Picks the box per call (a model already serving with the same settings first, else a box that can load it), proxies the call (streaming, uploads, binary), bounces a refused call to the next box, and logs one row per decision.
Latest version:
0.0.4
Required dependencies:
aiohttp
|
hugpy-wrapper
Optional dependencies:
pytest
Downloads last day:
93
Downloads last week:
260
Downloads last month:
0
7-day smoothing