PyPI page
Home page
Author:
putkoff
Summary:
The hugpy wrapper (import name: fitevict): a per-box evict-to-fit front door. An OpenAI /v1 server that resolves a called model to its weights (GGUF via llama.cpp, transformers chat/vision, and every non-chat engine: embeddings, whisper ASR, TTS, diffusers image + video, summarization, keywords), places it on the GPU by evicting what doesn't fit, serves it, and logs every call.
Latest version:
0.1.116
Optional dependencies:
accelerate
|
bitsandbytes
|
compressed-tensors
|
huggingface-hub
|
peft
|
protobuf
|
pytest
|
safetensors
|
sentencepiece
|
tiktoken
|
tokenizers
|
torch
|
transformers
Downloads last day:
652
Downloads last week:
2,067
Downloads last month:
1,048
7-day smoothing