PyPI page
Home page
Author:
Sehastrajit
Summary:
A vLLM-style, production-grade LLM inference server built natively for Windows (single GPU, single machine)
Latest version:
0.1.1
Required dependencies:
accelerate
|
fastapi
|
gguf
|
httpx
|
huggingface-hub
|
ninja
|
numpy
|
outlines-core
|
prometheus-client
|
pydantic
|
safetensors
|
tokenizers
|
torch
|
transformers
|
uvicorn
Optional dependencies:
matplotlib
|
pytest
|
pytest-asyncio
|
pywin32
|
ruff
Downloads last day:
1
Downloads last week:
10
Downloads last month:
278
7-day smoothing