PyPI page
Home page
Author:
lvyufeng
License:
MIT
Summary:
Inference engine for large language models on consumer GPUs with deep optimization for older hardware
Latest version:
0.1.1
Required dependencies:
cmake
|
ninja
|
pybind11
|
safetensors
|
setuptools
|
torch
|
tqdm
|
transformers
Optional dependencies:
pocketllm
|
triton
Downloads last day:
0
Downloads last week:
11
Downloads last month:
163
7-day smoothing