PyPI page
Author:
turboderp
Summary:
Inference library for running local LLMs on modern consumer GPUs
Latest version:
1.6.2
Required dependencies:
llguidance
|
marisa-trie
|
ninja
|
numpy
|
pillow
|
pydantic
|
pyyaml
|
rich
|
safetensors
|
tokenizers
|
torch
|
triton-windows
|
typing-extensions
Optional dependencies:
blessed
|
datasets
|
gguf
|
human-eval
|
llama-cpp-python
|
matplotlib
|
numpy
|
pandas
|
prompt-toolkit
|
pydantic
|
pyperclip
|
requests
|
seaborn
|
tabulate
|
torch
|
transformers
Downloads last day:
73
Downloads last week:
704
Downloads last month:
2,655
7-day smoothing