PyPI page
Home page
Author:
PygmalionAI
License:
GNU AFFERO GENERAL PUBLIC LICENSE Version 3, 19 November 2007 Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/> Everyone is permitted to copy and distribute verbatim copies of this ...
Summary:
Serving LLMs at Scale
Latest version:
0.24.1
Required dependencies:
aiohttp
|
anthropic
|
apache-tvm-ffi
|
blake3
|
cachetools
|
cbor2
|
cloudpickle
|
compressed-tensors
|
depyf
|
einops
|
fastapi
|
fastsafetensors
|
filelock
|
flashinfer-python
|
huggingface-hub
|
humming-kernels
|
ijson
|
instanttensor
|
jsonschema
|
lark
|
llguidance
|
lm-format-enforcer
|
mcp
|
mistral-common
|
model-hosting-container-standards
|
msgspec
|
ninja
|
numba
|
numpy
|
nvidia-cudnn-frontend
|
nvidia-cutlass-dsl
|
nvtx
|
openai
|
openai-harmony
|
opencv-python-headless
|
opentelemetry-api
|
opentelemetry-exporter-otlp
|
opentelemetry-sdk
|
opentelemetry-semantic-conventions-ai
|
outlines-core
|
partial-json-parser
|
pillow
|
prometheus-client
|
prometheus-fastapi-instrumentator
|
protobuf
|
psutil
|
py-cpuinfo
|
pybase64
|
pydantic
|
pynvvideocodec
|
python-json-logger
|
pyyaml
|
pyzmq
|
quack-kernels
|
regex
|
requests
|
safetensors
|
sentencepiece
|
setproctitle
|
setuptools
|
six
|
starlette
|
tiktoken
|
tilelang
|
tokenizers
|
tokenspeed-mla
|
torch
|
torchaudio
|
torchcodec
|
torchvision
|
tqdm
|
transformers
|
typing-extensions
|
watchfiles
|
xgrammar
Optional dependencies:
aphrodite-gguf-plugin
|
av
|
b12x
|
datasets
|
fastsafetensors
|
helion
|
instanttensor
|
matplotlib
|
mistral-common
|
nvidia-deepstream-videodecode-cu13
|
opentelemetry-api
|
opentelemetry-exporter-otlp
|
opentelemetry-sdk
|
opentelemetry-semantic-conventions-ai
|
pandas
|
plotly
|
runai-model-streamer
|
scipy
|
seaborn
|
smg-grpc-servicer
|
soundfile
|
soxr
|
tensorizer
|
zentorch
Downloads last day:
92
Downloads last week:
255
Downloads last month:
901
7-day smoothing