PyPI page
Home page
Author:
NVIDIA Corporation
License:
Apache License 2.0
Summary:
TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs.
Latest version:
1.2.0
Required dependencies:
accelerate
|
aenum
|
aiohttp
|
apache-tvm-ffi
|
backoff
|
blake3
|
blobfile
|
build
|
click
|
click_option_group
|
colored
|
cuda-python
|
datasets
|
diffusers
|
einops
|
etcd-sdk-python
|
evaluate
|
fastapi
|
flashinfer-python
|
h5py
|
jsonschema
|
lark
|
llguidance
|
matplotlib
|
meson
|
mistral-common
|
mpi4py
|
mpmath
|
ninja
|
numexpr
|
numpy
|
nvidia-cuda-nvrtc
|
nvidia-cutlass-dsl
|
nvidia-ml-py
|
nvidia-modelopt
|
nvidia-nccl-cu13
|
nvtx
|
omegaconf
|
onnx
|
onnx_graphsurgeon
|
openai
|
openai-harmony
|
opencv-python-headless
|
optimum
|
ordered-set
|
pandas
|
partial_json_parser
|
patchelf
|
peft
|
pillow
|
plotly
|
polygraphy
|
prometheus_client
|
prometheus_fastapi_instrumentator
|
protobuf
|
psutil
|
pulp
|
pydantic
|
pydantic-settings
|
pyzmq
|
sentencepiece
|
setuptools
|
soundfile
|
starlette
|
strenum
|
tensorrt
|
tiktoken
|
torch
|
torch-c-dlpack-ext
|
torchao
|
torchvision
|
transformers
|
triton
|
urllib3
|
uvicorn
|
wheel
|
xgrammar
Optional dependencies:
aiperf
|
bandit
|
cloudpickle
|
docstring_parser
|
einops
|
fuzzywuzzy
|
genai-perf
|
graphviz
|
jieba
|
jsonlines
|
lm_eval
|
mako
|
mypy
|
nanobind
|
opentelemetry-api
|
opentelemetry-exporter-otlp
|
opentelemetry-sdk
|
opentelemetry-semantic-conventions-ai
|
oyaml
|
parameterized
|
pre-commit
|
pybind11
|
pybind11-stubgen
|
pytest
|
pytest-asyncio
|
pytest-cov
|
pytest-csv
|
pytest-env
|
pytest-forked
|
pytest-mock
|
pytest-rerunfailures
|
pytest-split
|
pytest-threadleak
|
pytest-timeout
|
pytest-xdist
|
rouge
|
rouge_score
|
ruff
|
typing-extensions
Downloads last day:
264
Downloads last week:
2,969
Downloads last month:
11,707