PyPI page
Home page
Author:
Robert Tand
Summary:
vLLM plugin for variable-rate NVFP4-CB / FP8-CB product-codebook LLM quantization served by dedicated CUDA decode and prefill kernels.
Latest version:
0.9.1
Required dependencies:
huggingface-hub
|
safetensors
|
torch
Optional dependencies:
pytest
|
vllm
Downloads last day:
6
Downloads last week:
25
Downloads last month:
203
7-day smoothing