runtoken

PyPI page
Home page
Author: Thibault Jaigu
License: MIT
Summary: A blazing-fast BPE tokenizer for LLMs. Drop-in tiktoken replacement, 20-80x faster.
Latest version: 0.1.2


Downloads last day: 0
Downloads last week: 4
Downloads last month: 42
7-day smoothing