PyPI page
Home page
Author:
Romani Isa
Summary:
An ultra-fast, lightweight BPE tokenizer and trainer with a pure-C core.
Latest version:
1.0.0
Required dependencies:
regex
Optional dependencies:
huggingface-hub
|
mypy
|
pytest
|
pytest-cov
|
ruff
|
tiktoken
|
tinybpe
|
types-regex
Downloads last day:
0
Downloads last week:
4
Downloads last month:
254
7-day smoothing