PyPI page
Home page
Author:
None
Summary:
An ultra-fast, lightweight BPE tokenizer and trainer with a pure-C core.
Latest version:
1.0.0
Required dependencies:
regex
Optional dependencies:
huggingface_hub
|
mypy
|
pytest
|
pytest-cov
|
ruff
|
tiktoken
|
tinybpe
|
types-regex
Downloads last day:
4
Downloads last week:
29
Downloads last month:
1,408