PyPI page
Home page
Author:
None
Summary:
R-BPE: Improving BPE-Tokenizers with Token Reuse
Latest version:
0.1.2
Required dependencies:
click
|
datasets
|
torch
|
tqdm
|
transformers
Optional dependencies:
pre-commit
|
pytest
|
ruff
Downloads last day:
3
Downloads last week:
17
Downloads last month:
121