PyPI page
Home page
Author:
None
License:
MIT
Summary:
A BPE (Byte Pair Encoding) tokenizer written in Rust with Python bindings
Latest version:
0.1.0
Optional dependencies:
pytest
|
pytest-cov
Downloads last day:
1,613
Downloads last week:
10,336
Downloads last month:
51,369