puretiktoken

PyPI page
Home page
Author: adam2go
License: MIT
Summary: tiktoken in pure Python: byte-for-byte compatible BPE tokenizer with no Rust extension
Latest version: 0.2.1


Downloads last day: 3
Downloads last week: 25
Downloads last month: 426