PyPI page
Home page
Author:
Inigo
Summary:
Benchmarking tool for tokenizer efficiency across languages and models
Latest version:
0.1.1
Required dependencies:
anthropic
|
datasets
|
google-generativeai
|
huggingface-hub
|
jinja2
|
pyyaml
|
rich
|
tiktoken
|
transformers
|
typer
Downloads last day:
2
Downloads last week:
30
Downloads last month:
191