PyPI page
Home page
Author:
Faraazuddin Mohammed
License:
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTI...
Summary:
Open benchmark of LLM tokenization across 5 providers — offline vs empirical calibration deltas.
Latest version:
0.0.3
Required dependencies:
anthropic
|
certifi
|
cohere
|
datasets
|
google-generativeai
|
httpx
|
mistral-common
|
pandas
|
pyarrow
|
pydantic
|
tenacity
|
tiktoken
|
tqdm
Optional dependencies:
build
|
jsonschema
|
jupyter
|
matplotlib
|
mypy
|
pytest
|
ruff
|
seaborn
|
twine
Downloads last day:
6
Downloads last week:
148
Downloads last month:
264