PyPI page
Home page
Author:
None
Summary:
A flexible data transformation tool for ML training formats (SFT, RLHF, Pretrain)
Latest version:
0.7.2
Required dependencies:
fastexcel
|
numpy
|
orjson
|
polars
|
pyarrow
|
pyyaml
|
rich
|
textual
|
tiktoken
|
typer
|
xlsxwriter
Optional dependencies:
black
|
datasets
|
datasketch
|
flake8
|
flaxkv2
|
huggingface-hub
|
isort
|
mypy
|
myst-parser
|
pandas
|
pytest
|
pytest-asyncio
|
pytest-cov
|
scikit-learn
|
sphinx
|
sphinx-rtd-theme
|
tokenizers
|
toolong
Downloads last day:
11
Downloads last week:
993
Downloads last month:
1,901