PyPI page
Home page
Author:
License:
Apache-2.0
Summary:
SMASHED is a toolkit designed to apply transformations to samples in datasets, such as fields extraction, tokenization, prompting, batching, and more. Supports datasets from Huggingface, torchdata iterables, or simple lists of dictionaries.
Latest version:
0.21.5
Required dependencies:
ftfy
|
glom
|
jinja2
|
necessary
|
numpy
|
platformdirs
|
trouting
Optional dependencies:
autopep8
|
black
|
blingfire
|
boto3
|
datasets
|
dill
|
flake8
|
flake8-pyi
|
flake8-pyproject
|
ipdb
|
ipython
|
isort
|
moto
|
mypy
|
promptsource
|
pytest
|
smart-open
|
smashed
|
torch
|
torchdata
|
transformers
Downloads last day:
53
Downloads last week:
701
Downloads last month:
2,240