PyPI page
Home page
Author:
License:
Apache-2.0
Summary:
SMASHED is a toolkit designed to apply transformations to samples in datasets, such as fields extraction, tokenization, prompting, batching, and more. Supports datasets from Huggingface, torchdata iterables, or simple lists of dictionaries.
Latest version:
0.21.5
Required dependencies:
ftfy
|
glom
|
jinja2
|
necessary
|
numpy
|
platformdirs
|
trouting
Optional dependencies:
autopep8
|
black
|
blingfire
|
boto3
|
datasets
|
dill
|
flake8
|
flake8-pyi
|
flake8-pyproject
|
ipdb
|
ipython
|
isort
|
moto
|
mypy
|
promptsource
|
pytest
|
smart-open
|
smashed
|
torch
|
torchdata
|
transformers
Downloads last day:
138
Downloads last week:
486
Downloads last month:
2,673