PyPI page
Home page
Author:
License:
Apache-2.0
Summary:
SMASHED is a toolkit designed to apply transformations to samples in datasets, such as fields extraction, tokenization, prompting, batching, and more. Supports datasets from Huggingface, torchdata iterables, or simple lists of dictionaries.
Latest version:
0.21.5
Required dependencies:
ftfy
|
glom
|
jinja2
|
necessary
|
numpy
|
platformdirs
|
trouting
Optional dependencies:
autopep8
|
black
|
blingfire
|
boto3
|
datasets
|
dill
|
flake8
|
flake8-pyi
|
flake8-pyproject
|
ipdb
|
ipython
|
isort
|
moto
|
mypy
|
promptsource
|
pytest
|
smart-open
|
smashed
|
torch
|
torchdata
|
transformers
Downloads last day:
160
Downloads last week:
402
Downloads last month:
2,110