PyPI page
Home page
Author:
Ehtisham Mubarik
License:
MIT
Summary:
Turn a web crawl into an ML-ready dataset: extract, filter, deduplicate, embed, shard.
Latest version:
0.1.2
Optional dependencies:
pyarrow
|
pytest
|
pytest-cov
|
scrapy
|
sentence-transformers
|
torch
|
websieve
Downloads last day:
0
Downloads last week:
2
Downloads last month:
45
7-day smoothing