PyPI page
Home page
Author:
ysl
License:
MIT
Summary:
A simple crawler framework that implements both single run and distributed run based on Celery, allowing you to write your crawler code as you please
Latest version:
0.2.4
Required dependencies:
aiohttp
|
argparse
|
beautifulsoup4
|
celery
|
chardet
|
elasticsearch
|
httpx
|
jmespath
|
js2py
|
loguru
|
lxml
|
parsel
|
pyexecjs
|
pymongo
|
pytz
|
redis
|
requests
|
tenacity
|
tldextract
|
tqdm
|
watchdog
Downloads last day:
7
Downloads last week:
18
Downloads last month:
34