PyPI page
Home page
Author:
yanjlee
Summary:
使用scrapy,redis, mongodb,graphite实现的一个分布式网络爬虫,底层存储mongodb集群,分布式使用redis实现,爬虫状态显示使用graphite实现。
Latest version:
1.0.1
Required dependencies:
apscheduler
|
base64
|
beautifulsoup4
|
bs4
|
certifi
|
clickhouse-driver
|
crypto
|
curl-cffi
|
drissionpage
|
execjs
|
faker
|
fastapi
|
fuzzywuzzy
|
hashlib
|
httpx
|
jinja2
|
langchain
|
langchain-community
|
loguru
|
pandas
|
pillow
|
playwright
|
pyexecjs
|
redis
|
requests
|
suiutils-py
|
uvicorn
Downloads last day:
0
Downloads last week:
8
Downloads last month:
37