PyPI page
Home page
Author:
None
Summary:
Async web crawler powered by Playwright headless Chromium — JavaScript rendering, AI-powered extraction (OpenAI / Anthropic / Google), CSS rule engine with visual preview, and a full PySide6 desktop GUI. SQLite WAL crash recovery, proxy circuit breaker, RFC 9309 robots.txt, ReDoS-immune regex, 3,822 tests. pip install astrocrawl[full,gui,fast].
Latest version:
0.1.4
Required dependencies:
aiohttp
|
aiosqlite
|
beautifulsoup4
|
cssselect
|
google-re2
|
lxml
|
playwright
|
publicsuffixlist
|
pydantic
|
tiktoken
Optional dependencies:
anthropic
|
google-genai
|
mypy
|
openai
|
orjson
|
psutil
|
pyside6
|
pytest
|
pytest-asyncio
|
pytest-cov
|
pyyaml
|
ruff
Downloads last day:
122
Downloads last week:
594
Downloads last month:
594