PyPI page
Home page
Author:
iD8 (AIMLPM)
Summary:
Fast Python web crawler for AI & RAG ingestion — crawl, extract, and embed website content with one tool.
Latest version:
0.11.0
Required dependencies:
beautifulsoup4
|
certifi
|
lxml
|
markdownify
|
requests
|
sentence-transformers
|
sentencepiece
|
tenacity
|
torch
|
transformers
Optional dependencies:
anthropic
|
google-genai
|
httpx
|
langchain-core
|
mcp
|
openai
|
playwright
|
sentence-transformers
|
sentencepiece
|
supabase
|
torch
|
trafilatura
|
transformers
Downloads last day:
192
Downloads last week:
997
Downloads last month:
2,684