The challenge
10 different portals with different layouts, search forms and pagination styles.
The solution
A multi-portal framework: config.py holds keywords-to-category mappings, county lists, proxy/captcha/ZenRows/BatchData credentials. scrapers/ has per-portal modules (civil_core.py + public_notice.py + per-county trackers with proxy / without-proxy variants). run_test.py runs any subset. Output persists to SQLite and exports to Google Sheets via a service account.
Results
10+
Portals
Resumable
Resilience
4 (probate/foreclosure/sheriff/tax)
Categories
BatchData
Enrichment