The goal
Walk a logged-in ICCSafe digital code (e.g. IMC2024V3.0), export every section as a PDF, and merge to a single bookable PDF — with resumable progress and clean formatting.
The solution
A single-CLI Playwright scraper that walks the sidebar tree, injects print-only CSS to strip nav/drawer/headers, calls page.pdf() per section, and merges with pypdf. Session reuse via --save-storage-state / --storage-state auth.json. Resumable via state.json.
Highlights
- Premium bypass without writing a login handler (storage_state)
- Print-only CSS yields clean PDFs without changing the live page
- Stable sidebar keys enable reliable resume across runs
The challenge
Without Premium, generated PDFs say 'Printing is a feature of Digital Codes Premium'.
Playwright storage_state — one headful run to capture auth.json, all subsequent runs pass --storage-state auth.json to inherit cookies/localStorage/permissions.
page.pdf() captures the entire page including nav and drawer, producing ugly PDFs.
Inject @media print CSS to hide #chapter-wrapper, .v-navigation-drawer, .v-app-bar, nav, [role='navigation'] before each page.pdf().
Full codes span hundreds of sections — a single network hiccup shouldn't restart.
state.json records processed keys; restart skips them. Stable keys (li_id → aria-label → text) map the same section reliably.
The Vuetify SPA renders content async — calling page.pdf() too early captures a spinner.
Wait for content selectors + stabilize before rendering; retry loop with longer timeout if first render fails.
What was delivered
- iccsafe_scrape_to_pdf.py single-CLI scraper
- render_local_html_to_pdf.py helper
- requirements.txt + Playwright Chromium install
- README with two-stage storage-state flow
- Sample outputs under out_pdf4/, out_test2/
Results
codes.iccsafe.org
Target
Per-section + merged PDF
Output
state.json resumable
Recovery
What it taught me
- storage_state is the cleanest way to automate premium-gated document sites without writing a real login
- Print-only CSS gives much cleaner PDFs than scraping sanitized HTML and reflowing
- Stable keys (li_id → aria-label → text) are the right granularity for resume state