# Senior Developer — MVP Happy-Path TODO **Owner:** Senior developer **Status:** Ready to implement **Goal:** Make the FK-scraped data actually flow into the overview, and own the contract/config/integration decisions the junior can't make alone. **Date:** 2026-06-12 > **Context:** The happy path is mostly frontend wiring (delegated to the junior — see `JUNIOR_MVP_TODO.md`). FK scraping is now legally cleared and the scraper itself is implemented (`backend/app/scrapers/fk_scraper.py`, Playwright, real captured selectors). Your job is the few things that need backend seniority: **persisting scraped data, the read contract, enabling the scraper safely, and the end-to-end review.** > > **Preserve** the existing scraper implementation, captured selectors (`_SEL`), and the `FetchDataService` contract. Don't rewrite them. --- ## S1 — Persist the scraped snapshot *(the critical gap)* **Problem:** The scraper is **write-only and never writes**. `FKScraperService.get_latest` / `list_history` raise `NotImplementedError` (`fk_scraper.py:117-125`), and a finished session only sets `state.result` in memory and returns it in the `poll` response (`fk_scraper.py:162-163`). Nothing persists a `ParentalLeaveFetch(source='scraped')` to the DB. So once the session ends, the dashboard's read (`get_latest` via `ManualFetchService`) returns nothing — the overview can never show scraped data after the fact. **Do:** - When a session reaches `done` (after `_do_extract` builds the `ParentalLeaveFetch`, `fk_scraper.py:273-290`), **persist it via `ParentalLeaveRepository`** so subsequent reads return it. `ManualFetchService` already shows the create/validate shape — reuse the same repo + invariants (`days_left >= 0`, `reserved_days_left <= reserved_days_total`). - **The hard part is DB access from the background asyncio task.** The session runs in a detached `asyncio.create_task(_run_session(...))` (`fk_scraper.py:145`), outside the request scope, so it has no `ConnectionDep`. The app uses a single process-wide `aiosqlite` connection on `app.state.db_connection` (`dependencies.py:get_connection`). Decide and implement a clean approach, e.g.: - pass a repository (or a connection/factory) into `start_session()` from the router (which *does* have `ParentalLeaveRepositoryDep`), and hand it down to the session state; **or** - inject a small persistence callback. - Mind aiosqlite concurrency: a single shared connection serializes; confirm a long-running background task writing mid-session doesn't deadlock or race the request handlers. If it's risky, open a short-lived dedicated connection for the write. - Validate before persisting; on invariant violation set status `error` with a message (don't persist garbage) — consistent with `data_refresh_plan.md §2`. - Keep the "never persist cookies/session/QR" rule intact — this persists only the **extracted numbers**. **Acceptance:** - After a successful scrape for (personal_data_uuid, child_id), a read returns the freshly scraped `ParentalLeaveFetch` with `source='scraped'`. - A new integration test covers: run a session to `done` (scraper internals mocked) → row is persisted → read returns it. --- ## S2 — Reconcile the parental-leave read contract for the overview **Problem:** The frontend wants a workspace-centric read, but the backend route is `GET /api/parental-leave/{personal_data_uuid}/{child_id}` (`routers/fetch_data.py`). The junior's `getParentalLeave('/workspaces/{id}/parental-leave')` is wrong on both prefix and shape. **Do:** - Pick one and implement it, then tell the junior the exact path/params for `api.ts`: - **Option A (recommended):** keep the existing per-(personal_data_uuid, child_id) route; the frontend already has both IDs from `/auth/context` + selected child. Just correct the client path. Least backend change. - **Option B:** add a convenience route `GET /api/parental-leave/workspace/{workspace_id}/{child_id}` that resolves the caller's `personal_data_uuid` from `CurrentUserDep`. Nicer for the client, small addition. - Whichever you choose, make sure it returns the **latest** snapshot regardless of `source` ('scraped' or 'manual') so S1's data shows up. **Acceptance:** - The overview can fetch the latest parental-leave for the selected child via a route that actually exists, and it returns scraped data after S1. - The chosen path is written down for the junior (J3 depends on it). --- ## S3 — Enable the scraper for the real app (legal gate cleared) **Problem:** `settings.scraper_enabled` defaults to `False` and the endpoints 501 when off (`routers/fetch_data.py`). `npm run login` flips it for the dev harness only — so the *real* app overview can't start a fetch. Legal sign-off is now done (confirmed by product owner). **Do:** - Make `scraper_enabled` **env-controlled and on for the running app** (e.g. default stays conservative but the deployed/dev env sets `SCRAPER_ENABLED=true`), so J5's overview button works outside the harness. Keep it a single switch. - Update the now-stale "DO NOT SHIP / pending legal review" comments in `fk_scraper.py` and `fetch_data.py` to reflect that legal has cleared, **but keep the operational guardrails**: sandboxed browser, no credential/session/QR persistence, no personal data in logs, per-session user consent (`ARCHITECTURE.md §2.3`). - Confirm `playwright install chromium` is documented/run wherever the backend runs (browser binaries aren't in the lockfile). **Acceptance:** - Starting a fetch from the real app (not just `/dev/fetch`) reaches the QR step. - Guardrail comments are accurate; no secrets/PII logged. --- ## S4 — End-to-end smoke test + review the junior's seams **Do (once J1–J5 land):** - Walk the full happy path in a browser: register → land on dashboard → add child → "Hämta från Försäkringskassan" → scan → overview shows scraped numbers → logout → login → still there. - Review the junior's **workspace bootstrap** (J2: `/auth/context` → `WorkspaceContext`, selected-child state) and the **`QRCodeModal`/`useScraperFetch`** extraction (J5) for correctness and for not leaking the in-memory token anywhere it shouldn't go. - Spot-check CORS (`main.py`) allows the real frontend origin, and that JWT access-token refresh from the httpOnly cookie works across a page reload (`AuthContext` refresh-on-mount). **Acceptance:** - The demo in `JUNIOR_MVP_TODO.md`'s "Definition of done" passes start to finish. - Backend test suite stays green (was 159 passing / ~52% per `SENIOR_REPORT_3.md`); add the S1 persistence test. --- ## Out of scope for this MVP round - Multi-child switching, manual-entry form submission, calendar interactivity, tips polish — all post-MVP. - Postgres repo impls — deferred until SQLite hits a wall (pattern already documented). - Refresh-token revocation table — stateless refresh is fine for MVP. ## Suggested split S1 and S2 are the unblockers — do them first so the junior's J3/J5 can complete. S3 can go in parallel. S4 is the closing review.