84 lines
6.9 KiB
Markdown
84 lines
6.9 KiB
Markdown
# Senior Developer — MVP Happy-Path TODO
|
||
|
||
**Owner:** Senior developer
|
||
**Status:** Ready to implement
|
||
**Goal:** Make the FK-scraped data actually flow into the overview, and own the contract/config/integration decisions the junior can't make alone.
|
||
**Date:** 2026-06-12
|
||
|
||
> **Context:** The happy path is mostly frontend wiring (delegated to the junior — see `JUNIOR_MVP_TODO.md`). FK scraping is now legally cleared and the scraper itself is implemented (`backend/app/scrapers/fk_scraper.py`, Playwright, real captured selectors). Your job is the few things that need backend seniority: **persisting scraped data, the read contract, enabling the scraper safely, and the end-to-end review.**
|
||
>
|
||
> **Preserve** the existing scraper implementation, captured selectors (`_SEL`), and the `FetchDataService` contract. Don't rewrite them.
|
||
|
||
---
|
||
|
||
## S1 — Persist the scraped snapshot *(the critical gap)*
|
||
|
||
**Problem:** The scraper is **write-only and never writes**. `FKScraperService.get_latest` / `list_history` raise `NotImplementedError` (`fk_scraper.py:117-125`), and a finished session only sets `state.result` in memory and returns it in the `poll` response (`fk_scraper.py:162-163`). Nothing persists a `ParentalLeaveFetch(source='scraped')` to the DB. So once the session ends, the dashboard's read (`get_latest` via `ManualFetchService`) returns nothing — the overview can never show scraped data after the fact.
|
||
|
||
**Do:**
|
||
- When a session reaches `done` (after `_do_extract` builds the `ParentalLeaveFetch`, `fk_scraper.py:273-290`), **persist it via `ParentalLeaveRepository`** so subsequent reads return it. `ManualFetchService` already shows the create/validate shape — reuse the same repo + invariants (`days_left >= 0`, `reserved_days_left <= reserved_days_total`).
|
||
- **The hard part is DB access from the background asyncio task.** The session runs in a detached `asyncio.create_task(_run_session(...))` (`fk_scraper.py:145`), outside the request scope, so it has no `ConnectionDep`. The app uses a single process-wide `aiosqlite` connection on `app.state.db_connection` (`dependencies.py:get_connection`). Decide and implement a clean approach, e.g.:
|
||
- pass a repository (or a connection/factory) into `start_session()` from the router (which *does* have `ParentalLeaveRepositoryDep`), and hand it down to the session state; **or**
|
||
- inject a small persistence callback.
|
||
- Mind aiosqlite concurrency: a single shared connection serializes; confirm a long-running background task writing mid-session doesn't deadlock or race the request handlers. If it's risky, open a short-lived dedicated connection for the write.
|
||
- Validate before persisting; on invariant violation set status `error` with a message (don't persist garbage) — consistent with `data_refresh_plan.md §2`.
|
||
- Keep the "never persist cookies/session/QR" rule intact — this persists only the **extracted numbers**.
|
||
|
||
**Acceptance:**
|
||
- After a successful scrape for (personal_data_uuid, child_id), a read returns the freshly scraped `ParentalLeaveFetch` with `source='scraped'`.
|
||
- A new integration test covers: run a session to `done` (scraper internals mocked) → row is persisted → read returns it.
|
||
|
||
---
|
||
|
||
## S2 — Reconcile the parental-leave read contract for the overview
|
||
|
||
**Problem:** The frontend wants a workspace-centric read, but the backend route is `GET /api/parental-leave/{personal_data_uuid}/{child_id}` (`routers/fetch_data.py`). The junior's `getParentalLeave('/workspaces/{id}/parental-leave')` is wrong on both prefix and shape.
|
||
|
||
**Do:**
|
||
- Pick one and implement it, then tell the junior the exact path/params for `api.ts`:
|
||
- **Option A (recommended):** keep the existing per-(personal_data_uuid, child_id) route; the frontend already has both IDs from `/auth/context` + selected child. Just correct the client path. Least backend change.
|
||
- **Option B:** add a convenience route `GET /api/parental-leave/workspace/{workspace_id}/{child_id}` that resolves the caller's `personal_data_uuid` from `CurrentUserDep`. Nicer for the client, small addition.
|
||
- Whichever you choose, make sure it returns the **latest** snapshot regardless of `source` ('scraped' or 'manual') so S1's data shows up.
|
||
|
||
**Acceptance:**
|
||
- The overview can fetch the latest parental-leave for the selected child via a route that actually exists, and it returns scraped data after S1.
|
||
- The chosen path is written down for the junior (J3 depends on it).
|
||
|
||
---
|
||
|
||
## S3 — Enable the scraper for the real app (legal gate cleared)
|
||
|
||
**Problem:** `settings.scraper_enabled` defaults to `False` and the endpoints 501 when off (`routers/fetch_data.py`). `npm run login` flips it for the dev harness only — so the *real* app overview can't start a fetch. Legal sign-off is now done (confirmed by product owner).
|
||
|
||
**Do:**
|
||
- Make `scraper_enabled` **env-controlled and on for the running app** (e.g. default stays conservative but the deployed/dev env sets `SCRAPER_ENABLED=true`), so J5's overview button works outside the harness. Keep it a single switch.
|
||
- Update the now-stale "DO NOT SHIP / pending legal review" comments in `fk_scraper.py` and `fetch_data.py` to reflect that legal has cleared, **but keep the operational guardrails**: sandboxed browser, no credential/session/QR persistence, no personal data in logs, per-session user consent (`ARCHITECTURE.md §2.3`).
|
||
- Confirm `playwright install chromium` is documented/run wherever the backend runs (browser binaries aren't in the lockfile).
|
||
|
||
**Acceptance:**
|
||
- Starting a fetch from the real app (not just `/dev/fetch`) reaches the QR step.
|
||
- Guardrail comments are accurate; no secrets/PII logged.
|
||
|
||
---
|
||
|
||
## S4 — End-to-end smoke test + review the junior's seams
|
||
|
||
**Do (once J1–J5 land):**
|
||
- Walk the full happy path in a browser: register → land on dashboard → add child → "Hämta från Försäkringskassan" → scan → overview shows scraped numbers → logout → login → still there.
|
||
- Review the junior's **workspace bootstrap** (J2: `/auth/context` → `WorkspaceContext`, selected-child state) and the **`QRCodeModal`/`useScraperFetch`** extraction (J5) for correctness and for not leaking the in-memory token anywhere it shouldn't go.
|
||
- Spot-check CORS (`main.py`) allows the real frontend origin, and that JWT access-token refresh from the httpOnly cookie works across a page reload (`AuthContext` refresh-on-mount).
|
||
|
||
**Acceptance:**
|
||
- The demo in `JUNIOR_MVP_TODO.md`'s "Definition of done" passes start to finish.
|
||
- Backend test suite stays green (was 159 passing / ~52% per `SENIOR_REPORT_3.md`); add the S1 persistence test.
|
||
|
||
---
|
||
|
||
## Out of scope for this MVP round
|
||
- Multi-child switching, manual-entry form submission, calendar interactivity, tips polish — all post-MVP.
|
||
- Postgres repo impls — deferred until SQLite hits a wall (pattern already documented).
|
||
- Refresh-token revocation table — stateless refresh is fine for MVP.
|
||
|
||
## Suggested split
|
||
S1 and S2 are the unblockers — do them first so the junior's J3/J5 can complete. S3 can go in parallel. S4 is the closing review.
|