kvitt/PLAN-M4.md

PLAN-M4: Server sync, audit, push, scheduler

1. Goal

Implement the sync protocol (pull and push), the append-only audit log with its field diff, sync_mutations idempotency and pruning, the FCM HTTP v1 sender, the recurring-expense scheduler and rotate-epoch. Every rule of §14.4 and every case of §9.5 has an integration test.

2. Read first

1. SPEC §14.4 Sync protocol v1: 14.4.1 Principles, 14.4.2 Pull, 14.4.3 Push, 14.4.4 Server processing of one mutation, 14.4.6 Rejection reasons, 14.4.7 Audit diff, 14.4.8 Epoch recovery and re-pairing, 14.4.9 Limits and timing (14.4.5 for the phone's expectations).
2. SPEC §14.6 Push payload v1.
3. SPEC §9.5 Recurring generation, §9.1 Splits (server check), §9.4 settlement validity, §9.6 Limits and validation, §9.10 Other edge cases.
4. SPEC §4 FR-9 Recurring expenses, FR-12 Audit log and timeline, FR-13 Push notifications, FR-15 Offline and sync.
5. SPEC §14.1 SQLite schema v1 and server-side rules; §14.3.3 Wire objects; §14.3.4 E-31, E-32.
6. SPEC §10.4 (rotate-epoch, KVITT_FCM_CREDENTIALS, KVITT_FCM_PROJECT_ID), §10.9 restore step 5, §6.3 (FCM auth row, Scheduler row), §8.7 Privacy and logging.
7. SPEC §11.2 (row M4), §11.3, §14.7, §14.8.
8. Files: contracts/examples/sync/*.json, contracts/examples/push/*.json, contracts/test-vectors/*.json, the M2a code under server/, CLAUDE.md.

3. Already established

• Pull: 409 cursor_ahead when since > current sequence; lowest server_version > since across the seven tables via UNION ALL, tombstones included, parent with children counts as one row; limit default 500, max 1000; next_since = highest version in page or since; has_more; every array present and sorted by server_version; epoch, server_time (§14.4.2, §9.6).
• Push: 1 to 100 mutations (batch_too_large), body ≤ 1 MiB; whole request in one BEGIN IMMEDIATE, one SAVEPOINT per mutation; steps 1 to 12 of §14.4.4 in order (replay, household, shape, clock skew > now + 10 min, load, tombstone, delete always wins, validate, insert, stale, update with diff or no-op, record in sync_mutations); pushes enqueued after commit; results in request order (§14.4.3, §14.4.4).
• Server recomputes shares with kvitt-domain and rejects mismatches validation_failed; mode default bp only checked to sum to 10000 (§9.1).
• Audit diff fields, order and value types per §14.4.7; create lists every field with old null, update only differing fields, delete only deleted_at; client_at = pushed updated_at (delete: deleted_at); scheduler and importer use server now for both; entries never updated or deleted (§14.4.7, AC-12.1).
• sync_mutations rows older than 90 days are pruned daily (§14.1, §14.4.9).
• Push: data-only FCM v1 message per target device, android.priority high, ttl 86400s, all values strings, types exactly the eight of §14.6, recipients per §14.6 (other user's non-revoked devices with a token; recurring to both users); more than 3 notifiable changes in one POST /sync → one sync_batch; importer sends none; no personal content (§14.6, FR-13).
• Sender: background task, bounded queue of 100 dropping the oldest with a warning; 3 attempts at 1 s, 5 s, 25 s on 429, 5xx, network errors; UNREGISTERED or 404 clears the device's fcm_token; disabled with a start-up warning when the credentials file is missing; OAuth2 via service-account JWT (RS256) at https://oauth2.googleapis.com/token, scope https://www.googleapis.com/auth/firebase.messaging, cached until 5 minutes before expiry (§14.6, §6.3).
• Scheduler: tokio task inside kvitt-server, runs at start and daily 06:00 Europe/Stockholm (DST-aware); candidate periods, clamping, one transaction per candidate, mode default resolves the account's current ratio at generation, audit create with source recurring, last_generated_period bump, push recurring_expense_created; unique index makes it idempotent; pause/resume and edit rules (§9.5, AC-9.2 to AC-9.6).
• rotate-epoch sets meta.sync_epoch to a new lower(hex(randomblob(16))), exit 0 (§10.4, §14.1).
• Environment variables only those of §10.4; the FCM endpoints are code constants, overridable only through code (constructor parameters) for the stub-server tests.

4. Owned paths

From SPEC §14.7, verbatim: kvitt-server: src/api/sync.rs, src/sync/**, src/audit/**, src/push/**, src/scheduler/**, src/cli/rotate_epoch.rs, tests/sync_*.rs, tests/scheduler_*.rs, tests/push_*.rs (owner M4; after landing: "later server plans"). Also from §14.7: M4 "may add a dependency line to a crate's Cargo.toml (second to land resolves Cargo.lock on rebase)" and "may add one line per new module to mod.rs files they need".

Edit nothing outside these paths; a needed change elsewhere is a contract request to the orchestrator.

Open points the orchestrator must settle in the start prompt's notes: (a) how serve (M2a's main.rs/lib.rs) starts the scheduler and push sender when M4 may only add mod.rs lines; (b) the shared recurring-generation function that M3's E-28/E-29 also need (see PLAN-M3 open point a); (c) the time of the daily sync_mutations pruning (the spec says only "daily").

5. Deliverables

• src/api/sync.rs: E-31 and E-32 handlers.
• src/sync/**: pull query and paging, push processing of §14.4.4.
• src/audit/**: audit writer and diff of §14.4.7, used by sync, scheduler and (later) importer.
• src/push/**: queue, FCM v1 sender, OAuth token cache, recipient selection, sync_batch aggregation.
• src/scheduler/**: recurring generation of §9.5, daily 06:00 Europe/Stockholm tick, sync_mutations pruning.
• src/cli/rotate_epoch.rs: the subcommand.
• tests/sync_*.rs: every rule of §14.4.2 and §14.4.4 steps 1 to 12, each rejection reason of §14.4.6, the §9.10 conflict cases, replay idempotency, batch limits, cursor_ahead, epoch change visible in pull, a 14-day/200-change offline scenario with a scripted clock (server side of AC-15.6).
• tests/scheduler_*.rs: every case of §9.5 (catch-up, clamping 29/30/31, February 2027, DST days, pause/resume without backfill, edit affects later only, deleted generated expense not recreated, archived account continues, idempotency, next_occurrence_date).
• tests/push_*.rs: payloads for every type against contracts/examples/push/, recipients, sync_batch threshold, retries and delays, token clearing on UNREGISTERED/404, queue overflow, disabled without credentials, all against a local stub HTTP server.

6. Steps

1. Worktree. From the main checkout /Users/jhag/projects/kvitt: git worktree add .claude/worktrees/agent-m4 -b agent-m4 main, then in .claude/worktrees/agent-m4: git reset --hard main. No git-ignored files are needed (the FCM key is not used; tests use a stub). Run tools/install-git-hooks.sh.
2. Read §2 in order. Confirm M2a has landed; if not, stop and report.
3. Versions: crates added (reqwest with rustls, jsonwebtoken, chrono-tz, a stub HTTP server for tests if not axum itself) at latest stable from crates.io / docs.rs; confirm the FCM HTTP v1 request and error format and the OAuth2 service-account flow from Google's official Firebase documentation; record each.
4. Audit writer and diff first, with unit tests for each §14.4.7 rule.
5. Push processing (§14.4.4) step by step, a test per step and per rejection reason; then pull (§14.4.2) with paging tests.
6. Push sender with the stub server tests.
7. Scheduler with an injectable clock; tests for every §9.5 case.
8. rotate-epoch; pruning.
9. Wiring into serve per open point (a).
10. Run the gate (§7). Commit on agent-m4, git rebase main (resolve Cargo.lock if M3 landed first), rerun the gate. Never merge, push or remove the worktree.

7. Gate

From SPEC §11.2, verbatim: "Server gate; integration tests for every sync rule of §14.4 and every case of §9.5; FCM sender tested against a local stub HTTP server". Command, from server/: cargo fmt --check && cargo clippy -- -D warnings && cargo test. Green means: all three exit 0 in this session; test result: lines show 0 failed; the report maps each §14.4.4 step, §14.4.6 reason and §9.5 case to a test name with no gaps.

8. HUMAN steps

From SPEC §11.2, verbatim: "none".

9. Out of scope

• REST endpoints E-2 to E-30 and E-33, landing page, pictures, rate limiting, bootstrap, pair-device (M3).
• The importer (M9); real FCM sends to devices (M8, M10); infra/; Android.
• Changing the sync protocol or push payload (contract change).

10. Report

• Branch agent-m4, head commit hash after rebasing on main.
• Gate output: fmt and clippy exit status, every test result: line.
• The rule → test mapping (§14.4.2, §14.4.4 steps, §14.4.6 reasons, §9.5 cases, §14.6 sender rules).
• Versions of crates added, with sources; FCM documentation pages consulted.
• How open points (a) to (c) were resolved.
• Numbering used or TODO(spec) occurrences.
• Contract requests, open questions, spec gaps (with section numbers).

11. Numbering

Decision ids, spec version and schema version are assigned by the orchestrator in the start prompt; use TODO(spec) if none was given.