kvitt/PLAN-M4.md

PLAN-M4: Server sync, audit, push, scheduler

1. Goal

Implement the sync protocol (pull and push), the append-only audit log with its field diff, sync_mutations idempotency and pruning, the FCM HTTP v1 sender, the recurring-expense scheduler and rotate-epoch. Every rule of §14.4 and every case of §9.5 has an integration test.

2. Read first

1. SPEC §14.4 Sync protocol v1: 14.4.1 Principles, 14.4.2 Pull, 14.4.3 Push, 14.4.4 Server processing of one mutation, 14.4.6 Rejection reasons, 14.4.7 Audit diff, 14.4.8 Epoch recovery and re-pairing, 14.4.9 Limits and timing (14.4.5 for the phone's expectations).
2. SPEC §14.6 Push payload v1.
3. SPEC §9.5 Recurring generation, §9.1 Splits (server check), §9.4 settlement validity, §9.6 Limits and validation, §9.10 Other edge cases.
4. SPEC §4 FR-9 Recurring expenses, FR-12 Audit log and timeline, FR-13 Push notifications, FR-15 Offline and sync.
5. SPEC §14.1 SQLite schema v1 and server-side rules; §14.3.3 Wire objects; §14.3.4 E-31, E-32.
6. SPEC §10.4 (rotate-epoch, KVITT_FCM_CREDENTIALS, KVITT_FCM_PROJECT_ID), §10.9 restore step 5, §6.3 (FCM auth row, Scheduler row), §8.7 Privacy and logging.
7. SPEC §11.2 (row M4), §11.3, §14.7, §14.8; §6.3 "Server seams (D-015)"; docs/decisions.md D-004, D-015, D-021.
8. Files: contracts/examples/sync/*.json, contracts/examples/push/*.json, contracts/test-vectors/*.json, the M2a code under server/ (including the shells src/tasks/mod.rs, src/audit/, src/push/), M3's generation function in src/api/recurring.rs once M3 has landed, CLAUDE.md.

3. Already established

• Pull: 409 cursor_ahead when since > current sequence; lowest server_version > since across the seven tables via UNION ALL, tombstones included, parent with children counts as one row; limit default 500, max 1000; next_since = highest version in page or since; has_more; every array present and sorted by server_version; epoch, server_time (§14.4.2, §9.6).
• Push: 1 to 100 mutations (batch_too_large), body ≤ 1 MiB; whole request in one BEGIN IMMEDIATE, one SAVEPOINT per mutation; steps 1 to 12 of §14.4.4 in order (replay, household, shape, clock skew > now + 10 min, load, tombstone, delete always wins, validate, insert, stale, update with diff or no-op, record in sync_mutations); pushes enqueued after commit; results in request order (§14.4.3, §14.4.4).
• Server recomputes shares with kvitt-domain and rejects mismatches validation_failed; mode default bp only checked to sum to 10000 (§9.1).
• Audit diff fields, order and value types per §14.4.7; create lists every field with old null, update only differing fields, delete only deleted_at; client_at = pushed updated_at (delete: deleted_at); scheduler and importer use server now for both; entries never updated or deleted (§14.4.7, AC-12.1).
• sync_mutations rows older than 90 days are pruned daily at 04:00 Europe/Stockholm (DST-aware) (§14.1, §14.4.9, D-021).
• Seams (D-015): serve calls tasks::start in src/tasks/mod.rs (M2a's stub, now M4's); M4 starts the scheduler, the push sender and the pruning from it. M2a defined the traits AuditWriter (src/audit/) and PushQueue (src/push/) with no-op implementations; M4 replaces the implementations with the real writer and diff and the FCM queue, keeping the trait signatures (a change to them is a contract request). M3 and M9 call the traits.
• Recurring generation (D-015): the function that generates the due periods of one template is M3's, in src/api/recurring.rs; M4's scheduler calls it for every template with deleted_at IS NULL and paused = 0. M4 lands after M3: if M3 has not landed when the scheduler is written, write the loop against the signature in M3's report and rebase on M3 before the gate.
• Push: data-only FCM v1 message per target device, android.priority high, ttl 86400s, all values strings, types exactly the eight of §14.6, recipients per §14.6 (other user's non-revoked devices with a token; recurring to both users); more than 3 notifiable changes in one POST /sync → one sync_batch; importer sends none; no personal content (§14.6, FR-13).
• Sender: background task, bounded queue of 100 dropping the oldest with a warning; 3 attempts at 1 s, 5 s, 25 s on 429, 5xx, network errors; UNREGISTERED or 404 clears the device's fcm_token; disabled with a start-up warning when the credentials file is missing; OAuth2 via service-account JWT (RS256) at https://oauth2.googleapis.com/token, scope https://www.googleapis.com/auth/firebase.messaging, cached until 5 minutes before expiry (§14.6, §6.3).
• Scheduler: tokio task inside kvitt-server, runs at start and daily 06:00 Europe/Stockholm (DST-aware), and generates through M3's function (candidate periods, clamping, one transaction per candidate, split resolution, audit, last_generated_period, push are inside that function) (§9.5, AC-9.2 to AC-9.6).
• rotate-epoch sets meta.sync_epoch to a new lower(hex(randomblob(16))), exit 0 (§10.4, §14.1).
• Environment variables only those of §10.4; the FCM endpoints are code constants, overridable only through code (constructor parameters) for the stub-server tests.

4. Owned paths

From SPEC §14.7 (v0.3): kvitt-server: src/api/sync.rs, src/tasks/**, src/sync/**, src/audit/**, src/push/**, src/scheduler/**, src/cli/rotate_epoch.rs, tests/sync_*.rs, tests/scheduler_*.rs, tests/push_*.rs (owner M4, shells from M2a; after landing: "later server plans"). Also from §14.7: M4 may add dependency lines to a crate's Cargo.toml (second to land resolves Cargo.lock on rebase) and one line per new module to mod.rs files it needs.

Edit nothing outside these paths; a needed change elsewhere is a contract request to the orchestrator.

Never edit SPEC.md or docs/decisions.md: put proposed decision entries in the report.

Resolved open points (SPEC v0.3; the start prompt may repeat them):

• (a) serve calls tasks::start (src/tasks/mod.rs, created by M2a, owned by M4); everything M4 runs in the background starts there (D-015).
• (b) The generation function is M3's; M4's scheduler calls it; M4 replaces the AuditWriter and PushQueue implementations that M3 calls; M4 lands after M3 (D-015).
• (c) Pruning runs daily at 04:00 Europe/Stockholm (D-021).

5. Deliverables

• src/api/sync.rs: E-31 and E-32 handlers.
• src/sync/**: pull query and paging, push processing of §14.4.4.
• src/audit/**: the real AuditWriter implementation and the diff of §14.4.7, used by sync, M3's generation function and the importer.
• src/push/**: the real PushQueue implementation: queue, FCM v1 sender, OAuth token cache, recipient selection, sync_batch aggregation.
• src/tasks/**: start launching the scheduler, the push sender and the pruning.
• src/scheduler/**: the daily 06:00 Europe/Stockholm tick (and one run at start) calling M3's generation function for every active template; the daily 04:00 sync_mutations pruning.
• src/cli/rotate_epoch.rs: the subcommand.
• tests/sync_*.rs: every rule of §14.4.2 and §14.4.4 steps 1 to 12, each rejection reason of §14.4.6, the §9.10 conflict cases, replay idempotency, batch limits, cursor_ahead, epoch change visible in pull, a 14-day/200-change offline scenario with a scripted clock (server side of AC-15.6).
• tests/scheduler_*.rs: the scheduler cases of §9.5: run at start, next 06:00 tick computed across both DST changes, catch-up after downtime through M3's function, paused and deleted templates skipped; the 04:00 pruning (rows of 90 days and older removed, newer kept). The per-template generation cases are M3's tests.
• tests/push_*.rs: payloads for every type against contracts/examples/push/, recipients, sync_batch threshold, retries and delays, token clearing on UNREGISTERED/404, queue overflow, disabled without credentials, all against a local stub HTTP server.

6. Steps

1. Worktree. From the main checkout /Users/jhag/projects/kvitt: git worktree add .claude/worktrees/agent-m4 -b agent-m4 main, then in .claude/worktrees/agent-m4: git reset --hard main. No git-ignored files are needed (the FCM key is not used; tests use a stub). Run tools/install-git-hooks.sh.
2. Read §2 in order. Confirm M2a has landed; if not, stop and report.
3. Versions: crates added (reqwest with rustls, jsonwebtoken, chrono-tz, a stub HTTP server for tests if not axum itself) at latest stable from crates.io / docs.rs; confirm the FCM HTTP v1 request and error format and the OAuth2 service-account flow from Google's official Firebase documentation; record each.
4. Audit writer and diff first, with unit tests for each §14.4.7 rule.
5. Push processing (§14.4.4) step by step, a test per step and per rejection reason; then pull (§14.4.2) with paging tests.
6. Push sender with the stub server tests.
7. Scheduler with an injectable clock, calling M3's generation function; tests for the scheduler cases.
8. rotate-epoch; pruning at 04:00.
9. Wiring through tasks::start (resolved point a).
10. Run the gate (§7). Commit on agent-m4, git rebase main (resolve Cargo.lock if M3 landed first), rerun the gate. Never merge, push or remove the worktree.

7. Gate

From SPEC §11.2 (v0.3), verbatim: "Server gate; integration tests for every sync rule of §14.4 and the scheduler cases of §9.5 (run at start, daily 06:00 on DST days, catch-up through M3's generation function); FCM sender tested against a local stub HTTP server". Command, from server/: cargo fmt --check && cargo clippy -- -D warnings && cargo test. Green means: all three exit 0 in this session; test result: lines show 0 failed; the report maps each §14.4.4 step, §14.4.6 reason and §9.5 scheduler case to a test name with no gaps.

8. HUMAN steps

From SPEC §11.2, verbatim: "none".

9. Out of scope

• REST endpoints E-2 to E-30 and E-33, landing page, pictures, rate limiting, bootstrap, pair-device (M3).
• The importer (M9); real FCM sends to devices (M8, M10); infra/; Android.
• Changing the sync protocol or push payload (contract change).

10. Report

• Branch agent-m4, head commit hash after rebasing on main.
• Gate output: fmt and clippy exit status, every test result: line.
• The rule → test mapping (§14.4.2, §14.4.4 steps, §14.4.6 reasons, §9.5 cases, §14.6 sender rules).
• Versions of crates added, with sources; FCM documentation pages consulted.
• How resolved points (a) to (c) were applied.
• Proposed decision entries (never written to SPEC.md or docs/decisions.md by this agent) or TODO(spec) occurrences.
• Contract requests, open questions, spec gaps (with section numbers).

11. Numbering

Decision ids, spec version and schema version are assigned by the orchestrator in the start prompt; use TODO(spec) if none was given.