From 86e08a12cf324882032aae81f520516424d0d03a Mon Sep 17 00:00:00 2001 From: Jacob Hagstedt Date: Sun, 18 Oct 2026 07:51:04 +0200 Subject: [PATCH] SPEC v0.7: M9 importer decisions (D-044 to D-046), category table, HANDOFF rewritten MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit D-044 adds nine Swedish Splitwise category names to the SPEC §10.8 table and splitwise_categories.csv. D-045 records the reconciliation against the totals rows; D-046 the importer CLI semantics and exit codes. M9 marked landed at ab298ab, the M11 rehearsal recorded, server/README.md and prompts/README.md updated. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01TQd3pEP9UeN2uifbHknQ7B --- SPEC.md | 53 +++++++++++-------- docs/HANDOFF.md | 30 +++++------ docs/decisions.md | 3 ++ prompts/README.md | 2 +- server/README.md | 7 ++- .../data/splitwise_categories.csv | 9 ++++ 6 files changed, 61 insertions(+), 43 deletions(-) diff --git a/SPEC.md b/SPEC.md index 2e1dde2..6aa98f5 100644 --- a/SPEC.md +++ b/SPEC.md @@ -1,6 +1,6 @@ # Kvitt: specification -Version **0.6** (2026-10-18). Schema version **1**. Sync protocol version **1**. Push payload version **1**. +Version **0.7** (2026-10-18). Schema version **1**. Sync protocol version **1**. Push payload version **1**. This file is the single source of truth for Kvitt. It is written for implementation agents that have no other context. If code, a plan or `CLAUDE.md` disagrees with this file, this file is right. Words in `code` are literal identifiers, paths, enum values or error codes and must be used exactly as written. Anything not decided here is listed in §12 as `TODO(spec)`; do not invent a value for it. @@ -283,7 +283,7 @@ The server starts with an empty database. The household is created from the comm * AC-17.2 Re-running the same import creates nothing new (stable import key, §10.8). * AC-17.3 Splitwise categories map to Kvitt categories by the fixed table in §10.8; unknown names map to `other` and are listed in the summary. * AC-17.4 Splitwise payments become settlements with one allocation. -* AC-17.5 `--dry-run` prints the summary (rows read, expenses, settlements, skipped duplicates, unknown categories, non-SEK rows) and writes nothing. +* AC-17.5 `--dry-run` prints the summary (rows read, expenses, settlements, skipped duplicates, unknown categories, non-SEK rows, the reconciliation of §10.8) and writes nothing; it runs the whole import transaction and rolls it back, with the same exit status a real run would have (D-046). * AC-17.6 Imported rows produce audit entries with `source` = `import` and no pushes. ### FR-18 Backups and restore @@ -974,35 +974,40 @@ Command (run on the Pi with the server stopped, or on the Mac against a copy of ``` kvitt-import --db /data/kvitt.sqlite --file --account "" \ - --user "=" \ - --user "=" \ + --user "=" \ + --user "=" \ [--fx =]... [--dry-run] ``` -* One CSV file per Splitwise group (Splitwise's group "Export as spreadsheet"). The account is created with default 50/50 if no non-deleted account with that name exists; otherwise rows go into the existing one. +* One CSV file per Splitwise group (Splitwise's group "Export as spreadsheet"). The account is created with default 50/50 if no non-deleted account with that name exists; otherwise rows go into the existing one, and the existing account's stored name (not the spelling given to `--account`) is the account name used in the import key (D-046). +* `--user` (exactly two, distinct Splitwise names, one per household user) maps a member column to a Kvitt user given as a user id or as a display name that exactly one user has; anything else is a usage error (D-046). +* The importer never creates or migrates a database: `--db` must name an existing file whose `meta.schema_version` is `1`, else it exits 1 (run `kvitt-server migrate` first) (D-046). +* The whole run is one `BEGIN IMMEDIATE` transaction: find or create the account, skip rows whose `import_key` exists, validate and insert the rest with the server's own push checks, write the audit entries, then reconcile (below). It commits only when every row is valid, the reconciliation is accepted and `--dry-run` is absent; otherwise it rolls back and nothing is written. A dry run runs the same transaction and rolls it back (D-046). +* Exit status (D-046): `0` imported (or, with `--dry-run`, would import); `1` database or file error; `2` usage (bad arguments, `--user` not resolving to the two household users); `3` the file cannot be imported (header, invalid rows, non-SEK rows without `--fx`); `4` reconciliation mismatch or missing totals row. Nothing is written unless the status is `0` and `--dry-run` is absent. * Header (confirmed against both real exports, D-003; the exact header lines are recorded in PLAN-M9, not here): Splitwise localises the export, so two variants exist and the importer must accept both. English: `Date`, `Description`, `Category`, `Cost`, `Currency`; Swedish: `Datum`, `Beskrivning`, `Kategori`, `Kostnad`, `Valuta`. Then one column per group member, named by the member's full Splitwise name, holding that member's net effect for the row (positive = paid more than their share). The member columns appear in different orders in different files, so the importer reads member names from the header and matches them to the `--user` arguments; it never relies on column positions. Any other header fails the import with a clear message, writing nothing. -* Fields containing commas are double-quoted (CSV quoting). The file ends with an empty line followed by a row whose `Description` is `Total balance` (English) or `Totalsumma` (Swedish); that row is ignored, as are blank rows. +* Fields containing commas are double-quoted (CSV quoting). The file ends with an empty line followed by a row whose `Description` is `Total balance` (English) or `Totalsumma` (Swedish), the totals row, holding each member's balance; it creates no row and is used only for the reconciliation. Blank rows are ignored. +* Reconciliation (D-045): after the rows are written, inside the transaction, each user's balance of the whole account (`net(u)`, §9.3, over every non-deleted row of the account, not only the imported ones) must equal the sum of that member's values in the file's totals rows. One totals row is allowed per currency; a non-SEK totals row is converted with the `--fx` rate like any other value. A mismatch, or a file without a totals row, rolls the run back and exits `4`; a dry run reports the same. Importing into an account that already holds other rows therefore rolls back, since those rows enter the comparison. When any `--fx` conversion took place, per-value rounding can make an exact match impossible, so a mismatch is reported in the summary but accepted. A re-run of an already imported file skips every row and reconciles again on the same balance. * `Date` is `YYYY-MM-DD`; `Cost` and member columns are decimal numbers with `.` as separator, converted to öre exactly by string parsing. -* For each expense row with members A and B and cost C (öre): the payer is the member with positive net. If `net_A > 0`: payer A, `share_B = net_A`, `share_A = C − net_A`, mode `exact`. Symmetric for B. If both nets are 0: payer is the owner, mode `full` on the owner. If `net_A + net_B ≠ 0` or a resulting share is negative, the row is reported as invalid and the import aborts (nothing written). -* Rows whose `Category` is `Payment` (English) or `Betalning` (Swedish) become settlements: `from` = the member with positive net, `to` = the other, amount = that net, one allocation to the account, `note` = the Splitwise description truncated to 200 code points. +* For each expense row with members A and B and cost C (öre): the payer is the member with positive net. If `net_A > 0`: payer A, `share_B = net_A`, `share_A = C − net_A`, mode `exact`. Symmetric for B. If both nets are 0: payer is the owner, mode `full` on the owner (the owner pays and bears the whole cost, D-046). If `net_A + net_B ≠ 0` or a resulting share is negative, the row is reported as invalid and the import aborts (nothing written). +* Rows whose `Category` is `Payment` (English) or `Betalning` (Swedish) become settlements: `from` = the member with positive net, `to` = the other, amount = that net, one allocation to the account, `note` = the Splitwise description truncated to 200 code points (D-046). * Rows whose `Currency` is not `SEK` abort the import unless a `--fx =` is given for that currency; then every amount is converted with `round_half_up(amount_ore × rate)` per value, and for expenses the exact shares are recomputed as `share_payer = C' − share_nonpayer'`. -* `import_key` = lowercase hex SHA-256 of `splitwise|v1||||||||` where `` is the 0-based occurrence index of an identical line earlier in the same file. Rows whose key exists are skipped. +* `import_key` = lowercase hex SHA-256 of `splitwise|v1||||||||` where `` is the 0-based occurrence index of an identical line earlier in the same file. Rows whose key exists are skipped. Idempotency uses the existing `import_key` columns of `expenses` and `settlements` (§14.1); no new column or table (D-046). * Every created row: `created_by_user_id` = `updated_by_user_id` = the owner; `updated_by_device_id` null; `created_at` = `updated_at` = the import time; one audit entry `create` with `source` = `import`; new `server_version`s from the global sequence; no pushes. -* Description is truncated to 100 code points; the full original goes into `notes` when truncated. -* Category mapping (case-insensitive exact match on the `Category` column, English and Swedish names, D-003). The importer commits this table as the data file `server/crates/kvitt-import/data/splitwise_categories.csv` (columns `splitwise_name,language,default_key`) and reads it from there; it is the only mapping. The Swedish names are those observed in the real Swedish export, each placed with its English Splitwise equivalent: +* Description is truncated to 100 code points; the full original goes into `notes` when truncated. An empty description, or one over 1000 code points (the `notes` limit, §9.6), makes the row invalid (exit `3`, nothing written) (D-046). +* Category mapping (case-insensitive exact match on the `Category` column, English and Swedish names, D-003). The importer commits this table as the data file `server/crates/kvitt-import/data/splitwise_categories.csv` (columns `splitwise_name,language,default_key`) and reads it from there; it is the only mapping. The Swedish names are those observed in the real Swedish export, each placed with its English Splitwise equivalent (D-044 added the nine found unmapped in the real run): | Splitwise category (English) | Splitwise category (Swedish) | Kvitt `default_key` | |---|---|---| | Groceries, Food and drink - Other, Liquor | Livsmedel, Mat och dryck - Övrigt, Alkohol | `groceries` | | Dining out | Restaurangbesök | `restaurants` | -| Electronics, Furniture, Household supplies, Maintenance, Mortgage, Rent, Services, Pets, Home - Other | Elektronik, Möbler, Förbrukningsvaror, Skötsel/underhåll, Avbetalning/Amortering, Hyra, Hem - Övrigt | `home` | -| Electricity, Heat/gas, Water, Trash, TV/Phone/Internet, Cleaning, Utilities - Other | TV/telefon/internet | `utilities` | -| Bicycle, Bus/train, Car, Gas/fuel, Parking, Taxi, Transportation - Other | Buss/tåg, Bil, Bensin/bränsle, Parkering, Taxi, Transport - Övrigt | `transport` | -| Hotel, Plane | Hotell | `travel` | -| Games, Movies, Music, Sports, Entertainment - Other | Musik, Filmer, Sport, Underhållning - Övrigt | `entertainment` | +| Electronics, Furniture, Household supplies, Maintenance, Mortgage, Rent, Services, Pets, Home - Other | Elektronik, Möbler, Förbrukningsvaror, Skötsel/underhåll, Avbetalning/Amortering, Hyra, Tjänster, Hem - Övrigt | `home` | +| Electricity, Heat/gas, Water, Trash, TV/Phone/Internet, Cleaning, Utilities - Other | TV/telefon/internet, Städning, Verktyg - Övrigt | `utilities` | +| Bicycle, Bus/train, Car, Gas/fuel, Parking, Taxi, Transportation - Other | Cykel, Buss/tåg, Bil, Bensin/bränsle, Parkering, Taxi, Transport - Övrigt | `transport` | +| Hotel, Plane | Hotell, Flyg | `travel` | +| Games, Movies, Music, Sports, Entertainment - Other | Spel, Musik, Filmer, Sport, Underhållning - Övrigt | `entertainment` | | Medical expenses | Sjukvård/medicin | `health` | -| Clothing, Gifts | Presenter | `shopping` | -| General, Childcare, Education, Insurance, Taxes, Life - Other, Other | Allmänt, Barnomsorg, Försäkringar, Skatter | `other` | +| Clothing, Gifts | Kläder, Presenter | `shopping` | +| General, Childcare, Education, Insurance, Taxes, Life - Other, Other | Allmänt, Barnomsorg, Utbildning, Försäkringar, Skatter, Livet - Övrigt | `other` | | any other value | any other value | `other`, counted as unmapped | * The summary lists each unmapped category name with its row count, and the total count of unmapped rows as a warning. @@ -1010,9 +1015,9 @@ kvitt-import --db /data/kvitt.sqlite --file --account " --account "" --user "…" --user "…"`, first with `--dry-run`, then without; `docker compose start server`; check `/health`. (`--entrypoint` is needed because the image's entrypoint is `kvitt-server`, §10.3.) +4. `docker compose stop server`; for each file run `docker compose run --rm --entrypoint kvitt-import server --db /data/kvitt.sqlite --file /import/ --account "" --user "…" --user "…"`, first with `--dry-run`, then without; each must exit `0` with the reconciliation `match` (D-045); the target accounts must not exist yet or be empty; `docker compose start server`; check `/health`. (`--entrypoint` is needed because the image's entrypoint is `kvitt-server`, §10.3.) 5. Delete the CSV files from `/opt/kvitt/import/`. ### 10.9 Backups and restore @@ -1093,9 +1098,9 @@ After M1, M2a and M2b land, these run in parallel without touching each other's | M7e | Feature: analytics | `:feature:analytics` screens S-11, S-12 | as M7a | none | M6 | | M7f | Feature: settings | `:feature:settings` screens S-13, S-14, S-15, S-19, S-20, S-21, S-22 | as M7a | none | M6 | | M8 | App integration | `:app` navigation and deep links, Hilt bindings from interfaces to `:data` and `:domain`, `:adapter:fcm`, notification building (AC-13.5), update banner wiring, revoke wipe, network security config (§8.5), manifest hardening (§8.8), release signing and version properties (§10.6), `google-services.json` copied with `infra/dev/copy-google-services.sh` | Android gate; on the owner's phone against the development server on the Mac (D-001): pair, add an expense offline, reconnect, see it synced | Tap through the pairing and the offline test as relayed by the orchestrator | M3, M4, M5, M7a-M7f, M2c-dev (Firebase, development server) | -| M9 | Splitwise importer | `kvitt-import` per §10.8 with tests over synthetic CSV fixtures | Server gate | Provide the CSV files (kept outside git) when OQ-2 is answered | M2a, M4 | +| M9 | Splitwise importer | Landed `ab298ab` (D-044 to D-046). `kvitt-import` per §10.8 with tests over synthetic CSV fixtures (`crates/kvitt-import/tests/import.rs`), reconciliation against the totals rows, the committed category table | Server gate | Provide the CSV files (kept outside git) when OQ-2 is answered | M2a, M4 | | M10 | First release | Release build signed, published, both phones paired, push working both ways, nightly backup on Drive, restore drill report, memory check | `docs/validation/first-release.md` and `docs/validation/restore-drill.md` complete; server RSS under 100 MB | Create the release keystore with the `keytool` command of §10.6, outside the repository; install the APK on both phones from the landing page; allow "Install unknown apps"; partner pairs with the invite; confirm notifications arrive | M2c-pi, M3, M4, M8 | -| M11 | Import history | Real import run on the Pi per §10.8 "Production run" (D-025), `docs/validation/splitwise-import.md` with per-account totals compared to Splitwise's balances (aggregates only) | Imported balances equal Splitwise's "Total balance" rows per group | Provide the CSVs; confirm the balances on the phone match Splitwise | M9, M10 | +| M11 | Import history | Rehearsed on 2026-10-18: the real run reconciled on both files in a throwaway database (Vardag 1746 rows, Hemma 37 rows; 21 rows unmapped before D-044, 0 expected after it). Real import run on the Pi per §10.8 "Production run" (D-025), `docs/validation/splitwise-import.md` with per-account totals compared to Splitwise's balances (aggregates only) | Imported balances equal Splitwise's "Total balance" rows per group | Provide the CSVs; confirm the balances on the phone match Splitwise | M9, M10 | ### 11.3 Notes per milestone @@ -1109,7 +1114,8 @@ After M1, M2a and M2b land, these run in parallel without touching each other's * **M8** is the only plan that edits `:app` after M2b. M5's contract requests (v0.5): `:app` provides `AppBuildInfo(versionCode, versionName, debug)` and includes `DataBindingsModule` in the release graph (D-041); WorkManager is configured with `KvittWorkerFactory` and the default initializer disabled; `:app` calls `SyncTriggers.onAppStart`, `onForeground` and `syncForPush`, and `UpdateRepository.check` on cold start and every resume; the `ImageLoader` that `:data` provides is set as Coil's singleton. It owns the `:app` items network security config (§8.5), manifest hardening (§8.8), release signing and version properties (§10.6), and copies `google-services.json` from `infra/secrets/` with `infra/dev/copy-google-services.sh` (§10.7). * **M10**: the orchestrator relays every HUMAN step in chat. The keystore is created with the command of §10.6 (D-024). * **M2c-pi**: the allowances to act on the Kvitt Pi, the Home Assistant Pi, Gitea and Google Drive are requested from the owner at the start of that sprint (D-023). -* **M11** runs the import per §10.8 "Production run" (D-025). +* **M9** (landed `ab298ab`) reuses the server's own push validation (`kvitt_server::api::validate`, `rows`) and the `SqlAuditWriter`, so imported rows pass the same checks as synced ones (§14.4.4). Its tests are one binary, `crates/kvitt-import/tests/import.rs`, over synthetic fixtures only; one test checks the §10.8 category table against `data/splitwise_categories.csv`. +* **M11** runs the import per §10.8 "Production run" (D-025). The rehearsal on 2026-10-18 ran both real files against a throwaway database and reconciled on both (Vardag 1746 rows, Hemma 37 rows); 21 rows were unmapped before D-044 and 0 are expected after it, to be confirmed by the next dry run. --- @@ -2440,3 +2446,4 @@ Rule: a change to any other contract file, or a semantic change to `contracts/op | 0.4 | 2026-10-17 | Decisions from the M2a (`fdf2093`) and M2b (`fad6fc9`) implementation reports. D-005: the eight normalisations of the OpenAPI equality test and the `CommonRules` modifier in `openapi.rs` (§11.3, §14.8). D-006: the wrong-bp-sum token is `bp_sum_not_10000` with an explicit serde rename in Rust; Kotlin emits the same token (§14.5.1, §14.5.3). D-007: the no-unwrap rule enforced per crate with `#![cfg_attr(not(test), deny(clippy::unwrap_used, clippy::expect_used))]` (CLAUDE.md). D-010: screenshot tests with Roborazzi 1.76.0 on Robolectric 4.17, `@Config(sdk = [36])`, `roborazzi.test.verify=true`, re-record with `recordRoborazziDebug` (§6.2, §11.2). D-011: `IncompleteCustomRangeException` for an incomplete `CUSTOM` range; `ALL_TIME` with the earliest expense after today gives today..today (§14.5.4). D-012: Room 3 (`androidx.room3` 3.0.3, KSP only, `SQLiteDriver` with `sqlite-bundled`, migrations on an `SQLiteConnection`); M2b created the entities and `KvittDatabase` v1, whose exported schema `1.json` is a contract file; M5 builds on them (§6.2, §11.2, §11.3, §14.2, §14.2.3, §14.7, §14.8). D-013: JDK 25 `--add-exports`/`--add-opens` for Robolectric in `build-logic` (§6.2). D-014: `failOnNoDiscoveredTests` off only for modules without `src/test` (§6.2). D-008 and D-009 unused. Clarifications without a decision id: exact Rust pin (1.99.1) shared by `rust-toolchain.toml` and the Dockerfile, `TARGETARCH` arm64 (shipped) and amd64, distroless pinned by digest (§6.3, §10.3); chrono stands (§6.3); `kvitt-domain` depends only on `serde` (§6.3, CLAUDE.md); `AuditWriter` and `PushQueue` as built (§6.3, §14.7); protocol check before token check (§6.3, §14.3.1); 405 with an empty body and the version header (§14.3.1); body limits as router layers (§8.6); `app_version_code` updated only from a present, parseable `X-Kvitt-App-Version` (§8.2); E-1 `schema_version` from a code constant; E-2 `initialized` means an admin user exists (§14.3.4); `healthcheck` uses the port of `KVITT_BIND`; `cargo run -p kvitt-server -- ` from the workspace root (§10.4); `minSdk` 36, `compileSdk` and `targetSdk` 37, Material 3 1.4.0 pinned beside the Compose BOM, AGP 9 `check` runs only the debug unit-test variant, version code and name from `kvitt.versionCode`/`kvitt.versionName` defaulting to `1`/`dev`, feature convention documented in `android/README.md` (§6.2); `FieldError.error` may be `validation_failed` for §9.6 rules without a domain token (§9.6, §14.2.4); `AnalyticsState` has no error field and a missing custom date resolves as today (§14.2.4, §14.5.4); `DrillDownState.title` stays a `String` of unlocalised parts and S-12 formats its own title (§14.2.4); a settlement row in an account's list shows that account's allocation (AC-11.7, §14.2.4); the `AnalyticsMeasure.Share` / `model.Share` name collision is accepted (§14.2.4, §14.5.4). | | 0.5 | 2026-10-17 | Decisions from the M3 (`503cee5`) and M5 (`c28c52b`) implementation reports. M3: D-030 picture version after a removal, `-removed` marker, next version = highest on disk + 1, first upload 1 (AC-4.4, §10.4, E-10); D-031 failed accept = any 4xx but 429, refused until 60 minutes after the first failure, fixed windows (AC-2.12); D-032 rate-limit keying by token hash, 401 counts against the IP, never-accepted tokens refused when the IP is over, `KVITT_TRUSTED_PROXY` re-resolved every 60 s, last `X-Forwarded-For` entry, `into_make_service_with_connect_info` added by M4 (§8.6, §10.4, §11.3); D-033 resume moves `last_generated_period` only to a period at or after `start_period` (§9.5); D-034 strings stored trimmed, transparency flattened onto white, `fcm_token` non-empty or null, revoke clears `fcm_token`, idempotent archive keeps `server_version`, new category `sort_order` over non-deleted (§8.6, §9.6, AC-3.5, AC-4.4, AC-7.3, E-17, E-20, E-21, E-25, E-26). M5 (renumbered: its D-030 to D-033 are D-040 to D-043): D-026 `cursor_ahead` is epoch recovery (§14.4.2); D-027 `clock_skew` gives a new `pending_mutation_id`, and the phone re-stamps `updated_at` on the next attempt (orchestrator decision, follow-up open) (§14.4.5); D-028 periodic work `kvitt-sync-periodic` (§14.4.1); D-029 `OnlineResult.Failed("network", null)` (§14.2.4); D-040 whole-request push refusals count as no progress, 429 retries, 426 stops (§14.4.5); D-041 `DataBindingsModule` (§6.2); D-042 Room tests on Robolectric, migrations from the exported JSON (§6.2); D-043 pairing status and 426 version code in DataStore (§8.2, §14.2.2). D-035 to D-039 reserved for M4. Clarifications without a decision id: control characters restricted only in `notes`, newlines accepted in names and descriptions (§8.6); landing page without download button while no APK is published, 404 page, `Referrer-Policy` and `Cache-Control` headers (AC-2.5, §13.3); generation sets the template's `updated_at` (§9.5); E-28 accepts archived accounts and categories (AC-6.4); E-14 on an accepted invite is 204 with no change; `/health` from the healthcheck counts against loopback (§8.6); M3's tests in one binary `tests/core_api/` (§14.7); sync triggers, batch and page sizes (AC-15.3, §14.4.1); 426 stops syncing until the installed version changes and `HomeState` has no update-required field (§14.2.4, §14.3.1); pictures always fetched at 512; `NeedsInstallPermission` once (AC-16.3); `Rejection.summary`, `FieldChange` display and `RecurringTemplate.draft.date` as documented in `:data` (§14.2.4); M8 contract requests (§11.3). | | 0.6 | 2026-10-18 | Decisions from the M4 (`bd6f10a`) implementation report. D-035: a mutation whose `mutation_id`, `entity_type` or `entity.id` is unusable fails the whole request with 400 `invalid_request`; every other shape error is a per-mutation `validation_failed` (§14.4.4 step 3). D-036: `POST /sync` with 0 mutations is 422 `batch_too_large`, the code now reads "outside 1 to 100" (§9.6, §14.3.2, §14.4.4). D-037: a delete of a never-synced row runs the full step-8 validation; every rejection carries the stored row or null as `current`, also at steps 3 and 4; step 11 keeps the stored `created_at`; pushed ids must be canonical lowercase hyphenated UUIDs (§14.4.3, §14.4.4). D-038: FCM retries are one send plus up to three retries at 1 s, 5 s, 25 s; 401 drops the cached access token and is not retried, other 4xx are not retried; retries block the single sender task up to 31 s per device (§14.6). D-039: pruning deletes `received_at` strictly older than 90 days; seam change to D-015: `tasks::start` returns the `AppState` with the real push queue and `serve` builds the router from it, `AppState::new` defaults to `SqlAuditWriter`, `NoopAuditWriter` removed, `NoopPushQueue` kept for push disabled (§6.3, §14.1, §14.4.9; PLAN-M4). Clarifications without a decision id: §14.4.4 step 8 order is field rules, domain rules, references; a negative `since` is `invalid_request` and a pull reads one page in a single read transaction (§9.6, §14.3.4, §14.4.2); push runs in one `BEGIN IMMEDIATE` with a savepoint per mutation; the push queue is bounded at 100 and drops the oldest with a warning; the FCM JWT is RS256 with `kid` = `private_key_id`, the access token cached until 5 minutes before expiry, messages data-only with string values; `UNREGISTERED` or 404 clears the device token only if unchanged; an unusable credentials file disables push like a missing one (§6.3, §14.6); the scheduler reads the injected clock at least every 30 s, runs a missed job once after a forward jump and re-plans after a backward jump of more than a day (§9.5); `rotate-epoch` prints the new epoch, exit 0 or 1 on a database error (§10.4); `serve` uses `into_make_service_with_connect_info` (D-032 fulfilled, §8.6, §11.3); `GET /i/` with an empty code is the HTML 404 (E-4); reqwest (rustls, aws-lc-rs, platform verifier) and jsonwebtoken added, so the arm64 image needs a C toolchain stage and the distroless image CA roots, to be confirmed by M2c-pi or M10; the dev image was rebuilt with them on the Mac on 2026-10-18 (§6.3, §10.3); M4's test binaries and scheduler path (§11.3, §14.7). | +| 0.7 | 2026-10-18 | Decisions from the M9 (`ab298ab`) implementation report and the importer rehearsal on the real exports. D-044: nine Swedish Splitwise category names added to the §10.8 table and to `splitwise_categories.csv` with the key of their English equivalent (Städning and Verktyg - Övrigt `utilities`, Tjänster `home`, Cykel `transport`, Flyg `travel`, Spel `entertainment`, Kläder `shopping`, Livet - Övrigt and Utbildning `other`). D-045: reconciliation: the run commits only if each user's balance of the whole account equals the sum of the file's totals rows (one per currency, converted with `--fx`); a mismatch or a missing totals row rolls back and exits 4, a dry run reports the same, an account already holding other rows rolls back, a mismatch after any `--fx` conversion is reported but accepted (§10.8, AC-17.5). D-046: CLI semantics: `--user` takes a user id or an exact unique display name; no database is created or migrated, `meta.schema_version` must be 1; a dry run runs the full transaction and rolls back; exit codes 0 ok, 1 database or file error, 2 usage, 3 cannot import, 4 reconciliation mismatch; the import key uses an existing account's stored name; an empty description or one over 1000 code points makes the row invalid; descriptions over 100 code points are cut with the full text in `notes`; settlement notes cut to 200; both nets zero is mode `full` on the owner; idempotency through the existing `import_key` columns (§10.8, AC-17.5). M9 marked landed, M11 rehearsal recorded (§11.2, §11.3). | diff --git a/docs/HANDOFF.md b/docs/HANDOFF.md index fb84d4c..63b48ff 100644 --- a/docs/HANDOFF.md +++ b/docs/HANDOFF.md @@ -3,28 +3,24 @@ Living document, rewritten in place (never a second copy). A new orchestrator session reads CLAUDE.md, then this. ## 1. Where things stand -- `main` at `bd6f10a` (M4) plus the SPEC v0.6 docs commit from `agent-spec6` once it lands. History: `5e1be38` plans and prompts, `c885aba` M2c-dev, `76ee957` SPEC v0.3, `fad6fc9` M2b, `fdf2093` M2a, `040bf41` SPEC v0.4, `503cee5` M3, `c28c52b` M5, `982451d` SPEC v0.5, `bd6f10a` M4. -- Landed: M0 spec, M1 contracts and vectors (`c6df2ab`), M2a server skeleton and `kvitt-domain` (`fdf2093`), M2b Android skeleton and `:domain` (`fad6fc9`), M2c-dev infra for development (`c885aba`), M3 server core API (`503cee5`), M5 Android data layer (`c28c52b`), SPEC v0.5 (`982451d`), M4 server sync, audit, push, scheduler (`bd6f10a`). With `agent-spec6`: SPEC v0.6 with M4's D-035 to D-039. -- Both gates exist: Android `tools/gradle-serial.sh check` (from `android/`), server `cargo fmt --check && cargo clippy -- -D warnings && cargo test` (from `server/`). -- Worktrees in flight under `.claude/worktrees/`: `agent-m9` (M9 Splitwise importer; decisions D-044 to D-049 reserved for it), `agent-spec6` (this docs change: SPEC v0.6). +- `main` at `ab298ab` (M9) plus the SPEC v0.7 docs commit from `agent-spec7` once it lands. History: `5e1be38` plans and prompts, `c885aba` M2c-dev, `76ee957` SPEC v0.3, `fad6fc9` M2b, `fdf2093` M2a, `040bf41` SPEC v0.4, `503cee5` M3, `c28c52b` M5, `982451d` SPEC v0.5, `bd6f10a` M4, `23a2f1e` SPEC v0.6, `ab298ab` M9. +- Landed: M0 spec, M1 contracts and vectors (`c6df2ab`), M2a server skeleton and `kvitt-domain` (`fdf2093`), M2b Android skeleton and `:domain` (`fad6fc9`), M2c-dev infra for development (`c885aba`), M3 server core API (`503cee5`), M4 server sync, audit, push, scheduler (`bd6f10a`), M5 Android data layer (`c28c52b`), M9 Splitwise importer (`ab298ab`), SPEC v0.6 (`23a2f1e`). With `agent-spec7`: SPEC v0.7 with M9's D-044 to D-046 and D-044's nine Swedish names in `server/crates/kvitt-import/data/splitwise_categories.csv`. +- Both gates exist: Android `tools/gradle-serial.sh check` (from `android/`), server `cargo fmt --check && cargo clippy --all-targets -- -D warnings && cargo test` (from `server/`). +- Worktrees in flight under `.claude/worktrees/`: `agent-spec7` (this docs change: SPEC v0.7). - The owner runs the UX sprint in a separate Claude session in `~/orca/workspaces/kvitt/UX-sprint` on branch `UX-sprint`; it commits only under `docs/ux/` and rebases on `main` before landing. U1 is committed on `UX-sprint` at `210ab5c` (`docs/ux/`: directions, ia, screens, flows, components, tokens, `strings.csv`); the owner has not yet picked a direction. -- Development stack (M2c-dev): running on the Mac with the M4 image, reachable at `http://192.168.1.210:8080`; the owner confirmed `/health` from the phone on the home Wi-Fi on 2026-10-17. The dev image was rebuilt with M4's new crates (reqwest on rustls/aws-lc-rs, jsonwebtoken) on 2026-10-18 and built cleanly. Firebase project `kvitt-1337` with apps `se.hagstedt.kvitt` and `se.hagstedt.kvitt.debug`; service account `kvitt-fcm`; `infra/secrets/fcm-service-account.json` and `infra/secrets/google-services.json` in the main checkout (git-ignored). -- Installed on the owner's phone (SM-S942B): the debug app `se.hagstedt.kvitt.debug` built from `main` at `c28c52b`, as of 2026-10-18 (M2b's `installDebug` check is done). +- Importer rehearsal (M11 preparation), 2026-10-18: the real run reconciled on both real files in a throwaway database (Vardag 1746 rows, Hemma 37 rows); 21 rows were unmapped before D-044, 0 are expected after it (confirm with the next dry run). The CSV files stay in `export/` (git-ignored). +- Development stack (M2c-dev): running on the Mac with the M4 image, reachable at `http://192.168.1.210:8080`; the owner confirmed `/health` from the phone on the home Wi-Fi on 2026-10-17. Firebase project `kvitt-1337` with apps `se.hagstedt.kvitt` and `se.hagstedt.kvitt.debug`; service account `kvitt-fcm`; `infra/secrets/fcm-service-account.json` and `infra/secrets/google-services.json` in the main checkout (git-ignored). +- Installed on the owner's phone (SM-S942B): the debug app `se.hagstedt.kvitt.debug` built from `main` at `c28c52b`, as of 2026-10-18. ## 2. Open work (one line each, with who owns it: agent or owner) -- Land `agent-spec6` (orchestrator): SPEC v0.6, decisions D-035 to D-039, PLAN-M4 pruning wording, prompts/README status. +- Land `agent-spec7` (orchestrator): SPEC v0.7, decisions D-044 to D-046, the category data file, `server/README.md`, prompts/README status. - Image build for the Pi (M2c-pi or M10): since M4 the arm64 server image needs a C toolchain in the build stage (aws-lc-rs) and CA roots in the distroless image (§10.3); confirm both and one FCM send from the shipped image. - D-027 re-stamp (orchestrator assigns: M8 as a contract request on `:data`, or a small `:data` agent): on the first sync attempt after a `clock_skew` rejection the phone re-stamps `updated_at` of each such pending row to its current clock before pushing. Not in `:data` at `c28c52b`. - Picture size (M6, if needed): `:data` always fetches E-11 at 512; a size parameter for the 128 variant is a contract request on `:core:model` and `:data`. -- Plans (SPEC §11.2 v0.6), in dependency order; prompt `prompts/.md`: - - M1, M2a, M2b, M2c-dev, M3, M4, M5: landed. +- Plans (SPEC §11.2 v0.7), in dependency order; prompt `prompts/.md`: + - M1, M2a, M2b, M2c-dev, M3, M4, M5, M9: landed. - U1 UX directions: committed on `UX-sprint` (`210ab5c`); waiting for the owner's pick. - - U2 UX design spec: owner's session, after the pick; owner approves; then the §9.7 colour change goes into the spec (docs agent) before M6. - - M9 Splitwise importer: agent in flight (`agent-m9`). - - Next waves: M6 Design system after U2 approved and the §9.7 colour change; M7a to M7f Features after M6, in parallel; M8 App integration after M7 (and M3, M4, M5). - - M2c-pi Infra on the Pi: after M8, before M10 (owner: answer OQ-3, grant the Pi, Home Assistant Pi, Gitea and Drive allowances at the start, D-023). - - M10 First release: after M2c-pi, M3, M4, M8 (owner: keystore per §10.6, D-024). - - M11 Import history: after M9 and M10. + - Next waves: U2 UX design spec (owner's session, after the pick; owner approves; then the §9.7 colour change goes into the spec by a docs agent), then M6 Design system, M7a to M7f Features after M6 in parallel, M8 App integration after M7, M2c-pi Infra on the Pi after M8 (owner: answer OQ-3, grant the Pi, Home Assistant Pi, Gitea and Drive allowances at the start, D-023), M10 First release after M2c-pi (owner: keystore per §10.6, D-024), M11 Import history after M9 and M10 (procedure §10.8 "Production run", D-025; each run must reconcile, D-045). ## 3. HUMAN steps pending (relay in chat) - UX direction pick from U1's directions in `docs/ux/` on `UX-sprint`, with comments (owner's UX-sprint session). @@ -36,8 +32,8 @@ Living document, rewritten in place (never a second copy). A new orchestrator se - Nothing. ## 6. Numbering -- Decisions: D-001 to D-043 used (D-008 and D-009 unused). Reserved: **D-044 to D-049 for M9** (in flight). Next free for anything else: **D-050**. -- Spec version: v0.6 written; next **v0.7**. +- Decisions: D-001 to D-046 used (D-008 and D-009 unused). Next free: **D-047**. +- Spec version: v0.7 written; next **v0.8**. - Server schema version **1**; next **2**. Room schema version **1** (`android/data/schemas/se.hagstedt.kvitt.data.db.KvittDatabase/1.json`); next **2**. - Sync protocol **1**, push payload **1**. diff --git a/docs/decisions.md b/docs/decisions.md index e485642..853ed4e 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -48,3 +48,6 @@ entry and the old one a strike-through plus a pointer. The spec is updated in th | D-041 | 2026-10-17 | `:data`'s `DataModule` provides implementations only; the interface bindings live in `DataBindingsModule` (`@DisableInstallInCheck`), which `:app` includes for release so the debug fakes do not duplicate bindings. (M5's D-031, renumbered.) | M5 report: Hilt fails on duplicate bindings when debug binds fakes. | SPEC §6.2, §11.3 (M8); PLAN-M8 | | D-042 | 2026-10-17 | Room tests run on Robolectric with the framework SQLite driver; migration tests build from the exported schema JSON rather than `room3-testing`. (M5's D-032, renumbered.) | M5 report: no device needed for the gate. | SPEC §6.2 | | D-043 | 2026-10-17 | Two extra DataStore keys: the pairing status in `auth.preferences_pb`, and the installed version code that received a 426 in `prefs.preferences_pb` (syncing stops until the installed version changes). The token is encrypted with Keystore key `kvitt_token_key`, AES-256-GCM (as §8.2). (M5's D-033, renumbered.) | M5 report. | SPEC §8.2, §14.2.2, §14.4.5 | +| D-044 | 2026-10-18 | Nine Swedish Splitwise category names, found unmapped in the importer rehearsal on the real Swedish export, are added to the §10.8 table and to `server/crates/kvitt-import/data/splitwise_categories.csv` with the key of their English equivalent: Städning → `utilities` (Cleaning); Verktyg - Övrigt → `utilities` (Utilities - Other); Tjänster → `home` (Services); Cykel → `transport` (Bicycle); Flyg → `travel` (Plane); Spel → `entertainment` (Games); Kläder → `shopping` (Clothing); Livet - Övrigt → `other` (Life - Other); Utbildning → `other` (Education). The test `category_table_matches_the_spec` keeps the two in step. | The rehearsal of 2026-10-18 left 21 rows of the Vardag file as `other`, counted unmapped, only because these names were missing from D-003's table. | SPEC §10.8; `server/crates/kvitt-import/data/splitwise_categories.csv` | +| D-045 | 2026-10-18 | Import reconciliation. Inside the import transaction, after the rows are written, each user's balance of the whole account (`net(u)`, §9.3) must equal the sum of that member's values in the file's totals rows (`Total balance` / `Totalsumma`), one totals row allowed per currency, a non-SEK one converted with its `--fx` rate. A mismatch or a missing totals row rolls the run back and exits 4; a dry run reports the same. The comparison covers the whole account, so importing into an account that already holds other rows rolls back. When any `--fx` conversion took place, a mismatch is reported in the summary but accepted (per-value rounding can make an exact match impossible). | M9 report (`ab298ab`): v0.6 ignored the totals row, and M11's gate ("imported balances equal Splitwise's totals") is cheapest checked by the importer itself, before anything is committed. | SPEC §10.8, AC-17.5, §11.2 (M11) | +| D-046 | 2026-10-18 | Importer CLI semantics. `--user =` takes a user id or an exact display name that exactly one user has; the two arguments must name the two household users. The importer never creates or migrates a database and requires `meta.schema_version` = `1`. A dry run runs the full transaction and rolls it back. Exit codes: 0 ok, 1 database or file error, 2 usage, 3 cannot import (header, invalid rows, non-SEK without `--fx`), 4 reconciliation mismatch. The import key uses an existing account's stored name, so a re-run with another spelling finds its rows. An empty description or one over 1000 code points makes the row invalid; a description over 100 code points is cut with the full text in `notes`; settlement notes are cut to 200 code points; both nets zero means the owner pays in mode `full`. Idempotency uses the existing `import_key` columns; no schema change. | M9 report (`ab298ab`): §10.8 left the user lookup, database handling, dry-run mechanics, exit codes and these edge cases open. | SPEC §10.8, AC-17.5; server/README.md | diff --git a/prompts/README.md b/prompts/README.md index 630aa31..7e7fe19 100644 --- a/prompts/README.md +++ b/prompts/README.md @@ -2,7 +2,7 @@ One ready-to-paste start prompt per plan (`PLAN-.md` at the repository root), plus `ORCHESTRATOR-prompt.md`. Before pasting, the orchestrator fills `{{NEXT_DECISION}}`, `{{SPEC_VERSION}}`, `{{SCHEMA_VERSION}}` (disjoint ranges for agents that run in parallel, CLAUDE.md › Numbering) and `{{ORCHESTRATOR_NOTES}}` (answers to the plan's open points, phone assignment, overrides). `U2.md` also takes `{{OWNER_PICK_AND_COMMENTS}}`. U1 and U2 run in the owner's own Claude session in `~/orca/workspaces/kvitt/UX-sprint` on branch `UX-sprint`. The numbers are for proposed entries in agents' reports: implementation agents never edit `SPEC.md` or `docs/decisions.md`; a docs agent records what the orchestrator accepts (CLAUDE.md › Orchestration). -Dependencies follow SPEC §11.1 and §11.2 (v0.6). Status as of 2026-10-18: M0 done, M1 landed (`c6df2ab`), M2c-dev landed (`c885aba`), M2a landed (`fdf2093`), M2b landed (`fad6fc9`), M3 landed (`503cee5`), M5 landed (`c28c52b`), M4 landed (`bd6f10a`); M9 in flight; U1 committed on `UX-sprint` (owner's direction pick pending). The plans' open points are resolved in SPEC v0.3 and recorded in each plan's §4; v0.4 records the M2a and M2b decisions (D-005 to D-014); v0.5 the M3 and M5 decisions (D-026 to D-034, D-040 to D-043); v0.6 the M4 decisions (D-035 to D-039). +Dependencies follow SPEC §11.1 and §11.2 (v0.7). Status as of 2026-10-18: M0 done, M1 landed (`c6df2ab`), M2c-dev landed (`c885aba`), M2a landed (`fdf2093`), M2b landed (`fad6fc9`), M3 landed (`503cee5`), M5 landed (`c28c52b`), M4 landed (`bd6f10a`), M9 landed (`ab298ab`); U1 committed on `UX-sprint` (owner's direction pick pending). The plans' open points are resolved in SPEC v0.3 and recorded in each plan's §4; v0.4 records the M2a and M2b decisions (D-005 to D-014); v0.5 the M3 and M5 decisions (D-026 to D-034, D-040 to D-043); v0.6 the M4 decisions (D-035 to D-039); v0.7 the M9 decisions (D-044 to D-046). ## Dependency order and waves diff --git a/server/README.md b/server/README.md index 4ef2e95..717c004 100644 --- a/server/README.md +++ b/server/README.md @@ -6,7 +6,7 @@ The Rust backend of Kvitt (SPEC §6.3): one Cargo workspace with three crates. |---|---|---| | `crates/kvitt-domain` | library, no I/O | Split, balance, allocation and settlement rules (SPEC §9.1, §9.3, §9.4, §14.5) and `next_occurrence_date` with the day clamping of §9.5. Passes every vector in `contracts/test-vectors/`. | | `crates/kvitt-server` | binary `kvitt-server` plus `lib.rs` | Config, SQLite pool and migrations, HTTP API, auth, CLI. | -| `crates/kvitt-import` | binary `kvitt-import` | Splitwise importer (SPEC §10.8, milestone M9). In M2a it prints its usage and exits 2. | +| `crates/kvitt-import` | binary `kvitt-import` plus `lib.rs` | Splitwise importer, implemented per SPEC §10.8 (M9, D-044 to D-046): one CSV export per run into one account, in one transaction that commits only when every row is valid and the account balance reconciles with the file's totals rows. Exit codes: 0 ok, 1 database or file error, 2 usage, 3 cannot import, 4 reconciliation mismatch. Category table in `crates/kvitt-import/data/splitwise_categories.csv`. | The Rust toolchain is pinned in `rust-toolchain.toml` (rustup installs it on first use); `Dockerfile` uses the same version. @@ -112,7 +112,7 @@ the wire contract makes `tests/openapi_contract.rs` fail against `contracts/open `Period` (`YYYY-MM`, `FromStr`/`Display`, `next()`, `days()`, `clamped_date(day_of_month)`), `next_candidate_period(start_period, end_period, last_generated_period) -> Result, RecurrenceError>` and `next_occurrence_date(&RecurrenceSchedule) -> Result, RecurrenceError>`. -* **Importer (M9):** `kvitt-import` already depends on `kvitt-domain` and `kvitt-server` (the library). +* **Importer (M9):** `kvitt-import` depends on `kvitt-domain` and `kvitt-server` (the library); it reuses the server's validation, row helpers and `SqlAuditWriter`, and never opens a database it would have to create or migrate (SPEC §10.8). ## Tests and the gate @@ -135,6 +135,9 @@ From `server/`: `cargo fmt --check && cargo clippy --all-targets -- -D warnings (an RSA key is generated per test; none is committed). * `crates/kvitt-server/tests/scheduler_daily/` (M4): 06:00 and 04:00 across both DST changes, run at start, clock jumps, catch-up through `generate_all_due`, paused and deleted templates skipped, 90-day pruning. +* `crates/kvitt-import/tests/import.rs` (M9): both header variants, split and settlement rules, `--fx`, the import key and + idempotent re-runs, reconciliation and its exit codes, dry run, the CLI, and the §10.8 category table against + `data/splitwise_categories.csv`; synthetic fixtures in `crates/kvitt-import/tests/fixtures/` only. The M4 binaries reuse M3's harness (`tests/core_api/common.rs`) and `tests/sync_api/m4.rs` through `#[path]`. No `unwrap`/`expect` in non-test code (a crate-level `deny` enforces it under clippy). diff --git a/server/crates/kvitt-import/data/splitwise_categories.csv b/server/crates/kvitt-import/data/splitwise_categories.csv index 510f7ac..adc1748 100644 --- a/server/crates/kvitt-import/data/splitwise_categories.csv +++ b/server/crates/kvitt-import/data/splitwise_categories.csv @@ -53,8 +53,12 @@ Förbrukningsvaror,sv,home Skötsel/underhåll,sv,home Avbetalning/Amortering,sv,home Hyra,sv,home +Tjänster,sv,home Hem - Övrigt,sv,home TV/telefon/internet,sv,utilities +Städning,sv,utilities +Verktyg - Övrigt,sv,utilities +Cykel,sv,transport Buss/tåg,sv,transport Bil,sv,transport Bensin/bränsle,sv,transport @@ -62,13 +66,18 @@ Parkering,sv,transport Taxi,sv,transport Transport - Övrigt,sv,transport Hotell,sv,travel +Flyg,sv,travel +Spel,sv,entertainment Musik,sv,entertainment Filmer,sv,entertainment Sport,sv,entertainment Underhållning - Övrigt,sv,entertainment Sjukvård/medicin,sv,health Presenter,sv,shopping +Kläder,sv,shopping Allmänt,sv,other Barnomsorg,sv,other +Utbildning,sv,other Försäkringar,sv,other Skatter,sv,other +Livet - Övrigt,sv,other