jacobh2/kvitt
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Pi: minimal self-contained compose project (D-125)

Browse Source 
The Kvitt Pi runs Kvitt from /home/jacobhagstedt/kvitt: compose.yaml, .env,
secrets/ and relative bind mounts ./data, ./backups, ./import. Both containers
run as the owner's uid/gid (KVITT_UID, KVITT_GID). No install script, no /opt,
no Docker daemon or system changes; operations are plain docker compose
commands (README "Pi: what the owner does", RESTORE.md).

Removed install.sh, deploy.sh, bootstrap.sh, backup-now.sh, install-apk.sh,
lib.sh. .env holds only the keys compose.yaml reads; RCLONE_REMOTE is fixed
to gdrive: in compose.yaml.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TQd3pEP9UeN2uifbHknQ7B
This commit is contained in:
Jacob Hagstedt
2026-10-20 19:18:53 +02:00
parent 200c99f2f6
commit 6e5981d62a
12 changed files with 225 additions and 663 deletions
Download Patch File Download Diff File Expand all files Collapse all files
infra/README.md
+130 -141
View File
@@ -2,7 +2,7 @@
This file covers the **development stack on the owner's Mac** (`infra/dev/`, milestone M2c-dev, D-001), the
**Firebase setup** (SPEC §10.7) and the **Pi stack** (`infra/pi/`, milestone M2c-pi, D-100): a self-contained
bundle the owner copies to the Kvitt Pi and runs there, behind the owner's Nginx Proxy Manager on the Home Assistant
compose project the owner copies to `/home/jacobhagstedt/kvitt` on the Kvitt Pi and runs there (D-125), behind the owner's Nginx Proxy Manager on the Home Assistant
Pi (D-120). No agent and no Mac script connects to the Pis; the owner runs every Pi command and every NPM setting
from the checklist in "Pi: what the owner does".
@@ -20,7 +20,7 @@ from the checklist in "Pi: what the owner does".
| `dev/lib.sh` | Helpers sourced by the scripts |
| `dev/secrets/` | Git-ignored: only `fcm-service-account.json`, copied there by `dev/up.sh` and mounted read-only at `/secrets` |
| `backup/Dockerfile`, `backup/backup.sh`, `backup/healthcheck.sh` | Backup image and job of SPEC §10.9 (shared with the Pi stack); Drive upload and D-002 deletion |
| `pi/` | **The Pi bundle** (copied to the Kvitt Pi): `compose.yaml`, `.env.example`, `install.sh`, `deploy.sh`, `bootstrap.sh`, `backup-now.sh`, `install-apk.sh`, `lib.sh`; git-ignored `pi/.env`, `pi/secrets/`, `pi/out/` |
| `pi/` | **The Pi project** (copied to `~/kvitt` on the Kvitt Pi): `compose.yaml`, `.env.example`; git-ignored `pi/.env`, `pi/secrets/`, `pi/out/` |
| `build-images.sh` | Mac fallback for the image build: linux/arm64, push to the Gitea registry |
| `publish-apk.sh` | Mac: release APK, mapping and `latest.json` into `pi/out/apk/` for the owner to copy to the Pi |
| `RESTORE.md` | Restore from a Drive backup, step by step (SPEC §10.9) |
@@ -153,7 +153,7 @@ gcloud iam service-accounts keys list --iam-account kvitt-fcm@kvitt-1337.iam.gse
Rotating the key: create a new one with
`gcloud iam service-accounts keys create infra/secrets/fcm-service-account.json --iam-account kvitt-fcm@kvitt-1337.iam.gserviceaccount.com --project kvitt-1337`,
restart the server, then delete the old key id with `gcloud iam service-accounts keys delete <key id> ...`.
Copying the key to the Pi (§10.7 step 7, mode 0440, group 10001) is done by `pi/install.sh` ("Pi: what the owner does", step 5).
The key goes to the Pi as `~/kvitt/secrets/fcm-service-account.json` ("Pi: what the owner does", step 2).
## Pi stack (SPEC §10.1 to §10.10, D-100)
@@ -163,21 +163,25 @@ Copying the key to the Pi (§10.7 step 7, mode 0440, group 10001) is done by `pi
|---|---|---|
| Gitea Actions (runner label `host`, a Raspberry Pi) | `.gitea/workflows/build-images.yaml`: builds `kvitt-server` and `kvitt-backup` for linux/arm64, pushes them to `gitea.lan/jacobh2/...` tagged `vMAJOR.MINOR.PATCH` and `latest`, pushes the git tag | Gitea, on a push to `server/**`, `infra/backup/**`, `.gitea/workflows/**`, or by hand (Actions, "Run workflow") |
| Owner's Mac | `infra/build-images.sh [tag]` (fallback image build), `infra/publish-apk.sh` (release APK into `infra/pi/out/apk/`) | The owner (or an agent, for the APK build only) |
| Kvitt Pi, `/opt/kvitt/` | `compose.yaml`, `.env`, `secrets/`, `import/`, `apk/`, and the scripts `deploy.sh`, `bootstrap.sh`, `backup-now.sh`, `install-apk.sh` (copied there by `install.sh`) | The owner, in their own SSH session |
| Home Assistant Pi | Nginx Proxy Manager (NPM): a proxy host for `<KVITT_PUBLIC_HOST>` with its own Let's Encrypt certificate, TLS on 443, plain HTTP to `<kvitt-pi-address>:8080` over the LAN (D-120) | The owner, in NPM's web UI (step 6) |
| Router (ASUS) | No new rule: the existing external TCP 443 forward to the Home Assistant Pi (NPM, for Home Assistant) serves Kvitt too, NPM picks the proxy host by name; external 80 already reaches NPM for Let's Encrypt; nothing is forwarded to the Kvitt Pi (D-124) | The owner checks it (step 7) |
| Kvitt Pi, `/home/jacobhagstedt/kvitt/` | `compose.yaml`, `.env`, `secrets/`, and the bind-mounted `data/` (SQLite, pictures, `apk/`), `backups/`, `import/`; nothing outside this folder (D-125) | The owner, in their own SSH session, with plain `docker compose` commands |
| Home Assistant Pi | Nginx Proxy Manager (NPM): a proxy host for `<KVITT_PUBLIC_HOST>` with its own Let's Encrypt certificate, TLS on 443, plain HTTP to `<kvitt-pi-address>:8080` over the LAN (D-120) | The owner, in NPM's web UI (step 4) |
| Router (ASUS) | No new rule: the existing external TCP 443 forward to the Home Assistant Pi (NPM, for Home Assistant) serves Kvitt too, NPM picks the proxy host by name; external 80 already reaches NPM for Let's Encrypt; nothing is forwarded to the Kvitt Pi (D-124) | The owner checks it (step 5) |
Stack (`pi/compose.yaml`, project name `kvitt`): `server` (publishes `8080:8080` on the Pi's LAN interfaces, the
only published port; `KVITT_PUBLIC_URL=https://${KVITT_PUBLIC_HOST}` (default port 443, D-124), `KVITT_TRUSTED_PROXY=${HA_HOST}`) and
`backup`; volumes `data`, `backups`; network `kvitt`; every service `restart: unless-stopped`, json-file logs
10 MB x 3, `TZ=Europe/Stockholm`. Images are `${KVITT_REGISTRY}/kvitt-server|kvitt-backup:${KVITT_TAG}` with
`KVITT_REGISTRY=gitea.lan/jacobh2` (D-104; the names the workflow pushes). Caddy and cert-sync were removed by
D-120: TLS terminates in NPM, which already runs on the Home Assistant Pi.
`backup`; relative bind mounts `./data:/data`, `./backups:/backups` (backup), `./import:/import:ro` (server), and one
secret file each, read-only: `secrets/fcm-service-account.json` for the server, `secrets/rclone.conf` for the backup
(D-117: the server never sees the Drive token); both containers run as `${KVITT_UID}:${KVITT_GID}`, the owner's
uid and gid, with a read-only root, a 16 MB tmpfs `/tmp`, no capabilities and `no-new-privileges`; every service
`restart: unless-stopped`, json-file logs 10 MB x 3, `TZ=Europe/Stockholm`; the compose default network. Images are
`${KVITT_REGISTRY}/kvitt-server|kvitt-backup:${KVITT_TAG}` with `KVITT_REGISTRY=gitea.lan/jacobh2` (D-104; the names
the workflow pushes). Caddy and cert-sync were removed by D-120: TLS terminates in NPM, which already runs on the Home
Assistant Pi. There is no install script and nothing on the Pi outside `~/kvitt` is changed (D-125).
**Client IPs (SPEC §8.5, §8.6).** NPM's request to the server carries `X-Forwarded-For: <what the client sent>,
<client address>`; the server takes the last entry only when the TCP peer is `KVITT_TRUSTED_PROXY`, so `HA_HOST`
must be NPM's LAN IPv4 address (the server accepts an IP literal or a resolvable name, not a CIDR; `install.sh`
requires the address because the container cannot resolve an mDNS name such as `homeassistant.local`). The
must be NPM's LAN IPv4 address (the server accepts an IP literal or a resolvable name, not a CIDR; use the
address, because the container cannot resolve an mDNS name such as `homeassistant.local`). The
router's forward keeps the internet client's source address, and Docker's published port keeps NPM's (DNAT, no
SNAT, for traffic arriving on the LAN interface), so the per-IP limits of D-032, D-115 and D-119 see real
addresses. The server does not use `X-Forwarded-Proto` or `Host`: links are built from `KVITT_PUBLIC_URL` (E-12).
@@ -188,34 +192,55 @@ address, and only `HA_HOST` may set `X-Forwarded-For`.
| Component | Version | Source |
|---|---|---|
| Nginx Proxy Manager | The owner's existing install on the Home Assistant Pi; the block of step 6 was written against NPM v2.16.0 (2026-09-24) templates | github.com/NginxProxyManager/nginx-proxy-manager, `backend/templates/proxy_host.conf` |
| Nginx Proxy Manager | The owner's existing install on the Home Assistant Pi; the block of step 4 was written against NPM v2.16.0 (2026-09-24) templates | github.com/NginxProxyManager/nginx-proxy-manager, `backend/templates/proxy_host.conf` |
| Alpine (backup) | `alpine:3.24.2@sha256:294b683cb724975bec92580e1e685676bd4b50bda910ddb8c51d4cabeaec77e6` | Docker Hub official image; `3.24` = `latest` |
| backup packages (Alpine v3.24) | sqlite 3.53.4, rclone 1.74.1, tar 1.35, tzdata 2026e (M2c-dev), supercronic 0.2.49-r1 | `apk policy` in the pinned image |
| Docker Engine on the Pi | Docker's Debian apt repository, arm64 packages (`docker-ce` 5:29.8.2 current): `docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin` | docs.docker.com/engine/install/raspberry-pi-os/: that page is for 32-bit (armhf) only; 64-bit Raspberry Pi OS uses the Debian instructions (docs.docker.com/engine/install/debian/) |
**Why supercronic, not busybox crond (D-104).** The backup sidecar runs as a non-root user (uid 10001). Busybox
**Why supercronic, not busybox crond (D-104).** The backup sidecar runs as a non-root user (the owner's uid on the Pi, D-125). Busybox
crond started as a non-root user in alpine 3.24.2 never ran a job (tested 2026-10-18: an every-minute job produced
no output in two minutes). supercronic is a cron for containers that runs jobs as the
user it runs as and honours `TZ`; the schedule is unchanged (`30 3 * * *`).
### Backup uid (D-105, SPEC §10.2 "M2c-pi reconciles the two")
### Container uid (D-105, D-125)
`backup` runs as **uid 10001**, the server's uid, on the Pi as on the Mac: `sqlite3 .backup` on a WAL database
must open (and write) `kvitt.sqlite-shm`, which the server creates as 10001 with mode 0644, so uid 1000 could not
back up the live database without loosening the server's files. Therefore `/opt/kvitt/secrets/rclone.conf` is
mode 0600 **owned by 10001:10001** (not 1000). rclone writes
`server` and `backup` run as the **same uid**, because `sqlite3 .backup` on a WAL database must open (and write)
`kvitt.sqlite-shm`, which the server creates with mode 0644 (D-105). On the Pi that uid is the owner's own
(`KVITT_UID`, `KVITT_GID` in `.env`, `id -u` / `id -g`), so everything in `data/` and `backups/` belongs to the
owner, the secret files can simply be the owner's with mode 0600, and a restore works with plain `mv` and `cp`
(D-125). The images need no particular uid: the server image is distroless with `USER 10001` only as its default
(compose's `user:` overrides it; the binary needs no account, no home and nothing writable outside `/data`), and the
backup image's scheduler supercronic runs as any uid. The owner's uid has no account inside either image; the
backup container gets `HOME=/tmp` for rclone. Folders the containers write to must exist before the first start
(`mkdir -p data/apk backups import`): a bind-mount source Docker creates itself belongs to root. rclone writes
refreshed access tokens back to its config; the mounted file is read-only, so `backup.sh` works on a private copy
in `/tmp` each run. `install.sh` gives the named volumes their owners once (`/data`, `/data/apk`, `/backups` to
10001) with a one-off root container (`docker compose run --user 0:0 --cap-add CHOWN`).
in the tmpfs `/tmp` each run. The development stack on the Mac keeps uid 10001 (`infra/dev/`).
### What was checked on the Mac
2026-10-20 (D-125, self-contained compose project):
* `podman-compose -f infra/pi/compose.yaml --env-file <filled sample> config`: exit 0.
* Rehearsal with Podman in a scratch copy of the project folder, `KVITT_UID`/`KVITT_GID` = the Mac user's 501:20
(not 10001), images `kvitt-server` from `server/Dockerfile` and `kvitt-backup` from `infra/backup/Dockerfile`:
both containers healthy with read-only roots; `GET /health` 200 with `db_ok: true`; `kvitt.sqlite`, its `-wal`
and `-shm` and `data/apk` owned by 501 on the host; `kvitt-server bootstrap` through `exec` printed a code, a
`https://<KVITT_PUBLIC_HOST>/i/` link and an expiry; an APK and `latest.json` copied into `data/apk/` were served
at once (`/apk/latest`, `/apk/files/kvitt-<n>.apk` 200 `application/vnd.android.package-archive`); `/backup.sh`
through `exec`: with `RCLONE_FOLDER_ID` empty it kept both files and wrote `last-success`; with a local rclone
remote under `/tmp` it uploaded, confirmed and deleted the local copies; with the remote on the read-only root it
failed, kept the files and left `last-success` unchanged; the archives belong to 501. The restore steps of
`RESTORE.md` (stop, `gunzip`, `integrity_check` `ok`, `mv` aside, `cp`, `tar -x`, `run --rm server
rotate-epoch`, start) ended with `/health` 200. Podman-compose ignores `create_host_path: false` (it created a
directory for a missing secret file); Docker Compose honours it and refuses to start instead, not checked here.
* Not checked (needs the Pi): Docker Engine and Docker Compose themselves, pulls from `gitea.lan`, Drive.
2026-10-19 (D-120, Nginx Proxy Manager instead of Caddy and cert-sync):
* `shellcheck -S style` on every script under `infra/` and `tools/`: clean.
* `podman-compose -f infra/pi/compose.yaml --env-file <filled sample> config` and the same for `infra/dev/`: exit 0.
* `nginx -t` (nginx:alpine 1.31.7) of the step 6 block inside a minimal server block, and inside a copy of NPM
v2.16.0's `nginx.conf` with the proxy host rendered as NPM's template renders it for the step 6 settings (Block
* `nginx -t` (nginx:alpine 1.31.7) of the step 4 block inside a minimal server block, and inside a copy of NPM
v2.16.0's `nginx.conf` with the proxy host rendered as NPM's template renders it for the step 4 settings (Block
Common Exploits, Force SSL, HTTP/2, HSTS, the block; NPM's regex `^(?:.*;)?\s*?location\s*?\/\s*?{` finds the
block's `location /`, so NPM leaves its own out): both successful.
* Rehearsal with Podman: the server image built from `server/Dockerfile` at this commit (arm64, hardened as on the
@@ -245,7 +270,7 @@ Same setup as the owner's working example: `runs-on: host`, `sh` steps, `apk add
Gitea gives a repository its owner's secrets) and `${{ github.actor }}`, a buildx builder whose `buildkitd.toml`
sets `http = true` for `gitea.lan` (created once and reused, so its cache survives), `actions/checkout@v4` with
`set-safe-directory` and `fetch-depth: 0`, the version from the newest `v1.0.*` git tag plus one, and the git tag
pushed after the build. That version is what `deploy.sh` takes.
pushed after the build. That version is the `KVITT_TAG` the Pi's `.env` takes.
The checkout is a cone-mode sparse checkout of `server` and `infra/backup` only, the two build contexts (neither
Dockerfile copies anything outside its context), so the action fetches with `--filter=blob:none` and writes only
those files instead of all 2118 (52 MB of Android screenshot goldens among them; about 95 s on the first runs).
@@ -258,7 +283,7 @@ act_runner masks the secret in the log. That first run got as far as the login (
all worked). `kvitt_cicd` needs only the `write:package` scope: the tag push uses the credentials `actions/checkout` stored (the job's own token).
If the "Tag and Push Version" step fails with a permission error, the job token cannot push tags on this Gitea;
then add `write:repository` to the token (owner decision below). The step "Print image digests" prints each
image's `sha256:` digest, which must match what `deploy.sh` prints on the Pi (D-117).
image's `sha256:` digest, which must match the digest the Pi pulled (Operations, "Image digests"; D-117).
The builder's `buildkitd.toml` is read only when the builder is created. After changing the registry setting in the
workflow, remove the builder once on the runner Pi: `docker buildx rm local-http-builder` (this also drops its
@@ -291,114 +316,76 @@ tags of the Gitea workflow (`v1.0.N`) are separate and say nothing about the APK
builds `:app:assembleRelease` through `tools/gradle-serial.sh` (JDK 25 on `PATH`), refuses an unsigned APK, and
writes into `infra/pi/out/apk/`: `kvitt-<version_code>.apk`, `kvitt-<version_code>.mapping.txt` (the R8 mapping,
for reading stack traces; never served) and `latest.json` (temp file, then rename); it keeps the newest 5 APKs
with their mappings. Then the owner, on the Mac:
with their mappings. Then the owner, on the Mac, copies the APK first and `latest.json` last into the server's
`data/apk/` (the server reads `latest.json` on every `GET /apk/latest`, so it must never name a file that is not
there yet):
```bash
cd /Users/jhag/projects/kvitt
scp infra/pi/out/apk/* <user>@<kvitt-pi-address>:/opt/kvitt/apk/
ssh <user>@<kvitt-pi-address> /opt/kvitt/install-apk.sh
scp infra/pi/out/apk/kvitt-<version_code>.apk jacobhagstedt@<kvitt-pi-address>:kvitt/data/apk/
scp infra/pi/out/apk/latest.json jacobhagstedt@<kvitt-pi-address>:kvitt/data/apk/
curl -s http://<kvitt-pi-address>:8080/apk/latest # shows the new version_code
```
`install-apk.sh` checks the SHA-256 against `latest.json`, copies the APKs into the `data` volume (`/data/apk/`)
and `latest.json` last with a rename, keeps the newest 5 there and prints `GET /apk/latest`.
The mapping files stay on the Mac. Older APKs on the Pi are removed by hand when wanted
(`ssh jacobhagstedt@<kvitt-pi-address> ls kvitt/data/apk`, then `rm` the old `kvitt-<n>.apk` files; about 10 MB each).
## Pi: what the owner does
Fill these in first (in `infra/pi/.env` on the Mac; the bundle carries the file to the Pi):
Kvitt runs from `/home/jacobhagstedt/kvitt` on the Kvitt Pi (D-125): `compose.yaml`, `.env`, `secrets/` and the
folders `data/`, `backups/`, `import/`. Nothing outside that folder is installed or changed. `<kvitt-pi-address>` is
the Kvitt Pi's LAN IPv4 address (static DHCP lease; ASUS router web UI, LAN, DHCP Server, client list), also NPM's
Forward Hostname / IP (step 4).
| Placeholder in `infra/pi/.env` | What | Where to find it |
|---|---|---|
| `KVITT_PUBLIC_HOST` | Kvitt's own public host name (D-120), for example a DuckDNS name or a name in your own domain, pointing at the home's public address like `icehack.asuscomm.com` does. Not `icehack.asuscomm.com` itself: that name is NPM's proxy host for Home Assistant, and NPM chooses the proxy host by name, not by port | Your choice; tell the orchestrator the name (it goes into the app and `docs/HANDOFF.md`) |
| `HA_HOST` | LAN IPv4 address of the Home Assistant Pi, where NPM runs (static DHCP lease); the server trusts `X-Forwarded-For` only from it. An address such as `192.168.1.20`, not `homeassistant.local` | Router client list, or Home Assistant, Settings, System, Network |
| `KVITT_TAG` | Image tag to run: the newest `v1.0.N` the Gitea workflow pushed (or the SHA from `build-images.sh`) | Gitea, the repository, Packages, `kvitt-server` |
| `KVITT_REGISTRY_USER`, `KVITT_REGISTRY_TOKEN` | Only if the Gitea packages are private: a Gitea user and a token with `read:package` | Gitea, Settings, Applications, Generate New Token |
| `KVITT_REGISTRY` | Already `gitea.lan/jacobh2` | |
| `RCLONE_REMOTE`, `RCLONE_FOLDER_ID` | Already `gdrive:` and `1jDp1dGE5R3H2DBcXCqIPN1iWzQnPkdlV` | |
### 1. Prerequisites (already on the Pi)
Not in `.env` (nothing on the Pi reads them; dropped from `.env` on 2026-10-20): in this README,
`<kvitt-pi-address>` is the Kvitt Pi's LAN IPv4 address (give it a static DHCP lease; ASUS router web UI, LAN, DHCP
Server, client list), also NPM's Forward Hostname / IP (step 6), and `<user>` is your user on the Kvitt Pi (the user
set in Raspberry Pi Imager).
* Docker Engine with the compose plugin, usable by `jacobhagstedt` without sudo: `docker compose version` prints a
version.
* Registry trust: the images come over plain HTTP from `gitea.lan`. If `docker compose pull` says "http: server
gave HTTP response to HTTPS client", add `gitea.lan` to `insecure-registries` in `/etc/docker/daemon.json` (the
Pi already has it for another project; Kvitt never edits it).
* If `docker compose pull` says "unauthorized" (private packages): once, `docker login gitea.lan` with your Gitea
user and a token with `read:package` (Gitea, Settings, Applications, Generate New Token).
### 1. Provision the Kvitt Pi
Raspberry Pi Imager on the Mac: Raspberry Pi 3, **Raspberry Pi OS Lite (64-bit)**; in its settings set the host
name, your user and password, the network, time zone Europe/Stockholm, and enable SSH with your public key. Boot
it, give it a static DHCP lease in the router, then from the Mac: `ssh <user>@<kvitt-pi-address>`.
### 2. Install Docker Engine on the Pi (docs.docker.com, Debian arm64 repository)
On the Pi:
```bash
sudo apt update && sudo apt install -y ca-certificates curl python3
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc
sudo tee /etc/apt/sources.list.d/docker.sources <<EOF
Types: deb
URIs: https://download.docker.com/linux/debian
Suites: $(. /etc/os-release && echo "$VERSION_CODENAME")
Components: stable
Architectures: $(dpkg --print-architecture)
Signed-By: /etc/apt/keyrings/docker.asc
EOF
sudo apt update
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo usermod -aG docker "$USER"
exit
```
Log in again; `docker compose version` prints a version and `docker run --rm hello-world` prints
"Hello from Docker!".
### 3. Fill the `.env` and stage the secrets, on the Mac
### 2. On the Mac: `.env`, secrets, copy
```bash
cd /Users/jhag/projects/kvitt
cp infra/pi/.env.example infra/pi/.env
open -e infra/pi/.env # replace every CHANGE_ME (table above), save
open -e infra/pi/.env # set HA_HOST; KVITT_UID/KVITT_GID = `id -u` / `id -g` on the Pi (1000 for the first user)
mkdir -p infra/pi/secrets
cp infra/secrets/fcm-service-account.json infra/secrets/rclone.conf infra/pi/secrets/
```
Only these two secret files go to the Pi. `infra/secrets/ha_ssh_key` and `ha_ssh_key.pub` are not used since
D-120: do not copy them (and no SSH add-on or `authorized_keys` entry on the Home Assistant Pi is needed).
`rclone.conf` is already generated from your `rclone authorize "drive"` (remote `[gdrive]`, `type = drive`,
`scope = drive`, the token); the Drive folder id is `RCLONE_FOLDER_ID` in `.env`, not in the conf.
`infra/pi/.env` and `infra/pi/secrets/` are git-ignored.
### 4. Copy the bundle to the Pi
On the Mac:
```bash
scp -r infra/pi <user>@<kvitt-pi-address>:kvitt-pi
scp -r infra/pi/compose.yaml infra/pi/.env infra/pi/secrets jacobhagstedt@<kvitt-pi-address>:kvitt/
rm -r infra/pi/secrets # the originals stay in infra/secrets/
```
### 5. Install and start
| Key in `infra/pi/.env` | Value |
|---|---|
| `HA_HOST` | LAN IPv4 address of the Home Assistant Pi (NPM), for example `192.168.1.20`, not `homeassistant.local` (router client list, or Home Assistant, Settings, System, Network) |
| `KVITT_UID`, `KVITT_GID` | `id -u` and `id -g` on the Kvitt Pi as `jacobhagstedt` (default 1000, 1000) |
| `KVITT_TAG` | `v1.0.0`, or the newest `v1.0.N` the Gitea workflow pushed (Gitea, the repository, Packages, `kvitt-server`) |
| `KVITT_PUBLIC_HOST`, `KVITT_REGISTRY`, `RCLONE_FOLDER_ID` | Already `kvitt.suorra.duckdns.org`, `gitea.lan/jacobh2`, `1jDp1dGE5R3H2DBcXCqIPN1iWzQnPkdlV` |
On the Pi:
`rclone.conf` is generated from your `rclone authorize "drive"` (remote `[gdrive]`, `scope = drive`); the Drive
folder id is `RCLONE_FOLDER_ID` in `.env`. `infra/pi/.env` and `infra/pi/secrets/` are git-ignored. To change
`compose.yaml` or `.env` later, copy just that file again the same way.
### 3. On the Pi: start
```bash
cd ~/kvitt-pi && ./install.sh
cd ~/kvitt
chmod 700 secrets && chmod 600 secrets/*
mkdir -p data/apk backups import
docker compose pull
docker compose up -d
docker compose ps
curl -s localhost:8080/health
```
It checks the `.env` (`KVITT_PUBLIC_HOST` a host name, `HA_HOST` an IPv4 address), creates
`/opt/kvitt/{secrets,import,apk}`, installs the two secret files (`fcm-service-account.json` 0440 root:10001;
`rclone.conf` 0600 10001:10001), adds `gitea.lan` to `insecure-registries` in `/etc/docker/daemon.json` (and
restarts Docker if that changed), copies `compose.yaml`, `.env` (0600) and the scripts to `/opt/kvitt`, pulls,
starts and waits. If an earlier bundle with Caddy and cert-sync was installed, it also removes its leftovers
(`/opt/kvitt/Caddyfile`, `secrets/ha_ssh_key`, `secrets/ha_known_hosts`, the containers and the volumes
`kvitt_certs` and `kvitt_caddy_data`) and says so. Expected end: `server: healthy`, a `docker compose ps` table with
`server` (ports `0.0.0.0:8080->8080/tcp`) and `backup`, the line `GET http://localhost:8080/health: {"status":"ok",...}`,
then "Not reachable through Nginx Proxy Manager yet" (expected on the first install, before step 6) or
`GET https://<KVITT_PUBLIC_HOST>/health (via NPM at <HA_HOST>:443): {"status":"ok",...}`, then "Kvitt is running.".
Then `rm -r ~/kvitt-pi/secrets` (the script reminds you). Running `install.sh` again is safe. Re-running
`install.sh` keeps the `KVITT_TAG` that `deploy.sh` set; change the tag only with `deploy.sh <tag>`.
Expected: `docker compose ps` lists `server` (ports `0.0.0.0:8080->8080/tcp`) and `backup`, both `(healthy)` within
a minute; `curl` prints `{"status":"ok",...,"db_ok":true}`.
### 6. Nginx Proxy Manager on the Home Assistant Pi
### 4. Nginx Proxy Manager on the Home Assistant Pi (already set up)
NPM terminates TLS for Kvitt with its own Let's Encrypt certificate for `<KVITT_PUBLIC_HOST>` and forwards plain
HTTP over the home LAN to the Kvitt Pi's port 8080 (plain HTTP between the two Pis is accepted, D-120). First make
@@ -486,8 +473,7 @@ the Kvitt Pi:
curl -fsS --resolve <KVITT_PUBLIC_HOST>:443:<HA_HOST> https://<KVITT_PUBLIC_HOST>/health
```
It prints `{"status":"ok",...}` (no `-k`: this also checks the certificate). `deploy.sh` and `install.sh` run the
same check.
It prints `{"status":"ok",...}` (no `-k`: this also checks the certificate).
**Why the block has its own `location /`.** nginx drops every `add_header` inherited from the server level in a
location that has an `add_header` of its own, and NPM's default `location /` has one (`X-Served-By`), which would
@@ -519,7 +505,7 @@ the connection comes from 10/8, 172.16/12, 192.168/16 (and the CDN ranges it dow
LAN can choose its own client address for the rate limits. Internet clients arrive with their public address (the
router's forward keeps the source address), so their `X-Real-IP` is ignored.
### 7. Router: no new rule
### 5. Router: no new rule
No new rule (D-124): external TCP 443 already reaches the Home Assistant Pi (NPM) for Home Assistant, and NPM picks
Kvitt's proxy host by name on the same port. Verify it exists: ASUS router web UI, WAN, Virtual Server / Port
@@ -528,38 +514,38 @@ forwarded to the Kvitt Pi: remove a rule to `<kvitt-pi-address>` if you made one
External Port `8443` if you made one for the earlier NPM version (D-120). Then on the phone with Wi-Fi off, open
`https://<KVITT_PUBLIC_HOST>/health` in Chrome: it shows `"status":"ok"` without a certificate warning (yes/no).
### 8. First backup and bootstrap
### 6. From outside, first backup, bootstrap
On the Pi:
On the phone with Wi-Fi off (or from the Mac), `https://kvitt.suorra.duckdns.org/health` shows `"status":"ok"`
(yes/no). Then on the Pi:
```bash
/opt/kvitt/backup-now.sh # ends with "last-success: <stamp>" and the newest kvitt-*.sqlite.gz and pictures-*.tar.gz on Drive
/opt/kvitt/bootstrap.sh # first install only: prints the admin invite code, link and expiry
cd ~/kvitt
docker compose exec backup /backup.sh # ends with "backup: ok <stamp>" after "uploaded and confirmed on Drive" twice
docker compose exec server kvitt-server bootstrap # first install only: prints the admin invite code, link and expiry
```
The invite link starts with `https://<KVITT_PUBLIC_HOST>/i/` (no port). The release app opens such links and connects to
that host only once its Android build carries the host name (`kvitt.publicHost`, D-121) and the default port 443
(D-124; builds made for port 8443 do not match a link without a port): tell the orchestrator before pairing a
release build.
release build. The partner's invite, once the admin is paired: `docker compose exec server kvitt-server invite-partner`.
### 9. Deploying a new version
### 7. Deploying a new version
After the Gitea workflow pushed `v1.0.N` (or `infra/build-images.sh` printed a tag), on the Pi:
```bash
/opt/kvitt/deploy.sh v1.0.N
cd ~/kvitt
docker compose exec backup /backup.sh # backup first, every deploy: any tag may carry a migration (SPEC §10.10)
sed -i 's/^KVITT_TAG=.*/KVITT_TAG=v1.0.N/' .env # or edit .env; note the previous tag for a rollback
docker compose pull && docker compose up -d
docker compose ps && curl -s localhost:8080/health
```
It backs up first (Drive confirmed) on every deploy, because any tag may carry a migration (SPEC §10.10), sets
`KVITT_TAG`, pulls, prints the `sha256:` digest of every image and appends it to `/opt/kvitt/deployed-digests`,
starts, waits for `server: healthy` and prints `/health` on port 8080 and through NPM (a failure through NPM is a
WARNING: the deploy succeeded, but Kvitt is not reachable from outside). The images come over plain HTTP from
`gitea.lan` (D-001), so compare the printed digests with the workflow's "Print image digests" step for that run; a
line starting `WARNING:` about a digest means a tag that was deployed before now points at a different image: do not
continue, roll back and find out why (D-117). If the server does not come up:
`/opt/kvitt/deploy.sh "$(cat /opt/kvitt/previous-tag)" --skip-backup`; if a migration had partially applied,
restore the pre-deploy backup (`infra/RESTORE.md`). A newer bundle with a changed `compose.yaml` is installed by
copying the bundle again (step 4, without the secrets) and running `./install.sh`.
Continue only when the backup ended with `backup: ok`. The images come over plain HTTP from `gitea.lan` (D-001), so
compare the pulled digests (Operations, "Image digests") with the workflow's "Print image digests" step for that
run (D-117). Rollback: set the previous tag in `.env` the same way and run `docker compose pull && docker compose
up -d`; if a migration had partially applied, restore the pre-deploy backup (`infra/RESTORE.md`).
### Fallback (SPEC §10.5.3): when NPM is down
@@ -576,23 +562,24 @@ copying the bundle again (step 4, without the secrets) and running `./install.sh
self-made root whose certificate would be uploaded to NPM as a Custom certificate; it is not prepared: tell the
orchestrator.
### Operations (SPEC §10.10), on the Pi in `/opt/kvitt`
### Operations (SPEC §10.10), on the Pi in `~/kvitt`
| Task | Command |
|---|---|
| Status | `docker compose ps` |
| Logs | `docker compose logs --tail 100 <server or backup>`; NPM's own log files are inside NPM on the Home Assistant Pi |
| Backup now | `/opt/kvitt/backup-now.sh` |
| Upgrade | `/opt/kvitt/deploy.sh <tag>` |
| Roll back | `/opt/kvitt/deploy.sh "$(cat /opt/kvitt/previous-tag)" --skip-backup` |
| Backup now | `docker compose exec backup /backup.sh` (ends with `backup: ok <stamp>`) |
| Upgrade | Step 7: backup, set `KVITT_TAG` in `.env`, `docker compose pull && docker compose up -d` |
| Roll back | Set the previous `KVITT_TAG` in `.env`, `docker compose pull && docker compose up -d` |
| Health through NPM | `curl -fsS --resolve <KVITT_PUBLIC_HOST>:443:<HA_HOST> https://<KVITT_PUBLIC_HOST>/health` |
| Headers through NPM | `curl -sS -D - -o /dev/null --resolve <KVITT_PUBLIC_HOST>:443:<HA_HOST> https://<KVITT_PUBLIC_HOST>/health` lists `content-security-policy`, `x-content-type-options`, `x-frame-options`, `referrer-policy`, `strict-transport-security` once each and no `x-served-by` |
| Certificate expiry and issuer | `echo \| openssl s_client -connect <HA_HOST>:443 -servername <KVITT_PUBLIC_HOST> 2>/dev/null \| openssl x509 -noout -enddate -issuer` (NPM renews it; NPM, SSL Certificates shows the same) |
| Partner invite | `docker compose exec server kvitt-server invite-partner` |
| Extra device invite | `docker compose exec server kvitt-server pair-device --user <user_id>` |
| Memory | `docker stats --no-stream` (server RSS under 100 MB with both phones syncing) |
| Disk | `df -h /` and `docker system df`; SQLite a few MB, pictures under 1 MB, APKs about 10 MB each since R8, 5 kept |
| Image digests deployed | `tail -n 8 /opt/kvitt/deployed-digests` |
| Client IPs are real (once, after step 7) | Open `https://<KVITT_PUBLIC_HOST>/health` on the phone with Wi-Fi off, then within a minute on the Kvitt Pi `sudo apt install -y conntrack` (once) and `sudo conntrack -L -p tcp --orig-port-dst 8080`: in every line the first `src=` is `<HA_HOST>` and the second half (from the second `src=`) has `dst=<HA_HOST>`, no `172.*` there, so the server sees NPM's address and trusts its `X-Forwarded-For`. If the second half shows a `172.*` address instead, every internet client shares one per-IP limit (SPEC §8.6): tell the orchestrator |
| Disk | `df -h ~` and `du -sh ~/kvitt/*`; SQLite a few MB, pictures under 1 MB, APKs about 10 MB each since R8, 5 kept |
| Image digests | `docker compose images` names the images; `docker image inspect --format '{{.RepoDigests}}' gitea.lan/jacobh2/kvitt-server:<tag>` (and `kvitt-backup`) prints the `sha256:` digest to compare with the workflow's |
| Client IPs are real (once, after step 5) | Open `https://<KVITT_PUBLIC_HOST>/health` on the phone with Wi-Fi off, then within a minute on the Kvitt Pi `sudo apt install -y conntrack` (once) and `sudo conntrack -L -p tcp --orig-port-dst 8080`: in every line the first `src=` is `<HA_HOST>` and the second half (from the second `src=`) has `dst=<HA_HOST>`, no `172.*` there, so the server sees NPM's address and trusts its `X-Forwarded-For`. If the second half shows a `172.*` address instead, every internet client shares one per-IP limit (SPEC §8.6): tell the orchestrator |
| Restore | `infra/RESTORE.md` |
**APK downloads (security review L-3, D-119).** `GET /apk/files/<name>` is public (the landing page links it) and
@@ -611,16 +598,18 @@ Backup encryption (security review M-1) is decided: no rclone `crypt`, the `driv
### Troubleshooting
* Step 6 check or `deploy.sh` says /health through NPM failed: "Could not resolve host" or "SSL certificate problem":
* The step 4 check says /health through NPM failed: "Could not resolve host" or "SSL certificate problem":
the proxy host has no certificate for `<KVITT_PUBLIC_HOST>` yet (SSL tab) or a different name; a `502 Bad Gateway`:
NPM cannot reach `<kvitt-pi-address>:8080` (wrong Forward Hostname / IP or Forward Port, or the server is down:
`docker compose ps`); the Home Assistant page instead of `{"status":"ok"}`: Domain Names does not match the name
you requested (NPM fell back to another host).
* "Request a new Certificate" fails in NPM: `<KVITT_PUBLIC_HOST>` does not point at the home's public address yet,
or external port 80 does not reach NPM.
* Pull fails with "http: server gave HTTP response to HTTPS client": `gitea.lan` is missing from
`/etc/docker/daemon.json` (`install.sh` adds it; check with `docker info | grep -A3 'Insecure Registries'`).
* Pull fails with "unauthorized": the packages are private; set `KVITT_REGISTRY_USER` and `KVITT_REGISTRY_TOKEN`
in `~/kvitt-pi/.env` and run `./install.sh` again.
* `install.sh` says `/opt/kvitt/secrets/<file>` is a directory: Docker created it because the file was missing at a
start; `sudo rmdir` it, put the file in `~/kvitt-pi/secrets/`, run `./install.sh`.
* Pull fails with "http: server gave HTTP response to HTTPS client": `gitea.lan` is missing from `insecure-registries`
(step 1; check with `docker info | grep -A3 'Insecure Registries'`).
* Pull fails with "unauthorized": the packages are private; `docker login gitea.lan` once (step 1).
* `docker compose up` says "bind source path does not exist" for a file in `secrets/`: the file was not copied;
copy it (step 2) and run `docker compose up -d` again.
* The server or backup log says "Permission denied" or "readonly database" on `/data` or `/backups`: the folder was
created by Docker as root, or `KVITT_UID`/`KVITT_GID` differ from the folder's owner: `ls -ln ~/kvitt` shows the
owner, `id -u` / `id -g` yours; `sudo chown -R "$(id -u):$(id -g)" ~/kvitt/data ~/kvitt/backups ~/kvitt/import` once.