F-r-ldraledigplannerare/observability-feature.md

# New observability feature

Collect errors and crashes from frontend and backend, run nightly batch analysis via Opus to categorize and suggest fixes, and surface tickets for manual resolution.

## High-level flow

1. **Capture**: Frontend/backend emit errors → POST to `/errors` endpoint → stored in SQLite
2. **Cleanup**: When a ticket is created to address an error group, delete the raw error logs for that group
3. **Nightly categorization job**: Query new errors → classify by category
4. **Nightly analysis job**: Group errors by category + location → send to Opus via Batches API → generate tickets
5. **Frontend dashboard**: View tickets, filter, search, edit, delete

---

## 1. Frontend error capture

### TypeScript SDK

Lightweight error capture library (part of Föräldraledig Plannerare React/Vite frontend).

**Functionality:**
- Catch global errors: `window.onerror`, `unhandledrejection`
- Provide explicit `captureError(error, context?)` function for manual logging
- Attach context: current URL, user session ID (if available), browser/device info
- Queue errors with exponential backoff + retry logic before POSTing
- POST to the app's existing `/errors` endpoint (backend defined below)
- No external dependencies; vanilla TypeScript

**Captured fields:**
- `error_message` (e.g., "Cannot read property 'x' of undefined")
- `stack_trace` (full JS stack)
- `url` (current page URL)
- `user_agent` (browser + OS)
- `timestamp`
- `session_id` (if available)
- `environment` (e.g., "production", "staging")

---

## 2. Backend error endpoint

### `/errors` POST endpoint

**Location:** Föräldraledig Plannerare existing backend

**Request body:**
```json
{
  "error_message": "string",
  "stack_trace": "string",
  "url": "string",
  "user_agent": "string",
  "timestamp": "ISO8601",
  "session_id": "string or null",
  "environment": "string"
}
```

**Behavior:**
- Validate and log the error
- Store in SQLite `errors` table (schema below)
- Return 200 OK immediately (don't block the client)
- Optionally rate-limit per session_id or IP to prevent spam

---

## 3. SQLite schema

### `errors` table

```sql
CREATE TABLE errors (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  timestamp TEXT NOT NULL,                    -- ISO8601, when error occurred
  environment TEXT NOT NULL,                  -- 'production', 'staging', etc.
  error_message TEXT NOT NULL,                -- e.g., "Cannot read property 'x'"
  stack_trace TEXT NOT NULL,                  -- Full stack trace
  endpoint TEXT,                              -- URL or backend route (e.g., '/api/users')
  location TEXT,                              -- File:line or component name (extracted from stack)
  user_agent TEXT,                            -- Browser + OS
  session_id TEXT,                            -- User session identifier
  error_level TEXT,                           -- 'error', 'warning', 'fatal'
  error_type TEXT,                            -- 'frontend' or 'backend'
  severity INTEGER DEFAULT 1,                 -- 1=low, 2=medium, 3=high (Opus can set this)
  error_hash TEXT,                            -- SHA256(stack_trace) for quick deduplication
  first_seen TEXT,                            -- ISO8601, first occurrence of this group
  last_seen TEXT,                             -- ISO8601, most recent occurrence
  occurrence_count INTEGER DEFAULT 1,         -- How many times in the time window
  category TEXT,                              -- Set by categorization job (NULL initially)
  created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,
  has_ticket INTEGER DEFAULT 0               -- 1 if a ticket exists for this error group, 0 otherwise
);

CREATE INDEX idx_timestamp ON errors(timestamp);
CREATE INDEX idx_category ON errors(category);
CREATE INDEX idx_error_hash ON errors(error_hash);
CREATE INDEX idx_has_ticket ON errors(has_ticket);
```

### `tickets` table

```sql
CREATE TABLE tickets (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  title TEXT NOT NULL,                        -- Auto-generated by Opus, editable by user
  description TEXT NOT NULL,                  -- How to fix, what changed, Opus analysis
  error_category TEXT NOT NULL,               -- The category this ticket addresses
  error_location TEXT NOT NULL,               -- Where in the code (e.g., 'frontend/auth', 'backend/api/users')
  error_type TEXT NOT NULL,                   -- 'frontend' or 'backend'
  error_count INTEGER,                        -- Total occurrences in the period
  opus_suggestion TEXT,                       -- Raw suggestion from Opus
  code_context TEXT,                          -- Git diff, recent commits, relevant file snippets
  created_at TEXT NOT NULL DEFAULT CURRENT_TIMESTAMP,
  status TEXT DEFAULT 'open',                 -- 'open', 'in_progress', 'resolved', 'ignored'
  resolved_at TEXT,                           -- When marked resolved
  related_errors TEXT                         -- JSON array of error IDs this ticket groups
);

CREATE INDEX idx_status ON tickets(status);
CREATE INDEX idx_created_at ON tickets(created_at);
```

---

## 4. Predefined error categories

The categorization job uses these predefined categories as a starting point. **Opus is encouraged to suggest new categories** if errors don't fit these patterns.

**Predefined categories:**

1. **Auth failure** — Authentication, authorization, token expiry, invalid credentials
2. **Null/undefined reference** — Null pointer, undefined property access, type mismatches
3. **Network error** — Fetch/HTTP failures, timeouts, 4xx/5xx responses, DNS issues
4. **Rate limit** — 429 responses, quota exceeded, throttling
5. **Validation error** — Input validation, schema mismatches, malformed data
6. **Resource not found** — 404, missing file/endpoint, nonexistent record
7. **Unhandled exception** — Unexpected errors, missing error handlers, caught but not handled
8. **Performance/timeout** — Slow operations, stack overflow, memory issues, request timeouts
9. **Data integrity** — Constraint violations, inconsistent state, corrupted records
10. **Configuration/setup** — Missing env vars, incorrect configuration, initialization failures

**New categories:** If Opus encounters errors that don't fit these, it should suggest a new category in the ticket description. Over time, review and add new predefined categories.

---

## 5. Nightly categorization job

**Frequency:** Nightly (e.g., 1 AM)

**Trigger:** Manually via cron or scheduled task runner (e.g., `APScheduler` in Python)

**Steps:**

1. Query all errors where `category IS NULL` and `created_at` >= last run timestamp
2. For each error:
   - Extract likely category from `error_message`, `stack_trace`, and `error_level`
   - Use simple regex or rule-based matching to assign a predefined category
   - If no match, leave `category = NULL` for the analysis job to handle
3. Commit changes to `errors.category`

**Example logic:**
- If message contains "Unauthorized" or "403" → "Auth failure"
- If message contains "null is not an object" or "Cannot read property" → "Null/undefined reference"
- If message contains "timeout" or "ECONNREFUSED" → "Network error"
- If HTTP status 429 → "Rate limit"

---

## 6. Nightly analysis and ticket generation job

**Frequency:** Nightly (e.g., 1:30 AM, after categorization job)

**Tool:** Python with Anthropic SDK, using the Batches API for cost efficiency

**Steps:**

### 6.1 Group errors

Query all errors where `has_ticket = 0` and `category IS NOT NULL` (use categorization job results).

**Grouping strategy:** Errors are grouped by **`(category, error_type, location)`** to avoid treating the same class of error (e.g., "Null/undefined reference in frontend/auth") as separate issues across occurrences.

- `category`: The classified error type (e.g., "Null/undefined reference")
- `error_type`: Whether error originated in frontend or backend
- `location`: Where in the code the error occurred (extracted from stack trace, e.g., `components/LoginForm.tsx` or `services/user_service.py`)

**Deduplication within a group:**
- Count total occurrences of errors in this group (sum `occurrence_count` from all matching error rows)
- Collect **representative error instances** (e.g., top 3 most recent or most severe)
- Discard duplicate raw instances within the group to avoid redundant analysis
- **Opus sees only a count + sample errors, not every instance**

**Output:** A list of error groups, each with:
- `category`
- `error_type`
- `location`
- `total_count` (total occurrences)
- `sample_errors` (list of representative error details)
- `time_window` (when these errors occurred)

### 6.2 Gather context for Opus

For each error group, collect **surrounding code and git context** so Opus can make informed suggestions:

**For frontend errors:**
- The exact error messages and stack traces from the sample errors
- Extract file path + line number from stack trace
- Fetch the relevant code snippet from the file (±10 lines from the error location)
- Get recent commits affecting that file: `git log --oneline -n 10 -- <file>`
- Get git diff of recent changes: `git diff HEAD~5 -- <file>` (last 5 commits)
- Browser/environment info from user_agent

**For backend errors:**
- The exact error messages and stack traces
- Extract file path + line number from stack trace
- Fetch the relevant code snippet from the file (±10 lines from the error location)
- Get recent commits affecting that file: `git log --oneline -n 10 -- <file>`
- Get git diff of recent changes: `git diff HEAD~5 -- <file>`
- Related logs or request context (if available)

**Data structure sent to Opus:**
```json
{
  "category": "Null/undefined reference",
  "error_type": "frontend",
  "location": "components/LoginForm.tsx",
  "total_count": 47,
  "time_window": "2026-09-18 to 2026-09-19",
  "sample_errors": [
    {
      "error_message": "Cannot read property 'email' of undefined",
      "stack_trace": "...",
      "timestamp": "2026-09-19T02:15:00Z",
      "url": "http://app/login"
    },
    {
      "error_message": "Cannot read property 'email' of undefined",
      "stack_trace": "...",
      "timestamp": "2026-09-19T01:30:00Z",
      "url": "http://app/login"
    }
  ],
  "code_context": {
    "file": "components/LoginForm.tsx",
    "snippet": "const LoginForm = ({ user }) => {\n  return <div>{user.email}</div>;\n}",
    "recent_commits": [
      "abc1234 - Fix login form prop handling",
      "def5678 - Update auth flow"
    ],
    "git_diff": "..."
  }
}
```

### 6.3 Send to Opus via Batches API

Batch all error groups into a single Batches API request (for cost efficiency).

**Prompt for Opus:**

```
You are analyzing error logs from a web application (Föräldraledig Plannerare).
For each error group below, provide:

Root cause analysis
Suggested fix (code changes or steps)
Affected systems/users
Priority (low, medium, high)

Error group:
{error data from 6.2}

Respond in JSON format:
{
"title": "Brief title for the issue",
"root_cause": "What caused this",
"suggested_fix": "How to fix it (code changes or steps)",
"affected_scope": "Frontend/backend, number of users, systems impacted",
"priority": "low|medium|high",
"reasoning": "Why this priority"
}
```


### 6.4 Parse Opus response and create tickets

For each error group with Opus analysis:

1. Create a row in `tickets` table with:
   - `title` = Opus's suggested title
   - `description` = Combination of root cause, suggested fix, and affected scope
   - `error_category` = The error group's category
   - `error_location` = The file/component location
   - `error_type` = frontend or backend
   - `error_count` = total_count from the group
   - `opus_suggestion` = Full raw response from Opus
   - `code_context` = Git diff and file snippets (for reference)
   - `related_errors` = JSON array of error IDs in this group

2. Update all error rows in the group: `has_ticket = 1`

3. **After ticket creation:** Delete all error rows where `has_ticket = 1` (cleanup raw errors once addressed by a ticket)

---

## 7. Frontend dashboard MVP

**Technology:** React + Vite (part of Föräldraledig Plannerare frontend)

**Access:** Run with `npm run observability` or similar command

**Features:**

### View
- **Tickets list**: All open/in-progress tickets (paginated, ~20 per page)
- **Ticket detail**: Expand to show full description, error context, code snippets, priority
- **Filter**:
  - By status: open, in_progress, resolved, ignored
  - By error_type: frontend, backend
  - By error_category: (all predefined categories + custom)
  - By date range
- **Search**: By ticket title or description (substring match)
- **Trends chart**: Line chart showing error count over time (last 7 days)

### Actions
- **Edit ticket**: User can update `title` and `description` (e.g., clarify the fix, add notes)
- **Change status**: Mark as in_progress, resolved, ignored
- **Delete ticket**: Remove a bad or duplicate ticket
- **View related errors**: Show the raw error logs (with timestamps, counts) that this ticket groups

### Data source
- All ticket data queried from backend `/tickets` GET endpoint (with optional filters)
- All error data queried from backend `/errors` GET endpoint (with filters)

---

## 8. Backend API endpoints (existing app backend)

### `POST /errors`
*Receive error reports from the frontend SDK*

Request: Error object (see Section 2)
Response: `{ success: true }`

---

### `GET /tickets`
*Fetch tickets with optional filters*

Query params:
- `status` (open, in_progress, resolved, ignored)
- `error_type` (frontend, backend)
- `error_category` (predefined category name)
- `date_from` (ISO8601)
- `date_to` (ISO8601)
- `search` (substring search in title/description)
- `page` (pagination, default 1)
- `limit` (per page, default 20)

Response:
```json
{
  "tickets": [
    {
      "id": 1,
      "title": "Login form crashes on null user object",
      "description": "...",
      "error_category": "Null/undefined reference",
      "error_type": "frontend",
      "error_count": 47,
      "status": "open",
      "created_at": "2026-09-19T01:45:00Z",
      "code_context": "..."
    }
  ],
  "total": 42,
  "page": 1
}
```

---

### `GET /errors`
*Fetch raw error logs (for dashboard context)*

Query params:
- `ticket_id` (optional; show errors related to a specific ticket)
- `category` (optional; filter by category)
- `error_type` (optional; frontend or backend)
- `date_from`, `date_to` (optional; filter by date range)
- `has_ticket` (optional; 0 or 1)
- `limit` (default 100)

Response:
```json
{
  "errors": [
    {
      "id": 123,
      "timestamp": "2026-09-19T02:15:00Z",
      "error_message": "Cannot read property 'email' of undefined",
      "location": "components/LoginForm.tsx",
      "occurrence_count": 5,
      "category": "Null/undefined reference"
    }
  ],
  "total": 200
}
```

---

### `PATCH /tickets/:id`
*Update a ticket (title, description, status)*

Request:
```json
{
  "title": "Updated title",
  "description": "Updated description",
  "status": "in_progress"
}
```

Response: Updated ticket object

---

### `DELETE /tickets/:id`
*Delete a ticket*

Response: `{ success: true }`

After deletion, the related error rows should have `has_ticket = 0` set again (so they can be re-analyzed in a future batch run if desired).

---

## 9. Summary of jobs and scheduling

| Job | Frequency | Trigger | Input | Output |
|-----|-----------|---------|-------|--------|
| **Categorization** | Nightly (1 AM) | Cron/scheduler | Uncategorized errors | Errors with `category` set |
| **Analysis + ticket generation** | Nightly (1:30 AM) | Cron/scheduler | Categorized errors + git context | New tickets, deletion of raw errors with tickets |

---

## 10. Implementation notes

- **Python setup**: Use Anthropic Python SDK for Batches API calls
- **Scheduling**: Use APScheduler or similar to run jobs on a schedule
- **Error deduplication**: Use SHA256(stack_trace) for quick lookups; grouping by (category, error_type, location) for analysis
- **Cost efficiency**: Batches API ensures Opus analysis is cost-effective at scale
- **Self-hosting**: All components (SDKs, backend, jobs, frontend) are self-hosted on the existing Föräldraledig Plannerare infrastructure