workspace/.claude/CLAUDE.md

# CLAUDE.md

Guide Claude Code when work in repo.

## Keeping This File Up to Date

Update when lesson learned. Update when:

- Non-obvious arch decision made or found
- Gotcha, footgun, surprising behavior hit (+ fix/workaround)
- New command, workflow, tool added
- Convention set that not obvious from code
- Integration detail clarified (IPC protocol behavior, agent tool call edge cases)

Do **not** add derivable-from-code things, generic best practices, or ephemeral task notes — durable knowledge only.

## graphify

This project has a graphify knowledge graph at graphify-out/.

Rules:
- Before answering architecture or codebase questions, read graphify-out/GRAPH_REPORT.md for god nodes and community structure
- If graphify-out/wiki/index.md exists, navigate it instead of reading raw files
- For cross-module "how does X relate to Y" questions, prefer `graphify query "<question>"`, `graphify path "<A>" "<B>"`, or `graphify explain "<concept>"` over grep — these traverse the graph's EXTRACTED + INFERRED edges instead of scanning files
- After modifying code files in this session, run `graphify update .` to keep the graph current (AST-only, no API cost)

## Repository Layout

**Monorepo with git submodules.** Each submodule independent repo with own `.claude/CLAUDE.md`.

| Directory | What | Submodule |
|-----------|------|-----------|
| **`adiuvAI/`** | Electron desktop app (TypeScript/React) | `git.muticolturano.com/adiuvAI/adiuvAI` |
| **`api/`** | FastAPI backend (Python) | `git.muticolturano.com/adiuvAI/api` |
| **`docs/`** | Planning docs & working memory (not a submodule) | -- |

After clone, run `git submodule update --init --recursive` to populate submodules.

---

## adiuvAI (Electron App)

> **Detailed docs**: `adiuvAI/.claude/CLAUDE.md` — commands, architecture, AI subsystem, design context, conventions.

### Commands

```bash
cd adiuvAI
npm run start            # Dev (Electron + Vite)
npm run lint             # ESLint
npm run knip             # Dead code analysis
npm run make             # Build installers (Win/Linux/macOS)
npm run package          # Package without installers
npm run dev:web          # Standalone web SPA dev
npm run build:web        # Build standalone SPA → dist-web/
npm run preview:web      # Preview built web SPA
npx drizzle-kit generate # Generate migration from schema
npx drizzle-kit push     # Push schema directly (dev only)
```

No test suite currently.

### Architecture

```
Renderer (React 19 + TanStack Router)
  ↓ custom ipcLink (NOT electron-trpc — incompatible with tRPC v11)
Preload (contextBridge: window.electronTRPC + window.electronAI)
  ↓ IPC channels
Main Process (Node.js)
  ├── tRPC router (CRUD + AI proxy procedures)
  ├── SQLite (better-sqlite3 + Drizzle ORM, WAL mode)
  └── Backend delegation layer (orchestrator.ts forwards to FastAPI WS)
```

**Local-first storage, cloud AI.** All user data (clients, projects, tasks, notes, timelines) in local SQLite. AI lives entirely on the FastAPI backend — Electron orchestrator is a thin delegation shell that forwards to `/api/v1/device` WS and dispatches v3 typed stream frames + tool-call ↔ DrizzleExecutor round-trips back to renderer.

**IPC channels**:
- `'trpc'` — bidirectional tRPC request/response (all CRUD + auth + scout + memory proxy)
- `'ai:stream'` — one-way v3 stream frames main → renderer
- `'ai:action'` — AI side-effects (e.g. agent auto-creates task)

**Main process layout (`src/main/`)**:
- `index.ts` — Window creation, app lifecycle, protocol handler
- `ipc.ts` — Custom tRPC↔IPC bridge
- `store.ts` — electron-store for `FormatPrefs` + `uiLanguage`; exports `getUiLanguage()`
- `router/index.ts` — All tRPC sub-routers (~1627 LOC)
- `db/schema.ts` — 10 tables: clients, projects, tasks, timelineEvents, timelineEventDependencies, notes, noteEdits, taskComments, scoutRuns, scoutRunActions
- `db/index.ts` — Drizzle + better-sqlite3 (WAL), singleton `getDb()`, `initDb()` migrations
- `db/notes-backfill.ts` — Startup backfill: generates `aiSummary` for notes with null summary
- `ai/orchestrator.ts` — Thin backend-delegation layer (~304 LOC). Connectivity/auth guard → `BackendClient.sendHomeRequest()` / `sendFloatingRequest()` → forwards v3 stream frames to renderer. Also schedules daily-brief regeneration.
- `ai/token.ts` — Two-tier token storage (safeStorage + electron-store fallback)
- `scouts/scout-scheduler.ts` — Local scout scheduling (filesystem scouts)
- `api/backend-client.ts` — WS client to FastAPI: handles tool-call round-trips, v3 stream frame dispatch, journey + scout proxies
- `api/drizzle-executor.ts` — Executes backend-issued tool calls against local SQLite. Wraps results through `formatRow()`/`formatRows()` using user FormatPrefs
- `auth/auth-manager.ts` — Login, register, logout, OAuth flow (singleton)
- `auth/backup-key.ts` — Device-specific AES-256 backup key (safeStorage, not password-derived)
- `auth/locale-defaults.ts` — Detects timezone, date/time format, language from OS locale

**tRPC routers** (in `appRouter`): `health`, `settings`, `clients`, `projects`, `tasks`, `timelineEvents`, `timelineEventDependencies`, `notes`, `noteEdits`, `taskComments`, `ai`, `auth`, `scout` (with `local` / `cloud` / `journey` sub-routers), `memory`.

**Renderer** (`src/renderer/`): file-based routing via TanStack Router (`routeTree.gen.ts` auto-generated). shadcn/ui new-york theme, neutral colors. Path alias `@/*` → `src/renderer/*`. Notes editor: Milkdown (`@milkdown/crepe`).

**Non-obvious details**:
- `electron-trpc` NOT used — custom IPC bridge (`ipc.ts` + `lib/ipcLink.ts`) because electron-trpc bundles tRPC v10 internals
- Vite configs use `.mts` extension to avoid ESM/CJS conflicts with electron-forge
- `forge.config.ts` has cross-compilation hooks (downloads platform-specific native binaries for better-sqlite3)
- DB has no foreign key constraints — cascade deletes in tRPC procedures
- Timestamps are milliseconds (`Date.getTime()`), not ISO strings
- Notes use `aiSummary` (≤250 char, backend `gpt-4o-mini` via `POST /api/v1/scouts/notes/summarize`) for AI navigation — LanceDB fully removed
- AI note edits go through `noteEdits` HITL table (`type: append|insert|replace`, `status: pending|approved|rejected`); backend tool `propose_note_edit` → drizzle-executor inserts row; user approves/rejects in UI; auto-reject on missing anchor
- `checkpoints` table replaced by `timelineEvents` + `timelineEventDependencies` (events are typed `milestone|checkpoint|activity`, with optional dep edges)
- `scoutRuns` + `scoutRunActions` populated by backend-client on tool_call/run_complete frames; UI reads via `scout.runs` / `scout.runActions`

**Settings Page (shared Electron + Web)**:
- Settings page runs in **both** Electron and standalone web SPA. Same React components — no duplication.
- **Platform Adapter**: `PlatformProvider` context (`src/renderer/lib/platform.tsx`) exposes `isElectron`/`isWeb`/`hasLocalAgents`/`hasFileDialog`. Components use `usePlatform()` to gate Electron-only features.
- **Web build**: `vite.web.config.mts` → `dist-web/`. Entry: `web.html` → `src/renderer/web-main.tsx` (uses `httpBatchLink` via `lib/httpLink.ts` instead of `ipcLink`).
- **Electron-only gating**: Device ID card and local scout filesystem gated behind `platform.isElectron`. On web: visible but disabled, not hidden.
- **Gotcha**: Do NOT add Electron-specific settings (server URL, native file pickers) without wrapping in `platform.isElectron`. Same component tree renders on web.

**Onboarding Wizard**:
- First-run wizard collects 5 fields: `job_role`, `industry`, `primary_use_case`, `tone_preference`, `language`. Plus `user_name` from `name`+`surname`.
- All fields stored as encrypted core memory (backend `MemoryMiddleware`), not local electron-store.
- `onboarding_completed_at` on `users` table (nullable TIMESTAMPTZ) gates flow — `null` = show wizard, non-null = skip.
- `AppShell.tsx` gates: if `profile.onboardingCompletedAt == null` → render `<OnboardingFlow>` instead of app.
- `auth.status` tRPC procedure auto-seeds `language` and `user_name` into MemoryCore if missing (fire-and-forget `.catch(() => {})`).
- Format prefs (timezone, dateFormat, timeFormat) stored in electron-store (`FormatPrefs`), not core memory — device-specific.
- `drizzle-executor.ts` wraps all query results through `formatRow()`/`formatRows()` using user FormatPrefs.
- Settings > Profile allows post-onboarding edit of all fields + format prefs.
- **Gotcha — shadcn Button `outline` variant in dark mode**: variant defines `dark:bg-input/30 dark:border-input dark:hover:bg-input/50` — overrides custom `className` background. Fix: switch between `variant="default"` and `variant="outline"` instead of className overrides.
- **Gotcha — locale codes vs human names**: `app.getLocale()` and `navigator.language` return codes like `en-US`. Use `Intl.DisplayNames(undefined, { type: 'language' })` to convert to "English". Must do in both main process (`locale-defaults.ts`) and renderer (`OnboardingFlow.tsx`).

**i18n (Internationalization)**:
- `i18next` + `react-i18next` with bundled JSON translations (no lazy loading).
- Config in `src/renderer/i18n.ts`. 5 languages: EN, IT, ES, FR, DE. `SUPPORTED_LANGUAGES` exported for UI selectors.
- Translation files: `src/renderer/locales/{en,it,es,fr,de}/translation.json`. Namespaces: `nav`, `auth`, `tasks`, `settings`, `common`, `errors`, `home`, `timeline`, `projects`, `scouts`.
- **`common.*` namespace** holds shared labels (`save`, `cancel`, `delete`, `edit`, `add`, `rename`, `saving`, `deleting`, `creating`, `renameDescription`, `deleteTitle`). Check `common.*` before adding new key.
- Pluralization uses i18next `_one`/`_other` suffixes.
- `LanguageSync` component in `src/renderer/index.tsx` reads persisted `uiLanguage` from electron-store via tRPC on startup, syncs to i18next.
- Language selector in `GeneralSection.tsx` (Settings > General). On change: (1) calls `i18n.changeLanguage()`, (2) persists to electron-store via `setUiLanguage` mutation, (3) writes to backend core memory so AI responds in same language.
- `getUiLanguage()` exported from `src/main/store.ts`.
- Static data arrays needing translation use `labelKey` pattern: store translation key, call `t(labelKey)` at render. Used in `NAV_ITEMS`, `COLUMNS`, `SECTIONS`, `SUGGESTION_CHIPS`.
- When adding new translated text: add key to **all 5** JSON files. Keep `common.*` consistent across all languages.

**Google OAuth (adiuvAI side)**:
- `adiuvai://` NOT accepted by Google as redirect URI — Google only accepts `http://localhost` or `https://`. API backend exposes `GET /auth/oauth/google/web-callback` which receives Google redirect and bounces to `adiuvai://oauth/callback?...`. Redirect URI in Google Cloud Console points to backend, not Electron app.
- `app.requestSingleInstanceLock()` required for `second-instance` event on Windows/Linux. If returns `false`, call `app.quit()` immediately.
- In dev (`process.defaultApp === true`), `setAsDefaultProtocolClient('adiuvai')` must include `[path.resolve(process.argv[1])]` as third arg so OS protocol registration includes entry script.
- `loginWithOAuth` uses `fetch()` directly (not `this.get()`) — authorize endpoint is public, `get()` throws when not authenticated.
- Backup key in `backup-key.ts` stored in `encryptedTokens` under key `backup_key`, reusing `getToken/setToken` from `token.ts`. Device-bound, never password-derived — social-login users can use backup features.

---

## api (FastAPI Backend)

### Commands

```bash
cd api

# Development
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000

# Production
gunicorn app.main:app -k uvicorn.workers.UvicornWorker -w 4 --timeout 120

# Database migrations
alembic upgrade head

# Testing
pytest                                       # all tests
pytest -v                                    # verbose
pytest tests/test_deep_agent.py              # single file
pytest tests/test_deep_agent.py -k test_name # single test

# Linting/formatting
ruff check .
ruff format .

# Docker (full stack)
docker compose up --build
```

### Architecture

```
FastAPI app (app/main.py)
├── Lifespan: APScheduler crons (memory hourly + audit weekly) when SCHEDULER_ENABLED
├── Middleware: TierRateLimit → Sanitizer → CORS
├── HTTP Routes (app/api/routes/) — all under /api/v1
│   ├── auth.py        — register, login, refresh, profile, OAuth, onboarding, password
│   ├── chat.py        — POST /chat, /chat/brief, /chat/embed
│   ├── scouts.py      — catalog, can-create, trigger, notes/summarize
│   ├── scout_setup.py — guided scout setup (journey)
│   ├── billing.py     — Stripe checkout, webhook, subscription, invoices
│   ├── device_ws.py   — WS /device (unified streaming endpoint: home, floating, brief, journey)
│   └── memory.py      — core / relational / forget-all
├── Agent System (app/agents/)
│   ├── task_agent.py
│   ├── project_agent.py
│   ├── note_agent.py
│   ├── timeline_agent.py
│   └── filesystem_agent.py
├── Core (app/core/)
│   ├── deep_agent.py            — main agent runner (run_home / run_floating / run_brief / run_journey)
│   ├── brief_agent.py           — daily brief generation
│   ├── agent_runner.py          — local + cloud agent run executor
│   ├── agent_session_buffer.py  — per-session conversation buffer
│   ├── agent_registry.py        — decorator-based agent registry
│   ├── llm.py                   — LiteLLM factory (multi-provider)
│   ├── memory_middleware.py     — encrypted core memory read/write
│   ├── memory_extraction.py     — LLM extraction from conversation tail
│   ├── memory_maintenance.py    — drain queue, contradiction audit, proactive mining
│   ├── note_summarizer.py       — gpt-4o-mini summary for notes
│   ├── output_formatter.py      — render agent output to user-facing markdown
│   ├── embeddings.py
│   ├── device_manager.py        — device registration / WS session tracking
│   ├── ws_context.py            — per-WS user context plumbing
│   ├── langfuse_client.py       — Langfuse prompt + tracing client
│   └── preprocessors/           — input preprocessors (e.g. email_html.py)
├── Auth (app/auth/oauth_providers.py) — GoogleOAuthProvider (httpx + manual PKCE)
├── Billing (app/billing/) — tier_manager + stripe_service
├── Integrations (app/integrations/) — gmail.py, ms_graph.py
└── Models (app/models.py) — SQLAlchemy 2.0 ORM
```

**HTTP route prefix**: every router included with `prefix="/api/v1"`. So `/api/v1/auth/...`, `/api/v1/chat`, `/api/v1/scouts/...`, `/api/v1/memory/...`, `/api/v1/device` (WS).

**ORM models** (`app/models.py`): `User`, `RefreshToken`, `OAuthAccount`, `Subscription`, `LocalScoutConfig`, `CloudScoutConfig`, `ScoutRunLog`, `MemoryCore`, `MemoryAssociative`, `MemoryEpisodic`, `MemoryProactive`, `ExtractionQueue`, `MemoryRelation`, `Plugin`. PostgreSQL (asyncpg + SQLAlchemy 2.0 async). Alembic migrations in `alembic/versions/`.

**Lifespan crons** (only if `settings.SCHEDULER_ENABLED`):
- `_memory_cron_tick` — hourly: drains Free-tier extraction queue + mines proactive patterns for Power+ users
- `_memory_audit_cron_tick` — weekly: contradiction scan + label canonicalization for all users (Phase 7)

**LLM routing**: backend agents own all intelligence. Tool calls describe client-side ops (JSON) → Electron `drizzle-executor` runs them against local SQLite → result returned to backend over WS. Tool loop cap inside agent runner prevents runaway iteration.

**Zero-trust data model**: backend never stores raw user content. PostgreSQL holds auth, billing, plugin metadata, encrypted memory (Core/Associative/Episodic/Proactive/Relational), scout configs, run logs.

**Config**: `app/config/settings.py` — all env vars via Pydantic Settings. Copy `.env.example` to `.env` for local dev.

**Testing**: pytest + pytest-asyncio. Fixtures in `tests/conftest.py`. Active suites: agent runner, auth, brief/deep agents, device WS, integrations, journey, memory (audit/extraction/middleware/models/proactive/relations), middleware, output formatter, preprocessors, schemas, ws_unified.

### Non-obvious details

- **Tier from DB, not JWT**: `get_current_user` decodes JWT but fetches authoritative tier from `subscriptions` — tier changes take effect immediately, no re-login needed
- **Refresh tokens hashed**: plaintext returned to client, stored as SHA-256 in DB — server can never retrieve plaintext (intentional)
- **WebSocket auth via query param**: `?token=<jwt>` instead of Bearer header (WebSocket handshake limitation)
- **Unified device WS**: `/api/v1/device` is the single bidirectional channel. Handles home requests, floating requests, daily briefs, journeys, heartbeats. Tool calls round-trip through the same socket
- **Prompt IP protection**: prompts kept server-side via Langfuse (`langfuse_client`). `SanitizerMiddleware` strips leaked fragments from responses
- **Agents don't execute operations**: tools return JSON describing client-side ops — Electron client executes against local SQLite
- **Alembic async/sync split**: app uses `postgresql+asyncpg`, Alembic CLI needs `postgresql+psycopg2` — `env.py` handles URL conversion
- **CORS includes `app://`**: Electron uses custom `app://` protocol, not http/https
- **Run-disconnect tracking**: `_mark_runs_disconnected` flips active runs when WS drops so client can resume cleanly

**Onboarding (API side)**:
- `PUT /auth/me/memory` — updates core memory k/v pairs, optionally marks onboarding complete (`mark_onboarded: true` sets `users.onboarding_completed_at`).
- `POST /auth/me/onboarding/reset` — nullifies `onboarding_completed_at` so wizard re-runs.
- `POST /auth/onboarding/normalize` — LLM-normalizes free-text onboarding inputs via `gpt-4o-mini`; returns inputs unchanged on error.
- `get_current_user()` in `auth.py` middleware decrypts core memory blocks, includes in `UserProfile.memory` dict.
- `users.onboarding_completed_at` — nullable TIMESTAMPTZ, returned as epoch ms (int) in UserProfile schema.

**i18n (API side)**:
- `_language_instruction()` in `app/core/deep_agent.py` reads user's `language` from `MemoryCore`, appends system prompt directive ("Always respond in {language}") to all `run_*` functions.
- Electron client writes chosen language to backend core memory on change — API picks up on next agent call.

**Google OAuth (api side)**:
- OAuth routes in `app/api/routes/auth.py`: `GET /auth/oauth/{provider}/authorize`, `POST /auth/oauth/{provider}/callback`, `GET /auth/oauth/{provider}/web-callback` (bounces to deep link, excluded from OpenAPI schema).
- Provider abstraction in `app/auth/oauth_providers.py` — `GoogleOAuthProvider` uses `httpx` directly (no `authlib`). PKCE S256 implemented manually via `generate_pkce_pair()`.
- `_pending_states` dict in `routes/auth.py` is **in-memory** — works for single-process dev, doesn't survive restarts, doesn't scale to multiple workers. Replace with Redis in production.
- `users.password_hash` is **nullable** — social-only users have `password_hash=None`. `await db.flush()` required before creating linked `OAuthAccount` to populate `new_user.id` before commit.
- `OAUTH_REDIRECT_URI` must point to **API backend** (e.g. `https://api.adiuvai.com/...`).
- **Unverified email + existing account = 409**: if `email_verified=False` and email already registered, callback returns 409. Without this guard, branch 3 would INSERT duplicate email and crash with DB constraint violation (500).
- **Testing OAuth routes**: mock `GoogleOAuthProvider.exchange_code` and `get_userinfo` with `patch.object(..., new=AsyncMock(...))` — works because FastAPI instantiates new provider per request. Use `monkeypatch.setattr(settings, "GOOGLE_AUTH_CLIENT_ID", ...)` to simulate configured credentials without restart.

### Tier System

Source of truth: `app/billing/tier_manager.py` (`FEATURES` + `RATE_LIMITS` dicts).

| Feature             | Free   | Pro       | Power     | Team      |
|---------------------|--------|-----------|-----------|-----------|
| Rate limit          | 20/min | 60/min    | 120/min   | 200/min   |
| Providers           | 1      | unlimited | unlimited | unlimited |
| Relational memory   | no     | yes       | yes       | yes       |
| Proactive mining    | no     | no        | yes       | yes       |

`tier_manager.get_tier()` falls back to `'power'` in dev (`settings.ENV == 'dev'`) when no subscription found, else `'free'`. Enforced in `app/api/middleware/rate_limit.py` (sliding window) and `tier_manager.check_feature()` calls scattered through agent + memory paths.

---

## Cross-Project Integration

Electron app and FastAPI backend communicate via **WebSocket** (`/api/v1/device`):

1. Electron connects with `?token=<jwt>` query param
2. Client sends typed request frames (home / floating / brief / journey_start / journey_message)
3. Server streams v3 typed frames (text deltas, tool_call, run_complete, error)
4. Tool call frames → Electron `drizzle-executor` runs against local SQLite → returns `tool_result` over same socket
5. Heartbeat loop keeps connection alive; backend marks runs disconnected on drop

There is no fully-local AI fallback — the Electron orchestrator is a thin delegation shell that requires connectivity + auth. If offline or logged out, `checkConnectivity()` short-circuits with a user-facing error.

---

## MCP Servers

- **Langfuse Docs** (`https://langfuse.com/api/mcp`) — workspace-level, prompt management docs
- **shadcn** (`npx shadcn@latest mcp`) — configured in `adiuvAI/` for UI component generation