Files
workspace/.claude/CLAUDE.md
Roberto 596e5551f8 chore: bump submodules — Phase 1 scouts rename complete
api: agent_ids → scout_ids in device_hello WS frame + tests
adiuvAI: CloudAgentConfig → CloudScoutConfig, agentIds → scoutIds
.claude/CLAUDE.md: update all scout-subsystem doc references

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-16 01:50:48 +02:00

22 KiB

CLAUDE.md

Guide Claude Code when work in repo.

Keeping This File Up to Date

Update when lesson learned. Update when:

  • Non-obvious arch decision made or found
  • Gotcha, footgun, surprising behavior hit (+ fix/workaround)
  • New command, workflow, tool added
  • Convention set that not obvious from code
  • Integration detail clarified (IPC protocol behavior, agent tool call edge cases)

Do not add derivable-from-code things, generic best practices, or ephemeral task notes — durable knowledge only.

graphify

This project has a graphify knowledge graph at graphify-out/.

Rules:

  • Before answering architecture or codebase questions, read graphify-out/GRAPH_REPORT.md for god nodes and community structure
  • If graphify-out/wiki/index.md exists, navigate it instead of reading raw files
  • For cross-module "how does X relate to Y" questions, prefer graphify query "<question>", graphify path "<A>" "<B>", or graphify explain "<concept>" over grep — these traverse the graph's EXTRACTED + INFERRED edges instead of scanning files
  • After modifying code files in this session, run graphify update . to keep the graph current (AST-only, no API cost)

Repository Layout

Monorepo with git submodules. Each submodule independent repo with own .claude/CLAUDE.md.

Directory What Submodule
adiuvAI/ Electron desktop app (TypeScript/React) git.muticolturano.com/adiuvAI/adiuvAI
api/ FastAPI backend (Python) git.muticolturano.com/adiuvAI/api
docs/ Planning docs & working memory (not a submodule) --

After clone, run git submodule update --init --recursive to populate submodules.


adiuvAI (Electron App)

Detailed docs: adiuvAI/.claude/CLAUDE.md — commands, architecture, AI subsystem, design context, conventions.

Commands

cd adiuvAI
npm run start            # Dev (Electron + Vite)
npm run lint             # ESLint
npm run knip             # Dead code analysis
npm run make             # Build installers (Win/Linux/macOS)
npm run package          # Package without installers
npm run dev:web          # Standalone web SPA dev
npm run build:web        # Build standalone SPA → dist-web/
npm run preview:web      # Preview built web SPA
npx drizzle-kit generate # Generate migration from schema
npx drizzle-kit push     # Push schema directly (dev only)

No test suite currently.

Architecture

Renderer (React 19 + TanStack Router)
  ↓ custom ipcLink (NOT electron-trpc — incompatible with tRPC v11)
Preload (contextBridge: window.electronTRPC + window.electronAI)
  ↓ IPC channels
Main Process (Node.js)
  ├── tRPC router (CRUD + AI proxy procedures)
  ├── SQLite (better-sqlite3 + Drizzle ORM, WAL mode)
  └── Backend delegation layer (orchestrator.ts forwards to FastAPI WS)

Local-first storage, cloud AI. All user data (clients, projects, tasks, notes, timelines) in local SQLite. AI lives entirely on the FastAPI backend — Electron orchestrator is a thin delegation shell that forwards to /api/v1/device WS and dispatches v3 typed stream frames + tool-call ↔ DrizzleExecutor round-trips back to renderer.

IPC channels:

  • 'trpc' — bidirectional tRPC request/response (all CRUD + auth + scout + memory proxy)
  • 'ai:stream' — one-way v3 stream frames main → renderer
  • 'ai:action' — AI side-effects (e.g. agent auto-creates task)

Main process layout (src/main/):

  • index.ts — Window creation, app lifecycle, protocol handler
  • ipc.ts — Custom tRPC↔IPC bridge
  • store.ts — electron-store for FormatPrefs + uiLanguage; exports getUiLanguage()
  • router/index.ts — All tRPC sub-routers (~1627 LOC)
  • db/schema.ts — 10 tables: clients, projects, tasks, timelineEvents, timelineEventDependencies, notes, noteEdits, taskComments, scoutRuns, scoutRunActions
  • db/index.ts — Drizzle + better-sqlite3 (WAL), singleton getDb(), initDb() migrations
  • db/notes-backfill.ts — Startup backfill: generates aiSummary for notes with null summary
  • ai/orchestrator.ts — Thin backend-delegation layer (~304 LOC). Connectivity/auth guard → BackendClient.sendHomeRequest() / sendFloatingRequest() → forwards v3 stream frames to renderer. Also schedules daily-brief regeneration.
  • ai/token.ts — Two-tier token storage (safeStorage + electron-store fallback)
  • scouts/scout-scheduler.ts — Local scout scheduling (filesystem scouts)
  • api/backend-client.ts — WS client to FastAPI: handles tool-call round-trips, v3 stream frame dispatch, journey + scout proxies
  • api/drizzle-executor.ts — Executes backend-issued tool calls against local SQLite. Wraps results through formatRow()/formatRows() using user FormatPrefs
  • auth/auth-manager.ts — Login, register, logout, OAuth flow (singleton)
  • auth/backup-key.ts — Device-specific AES-256 backup key (safeStorage, not password-derived)
  • auth/locale-defaults.ts — Detects timezone, date/time format, language from OS locale

tRPC routers (in appRouter): health, settings, clients, projects, tasks, timelineEvents, timelineEventDependencies, notes, noteEdits, taskComments, ai, auth, scout (with local / cloud / journey sub-routers), memory.

Renderer (src/renderer/): file-based routing via TanStack Router (routeTree.gen.ts auto-generated). shadcn/ui new-york theme, neutral colors. Path alias @/*src/renderer/*. Notes editor: Milkdown (@milkdown/crepe).

Non-obvious details:

  • electron-trpc NOT used — custom IPC bridge (ipc.ts + lib/ipcLink.ts) because electron-trpc bundles tRPC v10 internals
  • Vite configs use .mts extension to avoid ESM/CJS conflicts with electron-forge
  • forge.config.ts has cross-compilation hooks (downloads platform-specific native binaries for better-sqlite3)
  • DB has no foreign key constraints — cascade deletes in tRPC procedures
  • Timestamps are milliseconds (Date.getTime()), not ISO strings
  • Notes use aiSummary (≤250 char, backend gpt-4o-mini via POST /api/v1/scouts/notes/summarize) for AI navigation — LanceDB fully removed
  • AI note edits go through noteEdits HITL table (type: append|insert|replace, status: pending|approved|rejected); backend tool propose_note_edit → drizzle-executor inserts row; user approves/rejects in UI; auto-reject on missing anchor
  • checkpoints table replaced by timelineEvents + timelineEventDependencies (events are typed milestone|checkpoint|activity, with optional dep edges)
  • scoutRuns + scoutRunActions populated by backend-client on tool_call/run_complete frames; UI reads via scout.runs / scout.runActions

Settings Page (shared Electron + Web):

  • Settings page runs in both Electron and standalone web SPA. Same React components — no duplication.
  • Platform Adapter: PlatformProvider context (src/renderer/lib/platform.tsx) exposes isElectron/isWeb/hasLocalAgents/hasFileDialog. Components use usePlatform() to gate Electron-only features.
  • Web build: vite.web.config.mtsdist-web/. Entry: web.htmlsrc/renderer/web-main.tsx (uses httpBatchLink via lib/httpLink.ts instead of ipcLink).
  • Electron-only gating: Device ID card and local scout filesystem gated behind platform.isElectron. On web: visible but disabled, not hidden.
  • Gotcha: Do NOT add Electron-specific settings (server URL, native file pickers) without wrapping in platform.isElectron. Same component tree renders on web.

Onboarding Wizard:

  • First-run wizard collects 5 fields: job_role, industry, primary_use_case, tone_preference, language. Plus user_name from name+surname.
  • All fields stored as encrypted core memory (backend MemoryMiddleware), not local electron-store.
  • onboarding_completed_at on users table (nullable TIMESTAMPTZ) gates flow — null = show wizard, non-null = skip.
  • AppShell.tsx gates: if profile.onboardingCompletedAt == null → render <OnboardingFlow> instead of app.
  • auth.status tRPC procedure auto-seeds language and user_name into MemoryCore if missing (fire-and-forget .catch(() => {})).
  • Format prefs (timezone, dateFormat, timeFormat) stored in electron-store (FormatPrefs), not core memory — device-specific.
  • drizzle-executor.ts wraps all query results through formatRow()/formatRows() using user FormatPrefs.
  • Settings > Profile allows post-onboarding edit of all fields + format prefs.
  • Gotcha — shadcn Button outline variant in dark mode: variant defines dark:bg-input/30 dark:border-input dark:hover:bg-input/50 — overrides custom className background. Fix: switch between variant="default" and variant="outline" instead of className overrides.
  • Gotcha — locale codes vs human names: app.getLocale() and navigator.language return codes like en-US. Use Intl.DisplayNames(undefined, { type: 'language' }) to convert to "English". Must do in both main process (locale-defaults.ts) and renderer (OnboardingFlow.tsx).

i18n (Internationalization):

  • i18next + react-i18next with bundled JSON translations (no lazy loading).
  • Config in src/renderer/i18n.ts. 5 languages: EN, IT, ES, FR, DE. SUPPORTED_LANGUAGES exported for UI selectors.
  • Translation files: src/renderer/locales/{en,it,es,fr,de}/translation.json. Namespaces: nav, auth, tasks, settings, common, errors, home, timeline, projects, scouts.
  • common.* namespace holds shared labels (save, cancel, delete, edit, add, rename, saving, deleting, creating, renameDescription, deleteTitle). Check common.* before adding new key.
  • Pluralization uses i18next _one/_other suffixes.
  • LanguageSync component in src/renderer/index.tsx reads persisted uiLanguage from electron-store via tRPC on startup, syncs to i18next.
  • Language selector in GeneralSection.tsx (Settings > General). On change: (1) calls i18n.changeLanguage(), (2) persists to electron-store via setUiLanguage mutation, (3) writes to backend core memory so AI responds in same language.
  • getUiLanguage() exported from src/main/store.ts.
  • Static data arrays needing translation use labelKey pattern: store translation key, call t(labelKey) at render. Used in NAV_ITEMS, COLUMNS, SECTIONS, SUGGESTION_CHIPS.
  • When adding new translated text: add key to all 5 JSON files. Keep common.* consistent across all languages.

Google OAuth (adiuvAI side):

  • adiuvai:// NOT accepted by Google as redirect URI — Google only accepts http://localhost or https://. API backend exposes GET /auth/oauth/google/web-callback which receives Google redirect and bounces to adiuvai://oauth/callback?.... Redirect URI in Google Cloud Console points to backend, not Electron app.
  • app.requestSingleInstanceLock() required for second-instance event on Windows/Linux. If returns false, call app.quit() immediately.
  • In dev (process.defaultApp === true), setAsDefaultProtocolClient('adiuvai') must include [path.resolve(process.argv[1])] as third arg so OS protocol registration includes entry script.
  • loginWithOAuth uses fetch() directly (not this.get()) — authorize endpoint is public, get() throws when not authenticated.
  • Backup key in backup-key.ts stored in encryptedTokens under key backup_key, reusing getToken/setToken from token.ts. Device-bound, never password-derived — social-login users can use backup features.

api (FastAPI Backend)

Commands

cd api

# Development
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000

# Production
gunicorn app.main:app -k uvicorn.workers.UvicornWorker -w 4 --timeout 120

# Database migrations
alembic upgrade head

# Testing
pytest                                       # all tests
pytest -v                                    # verbose
pytest tests/test_deep_agent.py              # single file
pytest tests/test_deep_agent.py -k test_name # single test

# Linting/formatting
ruff check .
ruff format .

# Docker (full stack)
docker compose up --build

Architecture

FastAPI app (app/main.py)
├── Lifespan: APScheduler crons (memory hourly + audit weekly) when SCHEDULER_ENABLED
├── Middleware: TierRateLimit → Sanitizer → CORS
├── HTTP Routes (app/api/routes/) — all under /api/v1
│   ├── auth.py        — register, login, refresh, profile, OAuth, onboarding, password
│   ├── chat.py        — POST /chat, /chat/brief, /chat/embed
│   ├── scouts.py      — catalog, can-create, trigger, notes/summarize
│   ├── scout_setup.py — guided scout setup (journey)
│   ├── billing.py     — Stripe checkout, webhook, subscription, invoices
│   ├── device_ws.py   — WS /device (unified streaming endpoint: home, floating, brief, journey)
│   └── memory.py      — core / relational / forget-all
├── Agent System (app/agents/)
│   ├── task_agent.py
│   ├── project_agent.py
│   ├── note_agent.py
│   ├── timeline_agent.py
│   └── filesystem_agent.py
├── Core (app/core/)
│   ├── deep_agent.py            — main agent runner (run_home / run_floating / run_brief / run_journey)
│   ├── brief_agent.py           — daily brief generation
│   ├── agent_runner.py          — local + cloud agent run executor
│   ├── agent_session_buffer.py  — per-session conversation buffer
│   ├── agent_registry.py        — decorator-based agent registry
│   ├── llm.py                   — LiteLLM factory (multi-provider)
│   ├── memory_middleware.py     — encrypted core memory read/write
│   ├── memory_extraction.py     — LLM extraction from conversation tail
│   ├── memory_maintenance.py    — drain queue, contradiction audit, proactive mining
│   ├── note_summarizer.py       — gpt-4o-mini summary for notes
│   ├── output_formatter.py      — render agent output to user-facing markdown
│   ├── embeddings.py
│   ├── device_manager.py        — device registration / WS session tracking
│   ├── ws_context.py            — per-WS user context plumbing
│   ├── langfuse_client.py       — Langfuse prompt + tracing client
│   └── preprocessors/           — input preprocessors (e.g. email_html.py)
├── Auth (app/auth/oauth_providers.py) — GoogleOAuthProvider (httpx + manual PKCE)
├── Billing (app/billing/) — tier_manager + stripe_service
├── Integrations (app/integrations/) — gmail.py, ms_graph.py
└── Models (app/models.py) — SQLAlchemy 2.0 ORM

HTTP route prefix: every router included with prefix="/api/v1". So /api/v1/auth/..., /api/v1/chat, /api/v1/scouts/..., /api/v1/memory/..., /api/v1/device (WS).

ORM models (app/models.py): User, RefreshToken, OAuthAccount, Subscription, LocalScoutConfig, CloudScoutConfig, ScoutRunLog, MemoryCore, MemoryAssociative, MemoryEpisodic, MemoryProactive, ExtractionQueue, MemoryRelation, Plugin. PostgreSQL (asyncpg + SQLAlchemy 2.0 async). Alembic migrations in alembic/versions/.

Lifespan crons (only if settings.SCHEDULER_ENABLED):

  • _memory_cron_tick — hourly: drains Free-tier extraction queue + mines proactive patterns for Power+ users
  • _memory_audit_cron_tick — weekly: contradiction scan + label canonicalization for all users (Phase 7)

LLM routing: backend agents own all intelligence. Tool calls describe client-side ops (JSON) → Electron drizzle-executor runs them against local SQLite → result returned to backend over WS. Tool loop cap inside agent runner prevents runaway iteration.

Zero-trust data model: backend never stores raw user content. PostgreSQL holds auth, billing, plugin metadata, encrypted memory (Core/Associative/Episodic/Proactive/Relational), scout configs, run logs.

Config: app/config/settings.py — all env vars via Pydantic Settings. Copy .env.example to .env for local dev.

Testing: pytest + pytest-asyncio. Fixtures in tests/conftest.py. Active suites: agent runner, auth, brief/deep agents, device WS, integrations, journey, memory (audit/extraction/middleware/models/proactive/relations), middleware, output formatter, preprocessors, schemas, ws_unified.

Non-obvious details

  • Tier from DB, not JWT: get_current_user decodes JWT but fetches authoritative tier from subscriptions — tier changes take effect immediately, no re-login needed
  • Refresh tokens hashed: plaintext returned to client, stored as SHA-256 in DB — server can never retrieve plaintext (intentional)
  • WebSocket auth via query param: ?token=<jwt> instead of Bearer header (WebSocket handshake limitation)
  • Unified device WS: /api/v1/device is the single bidirectional channel. Handles home requests, floating requests, daily briefs, journeys, heartbeats. Tool calls round-trip through the same socket
  • Prompt IP protection: prompts kept server-side via Langfuse (langfuse_client). SanitizerMiddleware strips leaked fragments from responses
  • Agents don't execute operations: tools return JSON describing client-side ops — Electron client executes against local SQLite
  • Alembic async/sync split: app uses postgresql+asyncpg, Alembic CLI needs postgresql+psycopg2env.py handles URL conversion
  • CORS includes app://: Electron uses custom app:// protocol, not http/https
  • Run-disconnect tracking: _mark_runs_disconnected flips active runs when WS drops so client can resume cleanly

Onboarding (API side):

  • PUT /auth/me/memory — updates core memory k/v pairs, optionally marks onboarding complete (mark_onboarded: true sets users.onboarding_completed_at).
  • POST /auth/me/onboarding/reset — nullifies onboarding_completed_at so wizard re-runs.
  • POST /auth/onboarding/normalize — LLM-normalizes free-text onboarding inputs via gpt-4o-mini; returns inputs unchanged on error.
  • get_current_user() in auth.py middleware decrypts core memory blocks, includes in UserProfile.memory dict.
  • users.onboarding_completed_at — nullable TIMESTAMPTZ, returned as epoch ms (int) in UserProfile schema.

i18n (API side):

  • _language_instruction() in app/core/deep_agent.py reads user's language from MemoryCore, appends system prompt directive ("Always respond in {language}") to all run_* functions.
  • Electron client writes chosen language to backend core memory on change — API picks up on next agent call.

Google OAuth (api side):

  • OAuth routes in app/api/routes/auth.py: GET /auth/oauth/{provider}/authorize, POST /auth/oauth/{provider}/callback, GET /auth/oauth/{provider}/web-callback (bounces to deep link, excluded from OpenAPI schema).
  • Provider abstraction in app/auth/oauth_providers.pyGoogleOAuthProvider uses httpx directly (no authlib). PKCE S256 implemented manually via generate_pkce_pair().
  • _pending_states dict in routes/auth.py is in-memory — works for single-process dev, doesn't survive restarts, doesn't scale to multiple workers. Replace with Redis in production.
  • users.password_hash is nullable — social-only users have password_hash=None. await db.flush() required before creating linked OAuthAccount to populate new_user.id before commit.
  • OAUTH_REDIRECT_URI must point to API backend (e.g. https://api.adiuvai.com/...).
  • Unverified email + existing account = 409: if email_verified=False and email already registered, callback returns 409. Without this guard, branch 3 would INSERT duplicate email and crash with DB constraint violation (500).
  • Testing OAuth routes: mock GoogleOAuthProvider.exchange_code and get_userinfo with patch.object(..., new=AsyncMock(...)) — works because FastAPI instantiates new provider per request. Use monkeypatch.setattr(settings, "GOOGLE_AUTH_CLIENT_ID", ...) to simulate configured credentials without restart.

Tier System

Source of truth: app/billing/tier_manager.py (FEATURES + RATE_LIMITS dicts).

Feature Free Pro Power Team
Rate limit 20/min 60/min 120/min 200/min
Providers 1 unlimited unlimited unlimited
Relational memory no yes yes yes
Proactive mining no no yes yes

tier_manager.get_tier() falls back to 'power' in dev (settings.ENV == 'dev') when no subscription found, else 'free'. Enforced in app/api/middleware/rate_limit.py (sliding window) and tier_manager.check_feature() calls scattered through agent + memory paths.


Cross-Project Integration

Electron app and FastAPI backend communicate via WebSocket (/api/v1/device):

  1. Electron connects with ?token=<jwt> query param
  2. Client sends typed request frames (home / floating / brief / journey_start / journey_message)
  3. Server streams v3 typed frames (text deltas, tool_call, run_complete, error)
  4. Tool call frames → Electron drizzle-executor runs against local SQLite → returns tool_result over same socket
  5. Heartbeat loop keeps connection alive; backend marks runs disconnected on drop

There is no fully-local AI fallback — the Electron orchestrator is a thin delegation shell that requires connectivity + auth. If offline or logged out, checkConnectivity() short-circuits with a user-facing error.


MCP Servers

  • Langfuse Docs (https://langfuse.com/api/mcp) — workspace-level, prompt management docs
  • shadcn (npx shadcn@latest mcp) — configured in adiuvAI/ for UI component generation