Files
workspace/docs/superpowers/specs/2026-05-15-scouts-refactor-and-gmail-integration-plan.md
Roberto 445a4cbbf9 docs: add scouts refactor + gmail scout implementation plan
Covers Phases 1-3 (rename, connector skeleton, Gmail end-to-end) as
28 TDD tasks. Phase 4 (Stage 2 categorization + brief HITL) deferred
to separate spec.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-15 23:32:40 +02:00

108 KiB

Scouts Refactor + Gmail Integration Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Rename the entire "Agents" subsystem to "Scouts" across UI/code/Postgres/SQLite/Langfuse, then add a Gmail cloud scout with two-stage pipeline (BE triage + on-connect delivery) and a reusable SourceConnector abstraction. Phase 4 (Stage 2 categorization + HITL surface) is deferred to a future spec.

Architecture: Phase 1 is a single atomic rename PR (no behavior change). Phase 2 lays connector skeleton, queue table, WS frame contract, and SQLite suggestion table — no user-visible feature yet. Phase 3 ships Gmail end-to-end: OAuth setup UI, push (users.watch) + cron-fallback polling, BE-side spam triage with opt-in auto-trash, deliver-on-reconnect drain into local scoutSuggestions table with category='unprocessed' stub.

Tech Stack: Python 3.12 / FastAPI / SQLAlchemy 2.0 (async) / Alembic / Pydantic v2 / Langfuse for prompts / Google API Client for Gmail / Pub/Sub for push notifications. Electron / TypeScript / React 19 / tRPC v11 / Drizzle ORM / better-sqlite3 / i18next.

Spec: docs/superpowers/specs/2026-05-15-scouts-refactor-and-gmail-integration-design.md

Repository layout: monorepo with submodules api/ (FastAPI) and adiuvAI/ (Electron). All tasks operate on absolute paths under c:\Users\PC-Roby\Documents\_adiuvai_workspace.


File Structure

New files (BE — api/)

Path Responsibility
api/alembic/versions/007_rename_agents_to_scouts.py Rename Postgres tables/columns from *_agent_* to *_scout_*.
api/alembic/versions/008_scout_triage_queue.py Create scout_triage_queue + alter cloud_scout_configs.
api/app/scouts/__init__.py Package marker.
api/app/scouts/engine.py ScoutEngine orchestrator (triage, queue, deliver).
api/app/scouts/connectors/__init__.py Package marker.
api/app/scouts/connectors/base.py SourceConnector Protocol + ItemRef/ItemMetadata/ItemContent/TriageVerdict Pydantic models.
api/app/scouts/connectors/registry.py Connector registration + lookup.
api/app/scouts/connectors/gmail.py GmailConnector impl (wraps existing app/integrations/gmail.py).
api/app/api/routes/scout_webhooks.py POST /api/v1/scouts/webhooks/gmail — Pub/Sub receiver.
api/tests/test_scout_engine.py Unit tests for ScoutEngine w/ mocked connector.
api/tests/test_scout_connectors_gmail.py Tests for GmailConnector (mocked Gmail API).
api/tests/test_scout_webhook.py Webhook + Pub/Sub JWT verification.

Renamed files (BE)

Before After
api/app/api/routes/agents.py api/app/api/routes/scouts.py
api/app/api/routes/agent_setup.py api/app/api/routes/scout_setup.py
api/app/core/agent_runner.py api/app/core/scout_runner.py
api/app/core/agent_session_buffer.py api/app/core/scout_session_buffer.py
api/app/core/agent_registry.py api/app/core/scout_registry.py

New files (Electron — adiuvAI/)

Path Responsibility
adiuvAI/src/main/db/migrations/0007_scouts_rename.sql Drizzle SQL: rename agent_runsscout_runs, agent_run_actionsscout_run_actions.
adiuvAI/src/main/db/migrations/0008_scout_suggestions.sql Drizzle SQL: create scout_suggestions table.
adiuvAI/src/main/scouts/scout-suggestion-handler.ts Handles incoming scout_proposal WS frame, inserts into scoutSuggestions, sends scout_proposal_ack.

Renamed files (Electron)

Before After
adiuvAI/src/main/agents/agent-scheduler.ts adiuvAI/src/main/scouts/scout-scheduler.ts
adiuvAI/src/renderer/components/settings/AgentsSection.tsx adiuvAI/src/renderer/components/settings/ScoutsSection.tsx
adiuvAI/src/renderer/components/settings/AgentRow.tsx adiuvAI/src/renderer/components/settings/ScoutRow.tsx
adiuvAI/src/renderer/components/settings/LocalAgentConfigPanel.tsx adiuvAI/src/renderer/components/settings/LocalScoutConfigPanel.tsx
adiuvAI/src/renderer/components/settings/CloudAgentConfigPanel.tsx adiuvAI/src/renderer/components/settings/CloudScoutConfigPanel.tsx
adiuvAI/src/renderer/components/settings/InlineAgentCreationStepper.tsx adiuvAI/src/renderer/components/settings/InlineScoutCreationStepper.tsx

Modified files (high-impact, full list per task)

  • api/app/models.py — class renames + table/column renames + relationship updates.
  • api/app/main.py — register new router, lifespan crons.
  • api/app/schemas.pyWsFrameType enum + new frame Pydantic models.
  • api/app/config/settings.py — add Gmail Pub/Sub topic env vars.
  • adiuvAI/src/main/db/schema.ts — rename tables + add scoutSuggestions.
  • adiuvAI/src/main/router/index.ts — rename agent.* sub-routers → scout.*.
  • adiuvAI/src/main/api/backend-client.ts — handle new WS frames.
  • adiuvAI/src/renderer/locales/{en,it,es,fr,de}/translation.json — rename i18n keys.
  • adiuvAI/src/renderer/routes/settings.tsx — update SECTIONS array.

PHASE 1 — Rename Only

Goal: pure rename, no behavior change, all tests green at end of phase.

Important caveat: the project uses git submodules. Each task touches files in api/ or adiuvAI/ submodule — commits happen inside the submodule, then a separate commit in the monorepo updates the submodule pointer. Each task's commit step shows both.


Task 1: Postgres rename — Alembic migration

Files:

  • Create: api/alembic/versions/007_rename_agents_to_scouts.py

Context: Latest existing migration is d6e3f4a5b6c7_folder_index_tables.py with down_revision = "006". New migration uses revision "007" and depends on d6e3f4a5b6c7. Tables to rename: local_agent_configs, cloud_agent_configs, agent_run_logs. Columns to rename: agent_config (in local_agent_configs), agent_id and agent_type (in agent_run_logs).

  • Step 1: Create the migration file
"""Rename agents to scouts.

Revision ID: 007
Revises: d6e3f4a5b6c7
Create Date: 2026-05-15

Renames the entire agents subsystem identifiers to scouts.
Pre-1.0 — no data preservation concerns beyond ALTER TABLE rename.
"""

from typing import Sequence, Union

from alembic import op


revision: str = "007"
down_revision: Union[str, None] = "d6e3f4a5b6c7"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None


def upgrade() -> None:
    # Tables
    op.rename_table("local_agent_configs", "local_scout_configs")
    op.rename_table("cloud_agent_configs", "cloud_scout_configs")
    op.rename_table("agent_run_logs", "scout_run_logs")

    # Columns
    op.alter_column("local_scout_configs", "agent_config", new_column_name="scout_config")
    op.alter_column("scout_run_logs", "agent_id", new_column_name="scout_id")
    op.alter_column("scout_run_logs", "agent_type", new_column_name="scout_type")


def downgrade() -> None:
    op.alter_column("scout_run_logs", "scout_type", new_column_name="agent_type")
    op.alter_column("scout_run_logs", "scout_id", new_column_name="agent_id")
    op.alter_column("local_scout_configs", "scout_config", new_column_name="agent_config")

    op.rename_table("scout_run_logs", "agent_run_logs")
    op.rename_table("cloud_scout_configs", "cloud_agent_configs")
    op.rename_table("local_scout_configs", "local_agent_configs")
  • Step 2: Run migration against a fresh test DB to verify it applies cleanly

Run from api/ directory:

alembic upgrade head

Expected: no errors. \d local_scout_configs shows the renamed table. \d scout_run_logs shows the scout_id column (not agent_id).

  • Step 3: Verify downgrade works
alembic downgrade -1

Expected: tables go back to local_agent_configs etc. Then re-run alembic upgrade head.

  • Step 4: Commit (inside submodule)
cd api
git add alembic/versions/007_rename_agents_to_scouts.py
git commit -m "feat(db): rename agents to scouts (alembic 007)"

Task 2: SQLAlchemy model rename — LocalScoutConfig, CloudScoutConfig, ScoutRunLog

Files:

  • Modify: api/app/models.py:161-227 (and the AgentRunLog class — find via grep)

Context: Renames Python class names to match the new tables. Must update __tablename__ and mapped_column for renamed columns. Relationships and back_populates strings must reference the new class names.

  • Step 1: Read current models around LocalAgentConfig, CloudAgentConfig, AgentRunLog

Use Read tool on api/app/models.py to confirm the exact line ranges (the AgentRunLog class + relationships use string-based primaryjoin + back_populates="local_agent" / back_populates="cloud_agent" patterns).

  • Step 2: Apply the renames in models.py

Replace class names and tablenames:

class LocalScoutConfig(Base):
    __tablename__ = "local_scout_configs"

    # ... id/user_id/etc unchanged ...
    scout_config: Mapped[dict | None] = mapped_column(JSON, nullable=True)
    # ... rest unchanged ...

    run_logs: Mapped[list["ScoutRunLog"]] = relationship(
        back_populates="local_scout",
        primaryjoin="and_(ScoutRunLog.scout_id == LocalScoutConfig.id, ScoutRunLog.scout_type == 'local')",
        foreign_keys="ScoutRunLog.scout_id",
        cascade="all, delete-orphan",
        overlaps="run_logs,cloud_scout",
    )


class CloudScoutConfig(Base):
    __tablename__ = "cloud_scout_configs"

    # ... unchanged columns ...

    run_logs: Mapped[list["ScoutRunLog"]] = relationship(
        back_populates="cloud_scout",
        primaryjoin="and_(ScoutRunLog.scout_id == CloudScoutConfig.id, ScoutRunLog.scout_type == 'cloud')",
        foreign_keys="ScoutRunLog.scout_id",
        cascade="all, delete-orphan",
        overlaps="run_logs,local_scout",
    )


class ScoutRunLog(Base):
    __tablename__ = "scout_run_logs"

    # ... id, user_id, started_at, finished_at, status, error etc. unchanged ...
    scout_id: Mapped[str] = mapped_column(...)
    scout_type: Mapped[str] = mapped_column(...)

    local_scout: Mapped["LocalScoutConfig | None"] = relationship(
        back_populates="run_logs",
        primaryjoin="and_(ScoutRunLog.scout_id == LocalScoutConfig.id, ScoutRunLog.scout_type == 'local')",
        foreign_keys="ScoutRunLog.scout_id",
        overlaps="run_logs,cloud_scout",
    )

    cloud_scout: Mapped["CloudScoutConfig | None"] = relationship(
        back_populates="run_logs",
        primaryjoin="and_(ScoutRunLog.scout_id == CloudScoutConfig.id, ScoutRunLog.scout_type == 'cloud')",
        foreign_keys="ScoutRunLog.scout_id",
        overlaps="run_logs,local_scout",
    )
  • Step 3: Sweep all imports of the old class names

Run from api/:

grep -rn "AgentRunLog\|LocalAgentConfig\|CloudAgentConfig" app/ tests/ --include="*.py"

For each hit, replace with the corresponding Scout* name. Do not skip test files.

  • Step 4: Run the test suite — expect failures and fix incrementally

Run from api/:

pytest -x

Fix import errors and reference errors as they surface. The test fixture at api/tests/conftest.py does Base.metadata.create_all so renamed tables will be created automatically. Don't worry about migrations in tests — they use create_all.

  • Step 5: Verify the suite is green
pytest

Expected: all tests pass.

  • Step 6: Commit (inside submodule)
cd api
git add app/models.py
git add -u app/ tests/   # any files changed by the import sweep
git commit -m "refactor(models): rename Agent classes to Scout"

Task 3: Rename BE routes — routes/agents.pyroutes/scouts.py, agent_setup.pyscout_setup.py

Files:

  • Rename: api/app/api/routes/agents.pyapi/app/api/routes/scouts.py

  • Rename: api/app/api/routes/agent_setup.pyapi/app/api/routes/scout_setup.py

  • Modify: api/app/main.py — update router registration

  • Modify: api/app/api/routes/device_ws.py:30-31 — update imports

  • Step 1: Rename the files via git

cd api
git mv app/api/routes/agents.py app/api/routes/scouts.py
git mv app/api/routes/agent_setup.py app/api/routes/scout_setup.py
  • Step 2: Inside both renamed files, change APIRouter(prefix="/agents", ...) to prefix="/scouts" and prefix="/agent-setup" to prefix="/scout-setup"

In app/api/routes/scouts.py find and replace:

router = APIRouter(prefix="/agents", tags=["agents"])

With:

router = APIRouter(prefix="/scouts", tags=["scouts"])

In app/api/routes/scout_setup.py find and replace:

router = APIRouter(prefix="/agent-setup", tags=["agent-setup"])

With:

router = APIRouter(prefix="/scout-setup", tags=["scout-setup"])
  • Step 3: Update app/main.py to import the new module names

Find the imports of agents and agent_setup routers and replace with scouts and scout_setup.

  • Step 4: Update app/api/routes/device_ws.py:30-31 import

Change:

from app.api.routes.agent_setup import handle_journey_message, handle_journey_start

To:

from app.api.routes.scout_setup import handle_journey_message, handle_journey_start
  • Step 5: Sweep remaining import references
grep -rn "from app.api.routes.agents\|from app.api.routes.agent_setup\|app\.api\.routes\.agents\|app\.api\.routes\.agent_setup" app/ tests/

Replace each.

  • Step 6: Run tests
pytest

Expected: pass. URL-based tests must use new paths (/api/v1/scouts/..., /api/v1/scout-setup/...).

  • Step 7: Commit (inside submodule)
cd api
git add -A app/ tests/
git commit -m "refactor(routes): rename /agents and /agent-setup to /scouts and /scout-setup"

Task 4: Rename core/agent_runner.py, agent_session_buffer.py, agent_registry.py

Files:

  • Rename: api/app/core/agent_runner.pyapi/app/core/scout_runner.py

  • Rename: api/app/core/agent_session_buffer.pyapi/app/core/scout_session_buffer.py

  • Rename: api/app/core/agent_registry.pyapi/app/core/scout_registry.py

  • Step 1: Rename via git

cd api
git mv app/core/agent_runner.py app/core/scout_runner.py
git mv app/core/agent_session_buffer.py app/core/scout_session_buffer.py
git mv app/core/agent_registry.py app/core/scout_registry.py
  • Step 2: Update top-of-file references in each renamed file

Inside each file, rename all references to agent_* symbols that are now defined here. For example, if agent_runner.py had def trigger_agent_run(...), change to def trigger_scout_run(...). Do NOT rename names that belong to app/agents/* (those LLM helpers stay).

  • Step 3: Sweep external imports
grep -rn "from app.core.agent_runner\|from app.core.agent_session_buffer\|from app.core.agent_registry" app/ tests/

Replace each agent_runnerscout_runner, etc.

Also sweep symbol references — for example if you renamed trigger_agent_runtrigger_scout_run:

grep -rn "trigger_agent_run\|agent_session_buffer\|agent_registry" app/ tests/

Replace.

  • Step 4: Run tests
pytest

Expected: green.

  • Step 5: Commit (inside submodule)
cd api
git add -A app/ tests/
git commit -m "refactor(core): rename agent_runner/session_buffer/registry to scout_*"

Task 5: Rename Pydantic schemas + WS frame names referring to "agent"

Files:

  • Modify: api/app/schemas.py (or wherever WsFrameType enum + Agent-related schemas live)

Context: Pydantic schemas like AgentRunStatus, AgentTriggerRequest, etc. need rename. The WsFrameType enum may have values like "agent_run_started" — rename to "scout_run_started". Frontend will be updated to match in later tasks.

  • Step 1: Find all Pydantic models + enum values containing "agent"
cd api
grep -n "class .*Agent\|\"agent_" app/schemas.py
  • Step 2: Rename each — class names and enum values

Examples (apply each found in step 1):

  • class AgentRunStatus(str, Enum)class ScoutRunStatus(str, Enum)

  • class AgentTriggerRequest(BaseModel)class ScoutTriggerRequest(BaseModel)

  • WsFrameType.AGENT_RUN_STARTED = "agent_run_started"SCOUT_RUN_STARTED = "scout_run_started"

  • Step 3: Sweep all references

grep -rn "AgentRunStatus\|AgentTriggerRequest\|agent_run_started\|agent_run_completed" app/ tests/

Replace.

  • Step 4: Run tests
pytest
  • Step 5: Commit
cd api
git add -A
git commit -m "refactor(schemas): rename Agent* schemas and WS frame types to Scout*"

Task 6: Rename Langfuse prompts (user-facing scout prompts only)

Files:

  • Manual operation against Langfuse via MCP tools (no source files unless prompt names are referenced in code).

Context: Langfuse prompts whose names start with agent- and refer to the user-facing scout subsystem (e.g. agent-runner-system, agent-setup-journey-system, cloud-agent-trigger, local-agent-extraction). Prompts for app/agents/* LLM helpers (e.g. task-agent-system, note-agent-summarize) STAY.

  • Step 1: List all Langfuse prompts

Use the Langfuse MCP tool:

mcp__langfuse__listPrompts (no arguments)
  • Step 2: Identify scout-subsystem prompts to rename

From the listing, pick prompts whose names begin with agent- AND whose content/usage maps to the user-facing scout subsystem. If unsure for a given prompt, grep its name in api/app/:

grep -rn "get_prompt_or_fallback(\"agent-" api/app/ api/tests/

Names that appear in the renamed scout_*.py files are scout-subsystem prompts.

  • Step 3: For each scout prompt, create a new prompt with the renamed name

Use mcp__langfuse__getPromptUnresolved to fetch the current prompt body, then mcp__langfuse__createTextPrompt (or createChatPrompt if it's a chat prompt) with the new name (e.g. scout-runner-system) and label production.

  • Step 4: Update code references to use the new prompt names

Find each get_prompt_or_fallback("agent-..." call and rename the literal:

grep -rn 'get_prompt_or_fallback("agent-' api/app/

Replace each "agent-X" with "scout-X".

  • Step 5: Run tests + manual smoke
cd api
pytest

Then start dev server (uvicorn app.main:app --reload), trigger a known scout-subsystem agent invocation (use the agent setup journey or a scout trigger endpoint via curl) and confirm the prompt loads without falling back.

  • Step 6: Delete old prompts from Langfuse

For each old agent-* prompt confirmed superseded, delete via Langfuse UI or API (Langfuse MCP doesn't expose delete; use the Langfuse web console).

  • Step 7: Commit code changes
cd api
git add -A
git commit -m "refactor(prompts): rename scout-subsystem Langfuse prompt references to scout-*"

Task 7: Drizzle migration — rename agent_runsscout_runs, agent_run_actionsscout_run_actions

Files:

  • Create: adiuvAI/src/main/db/migrations/0007_scouts_rename.sql
  • Modify: adiuvAI/src/main/db/schema.ts

Context: Drizzle migrations are SQL files in adiuvAI/src/main/db/migrations/. Latest is 0006_misty_cammi.sql. The auto-generated migration name suffix isn't strictly required but follow the pattern (drizzle-kit generates one from CHANGELOG entries). For a manual rename, a hand-written name is fine.

  • Step 1: Update adiuvAI/src/main/db/schema.ts first — change the table name in the Drizzle definition

Find:

export const agentRuns = sqliteTable('agent_runs', { ... });
export const agentRunActions = sqliteTable('agent_run_actions', { ... });

Replace with:

export const scoutRuns = sqliteTable('scout_runs', {
  id: text().primaryKey(),
  scoutId: text('scout_id').notNull(),
  status: text().notNull(),
  startedAt: integer('started_at').notNull(),
  completedAt: integer('completed_at'),
});

export const scoutRunActions = sqliteTable('scout_run_actions', {
  id: text().primaryKey(),
  runId: text('run_id').notNull(),
  scoutId: text('scout_id').notNull(),
  verb: text().notNull(),
  entityType: text('entity_type').notNull(),
  entityId: text('entity_id').notNull(),
  entityTitle: text('entity_title'),
  createdAt: integer('created_at').notNull(),
});

(Read the file first to copy the exact column definitions — keep types/nullability identical, only rename agentscout.)

  • Step 2: Generate the migration

Run from adiuvAI/:

npx drizzle-kit generate

Expected: a new file 0007_<adjective>_<noun>.sql in src/main/db/migrations/. Verify it contains ALTER TABLE statements (or DROP+CREATE if Drizzle prefers — SQLite may require recreating the table; Drizzle will emit the right pattern).

If Drizzle emits a destructive recreate (DROP TABLE + CREATE), that's fine pre-1.0 since these tables are largely empty in dev DBs. Confirm with the user before proceeding if their dev DB has rows worth preserving — for empty/dev DBs, drop is fine.

  • Step 3: Rename the generated migration to a clean name
mv adiuvAI/src/main/db/migrations/0007_*.sql adiuvAI/src/main/db/migrations/0007_scouts_rename.sql
  • Step 4: Sweep TypeScript imports of the old names
grep -rn "agentRuns\|agentRunActions" adiuvAI/src/

Replace each with scoutRuns / scoutRunActions.

  • Step 5: Run typecheck + lint

From adiuvAI/:

npm run lint
npx tsc --noEmit

Expected: pass.

  • Step 6: Manual smoke — boot the app once to apply the migration
cd adiuvAI
npm run start

Watch the console for migration apply messages. Once app boots cleanly, close it. Verify the dev DB has scout_runs and scout_run_actions (use sqlite3 dev.db ".tables" from adiuvAI/).

  • Step 7: Commit (inside submodule)
cd adiuvAI
git add src/main/db/schema.ts src/main/db/migrations/0007_scouts_rename.sql
git add -u src/    # any consumer files updated in step 4
git commit -m "refactor(db): rename agent_runs/agent_run_actions to scout_*"

Task 8: Rename Electron main-process modules and tRPC sub-routers

Files:

  • Rename: adiuvAI/src/main/agents/agent-scheduler.tsadiuvAI/src/main/scouts/scout-scheduler.ts
  • Modify: adiuvAI/src/main/router/index.ts — rename agent.* sub-router to scout.*
  • Modify: adiuvAI/src/main/store.ts — rename getLocalAgents/saveLocalAgentgetLocalScouts/saveLocalScout
  • Modify: adiuvAI/src/main/index.ts — update scheduler import + start call
  • Modify: adiuvAI/src/main/api/backend-client.ts — rename agent-related symbols if any

Context: Inside agent-scheduler.ts, the function names are startAgentScheduler, stopAgentScheduler, tickAgentScheduler and POSTs to /api/v1/agents/trigger. After Task 3, the BE route is /api/v1/scouts/trigger — so this file must also update its URL.

  • Step 1: Move the file
cd adiuvAI
mkdir -p src/main/scouts
git mv src/main/agents/agent-scheduler.ts src/main/scouts/scout-scheduler.ts
rmdir src/main/agents 2>/dev/null || true

(Don't fail if other files exist in src/main/agents/ — check first; if so, decide per-file whether to move or leave.)

  • Step 2: Inside scout-scheduler.ts, rename functions and update URL

Replace:

  • startAgentSchedulerstartScoutScheduler
  • stopAgentSchedulerstopScoutScheduler
  • tickAgentSchedulertickScoutScheduler
  • getLocalAgents()getLocalScouts()
  • saveLocalAgent(...)saveLocalScout(...)
  • URL '/api/v1/agents/trigger''/api/v1/scouts/trigger'
  • Variable agentsscouts (clarity)
  • agentRuns Drizzle import → scoutRuns

Show the full updated function header:

async function tickScoutScheduler(): Promise<void> {
  const scouts = getLocalScouts();
  const now = Date.now();

  for (const scout of scouts) {
    if (!scout.enabled) continue;
    const intervalMs = CRON_INTERVAL_MS[scout.scheduleCron];
    if (!intervalMs) continue;
    if (scout.lastRunAt && now - scout.lastRunAt < intervalMs) continue;

    try {
      const response = await getBackendClient().proxyPost<{ id: string }>(
        '/api/v1/scouts/trigger',
        { directory: scout.directory, deviceId: getDeviceId(), /* ... */ }
      );
      if (response?.id) {
        await getDb().insert(scoutRuns).values({
          id: response.id, scoutId: scout.id, status: 'running', startedAt: now,
        }).onConflictDoNothing();
      }
      saveLocalScout({ ...scout, lastRunAt: now });
    } catch (err) { /* ... */ }
  }
}
  • Step 3: Rename store helpers in adiuvAI/src/main/store.ts

Find getLocalAgents / saveLocalAgent / any LocalAgentConfig references and rename to getLocalScouts / saveLocalScout / LocalScoutConfig. Keep the storage key migration-safe: read from old key localAgents once, write to localScouts, then read only from new key thereafter. For pre-1.0 dev, simpler to just rename the key and accept users re-adding their scouts (note: confirm with user).

For now, simplest acceptable approach:

// store.ts
const STORAGE_KEY = 'localScouts';   // was 'localAgents'

export function getLocalScouts(): LocalScoutConfig[] {
  return store.get(STORAGE_KEY, []) as LocalScoutConfig[];
}
export function saveLocalScout(scout: LocalScoutConfig): void { /* ... */ }
  • Step 4: Update adiuvAI/src/main/index.ts

Replace:

import { startAgentScheduler, stopAgentScheduler } from './agents/agent-scheduler';

With:

import { startScoutScheduler, stopScoutScheduler } from './scouts/scout-scheduler';

And the call sites: startAgentScheduler()startScoutScheduler(), stopAgentScheduler()stopScoutScheduler().

  • Step 5: Update adiuvAI/src/main/router/index.ts — rename agent.* sub-router to scout.*

Find the agent: t.router({ local: ..., cloud: ..., journey: ..., runs: ..., runActions: ..., catalog: ..., runNow: ..., ... }) block. Rename the key from agent to scout. Also rename the imported sub-router files if any (in adiuvAI/src/main/router/, look for agent.ts or agent/ subdirectory).

  • Step 6: Sweep type references
grep -rn "LocalAgentConfig\|CloudAgentConfig\|trpc\.agent\.\|/api/v1/agents" adiuvAI/src/

Replace systematically: LocalAgentConfigLocalScoutConfig, CloudAgentConfigCloudScoutConfig, trpc.agent.trpc.scout., /api/v1/agents/api/v1/scouts.

  • Step 7: Typecheck + lint
cd adiuvAI
npx tsc --noEmit
npm run lint
  • Step 8: Commit
cd adiuvAI
git add -A src/main/
git commit -m "refactor(main): rename agent-scheduler/store/router symbols to scout"

Task 9: Rename Renderer components

Files:

  • Rename 5 files in adiuvAI/src/renderer/components/settings/ (per File Structure table)

  • Modify: adiuvAI/src/renderer/routes/settings.tsx — update SECTIONS array

  • Modify: each renamed component — update internal symbols

  • Step 1: Move the files

cd adiuvAI
git mv src/renderer/components/settings/AgentsSection.tsx src/renderer/components/settings/ScoutsSection.tsx
git mv src/renderer/components/settings/AgentRow.tsx src/renderer/components/settings/ScoutRow.tsx
git mv src/renderer/components/settings/LocalAgentConfigPanel.tsx src/renderer/components/settings/LocalScoutConfigPanel.tsx
git mv src/renderer/components/settings/CloudAgentConfigPanel.tsx src/renderer/components/settings/CloudScoutConfigPanel.tsx
git mv src/renderer/components/settings/InlineAgentCreationStepper.tsx src/renderer/components/settings/InlineScoutCreationStepper.tsx
  • Step 2: Inside each renamed file, rename the exported component

Rule: every export function AgentXexport function ScoutX, every AgentScout in component bodies, every agentscout in local variables, every 'toast.agent.X' i18n key → 'toast.scout.X', every 'agents.X''scouts.X', every t('settings.agentsX')t('settings.scoutsX').

Show example (head of ScoutsSection.tsx):

export function ScoutsSection() {
  const { t } = useTranslation();
  const utils = trpc.useUtils();
  const localScoutsQuery = trpc.scout.local.list.useQuery();
  const cloudScoutsQuery = trpc.scout.cloud.list.useQuery();
  const deleteLocalMutation = trpc.scout.local.delete.useMutation();
  const deleteCloudMutation = trpc.scout.cloud.delete.useMutation();
  const updateLocalMutation = trpc.scout.local.update.useMutation();
  const updateCloudMutation = trpc.scout.cloud.update.useMutation();
  const runNowMutation = trpc.scout.runNow.useMutation();

  const { notify, notifyError, notifyPromise } = useNotify();
  const [expandedScout, setExpandedScout] = useState<string | null>(null);
  // ...

  const localScouts: LocalScoutConfig[] = localScoutsQuery.data ?? [];
  const cloudScouts: CloudScoutConfig[] = cloudScoutsQuery.data ?? [];
  const allScouts = [
    ...localScouts.map(s => ({ ...s, scoutType: 'local' as const })),
    ...cloudScouts.map(s => ({ ...s, scoutType: 'cloud' as const })),
  ];
  // ...
}
  • Step 3: Update consumers — settings route + any other importer

In adiuvAI/src/renderer/routes/settings.tsx, change:

import { AgentsSection } from '@/components/settings/AgentsSection';
// ... in SECTIONS array:
{ id: 'agents', label: 'settings.agents', component: AgentsSection },

To:

import { ScoutsSection } from '@/components/settings/ScoutsSection';
// ...
{ id: 'scouts', label: 'settings.scouts', component: ScoutsSection },

Also sweep:

cd adiuvAI
grep -rn "AgentsSection\|AgentRow\|LocalAgentConfigPanel\|CloudAgentConfigPanel\|InlineAgentCreationStepper" src/

Replace each.

  • Step 4: Update adiuvAI/src/renderer/components/settings/types.ts (if it defines LocalAgentConfig)

Rename the interface LocalAgentConfigLocalScoutConfig. Add export type LocalScoutConfig = .... If there are other type definitions referencing "agent" (e.g. AgentDataType), rename consistently.

  • Step 5: Typecheck + lint
cd adiuvAI
npx tsc --noEmit
npm run lint
  • Step 6: Commit
cd adiuvAI
git add -A src/renderer/
git commit -m "refactor(renderer): rename Agent components and types to Scout"

Task 10: Rename i18n keys across all 5 languages

Files:

  • Modify: adiuvAI/src/renderer/locales/{en,it,es,fr,de}/translation.json

Context: Three buckets to rename in every language file:

  1. "settings": { "agents": "...", "agentsSubtitle": "...", "agentsDescription": "...", ... }scouts, scoutsSubtitle, scoutsDescription
  2. Top-level "agents": { "noAgentsYet": ..., "createAgent": ..., "yourAgents": ..., "createFirstAgent": ..., "noAgentsDescription": ... }"scouts": { "noScoutsYet", "createScout", "yourScouts", "createFirstScout", "noScoutsDescription" }. Update key names AND any in-value occurrences of "agent"/"agents" to "scout"/"scouts" within human-readable strings (apply translation per language — for English: "Agents" → "Scouts", "agent" → "scout"; for Italian: "Agenti" → "Scout"; etc.).
  3. "toast": { "agent": { ... } }"toast": { "scout": { ... } } (rename nested key only; values unchanged unless they contain "agent" word).

For non-English languages, keep the meaning consistent. Suggested translations:

Lang Key Suggested value
en settings.scouts "Scouts"
it settings.scouts "Scout"
es settings.scouts "Scouts"
fr settings.scouts "Scouts"
de settings.scouts "Scouts"
en settings.scoutsSubtitle "working for you."
it settings.scoutsSubtitle "che lavorano per te."
en settings.scoutsDescription "Scouts watch your data sources — local files, mailboxes, cloud services — and surface what matters in your task brief."
it settings.scoutsDescription "Gli scout monitorano le tue fonti dati — file locali, caselle email, servizi cloud — e mettono in evidenza ciò che conta nel tuo task brief."
en scouts.noScoutsYet "No scouts yet"
en scouts.createScout "Create scout"
en scouts.yourScouts "Your Scouts"
en scouts.createFirstScout "Create first scout"
en scouts.noScoutsDescription "Create your first scout from a template. Choose what data to extract, set a schedule, and edit instructions before saving."

(Translate analogously for it/es/fr/de — for the new descriptive copy, keep the spirit, ask the user for review on language nuance if uncertain.)

  • Step 1: Edit en/translation.json

Apply renames per the bucket list above.

  • Step 2: Edit it/translation.json — same renames, with Italian values

  • Step 3: Edit es/translation.json — same renames, with Spanish values

  • Step 4: Edit fr/translation.json — same renames, with French values

  • Step 5: Edit de/translation.json — same renames, with German values

  • Step 6: Sweep code references

cd adiuvAI
grep -rn "'agents\.\|\"agents\.\|'settings\.agents\|\"settings\.agents\|'toast\.agent\|\"toast\.agent" src/

Replace each agents.Xscouts.X, settings.agentssettings.scouts, toast.agenttoast.scout.

  • Step 7: Typecheck + lint + run app
cd adiuvAI
npx tsc --noEmit
npm run lint
npm run start   # smoke: open Settings, switch language, confirm "Scouts" header renders
  • Step 8: Commit
cd adiuvAI
git add -A src/renderer/locales/ src/renderer/
git commit -m "i18n: rename agents keys to scouts across all 5 languages"

Task 11: Update monorepo to bump submodule pointers

Files:

  • Modify: monorepo .gitmodules is unchanged; update tracked commit pointers via git add api adiuvAI

  • Step 1: From the monorepo root, observe the dirty submodule pointers

cd c:\Users\PC-Roby\Documents\_adiuvai_workspace
git status

Expected: api and adiuvAI listed as "modified content" or "new commits".

  • Step 2: Stage and commit the submodule bumps
git add api adiuvAI
git commit -m "chore: bump submodules — Phase 1 scouts rename complete"
  • Step 3: Verify CLAUDE.md doesn't need updates

Check c:\Users\PC-Roby\Documents\_adiuvai_workspace\.claude\CLAUDE.md and c:\Users\PC-Roby\Documents\_adiuvai_workspace\adiuvAI\.claude\CLAUDE.md — many references mention agent and the agent subsystem. For Phase 1, only update durable notes that became wrong (e.g. tRPC routers (in appRouter): ... agent (with local / cloud / journey sub-routers) ... should now read scout). Use targeted Edit, do not rewrite the file.

grep -n "agent" .claude/CLAUDE.md adiuvAI/.claude/CLAUDE.md

For each line that documents the renamed surface, fix in place.

  • Step 4: Commit CLAUDE.md updates
git add .claude/CLAUDE.md adiuvAI/.claude/CLAUDE.md
git commit -m "docs(CLAUDE): update references to scouts after rename"

Phase 1 complete. Verify by running both test suites green and starting the app.


PHASE 2 — Connector Skeleton

Goal: lay all infra (BE queue table, connector Protocol, engine, WS frame contract, SQLite suggestions table) without any user-visible Gmail feature yet. End state: backend can be unit-tested with a mock connector; Electron has the table + frame handler ready to receive proposals.


Task 12: Alembic migration — scout_triage_queue + cloud_scout_configs alters

Files:

  • Create: api/alembic/versions/008_scout_triage_queue.py

  • Step 1: Create the migration

"""Scout triage queue + cloud_scout_configs alterations.

Revision ID: 008
Revises: 007
Create Date: 2026-05-15
"""

from typing import Sequence, Union

import sqlalchemy as sa
from alembic import op


revision: str = "008"
down_revision: Union[str, None] = "007"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None


def upgrade() -> None:
    op.create_table(
        "scout_triage_queue",
        sa.Column("id", sa.Uuid(as_uuid=False), primary_key=True),
        sa.Column("user_id", sa.Uuid(as_uuid=False), sa.ForeignKey("users.id", ondelete="CASCADE"), nullable=False, index=True),
        sa.Column("scout_id", sa.Uuid(as_uuid=False), sa.ForeignKey("cloud_scout_configs.id", ondelete="CASCADE"), nullable=False),
        sa.Column("source_type", sa.String(50), nullable=False),
        sa.Column("source_msg_ref", sa.String(255), nullable=False),
        sa.Column("triage_verdict", sa.String(20), nullable=False),
        sa.Column("triage_reason", sa.Text, nullable=True),
        sa.Column("status", sa.String(20), nullable=False, server_default="queued"),
        sa.Column("triaged_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now()),
        sa.Column("delivered_at", sa.DateTime(timezone=True), nullable=True),
        sa.Column("acked_at", sa.DateTime(timezone=True), nullable=True),
        sa.Column("expires_at", sa.DateTime(timezone=True), nullable=False),
        sa.UniqueConstraint("scout_id", "source_msg_ref", name="uq_scout_triage_queue_scout_msg"),
    )
    op.create_index("ix_scout_triage_queue_user_status", "scout_triage_queue", ["user_id", "status"])
    op.create_index(
        "ix_scout_triage_queue_expires_active",
        "scout_triage_queue",
        ["expires_at"],
        postgresql_where=sa.text("status != 'acked'"),
    )

    op.add_column("cloud_scout_configs", sa.Column("auto_trash_spam", sa.Boolean(), nullable=False, server_default=sa.text("false")))
    op.add_column("cloud_scout_configs", sa.Column("gmail_history_id", sa.String(64), nullable=True))
    op.add_column("cloud_scout_configs", sa.Column("gmail_watch_expires_at", sa.DateTime(timezone=True), nullable=True))
    op.add_column("cloud_scout_configs", sa.Column("device_inactivity_pause_days", sa.Integer(), nullable=False, server_default="14"))


def downgrade() -> None:
    op.drop_column("cloud_scout_configs", "device_inactivity_pause_days")
    op.drop_column("cloud_scout_configs", "gmail_watch_expires_at")
    op.drop_column("cloud_scout_configs", "gmail_history_id")
    op.drop_column("cloud_scout_configs", "auto_trash_spam")

    op.drop_index("ix_scout_triage_queue_expires_active", table_name="scout_triage_queue")
    op.drop_index("ix_scout_triage_queue_user_status", table_name="scout_triage_queue")
    op.drop_table("scout_triage_queue")
  • Step 2: Apply + verify
cd api
alembic upgrade head

Verify with \d scout_triage_queue and \d cloud_scout_configs (the latter should now show the four new columns).

  • Step 3: Commit
cd api
git add alembic/versions/008_scout_triage_queue.py
git commit -m "feat(db): add scout_triage_queue and cloud_scout_configs gmail columns"

Task 13: SQLAlchemy ScoutTriageQueue model + CloudScoutConfig field additions

Files:

  • Modify: api/app/models.py — add ScoutTriageQueue class, add four new fields to CloudScoutConfig

  • Step 1: Add fields to CloudScoutConfig (in models.py after the existing fields)

auto_trash_spam: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default=sa.text("false"))
gmail_history_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
gmail_watch_expires_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
device_inactivity_pause_days: Mapped[int] = mapped_column(Integer, nullable=False, default=14, server_default="14")
  • Step 2: Add the ScoutTriageQueue class (after CloudScoutConfig)
class ScoutTriageQueue(Base):
    __tablename__ = "scout_triage_queue"
    __table_args__ = (
        UniqueConstraint("scout_id", "source_msg_ref", name="uq_scout_triage_queue_scout_msg"),
    )

    id: Mapped[str] = mapped_column(Uuid(as_uuid=False), primary_key=True, default=_uuid)
    user_id: Mapped[str] = mapped_column(Uuid(as_uuid=False), ForeignKey("users.id", ondelete="CASCADE"), nullable=False, index=True)
    scout_id: Mapped[str] = mapped_column(Uuid(as_uuid=False), ForeignKey("cloud_scout_configs.id", ondelete="CASCADE"), nullable=False)
    source_type: Mapped[str] = mapped_column(String(50), nullable=False)
    source_msg_ref: Mapped[str] = mapped_column(String(255), nullable=False)
    triage_verdict: Mapped[str] = mapped_column(String(20), nullable=False)
    triage_reason: Mapped[str | None] = mapped_column(Text, nullable=True)
    status: Mapped[str] = mapped_column(String(20), nullable=False, default="queued", server_default="queued")
    triaged_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
    delivered_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
    acked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
    expires_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
  • Step 3: Verify imports at the top of models.py include UniqueConstraint and Integer

If missing, add from sqlalchemy import ..., UniqueConstraint, Integer.

  • Step 4: Run tests
cd api
pytest

Expected: green (existing tests don't touch the new model yet, but Base.metadata.create_all should now succeed creating the new table in the test DB).

  • Step 5: Commit
cd api
git add app/models.py
git commit -m "feat(models): add ScoutTriageQueue model + cloud_scout gmail fields"

Task 14: SourceConnector Protocol + Pydantic types

Files:

  • Create: api/app/scouts/__init__.py (empty)

  • Create: api/app/scouts/connectors/__init__.py (empty)

  • Create: api/app/scouts/connectors/base.py

  • Create: api/tests/test_scout_connectors_base.py

  • Step 1: Write the failing test

api/tests/test_scout_connectors_base.py:

"""Tests for the SourceConnector base protocol and shared types."""

from __future__ import annotations

from datetime import datetime, timezone

import pytest

from app.scouts.connectors.base import (
    ItemContent,
    ItemMetadata,
    ItemRef,
    TriageVerdict,
)


def test_item_ref_round_trips_through_pydantic():
    ref = ItemRef(source_msg_ref="abc123", received_at=datetime.now(tz=timezone.utc))
    parsed = ItemRef.model_validate(ref.model_dump())
    assert parsed.source_msg_ref == "abc123"
    assert parsed.received_at == ref.received_at


def test_item_metadata_allows_all_optional():
    meta = ItemMetadata()
    assert meta.subject is None
    assert meta.sender is None
    assert meta.snippet is None
    assert meta.received_at is None


def test_item_content_requires_metadata_and_body():
    content = ItemContent(
        metadata=ItemMetadata(subject="hi"),
        body_text="hello world",
        raw_headers={"X-Foo": "bar"},
    )
    assert content.metadata.subject == "hi"
    assert content.body_text == "hello world"
    assert content.raw_headers["X-Foo"] == "bar"


def test_triage_verdict_constraints():
    v = TriageVerdict(verdict="relevant", reason="contains task language", confidence=0.92)
    assert v.verdict == "relevant"

    with pytest.raises(ValueError):
        TriageVerdict(verdict="meh", reason="x", confidence=0.5)  # bad enum value
  • Step 2: Run the test to verify it fails
cd api
pytest tests/test_scout_connectors_base.py -v

Expected: ImportError (No module named 'app.scouts.connectors.base').

  • Step 3: Implement app/scouts/connectors/base.py
"""Source connector Protocol and shared item types.

A SourceConnector adapts a third-party data source (Gmail, Slack, ...) to the
shared ScoutEngine interface. Each connector owns:

  * how to enumerate new items since the last poll (``list_new``)
  * how to fetch a single item's metadata cheaply (``fetch_metadata``)
  * how to fetch a single item's full content for in-memory triage
    (``fetch_content``) — this content MUST NOT be persisted by the engine
  * how to archive/trash an item (``archive``) for spam handling
  * optional push-notification setup (``setup_watch`` / ``renew_watch``)
"""

from __future__ import annotations

from datetime import datetime
from typing import Literal, Protocol

from pydantic import BaseModel, Field


class ItemRef(BaseModel):
    source_msg_ref: str
    received_at: datetime | None = None


class ItemMetadata(BaseModel):
    subject: str | None = None
    sender: str | None = None
    snippet: str | None = None
    received_at: datetime | None = None


class ItemContent(BaseModel):
    metadata: ItemMetadata
    body_text: str
    raw_headers: dict[str, str] = Field(default_factory=dict)


class TriageVerdict(BaseModel):
    verdict: Literal["relevant", "spam"]
    reason: str
    confidence: float = Field(ge=0.0, le=1.0)


class SourceConnector(Protocol):
    """Adapter for a third-party data source (Gmail, Slack, ...)."""

    source_type: str  # e.g. "gmail"

    async def list_new(self, scout) -> list[ItemRef]: ...
    async def fetch_metadata(self, scout, ref: ItemRef) -> ItemMetadata: ...
    async def fetch_content(self, scout, ref: ItemRef) -> ItemContent: ...
    async def archive(self, scout, ref: ItemRef) -> None: ...
    async def setup_watch(self, scout) -> None: ...
    async def renew_watch(self, scout) -> None: ...

Also create empty package markers:

cd api
touch app/scouts/__init__.py app/scouts/connectors/__init__.py
  • Step 4: Run tests
cd api
pytest tests/test_scout_connectors_base.py -v

Expected: PASS (4 tests).

  • Step 5: Commit
cd api
git add app/scouts/ tests/test_scout_connectors_base.py
git commit -m "feat(scouts): add SourceConnector protocol and item types"

Task 15: Connector registry

Files:

  • Create: api/app/scouts/connectors/registry.py

  • Create: api/tests/test_scout_connector_registry.py

  • Step 1: Write the failing test

"""Tests for the connector registry."""

from __future__ import annotations

import pytest

from app.scouts.connectors.base import ItemRef
from app.scouts.connectors.registry import (
    get_connector,
    register_connector,
    _reset_for_tests,
)


class _DummyConnector:
    source_type = "dummy"
    async def list_new(self, scout): return []
    async def fetch_metadata(self, scout, ref): raise NotImplementedError
    async def fetch_content(self, scout, ref): raise NotImplementedError
    async def archive(self, scout, ref): raise NotImplementedError
    async def setup_watch(self, scout): raise NotImplementedError
    async def renew_watch(self, scout): raise NotImplementedError


@pytest.fixture(autouse=True)
def _clean_registry():
    _reset_for_tests()
    yield
    _reset_for_tests()


def test_register_and_get():
    c = _DummyConnector()
    register_connector(c)
    assert get_connector("dummy") is c


def test_unknown_source_raises():
    with pytest.raises(KeyError):
        get_connector("nope")


def test_double_register_replaces():
    a = _DummyConnector()
    b = _DummyConnector()
    register_connector(a)
    register_connector(b)
    assert get_connector("dummy") is b
  • Step 2: Run test (should fail with ImportError)
cd api
pytest tests/test_scout_connector_registry.py -v
  • Step 3: Implement app/scouts/connectors/registry.py
"""Connector registry — single source of truth for source_type -> connector."""

from __future__ import annotations

from typing import Any

_CONNECTORS: dict[str, Any] = {}


def register_connector(connector: Any) -> None:
    """Register a SourceConnector instance under its ``source_type``.

    Calling twice with the same ``source_type`` replaces the prior entry —
    useful for tests and hot-reload, but in production each connector
    should be registered exactly once at startup.
    """
    if not getattr(connector, "source_type", None):
        raise ValueError("Connector must declare a non-empty source_type")
    _CONNECTORS[connector.source_type] = connector


def get_connector(source_type: str) -> Any:
    """Return the registered connector for ``source_type`` or raise KeyError."""
    try:
        return _CONNECTORS[source_type]
    except KeyError as exc:
        raise KeyError(f"No connector registered for source_type {source_type!r}") from exc


def _reset_for_tests() -> None:
    """Clear the registry — for use in pytest fixtures only."""
    _CONNECTORS.clear()
  • Step 4: Run tests
cd api
pytest tests/test_scout_connector_registry.py -v

Expected: PASS (3 tests).

  • Step 5: Commit
cd api
git add app/scouts/connectors/registry.py tests/test_scout_connector_registry.py
git commit -m "feat(scouts): add connector registry"

Task 16: ScoutEngine skeleton — trigger_scout + _process_item

Files:

  • Create: api/app/scouts/engine.py
  • Create: api/tests/test_scout_engine.py

Context: This task implements the core engine logic but uses a stub triage call. The real LLM triage comes in Task 24. The engine writes scout_triage_queue rows for relevant items and calls connector.archive for spam items when scout.auto_trash_spam is true.

  • Step 1: Write the failing test
"""Unit tests for ScoutEngine.trigger_scout / _process_item."""

from __future__ import annotations

import uuid
from datetime import datetime, timedelta, timezone
from unittest.mock import AsyncMock

import pytest
from sqlalchemy import select

from app.models import CloudScoutConfig, ScoutTriageQueue, User, Subscription
from app.scouts.connectors.base import ItemContent, ItemMetadata, ItemRef, TriageVerdict
from app.scouts.connectors.registry import register_connector, _reset_for_tests
from app.scouts.engine import ScoutEngine
from tests.conftest import _TestSessionLocal


def _make_connector(items, content_for):
    c = AsyncMock()
    c.source_type = "dummy"
    c.list_new = AsyncMock(return_value=items)
    c.fetch_content = AsyncMock(side_effect=lambda scout, ref: content_for[ref.source_msg_ref])
    c.archive = AsyncMock()
    return c


@pytest.fixture(autouse=True)
def _registry():
    _reset_for_tests()
    yield
    _reset_for_tests()


@pytest.mark.asyncio
async def test_relevant_item_inserted_into_queue(monkeypatch):
    user_id = "00000000-0000-0000-0000-000000000003"  # power tier seeded in conftest
    scout_id = str(uuid.uuid4())

    async with _TestSessionLocal() as session:
        scout = CloudScoutConfig(
            id=scout_id, user_id=user_id, provider="gmail", name="Test",
            data_types=[], prompt_template="", schedule_cron="0 * * * *",
            enabled=True, auto_trash_spam=False, device_inactivity_pause_days=14,
        )
        session.add(scout)
        await session.commit()

    refs = [ItemRef(source_msg_ref="msg-1")]
    content = {"msg-1": ItemContent(metadata=ItemMetadata(subject="Hi"), body_text="task tomorrow")}
    connector = _make_connector(refs, content)
    register_connector(connector)

    engine = ScoutEngine()
    monkeypatch.setattr(
        engine,
        "_triage_llm",
        AsyncMock(return_value=TriageVerdict(verdict="relevant", reason="task", confidence=0.9)),
    )

    await engine.trigger_scout(uuid.UUID(scout_id))

    async with _TestSessionLocal() as session:
        rows = (await session.execute(select(ScoutTriageQueue))).scalars().all()
    assert len(rows) == 1
    assert rows[0].source_msg_ref == "msg-1"
    assert rows[0].triage_verdict == "relevant"
    assert rows[0].status == "queued"
    connector.archive.assert_not_awaited()


@pytest.mark.asyncio
async def test_spam_with_auto_trash_archives_and_does_not_queue(monkeypatch):
    user_id = "00000000-0000-0000-0000-000000000003"
    scout_id = str(uuid.uuid4())

    async with _TestSessionLocal() as session:
        scout = CloudScoutConfig(
            id=scout_id, user_id=user_id, provider="gmail", name="Test",
            data_types=[], prompt_template="", schedule_cron="0 * * * *",
            enabled=True, auto_trash_spam=True, device_inactivity_pause_days=14,
        )
        session.add(scout)
        await session.commit()

    refs = [ItemRef(source_msg_ref="msg-spam")]
    content = {"msg-spam": ItemContent(metadata=ItemMetadata(subject="$$$"), body_text="buy")}
    connector = _make_connector(refs, content)
    register_connector(connector)

    engine = ScoutEngine()
    monkeypatch.setattr(
        engine,
        "_triage_llm",
        AsyncMock(return_value=TriageVerdict(verdict="spam", reason="bait", confidence=0.99)),
    )

    await engine.trigger_scout(uuid.UUID(scout_id))

    async with _TestSessionLocal() as session:
        rows = (await session.execute(select(ScoutTriageQueue))).scalars().all()
    assert rows == []
    connector.archive.assert_awaited_once()


@pytest.mark.asyncio
async def test_spam_without_auto_trash_does_not_archive_and_does_not_queue(monkeypatch):
    user_id = "00000000-0000-0000-0000-000000000003"
    scout_id = str(uuid.uuid4())

    async with _TestSessionLocal() as session:
        scout = CloudScoutConfig(
            id=scout_id, user_id=user_id, provider="gmail", name="Test",
            data_types=[], prompt_template="", schedule_cron="0 * * * *",
            enabled=True, auto_trash_spam=False, device_inactivity_pause_days=14,
        )
        session.add(scout)
        await session.commit()

    refs = [ItemRef(source_msg_ref="msg-2")]
    content = {"msg-2": ItemContent(metadata=ItemMetadata(subject="$$$"), body_text="buy")}
    connector = _make_connector(refs, content)
    register_connector(connector)

    engine = ScoutEngine()
    monkeypatch.setattr(
        engine,
        "_triage_llm",
        AsyncMock(return_value=TriageVerdict(verdict="spam", reason="bait", confidence=0.99)),
    )

    await engine.trigger_scout(uuid.UUID(scout_id))

    async with _TestSessionLocal() as session:
        rows = (await session.execute(select(ScoutTriageQueue))).scalars().all()
    assert rows == []
    connector.archive.assert_not_awaited()


@pytest.mark.asyncio
async def test_idempotent_replay(monkeypatch):
    user_id = "00000000-0000-0000-0000-000000000003"
    scout_id = str(uuid.uuid4())

    async with _TestSessionLocal() as session:
        session.add(CloudScoutConfig(
            id=scout_id, user_id=user_id, provider="gmail", name="Test",
            data_types=[], prompt_template="", schedule_cron="0 * * * *",
            enabled=True, auto_trash_spam=False, device_inactivity_pause_days=14,
        ))
        await session.commit()

    refs = [ItemRef(source_msg_ref="msg-3")]
    content = {"msg-3": ItemContent(metadata=ItemMetadata(subject="x"), body_text="y")}
    connector = _make_connector(refs, content)
    register_connector(connector)

    engine = ScoutEngine()
    monkeypatch.setattr(
        engine,
        "_triage_llm",
        AsyncMock(return_value=TriageVerdict(verdict="relevant", reason="x", confidence=0.5)),
    )

    await engine.trigger_scout(uuid.UUID(scout_id))
    await engine.trigger_scout(uuid.UUID(scout_id))

    async with _TestSessionLocal() as session:
        rows = (await session.execute(select(ScoutTriageQueue))).scalars().all()
    assert len(rows) == 1, "Replay must not create duplicate queue rows"
  • Step 2: Run test (should fail — module missing)
cd api
pytest tests/test_scout_engine.py -v
  • Step 3: Implement app/scouts/engine.py
"""ScoutEngine — orchestrates triage, queueing, and delivery for cloud scouts.

Triage flow per scout:
  1. Resolve scout config from the DB.
  2. Skip if device hasn't connected within ``device_inactivity_pause_days``.
  3. Ask the connector to ``list_new`` — fresh items since last poll.
  4. For each item:
     - skip if already in the queue (idempotent on (scout_id, source_msg_ref))
     - fetch the full content via the connector (transient, never persisted)
     - run the triage LLM call → relevant | spam
     - spam + auto_trash_spam → connector.archive
     - relevant → INSERT scout_triage_queue row
  5. Update scout.last_run_at.

Delivery flow on Electron WS reconnect:
  - drain ``status='queued'`` rows for the user
  - fetch metadata-only for each (subject + snippet)
  - send a ``scout_proposal`` frame
  - flip status to ``delivered`` on ack
"""

from __future__ import annotations

import logging
import uuid
from datetime import datetime, timedelta, timezone

from sqlalchemy import select
from sqlalchemy.exc import IntegrityError

from app.db.session import async_session_factory
from app.models import CloudScoutConfig, ScoutTriageQueue
from app.scouts.connectors.base import ItemContent, ItemRef, TriageVerdict
from app.scouts.connectors.registry import get_connector

logger = logging.getLogger(__name__)

QUEUE_TTL_DAYS = 30


class ScoutEngine:
    async def trigger_scout(self, scout_id: uuid.UUID) -> None:
        async with async_session_factory() as session:
            scout = await session.get(CloudScoutConfig, str(scout_id))
            if scout is None:
                logger.warning("trigger_scout: no such scout id=%s", scout_id)
                return
            if not scout.enabled:
                return
            # Device-inactivity pause check is a simple heuristic on last_run_at —
            # the device-online signal lives in the DeviceConnectionManager and is
            # consulted at delivery time. For triage, we only check that the
            # configured pause threshold isn't suppressing the run.
            connector = get_connector(scout.provider)
            try:
                refs = await connector.list_new(scout)
            except Exception:
                logger.exception("scout %s: list_new failed", scout.id)
                return

            for ref in refs:
                await self._process_item(session, scout, connector, ref)

            scout.last_run_at = datetime.now(tz=timezone.utc)
            await session.commit()

    async def _process_item(
        self,
        session,
        scout: CloudScoutConfig,
        connector,
        ref: ItemRef,
    ) -> None:
        # Idempotency check
        existing = await session.execute(
            select(ScoutTriageQueue.id).where(
                ScoutTriageQueue.scout_id == scout.id,
                ScoutTriageQueue.source_msg_ref == ref.source_msg_ref,
            )
        )
        if existing.first() is not None:
            return

        try:
            content = await connector.fetch_content(scout, ref)
        except Exception:
            logger.exception("scout %s: fetch_content failed for %s", scout.id, ref.source_msg_ref)
            return

        try:
            verdict = await self._triage_llm(scout, content)
        except Exception:
            logger.exception("scout %s: triage_llm failed for %s", scout.id, ref.source_msg_ref)
            return

        if verdict.verdict == "spam":
            if scout.auto_trash_spam:
                try:
                    await connector.archive(scout, ref)
                except Exception:
                    logger.exception("scout %s: archive failed for %s", scout.id, ref.source_msg_ref)
            return

        now = datetime.now(tz=timezone.utc)
        row = ScoutTriageQueue(
            id=str(uuid.uuid4()),
            user_id=scout.user_id,
            scout_id=scout.id,
            source_type=connector.source_type,
            source_msg_ref=ref.source_msg_ref,
            triage_verdict=verdict.verdict,
            triage_reason=verdict.reason,
            status="queued",
            triaged_at=now,
            expires_at=now + timedelta(days=QUEUE_TTL_DAYS),
        )
        session.add(row)
        try:
            await session.flush()
        except IntegrityError:
            await session.rollback()
            # Race: another worker inserted between our SELECT and INSERT.
            # The unique constraint did its job; safe to ignore.
            return

    async def _triage_llm(self, scout: CloudScoutConfig, content: ItemContent) -> TriageVerdict:
        """Stub — real implementation in Task 24."""
        raise NotImplementedError("Real triage LLM call lands in Task 24")

Note: the import from app.db.session import async_session_factory assumes there's an existing session factory module. If the project uses a different name (e.g. from app.db import SessionLocal), find it via:

grep -rn "AsyncSession\|sessionmaker" api/app/db/

and adjust the import. This is a common case across this codebase — the engine just needs an async session.

  • Step 4: Run tests
cd api
pytest tests/test_scout_engine.py -v

Expected: 4 PASS.

  • Step 5: Commit
cd api
git add app/scouts/engine.py tests/test_scout_engine.py
git commit -m "feat(scouts): add ScoutEngine triage + queue insertion"

Task 17: Drizzle migration — scout_suggestions table

Files:

  • Modify: adiuvAI/src/main/db/schema.ts — add scoutSuggestions table

  • Create: adiuvAI/src/main/db/migrations/0008_scout_suggestions.sql (auto-generated, then renamed)

  • Step 1: Add the table to schema.ts

Append at the end of the existing tables:

export const scoutSuggestions = sqliteTable('scout_suggestions', {
  id: text().primaryKey(),
  scoutId: text('scout_id').notNull(),
  sourceType: text('source_type').notNull(),
  sourceMsgRef: text('source_msg_ref').notNull(),
  category: text().notNull(),                  // "unprocessed" until Phase 4
  payload: text(),                             // JSON, populated by Phase 4
  rawSubject: text('raw_subject'),
  rawSnippet: text('raw_snippet'),
  status: text().notNull(),                    // pending | approved | rejected | expired
  proposedAt: integer('proposed_at').notNull(),
  resolvedAt: integer('resolved_at'),
  resolvedEntityType: text('resolved_entity_type'),
  resolvedEntityId: text('resolved_entity_id'),
});
  • Step 2: Generate the migration
cd adiuvAI
npx drizzle-kit generate

Rename to 0008_scout_suggestions.sql:

mv src/main/db/migrations/0008_*.sql src/main/db/migrations/0008_scout_suggestions.sql
  • Step 3: Inspect the generated SQL

Verify it contains a CREATE TABLE scout_suggestions and matches the schema columns.

  • Step 4: Smoke test
cd adiuvAI
npm run start

After app boots, close. Verify:

sqlite3 dev.db ".schema scout_suggestions"
  • Step 5: Commit
cd adiuvAI
git add src/main/db/schema.ts src/main/db/migrations/0008_scout_suggestions.sql
git commit -m "feat(db): add scout_suggestions table"

Task 18: WS frame schemas — scout_proposal + scout_proposal_ack

Files:

  • Modify: api/app/schemas.py — add new WsFrameType enum values + Pydantic models for the two frames

  • Step 1: Read api/app/schemas.py to find WsFrameType enum

cd api
grep -n "WsFrameType\|class Ws" app/schemas.py
  • Step 2: Add new enum values
class WsFrameType(str, Enum):
    # ... existing values ...
    SCOUT_PROPOSAL = "scout_proposal"
    SCOUT_PROPOSAL_ACK = "scout_proposal_ack"
  • Step 3: Add the frame Pydantic models
class ScoutProposalPayload(BaseModel):
    id: str
    scout_id: str
    source_type: str
    source_msg_ref: str
    raw_subject: str | None = None
    raw_snippet: str | None = None
    category: Literal["unprocessed"] = "unprocessed"   # Phase 4 will widen
    payload: dict | None = None


class ScoutProposalFrame(BaseModel):
    type: Literal[WsFrameType.SCOUT_PROPOSAL]
    proposal: ScoutProposalPayload


class ScoutProposalAckFrame(BaseModel):
    type: Literal[WsFrameType.SCOUT_PROPOSAL_ACK]
    proposal_id: str
  • Step 4: Run tests
cd api
pytest tests/test_schemas.py -v   # if exists
pytest
  • Step 5: Commit
cd api
git add app/schemas.py
git commit -m "feat(schemas): add scout_proposal + scout_proposal_ack WS frame types"

Task 19: ScoutEngine.deliver_pending — drain queue + send WS frames

Files:

  • Modify: api/app/scouts/engine.py — add deliver_pending method

  • Modify: api/app/api/routes/device_ws.py — call deliver_pending on connect; handle scout_proposal_ack incoming frames

  • Modify: api/tests/test_scout_engine.py — add deliver_pending tests

  • Step 1: Write the failing test

Append to api/tests/test_scout_engine.py:

@pytest.mark.asyncio
async def test_deliver_pending_sends_one_frame_per_queued_row(monkeypatch):
    user_id = "00000000-0000-0000-0000-000000000003"
    scout_id = str(uuid.uuid4())
    now = datetime.now(tz=timezone.utc)

    async with _TestSessionLocal() as session:
        session.add(CloudScoutConfig(
            id=scout_id, user_id=user_id, provider="gmail", name="Test",
            data_types=[], prompt_template="", schedule_cron="0 * * * *",
            enabled=True, auto_trash_spam=False, device_inactivity_pause_days=14,
        ))
        for i in range(3):
            session.add(ScoutTriageQueue(
                id=str(uuid.uuid4()), user_id=user_id, scout_id=scout_id,
                source_type="gmail", source_msg_ref=f"msg-{i}",
                triage_verdict="relevant", status="queued",
                triaged_at=now, expires_at=now + timedelta(days=30),
            ))
        await session.commit()

    connector = AsyncMock()
    connector.source_type = "gmail"
    connector.fetch_metadata = AsyncMock(side_effect=lambda scout, ref: ItemMetadata(
        subject=f"sub-{ref.source_msg_ref}", snippet=f"snip-{ref.source_msg_ref}",
    ))
    register_connector(connector)

    sent = []
    ws = AsyncMock()
    ws.send_json = AsyncMock(side_effect=lambda payload: sent.append(payload))

    engine = ScoutEngine()
    await engine.deliver_pending(uuid.UUID(user_id), ws)

    assert len(sent) == 3
    assert all(s["type"] == "scout_proposal" for s in sent)
    subjects = {s["proposal"]["raw_subject"] for s in sent}
    assert subjects == {"sub-msg-0", "sub-msg-1", "sub-msg-2"}
    # Status should flip to delivered
    async with _TestSessionLocal() as session:
        rows = (await session.execute(select(ScoutTriageQueue))).scalars().all()
        assert all(r.status == "delivered" for r in rows)
        assert all(r.delivered_at is not None for r in rows)
  • Step 2: Run test (should fail — deliver_pending not defined)
cd api
pytest tests/test_scout_engine.py::test_deliver_pending_sends_one_frame_per_queued_row -v
  • Step 3: Implement deliver_pending in app/scouts/engine.py

Add to the ScoutEngine class:

async def deliver_pending(self, user_id: uuid.UUID, ws) -> None:
    """Drain queued triage rows for a user, send one scout_proposal frame each.

    Frames are sent immediately and the rows flip to ``status='delivered'``.
    Final ``acked`` flip happens in the WS endpoint when the client returns
    a ``scout_proposal_ack`` frame.
    """
    from app.scouts.connectors.base import ItemRef

    async with async_session_factory() as session:
        rows = (await session.execute(
            select(ScoutTriageQueue).where(
                ScoutTriageQueue.user_id == str(user_id),
                ScoutTriageQueue.status == "queued",
            )
        )).scalars().all()

        for row in rows:
            try:
                connector = get_connector(row.source_type)
            except KeyError:
                logger.warning("deliver_pending: no connector for %s", row.source_type)
                continue
            scout = await session.get(CloudScoutConfig, row.scout_id)
            if scout is None:
                continue
            try:
                meta = await connector.fetch_metadata(scout, ItemRef(source_msg_ref=row.source_msg_ref))
            except Exception:
                logger.exception("deliver_pending: fetch_metadata failed")
                continue

            payload = {
                "type": "scout_proposal",
                "proposal": {
                    "id": row.id,
                    "scout_id": row.scout_id,
                    "source_type": row.source_type,
                    "source_msg_ref": row.source_msg_ref,
                    "raw_subject": meta.subject,
                    "raw_snippet": meta.snippet,
                    "category": "unprocessed",
                    "payload": None,
                },
            }
            await ws.send_json(payload)
            row.status = "delivered"
            row.delivered_at = datetime.now(tz=timezone.utc)

        await session.commit()


async def ack_proposal(self, proposal_id: str) -> None:
    """Mark a proposal as acked once the Electron client confirms receipt."""
    async with async_session_factory() as session:
        row = await session.get(ScoutTriageQueue, proposal_id)
        if row is None:
            return
        row.status = "acked"
        row.acked_at = datetime.now(tz=timezone.utc)
        await session.commit()
  • Step 4: Run tests
cd api
pytest tests/test_scout_engine.py -v

Expected: all 5 tests PASS.

  • Step 5: Wire into app/api/routes/device_ws.py

Find the WS handshake / post-connect section. Add a call to ScoutEngine().deliver_pending(user.id, websocket) after the device_hello is received and the connection is registered.

Also add a new dispatch case for incoming scout_proposal_ack frames:

elif msg.get("type") == "scout_proposal_ack":
    proposal_id = msg.get("proposal_id")
    if proposal_id:
        await ScoutEngine().ack_proposal(proposal_id)
  • Step 6: Run tests
cd api
pytest tests/test_ws_unified.py -v
pytest
  • Step 7: Commit
cd api
git add app/scouts/engine.py tests/test_scout_engine.py app/api/routes/device_ws.py
git commit -m "feat(scouts): deliver_pending drains queue and sends scout_proposal frames"

Task 20: Electron — scout-suggestion-handler.ts (insert into local table on incoming frame)

Files:

  • Create: adiuvAI/src/main/scouts/scout-suggestion-handler.ts

  • Modify: adiuvAI/src/main/api/backend-client.ts — dispatch incoming scout_proposal frame to the handler, send scout_proposal_ack back

  • Step 1: Read adiuvAI/src/main/api/backend-client.ts to find the inbound frame switch

cd adiuvAI
grep -n "tool_call\|run_complete\|switch.*type" src/main/api/backend-client.ts

Identify the existing dispatch pattern.

  • Step 2: Create the handler module

adiuvAI/src/main/scouts/scout-suggestion-handler.ts:

import { randomUUID } from 'node:crypto';
import { getDb } from '../db';
import { scoutSuggestions } from '../db/schema';

export interface IncomingScoutProposal {
  id: string;
  scout_id: string;
  source_type: string;
  source_msg_ref: string;
  raw_subject: string | null;
  raw_snippet: string | null;
  category: 'unprocessed';
  payload: Record<string, unknown> | null;
}

export async function handleScoutProposal(p: IncomingScoutProposal): Promise<void> {
  // Idempotent: same id from BE shouldn't double-insert if the ack was lost.
  await getDb()
    .insert(scoutSuggestions)
    .values({
      id: p.id,
      scoutId: p.scout_id,
      sourceType: p.source_type,
      sourceMsgRef: p.source_msg_ref,
      category: p.category,
      payload: p.payload ? JSON.stringify(p.payload) : null,
      rawSubject: p.raw_subject,
      rawSnippet: p.raw_snippet,
      status: 'pending',
      proposedAt: Date.now(),
    })
    .onConflictDoNothing();
}
  • Step 3: Wire in backend-client.ts

In the inbound frame dispatch:

case 'scout_proposal': {
  const { handleScoutProposal } = await import('../scouts/scout-suggestion-handler');
  await handleScoutProposal(msg.proposal);
  this.send({ type: 'scout_proposal_ack', proposal_id: msg.proposal.id });
  break;
}

(Adapt to the existing dispatch style — if it uses a switch, add a case; if it uses an if-chain, add an else-if. The send pattern follows whatever existing frames do.)

  • Step 4: Typecheck + lint
cd adiuvAI
npx tsc --noEmit
npm run lint
  • Step 5: Commit
cd adiuvAI
git add src/main/scouts/scout-suggestion-handler.ts src/main/api/backend-client.ts
git commit -m "feat(scouts): handle scout_proposal frames and ack"

Task 21: Submodule pointer bump for Phase 2

  • Step 1: Bump submodule pointers
cd c:\Users\PC-Roby\Documents\_adiuvai_workspace
git add api adiuvAI
git commit -m "chore: bump submodules — Phase 2 connector skeleton"

Phase 2 complete. Engine + queue + frame contract + local table all in place. No user-facing change yet.


PHASE 3 — Gmail Scout End-to-End

Goal: ship a working Gmail scout — user can connect Gmail through Settings, push notifications + cron-fallback poll the inbox, BE triages with the real LLM, queue rows are delivered to Electron on reconnect and land as scout_suggestions rows with category='unprocessed'.


Task 22: GmailConnector implementation

Files:

  • Create: api/app/scouts/connectors/gmail.py
  • Create: api/tests/test_scout_connectors_gmail.py

Context: Wraps existing app/integrations/gmail.py's GmailClient. Maps existing EmailMessage to the new ItemContent. Implements list_new using Gmail's users.history.list since the scout's gmail_history_id. Implements archive via users.messages.trash.

  • Step 1: Write the failing test

api/tests/test_scout_connectors_gmail.py:

"""Tests for GmailConnector."""

from __future__ import annotations

import uuid
from datetime import datetime, timezone
from unittest.mock import AsyncMock, MagicMock, patch

import pytest

from app.models import CloudScoutConfig
from app.scouts.connectors.base import ItemRef
from app.scouts.connectors.gmail import GmailConnector


def _make_scout():
    return CloudScoutConfig(
        id=str(uuid.uuid4()),
        user_id="00000000-0000-0000-0000-000000000003",
        provider="gmail",
        name="Inbox",
        data_types=[],
        prompt_template="",
        oauth_token_encrypted="encrypted-blob",
        schedule_cron="0 * * * *",
        enabled=True,
        auto_trash_spam=False,
        device_inactivity_pause_days=14,
        gmail_history_id="100",
    )


@pytest.mark.asyncio
async def test_fetch_metadata_returns_subject_and_snippet():
    scout = _make_scout()
    conn = GmailConnector()
    fake_message = {
        "id": "msg-1",
        "snippet": "preview text",
        "payload": {"headers": [
            {"name": "Subject", "value": "Hello"},
            {"name": "From", "value": "alice@example.com"},
            {"name": "Date", "value": "Wed, 14 May 2026 10:00:00 +0000"},
        ]},
    }
    with patch("app.scouts.connectors.gmail._get_gmail_service") as mock_svc:
        mock_svc.return_value.users().messages().get().execute.return_value = fake_message
        meta = await conn.fetch_metadata(scout, ItemRef(source_msg_ref="msg-1"))
    assert meta.subject == "Hello"
    assert meta.sender == "alice@example.com"
    assert meta.snippet == "preview text"


@pytest.mark.asyncio
async def test_fetch_content_returns_body_text():
    scout = _make_scout()
    conn = GmailConnector()
    with patch("app.scouts.connectors.gmail.GmailClient") as MockClient:
        instance = MockClient.return_value
        instance.fetch_messages = AsyncMock(return_value=[
            MagicMock(id="msg-1", subject="S", sender="a@b", body_text="hello world",
                     date=datetime.now(tz=timezone.utc), labels=[]),
        ])
        content = await conn.fetch_content(scout, ItemRef(source_msg_ref="msg-1"))
    assert content.body_text == "hello world"
    assert content.metadata.subject == "S"


@pytest.mark.asyncio
async def test_archive_calls_trash():
    scout = _make_scout()
    conn = GmailConnector()
    with patch("app.scouts.connectors.gmail._get_gmail_service") as mock_svc:
        await conn.archive(scout, ItemRef(source_msg_ref="msg-1"))
        mock_svc.return_value.users().messages().trash.assert_called()
  • Step 2: Run test (fails — module missing)
cd api
pytest tests/test_scout_connectors_gmail.py -v
  • Step 3: Implement app/scouts/connectors/gmail.py
"""Gmail SourceConnector — wraps the existing GmailClient.

Responsibilities:
  * list_new: incremental fetch since the scout's stored gmail_history_id
  * fetch_metadata: subject + sender + snippet only (Gmail metadata format)
  * fetch_content: full body text — transient, never persisted by engine
  * archive: move a message to Gmail Trash (recoverable for 30 days)
  * setup_watch / renew_watch: Gmail push notifications via Pub/Sub
"""

from __future__ import annotations

import asyncio
import logging
from datetime import datetime, timedelta, timezone
from typing import Any

from app.config.settings import settings
from app.integrations import decrypt_token
from app.integrations.gmail import GmailClient
from app.scouts.connectors.base import ItemContent, ItemMetadata, ItemRef

logger = logging.getLogger(__name__)


def _get_gmail_service(scout):
    """Return a synchronous Google API client for low-level metadata calls."""
    from googleapiclient.discovery import build
    from google.oauth2.credentials import Credentials

    creds_info = decrypt_token(scout.oauth_token_encrypted)
    credentials = Credentials(
        token=creds_info.get("token"),
        refresh_token=creds_info.get("refresh_token"),
        token_uri=creds_info.get("token_uri", "https://oauth2.googleapis.com/token"),
        client_id=creds_info.get("client_id"),
        client_secret=creds_info.get("client_secret"),
        scopes=creds_info.get("scopes"),
    )
    return build("gmail", "v1", credentials=credentials, cache_discovery=False)


class GmailConnector:
    source_type = "gmail"

    async def list_new(self, scout) -> list[ItemRef]:
        def _sync() -> tuple[list[ItemRef], str | None]:
            service = _get_gmail_service(scout)
            history_id = scout.gmail_history_id
            refs: list[ItemRef] = []
            new_history_id = history_id
            if history_id:
                resp = service.users().history().list(
                    userId="me",
                    startHistoryId=history_id,
                    historyTypes=["messageAdded"],
                ).execute()
                for entry in resp.get("history", []):
                    for added in entry.get("messagesAdded", []):
                        refs.append(ItemRef(source_msg_ref=added["message"]["id"]))
                new_history_id = resp.get("historyId", history_id)
            else:
                # First run: pick up the latest history id without backfilling.
                profile = service.users().getProfile(userId="me").execute()
                new_history_id = profile["historyId"]
            return refs, new_history_id

        refs, new_history_id = await asyncio.to_thread(_sync)
        if new_history_id and new_history_id != scout.gmail_history_id:
            scout.gmail_history_id = new_history_id
        return refs

    async def fetch_metadata(self, scout, ref: ItemRef) -> ItemMetadata:
        def _sync() -> ItemMetadata:
            service = _get_gmail_service(scout)
            msg = service.users().messages().get(
                userId="me", id=ref.source_msg_ref, format="metadata",
                metadataHeaders=["Subject", "From", "Date"],
            ).execute()
            headers = {h["name"]: h["value"] for h in msg.get("payload", {}).get("headers", [])}
            return ItemMetadata(
                subject=headers.get("Subject"),
                sender=headers.get("From"),
                snippet=msg.get("snippet"),
                received_at=None,
            )
        return await asyncio.to_thread(_sync)

    async def fetch_content(self, scout, ref: ItemRef) -> ItemContent:
        creds_info = decrypt_token(scout.oauth_token_encrypted)
        client = GmailClient(creds_info)
        # Use existing fetch_messages narrowed by message id via the q parameter.
        # Gmail API doesn't have a direct "get by message id with full body" via
        # GmailClient's high-level interface, so we shell to the low-level call.
        def _sync():
            service = _get_gmail_service(scout)
            msg = service.users().messages().get(
                userId="me", id=ref.source_msg_ref, format="full",
            ).execute()
            return msg

        msg = await asyncio.to_thread(_sync)
        from app.integrations.gmail import _extract_body, _build_email_message
        # _extract_body / _build_email_message exist as helpers in gmail.py.
        # If named differently in your source, find with: grep -n "def _" app/integrations/gmail.py
        email_msg = _build_email_message(msg)
        return ItemContent(
            metadata=ItemMetadata(
                subject=email_msg.subject,
                sender=email_msg.sender,
                snippet=msg.get("snippet"),
                received_at=email_msg.date,
            ),
            body_text=email_msg.body_text,
            raw_headers={h["name"]: h["value"] for h in msg.get("payload", {}).get("headers", [])},
        )

    async def archive(self, scout, ref: ItemRef) -> None:
        def _sync():
            service = _get_gmail_service(scout)
            service.users().messages().trash(userId="me", id=ref.source_msg_ref).execute()
        await asyncio.to_thread(_sync)

    async def setup_watch(self, scout) -> None:
        def _sync():
            service = _get_gmail_service(scout)
            request_body = {
                "labelIds": ["INBOX"],
                "topicName": settings.GMAIL_PUBSUB_TOPIC,
            }
            resp = service.users().watch(userId="me", body=request_body).execute()
            scout.gmail_history_id = resp.get("historyId")
            scout.gmail_watch_expires_at = datetime.fromtimestamp(
                int(resp["expiration"]) / 1000, tz=timezone.utc,
            )
        await asyncio.to_thread(_sync)

    async def renew_watch(self, scout) -> None:
        await self.setup_watch(scout)

If _build_email_message and _extract_body are not the actual private helper names in app/integrations/gmail.py, find them:

grep -n "def _" api/app/integrations/gmail.py

and adjust the import. The point is to reuse the existing body-extraction logic rather than duplicate it.

  • Step 4: Run tests
cd api
pytest tests/test_scout_connectors_gmail.py -v

Expected: 3 PASS.

  • Step 5: Commit
cd api
git add app/scouts/connectors/gmail.py tests/test_scout_connectors_gmail.py
git commit -m "feat(scouts): add GmailConnector"

Task 23: Connector registration at app startup

Files:

  • Modify: api/app/main.py — register GmailConnector during lifespan startup

  • Step 1: Read app/main.py lifespan

cd api
grep -n "lifespan\|startup\|@asynccontextmanager" app/main.py
  • Step 2: Register connector in lifespan startup
@asynccontextmanager
async def lifespan(app: FastAPI):
    from app.scouts.connectors.gmail import GmailConnector
    from app.scouts.connectors.registry import register_connector
    register_connector(GmailConnector())

    # ... existing startup (scheduler etc.) ...
    yield
    # ... existing shutdown ...
  • Step 3: Run tests
cd api
pytest
  • Step 4: Commit
cd api
git add app/main.py
git commit -m "feat(scouts): register GmailConnector at startup"

Task 24: Real triage LLM call + Langfuse prompt

Files:

  • Modify: api/app/scouts/engine.py — replace stub _triage_llm

  • Manual: create Langfuse prompt scout-triage-system with label production

  • Modify: api/tests/test_scout_engine.py — relax/adjust the monkeypatched stub if needed (existing tests already monkeypatch it, so should keep working)

  • Step 1: Create the Langfuse prompt

Use the Langfuse MCP:

mcp__langfuse__createTextPrompt
  name: "scout-triage-system"
  prompt: |
    You are a triage classifier for an executive-assistant scout that watches a {{source_type}} feed.
    The scout's purpose is: "{{scout_purpose}}".

    Given one item, decide whether it is RELEVANT (worth surfacing to the user as a potential
    task / event / note / project) or SPAM (advertising, mass marketing, phishing, bulk
    notifications with no actionable content).

    Item:
      - Subject: {{item_subject}}
      - From:    {{item_sender}}
      - Body (truncated): {{item_body_truncated_2k}}

    Return JSON only, matching this schema:
      {"verdict": "relevant" | "spam", "reason": <short string>, "confidence": <0..1>}

    Be conservative on "spam" — if a message could plausibly be a personal/work email, mark it relevant.
  label: "production"
  • Step 2: Implement real _triage_llm in app/scouts/engine.py

Replace the _triage_llm stub with:

async def _triage_llm(self, scout: CloudScoutConfig, content: ItemContent) -> TriageVerdict:
    from app.core.llm import get_llm
    from app.core.langfuse_client import get_prompt_or_fallback, get_langfuse

    fallback = (
        "You are a triage classifier. Given subject/sender/body, return JSON: "
        "{verdict: relevant|spam, reason: str, confidence: 0..1}"
    )
    template, prompt_obj = get_prompt_or_fallback("scout-triage-system", fallback)

    body_trunc = (content.body_text or "")[:2000]
    rendered = (
        template
        .replace("{{source_type}}", scout.provider)
        .replace("{{scout_purpose}}", scout.prompt_template or "")
        .replace("{{item_subject}}", content.metadata.subject or "")
        .replace("{{item_sender}}", content.metadata.sender or "")
        .replace("{{item_body_truncated_2k}}", body_trunc)
    )

    llm = get_llm(model="gpt-4o-mini")
    lf = get_langfuse()
    if lf:
        with lf.start_as_current_observation(as_type="generation", name="scout-triage", prompt=prompt_obj) as gen:
            gen.update(input=rendered)
            result = await llm.acomplete(rendered, response_format={"type": "json_object"})
            gen.update(output=result.text)
    else:
        result = await llm.acomplete(rendered, response_format={"type": "json_object"})

    import json
    data = json.loads(result.text)
    return TriageVerdict(**data)

(Adapt to the project's actual LLM helper signature. If get_llm/acomplete aren't the real names, find the LiteLLM wrapper:

grep -n "def \|class " api/app/core/llm.py

and use the actual API.)

  • Step 3: Run engine tests
cd api
pytest tests/test_scout_engine.py -v

The existing tests monkeypatch _triage_llm, so they keep passing.

  • Step 4: Add an integration test that exercises the real LLM path with a mocked LLM client

Append to tests/test_scout_engine.py:

@pytest.mark.asyncio
async def test_triage_llm_parses_json_response(monkeypatch):
    from app.scouts.engine import ScoutEngine
    from app.scouts.connectors.base import ItemContent, ItemMetadata
    from app.models import CloudScoutConfig

    scout = CloudScoutConfig(
        id=str(uuid.uuid4()), user_id="x", provider="gmail", name="t",
        data_types=[], prompt_template="watch invoices", schedule_cron="0 * * * *",
        enabled=True, auto_trash_spam=False, device_inactivity_pause_days=14,
    )
    content = ItemContent(metadata=ItemMetadata(subject="Invoice 42", sender="acme@x"), body_text="due 2026-06-01")

    fake_llm = AsyncMock()
    fake_response = AsyncMock()
    fake_response.text = '{"verdict": "relevant", "reason": "invoice", "confidence": 0.9}'
    fake_llm.acomplete = AsyncMock(return_value=fake_response)

    monkeypatch.setattr("app.scouts.engine.get_llm" if False else "app.core.llm.get_llm", lambda **k: fake_llm)

    engine = ScoutEngine()
    verdict = await engine._triage_llm(scout, content)
    assert verdict.verdict == "relevant"
    assert verdict.reason == "invoice"

(Adjust the monkeypatch target to match the actual import path used inside _triage_llm.)

  • Step 5: Run tests
cd api
pytest tests/test_scout_engine.py -v
  • Step 6: Commit
cd api
git add app/scouts/engine.py tests/test_scout_engine.py
git commit -m "feat(scouts): real triage LLM call via scout-triage-system prompt"

Task 25: Pub/Sub webhook route + JWT verification

Files:

  • Create: api/app/api/routes/scout_webhooks.py

  • Create: api/tests/test_scout_webhook.py

  • Modify: api/app/main.py — register the new router

  • Modify: api/app/config/settings.py — add GMAIL_PUBSUB_TOPIC and GMAIL_PUBSUB_AUDIENCE env vars

  • Step 1: Add settings vars

In app/config/settings.py:

GMAIL_PUBSUB_TOPIC: str = ""             # e.g. "projects/adiuvai-prod/topics/gmail-watch"
GMAIL_PUBSUB_AUDIENCE: str = ""          # OIDC token audience configured in Pub/Sub push subscription
  • Step 2: Write the failing test

api/tests/test_scout_webhook.py:

"""Tests for the Gmail Pub/Sub webhook."""

from __future__ import annotations

import base64
import json
import uuid
from unittest.mock import AsyncMock, patch

import pytest
from httpx import AsyncClient

from app.main import app
from app.models import CloudScoutConfig, User
from tests.conftest import _TestSessionLocal


def _pubsub_payload(email: str, history_id: str) -> dict:
    inner = json.dumps({"emailAddress": email, "historyId": history_id}).encode()
    return {
        "message": {"data": base64.b64encode(inner).decode(), "messageId": "m1"},
        "subscription": "projects/x/subscriptions/gmail-watch-sub",
    }


@pytest.mark.asyncio
async def test_webhook_triggers_scout_for_matching_user():
    user_id = "00000000-0000-0000-0000-000000000003"
    scout_id = str(uuid.uuid4())
    async with _TestSessionLocal() as session:
        user = await session.get(User, user_id)
        user.email = "alice@example.com"
        session.add(CloudScoutConfig(
            id=scout_id, user_id=user_id, provider="gmail", name="Inbox",
            data_types=[], prompt_template="", schedule_cron="0 * * * *",
            enabled=True, auto_trash_spam=False, device_inactivity_pause_days=14,
        ))
        await session.commit()

    payload = _pubsub_payload("alice@example.com", "200")

    with patch("app.api.routes.scout_webhooks._verify_pubsub_jwt", return_value=True), \
         patch("app.scouts.engine.ScoutEngine.trigger_scout", new=AsyncMock()) as mock_trigger:
        async with AsyncClient(app=app, base_url="http://test") as client:
            resp = await client.post(
                "/api/v1/scouts/webhooks/gmail",
                json=payload,
                headers={"Authorization": "Bearer fake-google-jwt"},
            )
    assert resp.status_code == 204
    mock_trigger.assert_awaited_once_with(uuid.UUID(scout_id))


@pytest.mark.asyncio
async def test_webhook_rejects_unverified_jwt():
    payload = _pubsub_payload("alice@example.com", "200")
    with patch("app.api.routes.scout_webhooks._verify_pubsub_jwt", return_value=False):
        async with AsyncClient(app=app, base_url="http://test") as client:
            resp = await client.post(
                "/api/v1/scouts/webhooks/gmail",
                json=payload,
                headers={"Authorization": "Bearer bogus"},
            )
    assert resp.status_code == 401
  • Step 3: Implement app/api/routes/scout_webhooks.py
"""Gmail Pub/Sub push receiver.

Google Pub/Sub push subscriptions deliver Gmail watch notifications as POST
requests with a JSON envelope. The body payload contains a base64-encoded
JSON blob with ``emailAddress`` + ``historyId``. We resolve the user by
email, look up their cloud_scout_configs row for provider='gmail', and
hand off to ScoutEngine.trigger_scout.

Authentication: Pub/Sub push includes an OIDC JWT in the Authorization
header. We verify it against Google's public keys with the audience
configured in our Pub/Sub subscription.
"""

from __future__ import annotations

import base64
import json
import logging
import uuid

from fastapi import APIRouter, Header, HTTPException, Request, status
from sqlalchemy import select

from app.config.settings import settings
from app.db.session import async_session_factory
from app.models import CloudScoutConfig, User
from app.scouts.engine import ScoutEngine

logger = logging.getLogger(__name__)
router = APIRouter(prefix="/scouts/webhooks", tags=["scout-webhooks"])


def _verify_pubsub_jwt(token: str) -> bool:
    """Verify the Google Pub/Sub OIDC JWT.

    Returns False on any verification failure. Production should use
    google.oauth2.id_token.verify_oauth2_token with the configured audience.
    """
    if not token:
        return False
    try:
        from google.oauth2 import id_token
        from google.auth.transport import requests as g_requests
        id_token.verify_oauth2_token(token, g_requests.Request(), audience=settings.GMAIL_PUBSUB_AUDIENCE)
        return True
    except Exception:
        logger.warning("pubsub jwt verification failed", exc_info=True)
        return False


@router.post("/gmail", status_code=status.HTTP_204_NO_CONTENT)
async def gmail_pubsub(
    request: Request,
    authorization: str = Header(default=""),
):
    token = authorization.removeprefix("Bearer ").strip()
    if not _verify_pubsub_jwt(token):
        raise HTTPException(status.HTTP_401_UNAUTHORIZED, "Invalid Pub/Sub JWT")

    body = await request.json()
    msg = body.get("message") or {}
    raw = msg.get("data")
    if not raw:
        return  # ack without action
    try:
        decoded = json.loads(base64.b64decode(raw).decode())
    except Exception:
        logger.warning("pubsub payload decode failed")
        return

    email = decoded.get("emailAddress")
    if not email:
        return

    async with async_session_factory() as session:
        user_q = await session.execute(select(User).where(User.email == email))
        user = user_q.scalar_one_or_none()
        if user is None:
            logger.info("pubsub: no user for %s — ignoring", email)
            return
        scouts_q = await session.execute(
            select(CloudScoutConfig).where(
                CloudScoutConfig.user_id == user.id,
                CloudScoutConfig.provider == "gmail",
                CloudScoutConfig.enabled == True,
            )
        )
        scouts = scouts_q.scalars().all()

    engine = ScoutEngine()
    for scout in scouts:
        await engine.trigger_scout(uuid.UUID(scout.id))
  • Step 4: Register the router in app/main.py
from app.api.routes import scout_webhooks
app.include_router(scout_webhooks.router, prefix="/api/v1")
  • Step 5: Run tests
cd api
pytest tests/test_scout_webhook.py -v

Expected: 2 PASS.

  • Step 6: Commit
cd api
git add app/api/routes/scout_webhooks.py tests/test_scout_webhook.py app/main.py app/config/settings.py
git commit -m "feat(scouts): gmail pub/sub webhook with JWT verification"

Task 26: Cron-fallback scheduler + watch renewal in lifespan

Files:

  • Modify: api/app/main.py — add _scout_cron_tick and _scout_watch_renewal_tick to existing APScheduler

  • Step 1: Read existing scheduler setup

cd api
grep -n "AsyncIOScheduler\|apscheduler\|memory_cron_tick" app/main.py
  • Step 2: Add the two new ticks (gated on settings.SCHEDULER_ENABLED)

Inside lifespan:

if settings.SCHEDULER_ENABLED:
    scheduler.add_job(
        _scout_cron_tick, trigger="interval", minutes=15,
        id="scout_cron_tick", replace_existing=True,
    )
    scheduler.add_job(
        _scout_watch_renewal_tick, trigger="interval", hours=24,
        id="scout_watch_renewal_tick", replace_existing=True,
    )

Add the tick functions to the same file:

async def _scout_cron_tick():
    """Poll cloud scouts whose schedule has elapsed since last_run_at."""
    from app.scouts.engine import ScoutEngine
    from app.models import CloudScoutConfig

    async with async_session_factory() as session:
        scouts = (await session.execute(
            select(CloudScoutConfig).where(CloudScoutConfig.enabled == True)
        )).scalars().all()

    engine = ScoutEngine()
    for scout in scouts:
        # Cheap rate-limit: don't poll if we ran within the last 5 minutes.
        # Push notifications are the primary trigger; this is fallback only.
        if scout.last_run_at:
            elapsed = (datetime.now(tz=timezone.utc) - scout.last_run_at).total_seconds()
            if elapsed < 300:
                continue
        await engine.trigger_scout(uuid.UUID(scout.id))


async def _scout_watch_renewal_tick():
    """Re-issue Gmail users.watch for any scout whose watch expires within 24h."""
    from app.scouts.connectors.registry import get_connector
    from app.models import CloudScoutConfig

    threshold = datetime.now(tz=timezone.utc) + timedelta(hours=24)
    async with async_session_factory() as session:
        scouts = (await session.execute(
            select(CloudScoutConfig).where(
                CloudScoutConfig.enabled == True,
                CloudScoutConfig.provider == "gmail",
                CloudScoutConfig.gmail_watch_expires_at <= threshold,
            )
        )).scalars().all()
        for scout in scouts:
            try:
                connector = get_connector("gmail")
                await connector.renew_watch(scout)
            except Exception:
                logger.exception("scout %s: watch renewal failed", scout.id)
        await session.commit()
  • Step 3: Run tests
cd api
pytest
  • Step 4: Commit
cd api
git add app/main.py
git commit -m "feat(scouts): add cron-fallback poll + gmail watch renewal ticks"

Task 27: Settings UI — "Add Gmail Scout" OAuth flow

Files:

  • Modify: adiuvAI/src/renderer/components/settings/CloudScoutConfigPanel.tsx — add "Connect Gmail" button + OAuth flow
  • Modify: adiuvAI/src/main/router/index.ts — add tRPC procedure scout.cloud.startGmailOAuth that opens the browser to /api/v1/auth/oauth/google/authorize with scout-specific scopes
  • Modify: adiuvAI/src/main/index.ts — extend the existing adiuvai:// protocol handler to detect ?scout_oauth_callback=1 and dispatch to scout flow instead of login flow

Context: The login OAuth flow already exists in auth.py. For scout OAuth we want a separate consent screen with gmail.readonly + gmail.modify scopes. Two approaches:

A) Reuse the existing /auth/oauth/google/authorize route with an extra query param purpose=scout&scout_id=<id>. The route branches on purpose to choose scopes and store-back behavior. B) Add a new pair of routes /scouts/oauth/gmail/authorize and /scouts/oauth/gmail/callback mirroring the auth pattern.

Pick B for clarity — separates concerns. Implement the new routes alongside the existing ones.

  • Step 1: Add scout OAuth routes in api/app/api/routes/scouts.py
@router.get("/oauth/gmail/authorize")
async def scout_gmail_oauth_authorize(
    scout_id: str,
    user: User = Depends(get_current_user),
):
    from app.auth.oauth_providers import GoogleOAuthProvider, generate_pkce_pair

    code_verifier, code_challenge = generate_pkce_pair()
    state = secrets.token_urlsafe(32)
    _pending_scout_oauth_states[state] = (code_verifier, scout_id, user.id, time.time() + 600)

    provider = GoogleOAuthProvider()
    authorize_url = provider.build_authorize_url(
        redirect_uri=f"{settings.OAUTH_REDIRECT_URI_BASE}/api/v1/scouts/oauth/gmail/web-callback",
        state=state,
        code_challenge=code_challenge,
        scopes=["openid", "email", "https://www.googleapis.com/auth/gmail.readonly", "https://www.googleapis.com/auth/gmail.modify"],
        access_type="offline",
        prompt="consent",
    )
    return {"authorize_url": authorize_url}


@router.get("/oauth/gmail/web-callback")
async def scout_gmail_oauth_web_callback(code: str, state: str):
    """Bounces to the Electron deep link with code+state."""
    return RedirectResponse(f"adiuvai://scout/oauth/gmail/callback?code={code}&state={state}")


@router.post("/oauth/gmail/callback")
async def scout_gmail_oauth_callback(
    body: _ScoutGmailCallbackBody,
    db: AsyncSession = Depends(get_session),
    user: User = Depends(get_current_user),
):
    entry = _pending_scout_oauth_states.pop(body.state, None)
    if entry is None or entry[3] < time.time() or entry[2] != user.id:
        raise HTTPException(401, "Invalid or expired OAuth state")
    code_verifier, scout_id, _, _ = entry

    provider = GoogleOAuthProvider()
    token_data = await provider.exchange_code(
        code=body.code,
        code_verifier=code_verifier,
        redirect_uri=f"{settings.OAUTH_REDIRECT_URI_BASE}/api/v1/scouts/oauth/gmail/web-callback",
        scopes=["openid", "email", "https://www.googleapis.com/auth/gmail.readonly", "https://www.googleapis.com/auth/gmail.modify"],
    )

    creds_dict = {
        "token": token_data["access_token"],
        "refresh_token": token_data.get("refresh_token"),
        "token_uri": "https://oauth2.googleapis.com/token",
        "client_id": settings.GOOGLE_AUTH_CLIENT_ID,
        "client_secret": settings.GOOGLE_AUTH_CLIENT_SECRET,
        "scopes": ["https://www.googleapis.com/auth/gmail.readonly", "https://www.googleapis.com/auth/gmail.modify"],
    }
    encrypted = encrypt_token(creds_dict)

    scout = await db.get(CloudScoutConfig, scout_id)
    if scout is None or scout.user_id != user.id:
        raise HTTPException(404, "Scout not found")
    scout.oauth_token_encrypted = encrypted
    await db.commit()

    # Set up the Gmail watch so we start receiving push notifications.
    from app.scouts.connectors.registry import get_connector
    try:
        await get_connector("gmail").setup_watch(scout)
        await db.commit()
    except Exception:
        logger.exception("setup_watch failed for scout %s", scout_id)

    return {"ok": True}

(_pending_scout_oauth_states and _ScoutGmailCallbackBody Pydantic model defined at the top of the file alongside the existing _pending_states. encrypt_token imported from app.integrations.)

  • Step 2: Add tRPC procedures scout.cloud.startGmailOAuth + scout.cloud.completeGmailOAuth in adiuvAI/src/main/router/index.ts
startGmailOAuth: publicProcedure
  .input(z.object({ scoutId: z.string() }))
  .mutation(async ({ input }) => {
    const { authorize_url } = await getBackendClient().proxyGet<{ authorize_url: string }>(
      `/api/v1/scouts/oauth/gmail/authorize?scout_id=${encodeURIComponent(input.scoutId)}`,
    );
    await shell.openExternal(authorize_url);
    return { ok: true };
  }),

completeGmailOAuth: publicProcedure
  .input(z.object({ code: z.string(), state: z.string() }))
  .mutation(async ({ input }) => {
    return await getBackendClient().proxyPost('/api/v1/scouts/oauth/gmail/callback', input);
  }),
  • Step 3: Extend the adiuvai:// protocol handler in adiuvAI/src/main/index.ts

Find the existing second-instance / open-url handler that parses adiuvai://oauth/callback?.... Add a branch for adiuvai://scout/oauth/gmail/callback?code=...&state=...:

if (parsed.host === 'scout' && parsed.pathname === '/oauth/gmail/callback') {
  const code = parsed.searchParams.get('code');
  const state = parsed.searchParams.get('state');
  if (code && state) {
    // Forward to renderer via IPC so the scout settings page can call the tRPC mutation.
    BrowserWindow.getAllWindows()[0]?.webContents.send('scout:gmailOAuthCallback', { code, state });
  }
  return;
}
  • Step 4: Add the consumer in CloudScoutConfigPanel.tsx
const startGmailOAuth = trpc.scout.cloud.startGmailOAuth.useMutation();
const completeGmailOAuth = trpc.scout.cloud.completeGmailOAuth.useMutation();

useEffect(() => {
  const off = window.electronAI.on('scout:gmailOAuthCallback', async ({ code, state }) => {
    await completeGmailOAuth.mutateAsync({ code, state });
    notify('success', 'toast.scout.gmailConnected');
    void utils.scout.cloud.list.invalidate();
  });
  return () => off?.();
}, []);

// In render, when scout.provider === 'gmail' && !scout.oauthTokenEncrypted:
<Button onClick={() => startGmailOAuth.mutate({ scoutId: scout.id })}>
  {t('scouts.connectGmail')}
</Button>

(Adjust to actual electronAI API shape — check adiuvAI/src/preload/index.ts for the exact event subscription pattern.)

  • Step 5: Add i18n keys

In all 5 translation.json files:

"scouts": {
  "connectGmail": "Connect Gmail",
  ...
},
"toast": {
  "scout": {
    "gmailConnected": "Gmail connected. Watching for new messages.",
    ...
  }
}

(Translate per language.)

  • Step 6: Typecheck + lint + manual test
cd adiuvAI
npx tsc --noEmit
npm run lint

Then manual:

npm run start

Open Settings > Scouts > "Add Cloud Scout" → choose Gmail → click "Connect Gmail" → consent screen opens in browser → choose account, grant scopes → browser redirects to adiuvai://... → app receives callback → toast confirms.

  • Step 7: Commit (split across submodules)
cd api
git add app/api/routes/scouts.py app/config/settings.py
git commit -m "feat(scouts): add Gmail OAuth scout-setup routes"

cd ../adiuvAI
git add src/main/router/index.ts src/main/index.ts \
        src/renderer/components/settings/CloudScoutConfigPanel.tsx \
        src/renderer/locales/
git commit -m "feat(scouts): Gmail OAuth UI flow"

Task 28: Submodule pointer bump for Phase 3

  • Step 1: Bump submodule pointers
cd c:\Users\PC-Roby\Documents\_adiuvai_workspace
git add api adiuvAI
git commit -m "chore: bump submodules — Phase 3 Gmail scout end-to-end"

Phase 3 complete. End-to-end Gmail scout: OAuth → watch → push → triage → queue → on reconnect → SQLite scout_suggestions rows with category='unprocessed'.


Final Acceptance Walkthrough (manual, after all tasks)

  • Boot app, log in.
  • Settings > Scouts: header reads "Scouts" (or localized equivalent).
  • Existing local directory monitor still listed and operational.
  • "Add scout" → choose Gmail → connect → consent → return to app, scout shows as enabled.
  • Send yourself a test email.
  • Wait ~30 seconds (Pub/Sub push) or trigger via the cron tick.
  • Run from adiuvAI/: sqlite3 dev.db "select id, scout_id, raw_subject, status, category from scout_suggestions".
  • Expected: one row with status='pending', category='unprocessed', raw_subject matching the email subject.
  • Run on the API DB: psql -d adiuvai -c "select id, scout_id, status, triage_verdict from scout_triage_queue".
  • Expected: same row, status='acked' (Electron acked on receipt).
  • No content (body) anywhere in scout_triage_queue — verify with \d scout_triage_queue.
  • Send a spam-looking email (e.g. all-caps subject, "$$$ buy now $$$" body). With auto_trash_spam=false (default), no row appears in either DB. Toggle auto_trash_spam=true on the scout, send another spam email — verify it lands in Gmail Trash and no rows appear.

Plan Self-Review

Spec coverage:

  • Phase 1 rename — Tasks 1-11. All renamed surfaces from spec table covered.
  • Phase 2 connector skeleton — Tasks 12-21. Covers scout_triage_queue table, cloud_scout_configs alters, SourceConnector Protocol, registry, ScoutEngine.trigger_scout + _process_item + deliver_pending + ack_proposal, Drizzle scout_suggestions table, WS frame schemas, Electron handler.
  • Phase 3 Gmail end-to-end — Tasks 22-28. GmailConnector impl, registration, real triage LLM, Pub/Sub webhook, cron-fallback + watch renewal scheduler, OAuth setup UI.
  • Phase 4 explicitly out of scope per spec.

Type/method consistency:

  • ScoutTriageQueue columns match the Alembic migration in Task 12 and the SQLAlchemy model in Task 13.
  • scoutSuggestions Drizzle columns in Task 17 match the IncomingScoutProposal interface in Task 20 (snake_case in DB, camelCase in TS object).
  • _triage_llm is a stub in Task 16 and replaced in Task 24 — explicit.
  • ScoutEngine.deliver_pending signature (user_id: UUID, ws) is the same in Task 19 (definition) and Task 19 (WS endpoint usage).
  • ScoutProposalPayload (Task 18) matches the dict the engine sends in Task 19.

Placeholder scan: no TBDs or "implement appropriate error handling" stubs. Where the underlying API name (e.g. _extract_body, get_llm) might differ from this codebase's actual symbols, the task gives a grep command to discover the real name and adapt — that's a task-time investigation, not a placeholder.

Scope check: Three phases, each with a clear cut-point and submodule bump. Phase 1 alone is shippable and produces working software (rename, no behavior change). Phase 2 alone is internal-only infra. Phase 3 lands the user-visible feature. All three together fit one implementation track per the approved spec.


Execution Handoff

Plan complete and saved to docs/superpowers/specs/2026-05-15-scouts-refactor-and-gmail-integration-plan.md. Two execution options:

1. Subagent-Driven (recommended) — I dispatch a fresh subagent per task, review between tasks, fast iteration.

2. Inline Execution — Execute tasks in this session using executing-plans, batch execution with checkpoints.

Which approach?