Covers Phases 1-3 (rename, connector skeleton, Gmail end-to-end) as 28 TDD tasks. Phase 4 (Stage 2 categorization + brief HITL) deferred to separate spec. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
108 KiB
Scouts Refactor + Gmail Integration Plan
For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (
- [ ]) syntax for tracking.
Goal: Rename the entire "Agents" subsystem to "Scouts" across UI/code/Postgres/SQLite/Langfuse, then add a Gmail cloud scout with two-stage pipeline (BE triage + on-connect delivery) and a reusable SourceConnector abstraction. Phase 4 (Stage 2 categorization + HITL surface) is deferred to a future spec.
Architecture: Phase 1 is a single atomic rename PR (no behavior change). Phase 2 lays connector skeleton, queue table, WS frame contract, and SQLite suggestion table — no user-visible feature yet. Phase 3 ships Gmail end-to-end: OAuth setup UI, push (users.watch) + cron-fallback polling, BE-side spam triage with opt-in auto-trash, deliver-on-reconnect drain into local scoutSuggestions table with category='unprocessed' stub.
Tech Stack: Python 3.12 / FastAPI / SQLAlchemy 2.0 (async) / Alembic / Pydantic v2 / Langfuse for prompts / Google API Client for Gmail / Pub/Sub for push notifications. Electron / TypeScript / React 19 / tRPC v11 / Drizzle ORM / better-sqlite3 / i18next.
Spec: docs/superpowers/specs/2026-05-15-scouts-refactor-and-gmail-integration-design.md
Repository layout: monorepo with submodules api/ (FastAPI) and adiuvAI/ (Electron). All tasks operate on absolute paths under c:\Users\PC-Roby\Documents\_adiuvai_workspace.
File Structure
New files (BE — api/)
| Path | Responsibility |
|---|---|
api/alembic/versions/007_rename_agents_to_scouts.py |
Rename Postgres tables/columns from *_agent_* to *_scout_*. |
api/alembic/versions/008_scout_triage_queue.py |
Create scout_triage_queue + alter cloud_scout_configs. |
api/app/scouts/__init__.py |
Package marker. |
api/app/scouts/engine.py |
ScoutEngine orchestrator (triage, queue, deliver). |
api/app/scouts/connectors/__init__.py |
Package marker. |
api/app/scouts/connectors/base.py |
SourceConnector Protocol + ItemRef/ItemMetadata/ItemContent/TriageVerdict Pydantic models. |
api/app/scouts/connectors/registry.py |
Connector registration + lookup. |
api/app/scouts/connectors/gmail.py |
GmailConnector impl (wraps existing app/integrations/gmail.py). |
api/app/api/routes/scout_webhooks.py |
POST /api/v1/scouts/webhooks/gmail — Pub/Sub receiver. |
api/tests/test_scout_engine.py |
Unit tests for ScoutEngine w/ mocked connector. |
api/tests/test_scout_connectors_gmail.py |
Tests for GmailConnector (mocked Gmail API). |
api/tests/test_scout_webhook.py |
Webhook + Pub/Sub JWT verification. |
Renamed files (BE)
| Before | After |
|---|---|
api/app/api/routes/agents.py |
api/app/api/routes/scouts.py |
api/app/api/routes/agent_setup.py |
api/app/api/routes/scout_setup.py |
api/app/core/agent_runner.py |
api/app/core/scout_runner.py |
api/app/core/agent_session_buffer.py |
api/app/core/scout_session_buffer.py |
api/app/core/agent_registry.py |
api/app/core/scout_registry.py |
New files (Electron — adiuvAI/)
| Path | Responsibility |
|---|---|
adiuvAI/src/main/db/migrations/0007_scouts_rename.sql |
Drizzle SQL: rename agent_runs → scout_runs, agent_run_actions → scout_run_actions. |
adiuvAI/src/main/db/migrations/0008_scout_suggestions.sql |
Drizzle SQL: create scout_suggestions table. |
adiuvAI/src/main/scouts/scout-suggestion-handler.ts |
Handles incoming scout_proposal WS frame, inserts into scoutSuggestions, sends scout_proposal_ack. |
Renamed files (Electron)
| Before | After |
|---|---|
adiuvAI/src/main/agents/agent-scheduler.ts |
adiuvAI/src/main/scouts/scout-scheduler.ts |
adiuvAI/src/renderer/components/settings/AgentsSection.tsx |
adiuvAI/src/renderer/components/settings/ScoutsSection.tsx |
adiuvAI/src/renderer/components/settings/AgentRow.tsx |
adiuvAI/src/renderer/components/settings/ScoutRow.tsx |
adiuvAI/src/renderer/components/settings/LocalAgentConfigPanel.tsx |
adiuvAI/src/renderer/components/settings/LocalScoutConfigPanel.tsx |
adiuvAI/src/renderer/components/settings/CloudAgentConfigPanel.tsx |
adiuvAI/src/renderer/components/settings/CloudScoutConfigPanel.tsx |
adiuvAI/src/renderer/components/settings/InlineAgentCreationStepper.tsx |
adiuvAI/src/renderer/components/settings/InlineScoutCreationStepper.tsx |
Modified files (high-impact, full list per task)
api/app/models.py— class renames + table/column renames + relationship updates.api/app/main.py— register new router, lifespan crons.api/app/schemas.py—WsFrameTypeenum + new frame Pydantic models.api/app/config/settings.py— add Gmail Pub/Sub topic env vars.adiuvAI/src/main/db/schema.ts— rename tables + addscoutSuggestions.adiuvAI/src/main/router/index.ts— renameagent.*sub-routers →scout.*.adiuvAI/src/main/api/backend-client.ts— handle new WS frames.adiuvAI/src/renderer/locales/{en,it,es,fr,de}/translation.json— rename i18n keys.adiuvAI/src/renderer/routes/settings.tsx— update SECTIONS array.
PHASE 1 — Rename Only
Goal: pure rename, no behavior change, all tests green at end of phase.
Important caveat: the project uses git submodules. Each task touches files in api/ or adiuvAI/ submodule — commits happen inside the submodule, then a separate commit in the monorepo updates the submodule pointer. Each task's commit step shows both.
Task 1: Postgres rename — Alembic migration
Files:
- Create:
api/alembic/versions/007_rename_agents_to_scouts.py
Context: Latest existing migration is d6e3f4a5b6c7_folder_index_tables.py with down_revision = "006". New migration uses revision "007" and depends on d6e3f4a5b6c7. Tables to rename: local_agent_configs, cloud_agent_configs, agent_run_logs. Columns to rename: agent_config (in local_agent_configs), agent_id and agent_type (in agent_run_logs).
- Step 1: Create the migration file
"""Rename agents to scouts.
Revision ID: 007
Revises: d6e3f4a5b6c7
Create Date: 2026-05-15
Renames the entire agents subsystem identifiers to scouts.
Pre-1.0 — no data preservation concerns beyond ALTER TABLE rename.
"""
from typing import Sequence, Union
from alembic import op
revision: str = "007"
down_revision: Union[str, None] = "d6e3f4a5b6c7"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# Tables
op.rename_table("local_agent_configs", "local_scout_configs")
op.rename_table("cloud_agent_configs", "cloud_scout_configs")
op.rename_table("agent_run_logs", "scout_run_logs")
# Columns
op.alter_column("local_scout_configs", "agent_config", new_column_name="scout_config")
op.alter_column("scout_run_logs", "agent_id", new_column_name="scout_id")
op.alter_column("scout_run_logs", "agent_type", new_column_name="scout_type")
def downgrade() -> None:
op.alter_column("scout_run_logs", "scout_type", new_column_name="agent_type")
op.alter_column("scout_run_logs", "scout_id", new_column_name="agent_id")
op.alter_column("local_scout_configs", "scout_config", new_column_name="agent_config")
op.rename_table("scout_run_logs", "agent_run_logs")
op.rename_table("cloud_scout_configs", "cloud_agent_configs")
op.rename_table("local_scout_configs", "local_agent_configs")
- Step 2: Run migration against a fresh test DB to verify it applies cleanly
Run from api/ directory:
alembic upgrade head
Expected: no errors. \d local_scout_configs shows the renamed table. \d scout_run_logs shows the scout_id column (not agent_id).
- Step 3: Verify downgrade works
alembic downgrade -1
Expected: tables go back to local_agent_configs etc. Then re-run alembic upgrade head.
- Step 4: Commit (inside submodule)
cd api
git add alembic/versions/007_rename_agents_to_scouts.py
git commit -m "feat(db): rename agents to scouts (alembic 007)"
Task 2: SQLAlchemy model rename — LocalScoutConfig, CloudScoutConfig, ScoutRunLog
Files:
- Modify:
api/app/models.py:161-227(and theAgentRunLogclass — find via grep)
Context: Renames Python class names to match the new tables. Must update __tablename__ and mapped_column for renamed columns. Relationships and back_populates strings must reference the new class names.
- Step 1: Read current models around
LocalAgentConfig,CloudAgentConfig,AgentRunLog
Use Read tool on api/app/models.py to confirm the exact line ranges (the AgentRunLog class + relationships use string-based primaryjoin + back_populates="local_agent" / back_populates="cloud_agent" patterns).
- Step 2: Apply the renames in
models.py
Replace class names and tablenames:
class LocalScoutConfig(Base):
__tablename__ = "local_scout_configs"
# ... id/user_id/etc unchanged ...
scout_config: Mapped[dict | None] = mapped_column(JSON, nullable=True)
# ... rest unchanged ...
run_logs: Mapped[list["ScoutRunLog"]] = relationship(
back_populates="local_scout",
primaryjoin="and_(ScoutRunLog.scout_id == LocalScoutConfig.id, ScoutRunLog.scout_type == 'local')",
foreign_keys="ScoutRunLog.scout_id",
cascade="all, delete-orphan",
overlaps="run_logs,cloud_scout",
)
class CloudScoutConfig(Base):
__tablename__ = "cloud_scout_configs"
# ... unchanged columns ...
run_logs: Mapped[list["ScoutRunLog"]] = relationship(
back_populates="cloud_scout",
primaryjoin="and_(ScoutRunLog.scout_id == CloudScoutConfig.id, ScoutRunLog.scout_type == 'cloud')",
foreign_keys="ScoutRunLog.scout_id",
cascade="all, delete-orphan",
overlaps="run_logs,local_scout",
)
class ScoutRunLog(Base):
__tablename__ = "scout_run_logs"
# ... id, user_id, started_at, finished_at, status, error etc. unchanged ...
scout_id: Mapped[str] = mapped_column(...)
scout_type: Mapped[str] = mapped_column(...)
local_scout: Mapped["LocalScoutConfig | None"] = relationship(
back_populates="run_logs",
primaryjoin="and_(ScoutRunLog.scout_id == LocalScoutConfig.id, ScoutRunLog.scout_type == 'local')",
foreign_keys="ScoutRunLog.scout_id",
overlaps="run_logs,cloud_scout",
)
cloud_scout: Mapped["CloudScoutConfig | None"] = relationship(
back_populates="run_logs",
primaryjoin="and_(ScoutRunLog.scout_id == CloudScoutConfig.id, ScoutRunLog.scout_type == 'cloud')",
foreign_keys="ScoutRunLog.scout_id",
overlaps="run_logs,local_scout",
)
- Step 3: Sweep all imports of the old class names
Run from api/:
grep -rn "AgentRunLog\|LocalAgentConfig\|CloudAgentConfig" app/ tests/ --include="*.py"
For each hit, replace with the corresponding Scout* name. Do not skip test files.
- Step 4: Run the test suite — expect failures and fix incrementally
Run from api/:
pytest -x
Fix import errors and reference errors as they surface. The test fixture at api/tests/conftest.py does Base.metadata.create_all so renamed tables will be created automatically. Don't worry about migrations in tests — they use create_all.
- Step 5: Verify the suite is green
pytest
Expected: all tests pass.
- Step 6: Commit (inside submodule)
cd api
git add app/models.py
git add -u app/ tests/ # any files changed by the import sweep
git commit -m "refactor(models): rename Agent classes to Scout"
Task 3: Rename BE routes — routes/agents.py → routes/scouts.py, agent_setup.py → scout_setup.py
Files:
-
Rename:
api/app/api/routes/agents.py→api/app/api/routes/scouts.py -
Rename:
api/app/api/routes/agent_setup.py→api/app/api/routes/scout_setup.py -
Modify:
api/app/main.py— update router registration -
Modify:
api/app/api/routes/device_ws.py:30-31— update imports -
Step 1: Rename the files via git
cd api
git mv app/api/routes/agents.py app/api/routes/scouts.py
git mv app/api/routes/agent_setup.py app/api/routes/scout_setup.py
- Step 2: Inside both renamed files, change
APIRouter(prefix="/agents", ...)toprefix="/scouts"andprefix="/agent-setup"toprefix="/scout-setup"
In app/api/routes/scouts.py find and replace:
router = APIRouter(prefix="/agents", tags=["agents"])
With:
router = APIRouter(prefix="/scouts", tags=["scouts"])
In app/api/routes/scout_setup.py find and replace:
router = APIRouter(prefix="/agent-setup", tags=["agent-setup"])
With:
router = APIRouter(prefix="/scout-setup", tags=["scout-setup"])
- Step 3: Update
app/main.pyto import the new module names
Find the imports of agents and agent_setup routers and replace with scouts and scout_setup.
- Step 4: Update
app/api/routes/device_ws.py:30-31import
Change:
from app.api.routes.agent_setup import handle_journey_message, handle_journey_start
To:
from app.api.routes.scout_setup import handle_journey_message, handle_journey_start
- Step 5: Sweep remaining import references
grep -rn "from app.api.routes.agents\|from app.api.routes.agent_setup\|app\.api\.routes\.agents\|app\.api\.routes\.agent_setup" app/ tests/
Replace each.
- Step 6: Run tests
pytest
Expected: pass. URL-based tests must use new paths (/api/v1/scouts/..., /api/v1/scout-setup/...).
- Step 7: Commit (inside submodule)
cd api
git add -A app/ tests/
git commit -m "refactor(routes): rename /agents and /agent-setup to /scouts and /scout-setup"
Task 4: Rename core/agent_runner.py, agent_session_buffer.py, agent_registry.py
Files:
-
Rename:
api/app/core/agent_runner.py→api/app/core/scout_runner.py -
Rename:
api/app/core/agent_session_buffer.py→api/app/core/scout_session_buffer.py -
Rename:
api/app/core/agent_registry.py→api/app/core/scout_registry.py -
Step 1: Rename via git
cd api
git mv app/core/agent_runner.py app/core/scout_runner.py
git mv app/core/agent_session_buffer.py app/core/scout_session_buffer.py
git mv app/core/agent_registry.py app/core/scout_registry.py
- Step 2: Update top-of-file references in each renamed file
Inside each file, rename all references to agent_* symbols that are now defined here. For example, if agent_runner.py had def trigger_agent_run(...), change to def trigger_scout_run(...). Do NOT rename names that belong to app/agents/* (those LLM helpers stay).
- Step 3: Sweep external imports
grep -rn "from app.core.agent_runner\|from app.core.agent_session_buffer\|from app.core.agent_registry" app/ tests/
Replace each agent_runner → scout_runner, etc.
Also sweep symbol references — for example if you renamed trigger_agent_run → trigger_scout_run:
grep -rn "trigger_agent_run\|agent_session_buffer\|agent_registry" app/ tests/
Replace.
- Step 4: Run tests
pytest
Expected: green.
- Step 5: Commit (inside submodule)
cd api
git add -A app/ tests/
git commit -m "refactor(core): rename agent_runner/session_buffer/registry to scout_*"
Task 5: Rename Pydantic schemas + WS frame names referring to "agent"
Files:
- Modify:
api/app/schemas.py(or whereverWsFrameTypeenum + Agent-related schemas live)
Context: Pydantic schemas like AgentRunStatus, AgentTriggerRequest, etc. need rename. The WsFrameType enum may have values like "agent_run_started" — rename to "scout_run_started". Frontend will be updated to match in later tasks.
- Step 1: Find all Pydantic models + enum values containing "agent"
cd api
grep -n "class .*Agent\|\"agent_" app/schemas.py
- Step 2: Rename each — class names and enum values
Examples (apply each found in step 1):
-
class AgentRunStatus(str, Enum)→class ScoutRunStatus(str, Enum) -
class AgentTriggerRequest(BaseModel)→class ScoutTriggerRequest(BaseModel) -
WsFrameType.AGENT_RUN_STARTED = "agent_run_started"→SCOUT_RUN_STARTED = "scout_run_started" -
Step 3: Sweep all references
grep -rn "AgentRunStatus\|AgentTriggerRequest\|agent_run_started\|agent_run_completed" app/ tests/
Replace.
- Step 4: Run tests
pytest
- Step 5: Commit
cd api
git add -A
git commit -m "refactor(schemas): rename Agent* schemas and WS frame types to Scout*"
Task 6: Rename Langfuse prompts (user-facing scout prompts only)
Files:
- Manual operation against Langfuse via MCP tools (no source files unless prompt names are referenced in code).
Context: Langfuse prompts whose names start with agent- and refer to the user-facing scout subsystem (e.g. agent-runner-system, agent-setup-journey-system, cloud-agent-trigger, local-agent-extraction). Prompts for app/agents/* LLM helpers (e.g. task-agent-system, note-agent-summarize) STAY.
- Step 1: List all Langfuse prompts
Use the Langfuse MCP tool:
mcp__langfuse__listPrompts (no arguments)
- Step 2: Identify scout-subsystem prompts to rename
From the listing, pick prompts whose names begin with agent- AND whose content/usage maps to the user-facing scout subsystem. If unsure for a given prompt, grep its name in api/app/:
grep -rn "get_prompt_or_fallback(\"agent-" api/app/ api/tests/
Names that appear in the renamed scout_*.py files are scout-subsystem prompts.
- Step 3: For each scout prompt, create a new prompt with the renamed name
Use mcp__langfuse__getPromptUnresolved to fetch the current prompt body, then mcp__langfuse__createTextPrompt (or createChatPrompt if it's a chat prompt) with the new name (e.g. scout-runner-system) and label production.
- Step 4: Update code references to use the new prompt names
Find each get_prompt_or_fallback("agent-..." call and rename the literal:
grep -rn 'get_prompt_or_fallback("agent-' api/app/
Replace each "agent-X" with "scout-X".
- Step 5: Run tests + manual smoke
cd api
pytest
Then start dev server (uvicorn app.main:app --reload), trigger a known scout-subsystem agent invocation (use the agent setup journey or a scout trigger endpoint via curl) and confirm the prompt loads without falling back.
- Step 6: Delete old prompts from Langfuse
For each old agent-* prompt confirmed superseded, delete via Langfuse UI or API (Langfuse MCP doesn't expose delete; use the Langfuse web console).
- Step 7: Commit code changes
cd api
git add -A
git commit -m "refactor(prompts): rename scout-subsystem Langfuse prompt references to scout-*"
Task 7: Drizzle migration — rename agent_runs → scout_runs, agent_run_actions → scout_run_actions
Files:
- Create:
adiuvAI/src/main/db/migrations/0007_scouts_rename.sql - Modify:
adiuvAI/src/main/db/schema.ts
Context: Drizzle migrations are SQL files in adiuvAI/src/main/db/migrations/. Latest is 0006_misty_cammi.sql. The auto-generated migration name suffix isn't strictly required but follow the pattern (drizzle-kit generates one from CHANGELOG entries). For a manual rename, a hand-written name is fine.
- Step 1: Update
adiuvAI/src/main/db/schema.tsfirst — change the table name in the Drizzle definition
Find:
export const agentRuns = sqliteTable('agent_runs', { ... });
export const agentRunActions = sqliteTable('agent_run_actions', { ... });
Replace with:
export const scoutRuns = sqliteTable('scout_runs', {
id: text().primaryKey(),
scoutId: text('scout_id').notNull(),
status: text().notNull(),
startedAt: integer('started_at').notNull(),
completedAt: integer('completed_at'),
});
export const scoutRunActions = sqliteTable('scout_run_actions', {
id: text().primaryKey(),
runId: text('run_id').notNull(),
scoutId: text('scout_id').notNull(),
verb: text().notNull(),
entityType: text('entity_type').notNull(),
entityId: text('entity_id').notNull(),
entityTitle: text('entity_title'),
createdAt: integer('created_at').notNull(),
});
(Read the file first to copy the exact column definitions — keep types/nullability identical, only rename agent → scout.)
- Step 2: Generate the migration
Run from adiuvAI/:
npx drizzle-kit generate
Expected: a new file 0007_<adjective>_<noun>.sql in src/main/db/migrations/. Verify it contains ALTER TABLE statements (or DROP+CREATE if Drizzle prefers — SQLite may require recreating the table; Drizzle will emit the right pattern).
If Drizzle emits a destructive recreate (DROP TABLE + CREATE), that's fine pre-1.0 since these tables are largely empty in dev DBs. Confirm with the user before proceeding if their dev DB has rows worth preserving — for empty/dev DBs, drop is fine.
- Step 3: Rename the generated migration to a clean name
mv adiuvAI/src/main/db/migrations/0007_*.sql adiuvAI/src/main/db/migrations/0007_scouts_rename.sql
- Step 4: Sweep TypeScript imports of the old names
grep -rn "agentRuns\|agentRunActions" adiuvAI/src/
Replace each with scoutRuns / scoutRunActions.
- Step 5: Run typecheck + lint
From adiuvAI/:
npm run lint
npx tsc --noEmit
Expected: pass.
- Step 6: Manual smoke — boot the app once to apply the migration
cd adiuvAI
npm run start
Watch the console for migration apply messages. Once app boots cleanly, close it. Verify the dev DB has scout_runs and scout_run_actions (use sqlite3 dev.db ".tables" from adiuvAI/).
- Step 7: Commit (inside submodule)
cd adiuvAI
git add src/main/db/schema.ts src/main/db/migrations/0007_scouts_rename.sql
git add -u src/ # any consumer files updated in step 4
git commit -m "refactor(db): rename agent_runs/agent_run_actions to scout_*"
Task 8: Rename Electron main-process modules and tRPC sub-routers
Files:
- Rename:
adiuvAI/src/main/agents/agent-scheduler.ts→adiuvAI/src/main/scouts/scout-scheduler.ts - Modify:
adiuvAI/src/main/router/index.ts— renameagent.*sub-router toscout.* - Modify:
adiuvAI/src/main/store.ts— renamegetLocalAgents/saveLocalAgent→getLocalScouts/saveLocalScout - Modify:
adiuvAI/src/main/index.ts— update scheduler import + start call - Modify:
adiuvAI/src/main/api/backend-client.ts— rename agent-related symbols if any
Context: Inside agent-scheduler.ts, the function names are startAgentScheduler, stopAgentScheduler, tickAgentScheduler and POSTs to /api/v1/agents/trigger. After Task 3, the BE route is /api/v1/scouts/trigger — so this file must also update its URL.
- Step 1: Move the file
cd adiuvAI
mkdir -p src/main/scouts
git mv src/main/agents/agent-scheduler.ts src/main/scouts/scout-scheduler.ts
rmdir src/main/agents 2>/dev/null || true
(Don't fail if other files exist in src/main/agents/ — check first; if so, decide per-file whether to move or leave.)
- Step 2: Inside
scout-scheduler.ts, rename functions and update URL
Replace:
startAgentScheduler→startScoutSchedulerstopAgentScheduler→stopScoutSchedulertickAgentScheduler→tickScoutSchedulergetLocalAgents()→getLocalScouts()saveLocalAgent(...)→saveLocalScout(...)- URL
'/api/v1/agents/trigger'→'/api/v1/scouts/trigger' - Variable
agents→scouts(clarity) agentRunsDrizzle import →scoutRuns
Show the full updated function header:
async function tickScoutScheduler(): Promise<void> {
const scouts = getLocalScouts();
const now = Date.now();
for (const scout of scouts) {
if (!scout.enabled) continue;
const intervalMs = CRON_INTERVAL_MS[scout.scheduleCron];
if (!intervalMs) continue;
if (scout.lastRunAt && now - scout.lastRunAt < intervalMs) continue;
try {
const response = await getBackendClient().proxyPost<{ id: string }>(
'/api/v1/scouts/trigger',
{ directory: scout.directory, deviceId: getDeviceId(), /* ... */ }
);
if (response?.id) {
await getDb().insert(scoutRuns).values({
id: response.id, scoutId: scout.id, status: 'running', startedAt: now,
}).onConflictDoNothing();
}
saveLocalScout({ ...scout, lastRunAt: now });
} catch (err) { /* ... */ }
}
}
- Step 3: Rename store helpers in
adiuvAI/src/main/store.ts
Find getLocalAgents / saveLocalAgent / any LocalAgentConfig references and rename to getLocalScouts / saveLocalScout / LocalScoutConfig. Keep the storage key migration-safe: read from old key localAgents once, write to localScouts, then read only from new key thereafter. For pre-1.0 dev, simpler to just rename the key and accept users re-adding their scouts (note: confirm with user).
For now, simplest acceptable approach:
// store.ts
const STORAGE_KEY = 'localScouts'; // was 'localAgents'
export function getLocalScouts(): LocalScoutConfig[] {
return store.get(STORAGE_KEY, []) as LocalScoutConfig[];
}
export function saveLocalScout(scout: LocalScoutConfig): void { /* ... */ }
- Step 4: Update
adiuvAI/src/main/index.ts
Replace:
import { startAgentScheduler, stopAgentScheduler } from './agents/agent-scheduler';
With:
import { startScoutScheduler, stopScoutScheduler } from './scouts/scout-scheduler';
And the call sites: startAgentScheduler() → startScoutScheduler(), stopAgentScheduler() → stopScoutScheduler().
- Step 5: Update
adiuvAI/src/main/router/index.ts— renameagent.*sub-router toscout.*
Find the agent: t.router({ local: ..., cloud: ..., journey: ..., runs: ..., runActions: ..., catalog: ..., runNow: ..., ... }) block. Rename the key from agent to scout. Also rename the imported sub-router files if any (in adiuvAI/src/main/router/, look for agent.ts or agent/ subdirectory).
- Step 6: Sweep type references
grep -rn "LocalAgentConfig\|CloudAgentConfig\|trpc\.agent\.\|/api/v1/agents" adiuvAI/src/
Replace systematically: LocalAgentConfig → LocalScoutConfig, CloudAgentConfig → CloudScoutConfig, trpc.agent. → trpc.scout., /api/v1/agents → /api/v1/scouts.
- Step 7: Typecheck + lint
cd adiuvAI
npx tsc --noEmit
npm run lint
- Step 8: Commit
cd adiuvAI
git add -A src/main/
git commit -m "refactor(main): rename agent-scheduler/store/router symbols to scout"
Task 9: Rename Renderer components
Files:
-
Rename 5 files in
adiuvAI/src/renderer/components/settings/(per File Structure table) -
Modify:
adiuvAI/src/renderer/routes/settings.tsx— updateSECTIONSarray -
Modify: each renamed component — update internal symbols
-
Step 1: Move the files
cd adiuvAI
git mv src/renderer/components/settings/AgentsSection.tsx src/renderer/components/settings/ScoutsSection.tsx
git mv src/renderer/components/settings/AgentRow.tsx src/renderer/components/settings/ScoutRow.tsx
git mv src/renderer/components/settings/LocalAgentConfigPanel.tsx src/renderer/components/settings/LocalScoutConfigPanel.tsx
git mv src/renderer/components/settings/CloudAgentConfigPanel.tsx src/renderer/components/settings/CloudScoutConfigPanel.tsx
git mv src/renderer/components/settings/InlineAgentCreationStepper.tsx src/renderer/components/settings/InlineScoutCreationStepper.tsx
- Step 2: Inside each renamed file, rename the exported component
Rule: every export function AgentX → export function ScoutX, every Agent → Scout in component bodies, every agent → scout in local variables, every 'toast.agent.X' i18n key → 'toast.scout.X', every 'agents.X' → 'scouts.X', every t('settings.agentsX') → t('settings.scoutsX').
Show example (head of ScoutsSection.tsx):
export function ScoutsSection() {
const { t } = useTranslation();
const utils = trpc.useUtils();
const localScoutsQuery = trpc.scout.local.list.useQuery();
const cloudScoutsQuery = trpc.scout.cloud.list.useQuery();
const deleteLocalMutation = trpc.scout.local.delete.useMutation();
const deleteCloudMutation = trpc.scout.cloud.delete.useMutation();
const updateLocalMutation = trpc.scout.local.update.useMutation();
const updateCloudMutation = trpc.scout.cloud.update.useMutation();
const runNowMutation = trpc.scout.runNow.useMutation();
const { notify, notifyError, notifyPromise } = useNotify();
const [expandedScout, setExpandedScout] = useState<string | null>(null);
// ...
const localScouts: LocalScoutConfig[] = localScoutsQuery.data ?? [];
const cloudScouts: CloudScoutConfig[] = cloudScoutsQuery.data ?? [];
const allScouts = [
...localScouts.map(s => ({ ...s, scoutType: 'local' as const })),
...cloudScouts.map(s => ({ ...s, scoutType: 'cloud' as const })),
];
// ...
}
- Step 3: Update consumers — settings route + any other importer
In adiuvAI/src/renderer/routes/settings.tsx, change:
import { AgentsSection } from '@/components/settings/AgentsSection';
// ... in SECTIONS array:
{ id: 'agents', label: 'settings.agents', component: AgentsSection },
To:
import { ScoutsSection } from '@/components/settings/ScoutsSection';
// ...
{ id: 'scouts', label: 'settings.scouts', component: ScoutsSection },
Also sweep:
cd adiuvAI
grep -rn "AgentsSection\|AgentRow\|LocalAgentConfigPanel\|CloudAgentConfigPanel\|InlineAgentCreationStepper" src/
Replace each.
- Step 4: Update
adiuvAI/src/renderer/components/settings/types.ts(if it definesLocalAgentConfig)
Rename the interface LocalAgentConfig → LocalScoutConfig. Add export type LocalScoutConfig = .... If there are other type definitions referencing "agent" (e.g. AgentDataType), rename consistently.
- Step 5: Typecheck + lint
cd adiuvAI
npx tsc --noEmit
npm run lint
- Step 6: Commit
cd adiuvAI
git add -A src/renderer/
git commit -m "refactor(renderer): rename Agent components and types to Scout"
Task 10: Rename i18n keys across all 5 languages
Files:
- Modify:
adiuvAI/src/renderer/locales/{en,it,es,fr,de}/translation.json
Context: Three buckets to rename in every language file:
"settings": { "agents": "...", "agentsSubtitle": "...", "agentsDescription": "...", ... }→scouts,scoutsSubtitle,scoutsDescription- Top-level
"agents": { "noAgentsYet": ..., "createAgent": ..., "yourAgents": ..., "createFirstAgent": ..., "noAgentsDescription": ... }→"scouts": { "noScoutsYet", "createScout", "yourScouts", "createFirstScout", "noScoutsDescription" }. Update key names AND any in-value occurrences of "agent"/"agents" to "scout"/"scouts" within human-readable strings (apply translation per language — for English: "Agents" → "Scouts", "agent" → "scout"; for Italian: "Agenti" → "Scout"; etc.). "toast": { "agent": { ... } }→"toast": { "scout": { ... } }(rename nested key only; values unchanged unless they contain "agent" word).
For non-English languages, keep the meaning consistent. Suggested translations:
| Lang | Key | Suggested value |
|---|---|---|
| en | settings.scouts |
"Scouts" |
| it | settings.scouts |
"Scout" |
| es | settings.scouts |
"Scouts" |
| fr | settings.scouts |
"Scouts" |
| de | settings.scouts |
"Scouts" |
| en | settings.scoutsSubtitle |
"working for you." |
| it | settings.scoutsSubtitle |
"che lavorano per te." |
| en | settings.scoutsDescription |
"Scouts watch your data sources — local files, mailboxes, cloud services — and surface what matters in your task brief." |
| it | settings.scoutsDescription |
"Gli scout monitorano le tue fonti dati — file locali, caselle email, servizi cloud — e mettono in evidenza ciò che conta nel tuo task brief." |
| en | scouts.noScoutsYet |
"No scouts yet" |
| en | scouts.createScout |
"Create scout" |
| en | scouts.yourScouts |
"Your Scouts" |
| en | scouts.createFirstScout |
"Create first scout" |
| en | scouts.noScoutsDescription |
"Create your first scout from a template. Choose what data to extract, set a schedule, and edit instructions before saving." |
(Translate analogously for it/es/fr/de — for the new descriptive copy, keep the spirit, ask the user for review on language nuance if uncertain.)
- Step 1: Edit
en/translation.json
Apply renames per the bucket list above.
-
Step 2: Edit
it/translation.json— same renames, with Italian values -
Step 3: Edit
es/translation.json— same renames, with Spanish values -
Step 4: Edit
fr/translation.json— same renames, with French values -
Step 5: Edit
de/translation.json— same renames, with German values -
Step 6: Sweep code references
cd adiuvAI
grep -rn "'agents\.\|\"agents\.\|'settings\.agents\|\"settings\.agents\|'toast\.agent\|\"toast\.agent" src/
Replace each agents.X → scouts.X, settings.agents → settings.scouts, toast.agent → toast.scout.
- Step 7: Typecheck + lint + run app
cd adiuvAI
npx tsc --noEmit
npm run lint
npm run start # smoke: open Settings, switch language, confirm "Scouts" header renders
- Step 8: Commit
cd adiuvAI
git add -A src/renderer/locales/ src/renderer/
git commit -m "i18n: rename agents keys to scouts across all 5 languages"
Task 11: Update monorepo to bump submodule pointers
Files:
-
Modify: monorepo
.gitmodulesis unchanged; update tracked commit pointers viagit add api adiuvAI -
Step 1: From the monorepo root, observe the dirty submodule pointers
cd c:\Users\PC-Roby\Documents\_adiuvai_workspace
git status
Expected: api and adiuvAI listed as "modified content" or "new commits".
- Step 2: Stage and commit the submodule bumps
git add api adiuvAI
git commit -m "chore: bump submodules — Phase 1 scouts rename complete"
- Step 3: Verify CLAUDE.md doesn't need updates
Check c:\Users\PC-Roby\Documents\_adiuvai_workspace\.claude\CLAUDE.md and c:\Users\PC-Roby\Documents\_adiuvai_workspace\adiuvAI\.claude\CLAUDE.md — many references mention agent and the agent subsystem. For Phase 1, only update durable notes that became wrong (e.g. tRPC routers (in appRouter): ... agent (with local / cloud / journey sub-routers) ... should now read scout). Use targeted Edit, do not rewrite the file.
grep -n "agent" .claude/CLAUDE.md adiuvAI/.claude/CLAUDE.md
For each line that documents the renamed surface, fix in place.
- Step 4: Commit CLAUDE.md updates
git add .claude/CLAUDE.md adiuvAI/.claude/CLAUDE.md
git commit -m "docs(CLAUDE): update references to scouts after rename"
Phase 1 complete. Verify by running both test suites green and starting the app.
PHASE 2 — Connector Skeleton
Goal: lay all infra (BE queue table, connector Protocol, engine, WS frame contract, SQLite suggestions table) without any user-visible Gmail feature yet. End state: backend can be unit-tested with a mock connector; Electron has the table + frame handler ready to receive proposals.
Task 12: Alembic migration — scout_triage_queue + cloud_scout_configs alters
Files:
-
Create:
api/alembic/versions/008_scout_triage_queue.py -
Step 1: Create the migration
"""Scout triage queue + cloud_scout_configs alterations.
Revision ID: 008
Revises: 007
Create Date: 2026-05-15
"""
from typing import Sequence, Union
import sqlalchemy as sa
from alembic import op
revision: str = "008"
down_revision: Union[str, None] = "007"
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.create_table(
"scout_triage_queue",
sa.Column("id", sa.Uuid(as_uuid=False), primary_key=True),
sa.Column("user_id", sa.Uuid(as_uuid=False), sa.ForeignKey("users.id", ondelete="CASCADE"), nullable=False, index=True),
sa.Column("scout_id", sa.Uuid(as_uuid=False), sa.ForeignKey("cloud_scout_configs.id", ondelete="CASCADE"), nullable=False),
sa.Column("source_type", sa.String(50), nullable=False),
sa.Column("source_msg_ref", sa.String(255), nullable=False),
sa.Column("triage_verdict", sa.String(20), nullable=False),
sa.Column("triage_reason", sa.Text, nullable=True),
sa.Column("status", sa.String(20), nullable=False, server_default="queued"),
sa.Column("triaged_at", sa.DateTime(timezone=True), nullable=False, server_default=sa.func.now()),
sa.Column("delivered_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("acked_at", sa.DateTime(timezone=True), nullable=True),
sa.Column("expires_at", sa.DateTime(timezone=True), nullable=False),
sa.UniqueConstraint("scout_id", "source_msg_ref", name="uq_scout_triage_queue_scout_msg"),
)
op.create_index("ix_scout_triage_queue_user_status", "scout_triage_queue", ["user_id", "status"])
op.create_index(
"ix_scout_triage_queue_expires_active",
"scout_triage_queue",
["expires_at"],
postgresql_where=sa.text("status != 'acked'"),
)
op.add_column("cloud_scout_configs", sa.Column("auto_trash_spam", sa.Boolean(), nullable=False, server_default=sa.text("false")))
op.add_column("cloud_scout_configs", sa.Column("gmail_history_id", sa.String(64), nullable=True))
op.add_column("cloud_scout_configs", sa.Column("gmail_watch_expires_at", sa.DateTime(timezone=True), nullable=True))
op.add_column("cloud_scout_configs", sa.Column("device_inactivity_pause_days", sa.Integer(), nullable=False, server_default="14"))
def downgrade() -> None:
op.drop_column("cloud_scout_configs", "device_inactivity_pause_days")
op.drop_column("cloud_scout_configs", "gmail_watch_expires_at")
op.drop_column("cloud_scout_configs", "gmail_history_id")
op.drop_column("cloud_scout_configs", "auto_trash_spam")
op.drop_index("ix_scout_triage_queue_expires_active", table_name="scout_triage_queue")
op.drop_index("ix_scout_triage_queue_user_status", table_name="scout_triage_queue")
op.drop_table("scout_triage_queue")
- Step 2: Apply + verify
cd api
alembic upgrade head
Verify with \d scout_triage_queue and \d cloud_scout_configs (the latter should now show the four new columns).
- Step 3: Commit
cd api
git add alembic/versions/008_scout_triage_queue.py
git commit -m "feat(db): add scout_triage_queue and cloud_scout_configs gmail columns"
Task 13: SQLAlchemy ScoutTriageQueue model + CloudScoutConfig field additions
Files:
-
Modify:
api/app/models.py— addScoutTriageQueueclass, add four new fields toCloudScoutConfig -
Step 1: Add fields to
CloudScoutConfig(inmodels.pyafter the existing fields)
auto_trash_spam: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default=sa.text("false"))
gmail_history_id: Mapped[str | None] = mapped_column(String(64), nullable=True)
gmail_watch_expires_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
device_inactivity_pause_days: Mapped[int] = mapped_column(Integer, nullable=False, default=14, server_default="14")
- Step 2: Add the
ScoutTriageQueueclass (afterCloudScoutConfig)
class ScoutTriageQueue(Base):
__tablename__ = "scout_triage_queue"
__table_args__ = (
UniqueConstraint("scout_id", "source_msg_ref", name="uq_scout_triage_queue_scout_msg"),
)
id: Mapped[str] = mapped_column(Uuid(as_uuid=False), primary_key=True, default=_uuid)
user_id: Mapped[str] = mapped_column(Uuid(as_uuid=False), ForeignKey("users.id", ondelete="CASCADE"), nullable=False, index=True)
scout_id: Mapped[str] = mapped_column(Uuid(as_uuid=False), ForeignKey("cloud_scout_configs.id", ondelete="CASCADE"), nullable=False)
source_type: Mapped[str] = mapped_column(String(50), nullable=False)
source_msg_ref: Mapped[str] = mapped_column(String(255), nullable=False)
triage_verdict: Mapped[str] = mapped_column(String(20), nullable=False)
triage_reason: Mapped[str | None] = mapped_column(Text, nullable=True)
status: Mapped[str] = mapped_column(String(20), nullable=False, default="queued", server_default="queued")
triaged_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, server_default=func.now())
delivered_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
acked_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
expires_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False)
- Step 3: Verify imports at the top of
models.pyincludeUniqueConstraintandInteger
If missing, add from sqlalchemy import ..., UniqueConstraint, Integer.
- Step 4: Run tests
cd api
pytest
Expected: green (existing tests don't touch the new model yet, but Base.metadata.create_all should now succeed creating the new table in the test DB).
- Step 5: Commit
cd api
git add app/models.py
git commit -m "feat(models): add ScoutTriageQueue model + cloud_scout gmail fields"
Task 14: SourceConnector Protocol + Pydantic types
Files:
-
Create:
api/app/scouts/__init__.py(empty) -
Create:
api/app/scouts/connectors/__init__.py(empty) -
Create:
api/app/scouts/connectors/base.py -
Create:
api/tests/test_scout_connectors_base.py -
Step 1: Write the failing test
api/tests/test_scout_connectors_base.py:
"""Tests for the SourceConnector base protocol and shared types."""
from __future__ import annotations
from datetime import datetime, timezone
import pytest
from app.scouts.connectors.base import (
ItemContent,
ItemMetadata,
ItemRef,
TriageVerdict,
)
def test_item_ref_round_trips_through_pydantic():
ref = ItemRef(source_msg_ref="abc123", received_at=datetime.now(tz=timezone.utc))
parsed = ItemRef.model_validate(ref.model_dump())
assert parsed.source_msg_ref == "abc123"
assert parsed.received_at == ref.received_at
def test_item_metadata_allows_all_optional():
meta = ItemMetadata()
assert meta.subject is None
assert meta.sender is None
assert meta.snippet is None
assert meta.received_at is None
def test_item_content_requires_metadata_and_body():
content = ItemContent(
metadata=ItemMetadata(subject="hi"),
body_text="hello world",
raw_headers={"X-Foo": "bar"},
)
assert content.metadata.subject == "hi"
assert content.body_text == "hello world"
assert content.raw_headers["X-Foo"] == "bar"
def test_triage_verdict_constraints():
v = TriageVerdict(verdict="relevant", reason="contains task language", confidence=0.92)
assert v.verdict == "relevant"
with pytest.raises(ValueError):
TriageVerdict(verdict="meh", reason="x", confidence=0.5) # bad enum value
- Step 2: Run the test to verify it fails
cd api
pytest tests/test_scout_connectors_base.py -v
Expected: ImportError (No module named 'app.scouts.connectors.base').
- Step 3: Implement
app/scouts/connectors/base.py
"""Source connector Protocol and shared item types.
A SourceConnector adapts a third-party data source (Gmail, Slack, ...) to the
shared ScoutEngine interface. Each connector owns:
* how to enumerate new items since the last poll (``list_new``)
* how to fetch a single item's metadata cheaply (``fetch_metadata``)
* how to fetch a single item's full content for in-memory triage
(``fetch_content``) — this content MUST NOT be persisted by the engine
* how to archive/trash an item (``archive``) for spam handling
* optional push-notification setup (``setup_watch`` / ``renew_watch``)
"""
from __future__ import annotations
from datetime import datetime
from typing import Literal, Protocol
from pydantic import BaseModel, Field
class ItemRef(BaseModel):
source_msg_ref: str
received_at: datetime | None = None
class ItemMetadata(BaseModel):
subject: str | None = None
sender: str | None = None
snippet: str | None = None
received_at: datetime | None = None
class ItemContent(BaseModel):
metadata: ItemMetadata
body_text: str
raw_headers: dict[str, str] = Field(default_factory=dict)
class TriageVerdict(BaseModel):
verdict: Literal["relevant", "spam"]
reason: str
confidence: float = Field(ge=0.0, le=1.0)
class SourceConnector(Protocol):
"""Adapter for a third-party data source (Gmail, Slack, ...)."""
source_type: str # e.g. "gmail"
async def list_new(self, scout) -> list[ItemRef]: ...
async def fetch_metadata(self, scout, ref: ItemRef) -> ItemMetadata: ...
async def fetch_content(self, scout, ref: ItemRef) -> ItemContent: ...
async def archive(self, scout, ref: ItemRef) -> None: ...
async def setup_watch(self, scout) -> None: ...
async def renew_watch(self, scout) -> None: ...
Also create empty package markers:
cd api
touch app/scouts/__init__.py app/scouts/connectors/__init__.py
- Step 4: Run tests
cd api
pytest tests/test_scout_connectors_base.py -v
Expected: PASS (4 tests).
- Step 5: Commit
cd api
git add app/scouts/ tests/test_scout_connectors_base.py
git commit -m "feat(scouts): add SourceConnector protocol and item types"
Task 15: Connector registry
Files:
-
Create:
api/app/scouts/connectors/registry.py -
Create:
api/tests/test_scout_connector_registry.py -
Step 1: Write the failing test
"""Tests for the connector registry."""
from __future__ import annotations
import pytest
from app.scouts.connectors.base import ItemRef
from app.scouts.connectors.registry import (
get_connector,
register_connector,
_reset_for_tests,
)
class _DummyConnector:
source_type = "dummy"
async def list_new(self, scout): return []
async def fetch_metadata(self, scout, ref): raise NotImplementedError
async def fetch_content(self, scout, ref): raise NotImplementedError
async def archive(self, scout, ref): raise NotImplementedError
async def setup_watch(self, scout): raise NotImplementedError
async def renew_watch(self, scout): raise NotImplementedError
@pytest.fixture(autouse=True)
def _clean_registry():
_reset_for_tests()
yield
_reset_for_tests()
def test_register_and_get():
c = _DummyConnector()
register_connector(c)
assert get_connector("dummy") is c
def test_unknown_source_raises():
with pytest.raises(KeyError):
get_connector("nope")
def test_double_register_replaces():
a = _DummyConnector()
b = _DummyConnector()
register_connector(a)
register_connector(b)
assert get_connector("dummy") is b
- Step 2: Run test (should fail with ImportError)
cd api
pytest tests/test_scout_connector_registry.py -v
- Step 3: Implement
app/scouts/connectors/registry.py
"""Connector registry — single source of truth for source_type -> connector."""
from __future__ import annotations
from typing import Any
_CONNECTORS: dict[str, Any] = {}
def register_connector(connector: Any) -> None:
"""Register a SourceConnector instance under its ``source_type``.
Calling twice with the same ``source_type`` replaces the prior entry —
useful for tests and hot-reload, but in production each connector
should be registered exactly once at startup.
"""
if not getattr(connector, "source_type", None):
raise ValueError("Connector must declare a non-empty source_type")
_CONNECTORS[connector.source_type] = connector
def get_connector(source_type: str) -> Any:
"""Return the registered connector for ``source_type`` or raise KeyError."""
try:
return _CONNECTORS[source_type]
except KeyError as exc:
raise KeyError(f"No connector registered for source_type {source_type!r}") from exc
def _reset_for_tests() -> None:
"""Clear the registry — for use in pytest fixtures only."""
_CONNECTORS.clear()
- Step 4: Run tests
cd api
pytest tests/test_scout_connector_registry.py -v
Expected: PASS (3 tests).
- Step 5: Commit
cd api
git add app/scouts/connectors/registry.py tests/test_scout_connector_registry.py
git commit -m "feat(scouts): add connector registry"
Task 16: ScoutEngine skeleton — trigger_scout + _process_item
Files:
- Create:
api/app/scouts/engine.py - Create:
api/tests/test_scout_engine.py
Context: This task implements the core engine logic but uses a stub triage call. The real LLM triage comes in Task 24. The engine writes scout_triage_queue rows for relevant items and calls connector.archive for spam items when scout.auto_trash_spam is true.
- Step 1: Write the failing test
"""Unit tests for ScoutEngine.trigger_scout / _process_item."""
from __future__ import annotations
import uuid
from datetime import datetime, timedelta, timezone
from unittest.mock import AsyncMock
import pytest
from sqlalchemy import select
from app.models import CloudScoutConfig, ScoutTriageQueue, User, Subscription
from app.scouts.connectors.base import ItemContent, ItemMetadata, ItemRef, TriageVerdict
from app.scouts.connectors.registry import register_connector, _reset_for_tests
from app.scouts.engine import ScoutEngine
from tests.conftest import _TestSessionLocal
def _make_connector(items, content_for):
c = AsyncMock()
c.source_type = "dummy"
c.list_new = AsyncMock(return_value=items)
c.fetch_content = AsyncMock(side_effect=lambda scout, ref: content_for[ref.source_msg_ref])
c.archive = AsyncMock()
return c
@pytest.fixture(autouse=True)
def _registry():
_reset_for_tests()
yield
_reset_for_tests()
@pytest.mark.asyncio
async def test_relevant_item_inserted_into_queue(monkeypatch):
user_id = "00000000-0000-0000-0000-000000000003" # power tier seeded in conftest
scout_id = str(uuid.uuid4())
async with _TestSessionLocal() as session:
scout = CloudScoutConfig(
id=scout_id, user_id=user_id, provider="gmail", name="Test",
data_types=[], prompt_template="", schedule_cron="0 * * * *",
enabled=True, auto_trash_spam=False, device_inactivity_pause_days=14,
)
session.add(scout)
await session.commit()
refs = [ItemRef(source_msg_ref="msg-1")]
content = {"msg-1": ItemContent(metadata=ItemMetadata(subject="Hi"), body_text="task tomorrow")}
connector = _make_connector(refs, content)
register_connector(connector)
engine = ScoutEngine()
monkeypatch.setattr(
engine,
"_triage_llm",
AsyncMock(return_value=TriageVerdict(verdict="relevant", reason="task", confidence=0.9)),
)
await engine.trigger_scout(uuid.UUID(scout_id))
async with _TestSessionLocal() as session:
rows = (await session.execute(select(ScoutTriageQueue))).scalars().all()
assert len(rows) == 1
assert rows[0].source_msg_ref == "msg-1"
assert rows[0].triage_verdict == "relevant"
assert rows[0].status == "queued"
connector.archive.assert_not_awaited()
@pytest.mark.asyncio
async def test_spam_with_auto_trash_archives_and_does_not_queue(monkeypatch):
user_id = "00000000-0000-0000-0000-000000000003"
scout_id = str(uuid.uuid4())
async with _TestSessionLocal() as session:
scout = CloudScoutConfig(
id=scout_id, user_id=user_id, provider="gmail", name="Test",
data_types=[], prompt_template="", schedule_cron="0 * * * *",
enabled=True, auto_trash_spam=True, device_inactivity_pause_days=14,
)
session.add(scout)
await session.commit()
refs = [ItemRef(source_msg_ref="msg-spam")]
content = {"msg-spam": ItemContent(metadata=ItemMetadata(subject="$$$"), body_text="buy")}
connector = _make_connector(refs, content)
register_connector(connector)
engine = ScoutEngine()
monkeypatch.setattr(
engine,
"_triage_llm",
AsyncMock(return_value=TriageVerdict(verdict="spam", reason="bait", confidence=0.99)),
)
await engine.trigger_scout(uuid.UUID(scout_id))
async with _TestSessionLocal() as session:
rows = (await session.execute(select(ScoutTriageQueue))).scalars().all()
assert rows == []
connector.archive.assert_awaited_once()
@pytest.mark.asyncio
async def test_spam_without_auto_trash_does_not_archive_and_does_not_queue(monkeypatch):
user_id = "00000000-0000-0000-0000-000000000003"
scout_id = str(uuid.uuid4())
async with _TestSessionLocal() as session:
scout = CloudScoutConfig(
id=scout_id, user_id=user_id, provider="gmail", name="Test",
data_types=[], prompt_template="", schedule_cron="0 * * * *",
enabled=True, auto_trash_spam=False, device_inactivity_pause_days=14,
)
session.add(scout)
await session.commit()
refs = [ItemRef(source_msg_ref="msg-2")]
content = {"msg-2": ItemContent(metadata=ItemMetadata(subject="$$$"), body_text="buy")}
connector = _make_connector(refs, content)
register_connector(connector)
engine = ScoutEngine()
monkeypatch.setattr(
engine,
"_triage_llm",
AsyncMock(return_value=TriageVerdict(verdict="spam", reason="bait", confidence=0.99)),
)
await engine.trigger_scout(uuid.UUID(scout_id))
async with _TestSessionLocal() as session:
rows = (await session.execute(select(ScoutTriageQueue))).scalars().all()
assert rows == []
connector.archive.assert_not_awaited()
@pytest.mark.asyncio
async def test_idempotent_replay(monkeypatch):
user_id = "00000000-0000-0000-0000-000000000003"
scout_id = str(uuid.uuid4())
async with _TestSessionLocal() as session:
session.add(CloudScoutConfig(
id=scout_id, user_id=user_id, provider="gmail", name="Test",
data_types=[], prompt_template="", schedule_cron="0 * * * *",
enabled=True, auto_trash_spam=False, device_inactivity_pause_days=14,
))
await session.commit()
refs = [ItemRef(source_msg_ref="msg-3")]
content = {"msg-3": ItemContent(metadata=ItemMetadata(subject="x"), body_text="y")}
connector = _make_connector(refs, content)
register_connector(connector)
engine = ScoutEngine()
monkeypatch.setattr(
engine,
"_triage_llm",
AsyncMock(return_value=TriageVerdict(verdict="relevant", reason="x", confidence=0.5)),
)
await engine.trigger_scout(uuid.UUID(scout_id))
await engine.trigger_scout(uuid.UUID(scout_id))
async with _TestSessionLocal() as session:
rows = (await session.execute(select(ScoutTriageQueue))).scalars().all()
assert len(rows) == 1, "Replay must not create duplicate queue rows"
- Step 2: Run test (should fail — module missing)
cd api
pytest tests/test_scout_engine.py -v
- Step 3: Implement
app/scouts/engine.py
"""ScoutEngine — orchestrates triage, queueing, and delivery for cloud scouts.
Triage flow per scout:
1. Resolve scout config from the DB.
2. Skip if device hasn't connected within ``device_inactivity_pause_days``.
3. Ask the connector to ``list_new`` — fresh items since last poll.
4. For each item:
- skip if already in the queue (idempotent on (scout_id, source_msg_ref))
- fetch the full content via the connector (transient, never persisted)
- run the triage LLM call → relevant | spam
- spam + auto_trash_spam → connector.archive
- relevant → INSERT scout_triage_queue row
5. Update scout.last_run_at.
Delivery flow on Electron WS reconnect:
- drain ``status='queued'`` rows for the user
- fetch metadata-only for each (subject + snippet)
- send a ``scout_proposal`` frame
- flip status to ``delivered`` on ack
"""
from __future__ import annotations
import logging
import uuid
from datetime import datetime, timedelta, timezone
from sqlalchemy import select
from sqlalchemy.exc import IntegrityError
from app.db.session import async_session_factory
from app.models import CloudScoutConfig, ScoutTriageQueue
from app.scouts.connectors.base import ItemContent, ItemRef, TriageVerdict
from app.scouts.connectors.registry import get_connector
logger = logging.getLogger(__name__)
QUEUE_TTL_DAYS = 30
class ScoutEngine:
async def trigger_scout(self, scout_id: uuid.UUID) -> None:
async with async_session_factory() as session:
scout = await session.get(CloudScoutConfig, str(scout_id))
if scout is None:
logger.warning("trigger_scout: no such scout id=%s", scout_id)
return
if not scout.enabled:
return
# Device-inactivity pause check is a simple heuristic on last_run_at —
# the device-online signal lives in the DeviceConnectionManager and is
# consulted at delivery time. For triage, we only check that the
# configured pause threshold isn't suppressing the run.
connector = get_connector(scout.provider)
try:
refs = await connector.list_new(scout)
except Exception:
logger.exception("scout %s: list_new failed", scout.id)
return
for ref in refs:
await self._process_item(session, scout, connector, ref)
scout.last_run_at = datetime.now(tz=timezone.utc)
await session.commit()
async def _process_item(
self,
session,
scout: CloudScoutConfig,
connector,
ref: ItemRef,
) -> None:
# Idempotency check
existing = await session.execute(
select(ScoutTriageQueue.id).where(
ScoutTriageQueue.scout_id == scout.id,
ScoutTriageQueue.source_msg_ref == ref.source_msg_ref,
)
)
if existing.first() is not None:
return
try:
content = await connector.fetch_content(scout, ref)
except Exception:
logger.exception("scout %s: fetch_content failed for %s", scout.id, ref.source_msg_ref)
return
try:
verdict = await self._triage_llm(scout, content)
except Exception:
logger.exception("scout %s: triage_llm failed for %s", scout.id, ref.source_msg_ref)
return
if verdict.verdict == "spam":
if scout.auto_trash_spam:
try:
await connector.archive(scout, ref)
except Exception:
logger.exception("scout %s: archive failed for %s", scout.id, ref.source_msg_ref)
return
now = datetime.now(tz=timezone.utc)
row = ScoutTriageQueue(
id=str(uuid.uuid4()),
user_id=scout.user_id,
scout_id=scout.id,
source_type=connector.source_type,
source_msg_ref=ref.source_msg_ref,
triage_verdict=verdict.verdict,
triage_reason=verdict.reason,
status="queued",
triaged_at=now,
expires_at=now + timedelta(days=QUEUE_TTL_DAYS),
)
session.add(row)
try:
await session.flush()
except IntegrityError:
await session.rollback()
# Race: another worker inserted between our SELECT and INSERT.
# The unique constraint did its job; safe to ignore.
return
async def _triage_llm(self, scout: CloudScoutConfig, content: ItemContent) -> TriageVerdict:
"""Stub — real implementation in Task 24."""
raise NotImplementedError("Real triage LLM call lands in Task 24")
Note: the import from app.db.session import async_session_factory assumes there's an existing session factory module. If the project uses a different name (e.g. from app.db import SessionLocal), find it via:
grep -rn "AsyncSession\|sessionmaker" api/app/db/
and adjust the import. This is a common case across this codebase — the engine just needs an async session.
- Step 4: Run tests
cd api
pytest tests/test_scout_engine.py -v
Expected: 4 PASS.
- Step 5: Commit
cd api
git add app/scouts/engine.py tests/test_scout_engine.py
git commit -m "feat(scouts): add ScoutEngine triage + queue insertion"
Task 17: Drizzle migration — scout_suggestions table
Files:
-
Modify:
adiuvAI/src/main/db/schema.ts— addscoutSuggestionstable -
Create:
adiuvAI/src/main/db/migrations/0008_scout_suggestions.sql(auto-generated, then renamed) -
Step 1: Add the table to
schema.ts
Append at the end of the existing tables:
export const scoutSuggestions = sqliteTable('scout_suggestions', {
id: text().primaryKey(),
scoutId: text('scout_id').notNull(),
sourceType: text('source_type').notNull(),
sourceMsgRef: text('source_msg_ref').notNull(),
category: text().notNull(), // "unprocessed" until Phase 4
payload: text(), // JSON, populated by Phase 4
rawSubject: text('raw_subject'),
rawSnippet: text('raw_snippet'),
status: text().notNull(), // pending | approved | rejected | expired
proposedAt: integer('proposed_at').notNull(),
resolvedAt: integer('resolved_at'),
resolvedEntityType: text('resolved_entity_type'),
resolvedEntityId: text('resolved_entity_id'),
});
- Step 2: Generate the migration
cd adiuvAI
npx drizzle-kit generate
Rename to 0008_scout_suggestions.sql:
mv src/main/db/migrations/0008_*.sql src/main/db/migrations/0008_scout_suggestions.sql
- Step 3: Inspect the generated SQL
Verify it contains a CREATE TABLE scout_suggestions and matches the schema columns.
- Step 4: Smoke test
cd adiuvAI
npm run start
After app boots, close. Verify:
sqlite3 dev.db ".schema scout_suggestions"
- Step 5: Commit
cd adiuvAI
git add src/main/db/schema.ts src/main/db/migrations/0008_scout_suggestions.sql
git commit -m "feat(db): add scout_suggestions table"
Task 18: WS frame schemas — scout_proposal + scout_proposal_ack
Files:
-
Modify:
api/app/schemas.py— add newWsFrameTypeenum values + Pydantic models for the two frames -
Step 1: Read
api/app/schemas.pyto findWsFrameTypeenum
cd api
grep -n "WsFrameType\|class Ws" app/schemas.py
- Step 2: Add new enum values
class WsFrameType(str, Enum):
# ... existing values ...
SCOUT_PROPOSAL = "scout_proposal"
SCOUT_PROPOSAL_ACK = "scout_proposal_ack"
- Step 3: Add the frame Pydantic models
class ScoutProposalPayload(BaseModel):
id: str
scout_id: str
source_type: str
source_msg_ref: str
raw_subject: str | None = None
raw_snippet: str | None = None
category: Literal["unprocessed"] = "unprocessed" # Phase 4 will widen
payload: dict | None = None
class ScoutProposalFrame(BaseModel):
type: Literal[WsFrameType.SCOUT_PROPOSAL]
proposal: ScoutProposalPayload
class ScoutProposalAckFrame(BaseModel):
type: Literal[WsFrameType.SCOUT_PROPOSAL_ACK]
proposal_id: str
- Step 4: Run tests
cd api
pytest tests/test_schemas.py -v # if exists
pytest
- Step 5: Commit
cd api
git add app/schemas.py
git commit -m "feat(schemas): add scout_proposal + scout_proposal_ack WS frame types"
Task 19: ScoutEngine.deliver_pending — drain queue + send WS frames
Files:
-
Modify:
api/app/scouts/engine.py— adddeliver_pendingmethod -
Modify:
api/app/api/routes/device_ws.py— calldeliver_pendingon connect; handlescout_proposal_ackincoming frames -
Modify:
api/tests/test_scout_engine.py— adddeliver_pendingtests -
Step 1: Write the failing test
Append to api/tests/test_scout_engine.py:
@pytest.mark.asyncio
async def test_deliver_pending_sends_one_frame_per_queued_row(monkeypatch):
user_id = "00000000-0000-0000-0000-000000000003"
scout_id = str(uuid.uuid4())
now = datetime.now(tz=timezone.utc)
async with _TestSessionLocal() as session:
session.add(CloudScoutConfig(
id=scout_id, user_id=user_id, provider="gmail", name="Test",
data_types=[], prompt_template="", schedule_cron="0 * * * *",
enabled=True, auto_trash_spam=False, device_inactivity_pause_days=14,
))
for i in range(3):
session.add(ScoutTriageQueue(
id=str(uuid.uuid4()), user_id=user_id, scout_id=scout_id,
source_type="gmail", source_msg_ref=f"msg-{i}",
triage_verdict="relevant", status="queued",
triaged_at=now, expires_at=now + timedelta(days=30),
))
await session.commit()
connector = AsyncMock()
connector.source_type = "gmail"
connector.fetch_metadata = AsyncMock(side_effect=lambda scout, ref: ItemMetadata(
subject=f"sub-{ref.source_msg_ref}", snippet=f"snip-{ref.source_msg_ref}",
))
register_connector(connector)
sent = []
ws = AsyncMock()
ws.send_json = AsyncMock(side_effect=lambda payload: sent.append(payload))
engine = ScoutEngine()
await engine.deliver_pending(uuid.UUID(user_id), ws)
assert len(sent) == 3
assert all(s["type"] == "scout_proposal" for s in sent)
subjects = {s["proposal"]["raw_subject"] for s in sent}
assert subjects == {"sub-msg-0", "sub-msg-1", "sub-msg-2"}
# Status should flip to delivered
async with _TestSessionLocal() as session:
rows = (await session.execute(select(ScoutTriageQueue))).scalars().all()
assert all(r.status == "delivered" for r in rows)
assert all(r.delivered_at is not None for r in rows)
- Step 2: Run test (should fail —
deliver_pendingnot defined)
cd api
pytest tests/test_scout_engine.py::test_deliver_pending_sends_one_frame_per_queued_row -v
- Step 3: Implement
deliver_pendinginapp/scouts/engine.py
Add to the ScoutEngine class:
async def deliver_pending(self, user_id: uuid.UUID, ws) -> None:
"""Drain queued triage rows for a user, send one scout_proposal frame each.
Frames are sent immediately and the rows flip to ``status='delivered'``.
Final ``acked`` flip happens in the WS endpoint when the client returns
a ``scout_proposal_ack`` frame.
"""
from app.scouts.connectors.base import ItemRef
async with async_session_factory() as session:
rows = (await session.execute(
select(ScoutTriageQueue).where(
ScoutTriageQueue.user_id == str(user_id),
ScoutTriageQueue.status == "queued",
)
)).scalars().all()
for row in rows:
try:
connector = get_connector(row.source_type)
except KeyError:
logger.warning("deliver_pending: no connector for %s", row.source_type)
continue
scout = await session.get(CloudScoutConfig, row.scout_id)
if scout is None:
continue
try:
meta = await connector.fetch_metadata(scout, ItemRef(source_msg_ref=row.source_msg_ref))
except Exception:
logger.exception("deliver_pending: fetch_metadata failed")
continue
payload = {
"type": "scout_proposal",
"proposal": {
"id": row.id,
"scout_id": row.scout_id,
"source_type": row.source_type,
"source_msg_ref": row.source_msg_ref,
"raw_subject": meta.subject,
"raw_snippet": meta.snippet,
"category": "unprocessed",
"payload": None,
},
}
await ws.send_json(payload)
row.status = "delivered"
row.delivered_at = datetime.now(tz=timezone.utc)
await session.commit()
async def ack_proposal(self, proposal_id: str) -> None:
"""Mark a proposal as acked once the Electron client confirms receipt."""
async with async_session_factory() as session:
row = await session.get(ScoutTriageQueue, proposal_id)
if row is None:
return
row.status = "acked"
row.acked_at = datetime.now(tz=timezone.utc)
await session.commit()
- Step 4: Run tests
cd api
pytest tests/test_scout_engine.py -v
Expected: all 5 tests PASS.
- Step 5: Wire into
app/api/routes/device_ws.py
Find the WS handshake / post-connect section. Add a call to ScoutEngine().deliver_pending(user.id, websocket) after the device_hello is received and the connection is registered.
Also add a new dispatch case for incoming scout_proposal_ack frames:
elif msg.get("type") == "scout_proposal_ack":
proposal_id = msg.get("proposal_id")
if proposal_id:
await ScoutEngine().ack_proposal(proposal_id)
- Step 6: Run tests
cd api
pytest tests/test_ws_unified.py -v
pytest
- Step 7: Commit
cd api
git add app/scouts/engine.py tests/test_scout_engine.py app/api/routes/device_ws.py
git commit -m "feat(scouts): deliver_pending drains queue and sends scout_proposal frames"
Task 20: Electron — scout-suggestion-handler.ts (insert into local table on incoming frame)
Files:
-
Create:
adiuvAI/src/main/scouts/scout-suggestion-handler.ts -
Modify:
adiuvAI/src/main/api/backend-client.ts— dispatch incomingscout_proposalframe to the handler, sendscout_proposal_ackback -
Step 1: Read
adiuvAI/src/main/api/backend-client.tsto find the inbound frame switch
cd adiuvAI
grep -n "tool_call\|run_complete\|switch.*type" src/main/api/backend-client.ts
Identify the existing dispatch pattern.
- Step 2: Create the handler module
adiuvAI/src/main/scouts/scout-suggestion-handler.ts:
import { randomUUID } from 'node:crypto';
import { getDb } from '../db';
import { scoutSuggestions } from '../db/schema';
export interface IncomingScoutProposal {
id: string;
scout_id: string;
source_type: string;
source_msg_ref: string;
raw_subject: string | null;
raw_snippet: string | null;
category: 'unprocessed';
payload: Record<string, unknown> | null;
}
export async function handleScoutProposal(p: IncomingScoutProposal): Promise<void> {
// Idempotent: same id from BE shouldn't double-insert if the ack was lost.
await getDb()
.insert(scoutSuggestions)
.values({
id: p.id,
scoutId: p.scout_id,
sourceType: p.source_type,
sourceMsgRef: p.source_msg_ref,
category: p.category,
payload: p.payload ? JSON.stringify(p.payload) : null,
rawSubject: p.raw_subject,
rawSnippet: p.raw_snippet,
status: 'pending',
proposedAt: Date.now(),
})
.onConflictDoNothing();
}
- Step 3: Wire in
backend-client.ts
In the inbound frame dispatch:
case 'scout_proposal': {
const { handleScoutProposal } = await import('../scouts/scout-suggestion-handler');
await handleScoutProposal(msg.proposal);
this.send({ type: 'scout_proposal_ack', proposal_id: msg.proposal.id });
break;
}
(Adapt to the existing dispatch style — if it uses a switch, add a case; if it uses an if-chain, add an else-if. The send pattern follows whatever existing frames do.)
- Step 4: Typecheck + lint
cd adiuvAI
npx tsc --noEmit
npm run lint
- Step 5: Commit
cd adiuvAI
git add src/main/scouts/scout-suggestion-handler.ts src/main/api/backend-client.ts
git commit -m "feat(scouts): handle scout_proposal frames and ack"
Task 21: Submodule pointer bump for Phase 2
- Step 1: Bump submodule pointers
cd c:\Users\PC-Roby\Documents\_adiuvai_workspace
git add api adiuvAI
git commit -m "chore: bump submodules — Phase 2 connector skeleton"
Phase 2 complete. Engine + queue + frame contract + local table all in place. No user-facing change yet.
PHASE 3 — Gmail Scout End-to-End
Goal: ship a working Gmail scout — user can connect Gmail through Settings, push notifications + cron-fallback poll the inbox, BE triages with the real LLM, queue rows are delivered to Electron on reconnect and land as scout_suggestions rows with category='unprocessed'.
Task 22: GmailConnector implementation
Files:
- Create:
api/app/scouts/connectors/gmail.py - Create:
api/tests/test_scout_connectors_gmail.py
Context: Wraps existing app/integrations/gmail.py's GmailClient. Maps existing EmailMessage to the new ItemContent. Implements list_new using Gmail's users.history.list since the scout's gmail_history_id. Implements archive via users.messages.trash.
- Step 1: Write the failing test
api/tests/test_scout_connectors_gmail.py:
"""Tests for GmailConnector."""
from __future__ import annotations
import uuid
from datetime import datetime, timezone
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
from app.models import CloudScoutConfig
from app.scouts.connectors.base import ItemRef
from app.scouts.connectors.gmail import GmailConnector
def _make_scout():
return CloudScoutConfig(
id=str(uuid.uuid4()),
user_id="00000000-0000-0000-0000-000000000003",
provider="gmail",
name="Inbox",
data_types=[],
prompt_template="",
oauth_token_encrypted="encrypted-blob",
schedule_cron="0 * * * *",
enabled=True,
auto_trash_spam=False,
device_inactivity_pause_days=14,
gmail_history_id="100",
)
@pytest.mark.asyncio
async def test_fetch_metadata_returns_subject_and_snippet():
scout = _make_scout()
conn = GmailConnector()
fake_message = {
"id": "msg-1",
"snippet": "preview text",
"payload": {"headers": [
{"name": "Subject", "value": "Hello"},
{"name": "From", "value": "alice@example.com"},
{"name": "Date", "value": "Wed, 14 May 2026 10:00:00 +0000"},
]},
}
with patch("app.scouts.connectors.gmail._get_gmail_service") as mock_svc:
mock_svc.return_value.users().messages().get().execute.return_value = fake_message
meta = await conn.fetch_metadata(scout, ItemRef(source_msg_ref="msg-1"))
assert meta.subject == "Hello"
assert meta.sender == "alice@example.com"
assert meta.snippet == "preview text"
@pytest.mark.asyncio
async def test_fetch_content_returns_body_text():
scout = _make_scout()
conn = GmailConnector()
with patch("app.scouts.connectors.gmail.GmailClient") as MockClient:
instance = MockClient.return_value
instance.fetch_messages = AsyncMock(return_value=[
MagicMock(id="msg-1", subject="S", sender="a@b", body_text="hello world",
date=datetime.now(tz=timezone.utc), labels=[]),
])
content = await conn.fetch_content(scout, ItemRef(source_msg_ref="msg-1"))
assert content.body_text == "hello world"
assert content.metadata.subject == "S"
@pytest.mark.asyncio
async def test_archive_calls_trash():
scout = _make_scout()
conn = GmailConnector()
with patch("app.scouts.connectors.gmail._get_gmail_service") as mock_svc:
await conn.archive(scout, ItemRef(source_msg_ref="msg-1"))
mock_svc.return_value.users().messages().trash.assert_called()
- Step 2: Run test (fails — module missing)
cd api
pytest tests/test_scout_connectors_gmail.py -v
- Step 3: Implement
app/scouts/connectors/gmail.py
"""Gmail SourceConnector — wraps the existing GmailClient.
Responsibilities:
* list_new: incremental fetch since the scout's stored gmail_history_id
* fetch_metadata: subject + sender + snippet only (Gmail metadata format)
* fetch_content: full body text — transient, never persisted by engine
* archive: move a message to Gmail Trash (recoverable for 30 days)
* setup_watch / renew_watch: Gmail push notifications via Pub/Sub
"""
from __future__ import annotations
import asyncio
import logging
from datetime import datetime, timedelta, timezone
from typing import Any
from app.config.settings import settings
from app.integrations import decrypt_token
from app.integrations.gmail import GmailClient
from app.scouts.connectors.base import ItemContent, ItemMetadata, ItemRef
logger = logging.getLogger(__name__)
def _get_gmail_service(scout):
"""Return a synchronous Google API client for low-level metadata calls."""
from googleapiclient.discovery import build
from google.oauth2.credentials import Credentials
creds_info = decrypt_token(scout.oauth_token_encrypted)
credentials = Credentials(
token=creds_info.get("token"),
refresh_token=creds_info.get("refresh_token"),
token_uri=creds_info.get("token_uri", "https://oauth2.googleapis.com/token"),
client_id=creds_info.get("client_id"),
client_secret=creds_info.get("client_secret"),
scopes=creds_info.get("scopes"),
)
return build("gmail", "v1", credentials=credentials, cache_discovery=False)
class GmailConnector:
source_type = "gmail"
async def list_new(self, scout) -> list[ItemRef]:
def _sync() -> tuple[list[ItemRef], str | None]:
service = _get_gmail_service(scout)
history_id = scout.gmail_history_id
refs: list[ItemRef] = []
new_history_id = history_id
if history_id:
resp = service.users().history().list(
userId="me",
startHistoryId=history_id,
historyTypes=["messageAdded"],
).execute()
for entry in resp.get("history", []):
for added in entry.get("messagesAdded", []):
refs.append(ItemRef(source_msg_ref=added["message"]["id"]))
new_history_id = resp.get("historyId", history_id)
else:
# First run: pick up the latest history id without backfilling.
profile = service.users().getProfile(userId="me").execute()
new_history_id = profile["historyId"]
return refs, new_history_id
refs, new_history_id = await asyncio.to_thread(_sync)
if new_history_id and new_history_id != scout.gmail_history_id:
scout.gmail_history_id = new_history_id
return refs
async def fetch_metadata(self, scout, ref: ItemRef) -> ItemMetadata:
def _sync() -> ItemMetadata:
service = _get_gmail_service(scout)
msg = service.users().messages().get(
userId="me", id=ref.source_msg_ref, format="metadata",
metadataHeaders=["Subject", "From", "Date"],
).execute()
headers = {h["name"]: h["value"] for h in msg.get("payload", {}).get("headers", [])}
return ItemMetadata(
subject=headers.get("Subject"),
sender=headers.get("From"),
snippet=msg.get("snippet"),
received_at=None,
)
return await asyncio.to_thread(_sync)
async def fetch_content(self, scout, ref: ItemRef) -> ItemContent:
creds_info = decrypt_token(scout.oauth_token_encrypted)
client = GmailClient(creds_info)
# Use existing fetch_messages narrowed by message id via the q parameter.
# Gmail API doesn't have a direct "get by message id with full body" via
# GmailClient's high-level interface, so we shell to the low-level call.
def _sync():
service = _get_gmail_service(scout)
msg = service.users().messages().get(
userId="me", id=ref.source_msg_ref, format="full",
).execute()
return msg
msg = await asyncio.to_thread(_sync)
from app.integrations.gmail import _extract_body, _build_email_message
# _extract_body / _build_email_message exist as helpers in gmail.py.
# If named differently in your source, find with: grep -n "def _" app/integrations/gmail.py
email_msg = _build_email_message(msg)
return ItemContent(
metadata=ItemMetadata(
subject=email_msg.subject,
sender=email_msg.sender,
snippet=msg.get("snippet"),
received_at=email_msg.date,
),
body_text=email_msg.body_text,
raw_headers={h["name"]: h["value"] for h in msg.get("payload", {}).get("headers", [])},
)
async def archive(self, scout, ref: ItemRef) -> None:
def _sync():
service = _get_gmail_service(scout)
service.users().messages().trash(userId="me", id=ref.source_msg_ref).execute()
await asyncio.to_thread(_sync)
async def setup_watch(self, scout) -> None:
def _sync():
service = _get_gmail_service(scout)
request_body = {
"labelIds": ["INBOX"],
"topicName": settings.GMAIL_PUBSUB_TOPIC,
}
resp = service.users().watch(userId="me", body=request_body).execute()
scout.gmail_history_id = resp.get("historyId")
scout.gmail_watch_expires_at = datetime.fromtimestamp(
int(resp["expiration"]) / 1000, tz=timezone.utc,
)
await asyncio.to_thread(_sync)
async def renew_watch(self, scout) -> None:
await self.setup_watch(scout)
If _build_email_message and _extract_body are not the actual private helper names in app/integrations/gmail.py, find them:
grep -n "def _" api/app/integrations/gmail.py
and adjust the import. The point is to reuse the existing body-extraction logic rather than duplicate it.
- Step 4: Run tests
cd api
pytest tests/test_scout_connectors_gmail.py -v
Expected: 3 PASS.
- Step 5: Commit
cd api
git add app/scouts/connectors/gmail.py tests/test_scout_connectors_gmail.py
git commit -m "feat(scouts): add GmailConnector"
Task 23: Connector registration at app startup
Files:
-
Modify:
api/app/main.py— registerGmailConnectorduring lifespan startup -
Step 1: Read
app/main.pylifespan
cd api
grep -n "lifespan\|startup\|@asynccontextmanager" app/main.py
- Step 2: Register connector in lifespan startup
@asynccontextmanager
async def lifespan(app: FastAPI):
from app.scouts.connectors.gmail import GmailConnector
from app.scouts.connectors.registry import register_connector
register_connector(GmailConnector())
# ... existing startup (scheduler etc.) ...
yield
# ... existing shutdown ...
- Step 3: Run tests
cd api
pytest
- Step 4: Commit
cd api
git add app/main.py
git commit -m "feat(scouts): register GmailConnector at startup"
Task 24: Real triage LLM call + Langfuse prompt
Files:
-
Modify:
api/app/scouts/engine.py— replace stub_triage_llm -
Manual: create Langfuse prompt
scout-triage-systemwith labelproduction -
Modify:
api/tests/test_scout_engine.py— relax/adjust the monkeypatched stub if needed (existing tests already monkeypatch it, so should keep working) -
Step 1: Create the Langfuse prompt
Use the Langfuse MCP:
mcp__langfuse__createTextPrompt
name: "scout-triage-system"
prompt: |
You are a triage classifier for an executive-assistant scout that watches a {{source_type}} feed.
The scout's purpose is: "{{scout_purpose}}".
Given one item, decide whether it is RELEVANT (worth surfacing to the user as a potential
task / event / note / project) or SPAM (advertising, mass marketing, phishing, bulk
notifications with no actionable content).
Item:
- Subject: {{item_subject}}
- From: {{item_sender}}
- Body (truncated): {{item_body_truncated_2k}}
Return JSON only, matching this schema:
{"verdict": "relevant" | "spam", "reason": <short string>, "confidence": <0..1>}
Be conservative on "spam" — if a message could plausibly be a personal/work email, mark it relevant.
label: "production"
- Step 2: Implement real
_triage_llminapp/scouts/engine.py
Replace the _triage_llm stub with:
async def _triage_llm(self, scout: CloudScoutConfig, content: ItemContent) -> TriageVerdict:
from app.core.llm import get_llm
from app.core.langfuse_client import get_prompt_or_fallback, get_langfuse
fallback = (
"You are a triage classifier. Given subject/sender/body, return JSON: "
"{verdict: relevant|spam, reason: str, confidence: 0..1}"
)
template, prompt_obj = get_prompt_or_fallback("scout-triage-system", fallback)
body_trunc = (content.body_text or "")[:2000]
rendered = (
template
.replace("{{source_type}}", scout.provider)
.replace("{{scout_purpose}}", scout.prompt_template or "")
.replace("{{item_subject}}", content.metadata.subject or "")
.replace("{{item_sender}}", content.metadata.sender or "")
.replace("{{item_body_truncated_2k}}", body_trunc)
)
llm = get_llm(model="gpt-4o-mini")
lf = get_langfuse()
if lf:
with lf.start_as_current_observation(as_type="generation", name="scout-triage", prompt=prompt_obj) as gen:
gen.update(input=rendered)
result = await llm.acomplete(rendered, response_format={"type": "json_object"})
gen.update(output=result.text)
else:
result = await llm.acomplete(rendered, response_format={"type": "json_object"})
import json
data = json.loads(result.text)
return TriageVerdict(**data)
(Adapt to the project's actual LLM helper signature. If get_llm/acomplete aren't the real names, find the LiteLLM wrapper:
grep -n "def \|class " api/app/core/llm.py
and use the actual API.)
- Step 3: Run engine tests
cd api
pytest tests/test_scout_engine.py -v
The existing tests monkeypatch _triage_llm, so they keep passing.
- Step 4: Add an integration test that exercises the real LLM path with a mocked LLM client
Append to tests/test_scout_engine.py:
@pytest.mark.asyncio
async def test_triage_llm_parses_json_response(monkeypatch):
from app.scouts.engine import ScoutEngine
from app.scouts.connectors.base import ItemContent, ItemMetadata
from app.models import CloudScoutConfig
scout = CloudScoutConfig(
id=str(uuid.uuid4()), user_id="x", provider="gmail", name="t",
data_types=[], prompt_template="watch invoices", schedule_cron="0 * * * *",
enabled=True, auto_trash_spam=False, device_inactivity_pause_days=14,
)
content = ItemContent(metadata=ItemMetadata(subject="Invoice 42", sender="acme@x"), body_text="due 2026-06-01")
fake_llm = AsyncMock()
fake_response = AsyncMock()
fake_response.text = '{"verdict": "relevant", "reason": "invoice", "confidence": 0.9}'
fake_llm.acomplete = AsyncMock(return_value=fake_response)
monkeypatch.setattr("app.scouts.engine.get_llm" if False else "app.core.llm.get_llm", lambda **k: fake_llm)
engine = ScoutEngine()
verdict = await engine._triage_llm(scout, content)
assert verdict.verdict == "relevant"
assert verdict.reason == "invoice"
(Adjust the monkeypatch target to match the actual import path used inside _triage_llm.)
- Step 5: Run tests
cd api
pytest tests/test_scout_engine.py -v
- Step 6: Commit
cd api
git add app/scouts/engine.py tests/test_scout_engine.py
git commit -m "feat(scouts): real triage LLM call via scout-triage-system prompt"
Task 25: Pub/Sub webhook route + JWT verification
Files:
-
Create:
api/app/api/routes/scout_webhooks.py -
Create:
api/tests/test_scout_webhook.py -
Modify:
api/app/main.py— register the new router -
Modify:
api/app/config/settings.py— addGMAIL_PUBSUB_TOPICandGMAIL_PUBSUB_AUDIENCEenv vars -
Step 1: Add settings vars
In app/config/settings.py:
GMAIL_PUBSUB_TOPIC: str = "" # e.g. "projects/adiuvai-prod/topics/gmail-watch"
GMAIL_PUBSUB_AUDIENCE: str = "" # OIDC token audience configured in Pub/Sub push subscription
- Step 2: Write the failing test
api/tests/test_scout_webhook.py:
"""Tests for the Gmail Pub/Sub webhook."""
from __future__ import annotations
import base64
import json
import uuid
from unittest.mock import AsyncMock, patch
import pytest
from httpx import AsyncClient
from app.main import app
from app.models import CloudScoutConfig, User
from tests.conftest import _TestSessionLocal
def _pubsub_payload(email: str, history_id: str) -> dict:
inner = json.dumps({"emailAddress": email, "historyId": history_id}).encode()
return {
"message": {"data": base64.b64encode(inner).decode(), "messageId": "m1"},
"subscription": "projects/x/subscriptions/gmail-watch-sub",
}
@pytest.mark.asyncio
async def test_webhook_triggers_scout_for_matching_user():
user_id = "00000000-0000-0000-0000-000000000003"
scout_id = str(uuid.uuid4())
async with _TestSessionLocal() as session:
user = await session.get(User, user_id)
user.email = "alice@example.com"
session.add(CloudScoutConfig(
id=scout_id, user_id=user_id, provider="gmail", name="Inbox",
data_types=[], prompt_template="", schedule_cron="0 * * * *",
enabled=True, auto_trash_spam=False, device_inactivity_pause_days=14,
))
await session.commit()
payload = _pubsub_payload("alice@example.com", "200")
with patch("app.api.routes.scout_webhooks._verify_pubsub_jwt", return_value=True), \
patch("app.scouts.engine.ScoutEngine.trigger_scout", new=AsyncMock()) as mock_trigger:
async with AsyncClient(app=app, base_url="http://test") as client:
resp = await client.post(
"/api/v1/scouts/webhooks/gmail",
json=payload,
headers={"Authorization": "Bearer fake-google-jwt"},
)
assert resp.status_code == 204
mock_trigger.assert_awaited_once_with(uuid.UUID(scout_id))
@pytest.mark.asyncio
async def test_webhook_rejects_unverified_jwt():
payload = _pubsub_payload("alice@example.com", "200")
with patch("app.api.routes.scout_webhooks._verify_pubsub_jwt", return_value=False):
async with AsyncClient(app=app, base_url="http://test") as client:
resp = await client.post(
"/api/v1/scouts/webhooks/gmail",
json=payload,
headers={"Authorization": "Bearer bogus"},
)
assert resp.status_code == 401
- Step 3: Implement
app/api/routes/scout_webhooks.py
"""Gmail Pub/Sub push receiver.
Google Pub/Sub push subscriptions deliver Gmail watch notifications as POST
requests with a JSON envelope. The body payload contains a base64-encoded
JSON blob with ``emailAddress`` + ``historyId``. We resolve the user by
email, look up their cloud_scout_configs row for provider='gmail', and
hand off to ScoutEngine.trigger_scout.
Authentication: Pub/Sub push includes an OIDC JWT in the Authorization
header. We verify it against Google's public keys with the audience
configured in our Pub/Sub subscription.
"""
from __future__ import annotations
import base64
import json
import logging
import uuid
from fastapi import APIRouter, Header, HTTPException, Request, status
from sqlalchemy import select
from app.config.settings import settings
from app.db.session import async_session_factory
from app.models import CloudScoutConfig, User
from app.scouts.engine import ScoutEngine
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/scouts/webhooks", tags=["scout-webhooks"])
def _verify_pubsub_jwt(token: str) -> bool:
"""Verify the Google Pub/Sub OIDC JWT.
Returns False on any verification failure. Production should use
google.oauth2.id_token.verify_oauth2_token with the configured audience.
"""
if not token:
return False
try:
from google.oauth2 import id_token
from google.auth.transport import requests as g_requests
id_token.verify_oauth2_token(token, g_requests.Request(), audience=settings.GMAIL_PUBSUB_AUDIENCE)
return True
except Exception:
logger.warning("pubsub jwt verification failed", exc_info=True)
return False
@router.post("/gmail", status_code=status.HTTP_204_NO_CONTENT)
async def gmail_pubsub(
request: Request,
authorization: str = Header(default=""),
):
token = authorization.removeprefix("Bearer ").strip()
if not _verify_pubsub_jwt(token):
raise HTTPException(status.HTTP_401_UNAUTHORIZED, "Invalid Pub/Sub JWT")
body = await request.json()
msg = body.get("message") or {}
raw = msg.get("data")
if not raw:
return # ack without action
try:
decoded = json.loads(base64.b64decode(raw).decode())
except Exception:
logger.warning("pubsub payload decode failed")
return
email = decoded.get("emailAddress")
if not email:
return
async with async_session_factory() as session:
user_q = await session.execute(select(User).where(User.email == email))
user = user_q.scalar_one_or_none()
if user is None:
logger.info("pubsub: no user for %s — ignoring", email)
return
scouts_q = await session.execute(
select(CloudScoutConfig).where(
CloudScoutConfig.user_id == user.id,
CloudScoutConfig.provider == "gmail",
CloudScoutConfig.enabled == True,
)
)
scouts = scouts_q.scalars().all()
engine = ScoutEngine()
for scout in scouts:
await engine.trigger_scout(uuid.UUID(scout.id))
- Step 4: Register the router in
app/main.py
from app.api.routes import scout_webhooks
app.include_router(scout_webhooks.router, prefix="/api/v1")
- Step 5: Run tests
cd api
pytest tests/test_scout_webhook.py -v
Expected: 2 PASS.
- Step 6: Commit
cd api
git add app/api/routes/scout_webhooks.py tests/test_scout_webhook.py app/main.py app/config/settings.py
git commit -m "feat(scouts): gmail pub/sub webhook with JWT verification"
Task 26: Cron-fallback scheduler + watch renewal in lifespan
Files:
-
Modify:
api/app/main.py— add_scout_cron_tickand_scout_watch_renewal_tickto existing APScheduler -
Step 1: Read existing scheduler setup
cd api
grep -n "AsyncIOScheduler\|apscheduler\|memory_cron_tick" app/main.py
- Step 2: Add the two new ticks (gated on
settings.SCHEDULER_ENABLED)
Inside lifespan:
if settings.SCHEDULER_ENABLED:
scheduler.add_job(
_scout_cron_tick, trigger="interval", minutes=15,
id="scout_cron_tick", replace_existing=True,
)
scheduler.add_job(
_scout_watch_renewal_tick, trigger="interval", hours=24,
id="scout_watch_renewal_tick", replace_existing=True,
)
Add the tick functions to the same file:
async def _scout_cron_tick():
"""Poll cloud scouts whose schedule has elapsed since last_run_at."""
from app.scouts.engine import ScoutEngine
from app.models import CloudScoutConfig
async with async_session_factory() as session:
scouts = (await session.execute(
select(CloudScoutConfig).where(CloudScoutConfig.enabled == True)
)).scalars().all()
engine = ScoutEngine()
for scout in scouts:
# Cheap rate-limit: don't poll if we ran within the last 5 minutes.
# Push notifications are the primary trigger; this is fallback only.
if scout.last_run_at:
elapsed = (datetime.now(tz=timezone.utc) - scout.last_run_at).total_seconds()
if elapsed < 300:
continue
await engine.trigger_scout(uuid.UUID(scout.id))
async def _scout_watch_renewal_tick():
"""Re-issue Gmail users.watch for any scout whose watch expires within 24h."""
from app.scouts.connectors.registry import get_connector
from app.models import CloudScoutConfig
threshold = datetime.now(tz=timezone.utc) + timedelta(hours=24)
async with async_session_factory() as session:
scouts = (await session.execute(
select(CloudScoutConfig).where(
CloudScoutConfig.enabled == True,
CloudScoutConfig.provider == "gmail",
CloudScoutConfig.gmail_watch_expires_at <= threshold,
)
)).scalars().all()
for scout in scouts:
try:
connector = get_connector("gmail")
await connector.renew_watch(scout)
except Exception:
logger.exception("scout %s: watch renewal failed", scout.id)
await session.commit()
- Step 3: Run tests
cd api
pytest
- Step 4: Commit
cd api
git add app/main.py
git commit -m "feat(scouts): add cron-fallback poll + gmail watch renewal ticks"
Task 27: Settings UI — "Add Gmail Scout" OAuth flow
Files:
- Modify:
adiuvAI/src/renderer/components/settings/CloudScoutConfigPanel.tsx— add "Connect Gmail" button + OAuth flow - Modify:
adiuvAI/src/main/router/index.ts— add tRPC procedurescout.cloud.startGmailOAuththat opens the browser to/api/v1/auth/oauth/google/authorizewith scout-specific scopes - Modify:
adiuvAI/src/main/index.ts— extend the existingadiuvai://protocol handler to detect?scout_oauth_callback=1and dispatch to scout flow instead of login flow
Context: The login OAuth flow already exists in auth.py. For scout OAuth we want a separate consent screen with gmail.readonly + gmail.modify scopes. Two approaches:
A) Reuse the existing /auth/oauth/google/authorize route with an extra query param purpose=scout&scout_id=<id>. The route branches on purpose to choose scopes and store-back behavior.
B) Add a new pair of routes /scouts/oauth/gmail/authorize and /scouts/oauth/gmail/callback mirroring the auth pattern.
Pick B for clarity — separates concerns. Implement the new routes alongside the existing ones.
- Step 1: Add scout OAuth routes in
api/app/api/routes/scouts.py
@router.get("/oauth/gmail/authorize")
async def scout_gmail_oauth_authorize(
scout_id: str,
user: User = Depends(get_current_user),
):
from app.auth.oauth_providers import GoogleOAuthProvider, generate_pkce_pair
code_verifier, code_challenge = generate_pkce_pair()
state = secrets.token_urlsafe(32)
_pending_scout_oauth_states[state] = (code_verifier, scout_id, user.id, time.time() + 600)
provider = GoogleOAuthProvider()
authorize_url = provider.build_authorize_url(
redirect_uri=f"{settings.OAUTH_REDIRECT_URI_BASE}/api/v1/scouts/oauth/gmail/web-callback",
state=state,
code_challenge=code_challenge,
scopes=["openid", "email", "https://www.googleapis.com/auth/gmail.readonly", "https://www.googleapis.com/auth/gmail.modify"],
access_type="offline",
prompt="consent",
)
return {"authorize_url": authorize_url}
@router.get("/oauth/gmail/web-callback")
async def scout_gmail_oauth_web_callback(code: str, state: str):
"""Bounces to the Electron deep link with code+state."""
return RedirectResponse(f"adiuvai://scout/oauth/gmail/callback?code={code}&state={state}")
@router.post("/oauth/gmail/callback")
async def scout_gmail_oauth_callback(
body: _ScoutGmailCallbackBody,
db: AsyncSession = Depends(get_session),
user: User = Depends(get_current_user),
):
entry = _pending_scout_oauth_states.pop(body.state, None)
if entry is None or entry[3] < time.time() or entry[2] != user.id:
raise HTTPException(401, "Invalid or expired OAuth state")
code_verifier, scout_id, _, _ = entry
provider = GoogleOAuthProvider()
token_data = await provider.exchange_code(
code=body.code,
code_verifier=code_verifier,
redirect_uri=f"{settings.OAUTH_REDIRECT_URI_BASE}/api/v1/scouts/oauth/gmail/web-callback",
scopes=["openid", "email", "https://www.googleapis.com/auth/gmail.readonly", "https://www.googleapis.com/auth/gmail.modify"],
)
creds_dict = {
"token": token_data["access_token"],
"refresh_token": token_data.get("refresh_token"),
"token_uri": "https://oauth2.googleapis.com/token",
"client_id": settings.GOOGLE_AUTH_CLIENT_ID,
"client_secret": settings.GOOGLE_AUTH_CLIENT_SECRET,
"scopes": ["https://www.googleapis.com/auth/gmail.readonly", "https://www.googleapis.com/auth/gmail.modify"],
}
encrypted = encrypt_token(creds_dict)
scout = await db.get(CloudScoutConfig, scout_id)
if scout is None or scout.user_id != user.id:
raise HTTPException(404, "Scout not found")
scout.oauth_token_encrypted = encrypted
await db.commit()
# Set up the Gmail watch so we start receiving push notifications.
from app.scouts.connectors.registry import get_connector
try:
await get_connector("gmail").setup_watch(scout)
await db.commit()
except Exception:
logger.exception("setup_watch failed for scout %s", scout_id)
return {"ok": True}
(_pending_scout_oauth_states and _ScoutGmailCallbackBody Pydantic model defined at the top of the file alongside the existing _pending_states. encrypt_token imported from app.integrations.)
- Step 2: Add tRPC procedures
scout.cloud.startGmailOAuth+scout.cloud.completeGmailOAuthinadiuvAI/src/main/router/index.ts
startGmailOAuth: publicProcedure
.input(z.object({ scoutId: z.string() }))
.mutation(async ({ input }) => {
const { authorize_url } = await getBackendClient().proxyGet<{ authorize_url: string }>(
`/api/v1/scouts/oauth/gmail/authorize?scout_id=${encodeURIComponent(input.scoutId)}`,
);
await shell.openExternal(authorize_url);
return { ok: true };
}),
completeGmailOAuth: publicProcedure
.input(z.object({ code: z.string(), state: z.string() }))
.mutation(async ({ input }) => {
return await getBackendClient().proxyPost('/api/v1/scouts/oauth/gmail/callback', input);
}),
- Step 3: Extend the
adiuvai://protocol handler inadiuvAI/src/main/index.ts
Find the existing second-instance / open-url handler that parses adiuvai://oauth/callback?.... Add a branch for adiuvai://scout/oauth/gmail/callback?code=...&state=...:
if (parsed.host === 'scout' && parsed.pathname === '/oauth/gmail/callback') {
const code = parsed.searchParams.get('code');
const state = parsed.searchParams.get('state');
if (code && state) {
// Forward to renderer via IPC so the scout settings page can call the tRPC mutation.
BrowserWindow.getAllWindows()[0]?.webContents.send('scout:gmailOAuthCallback', { code, state });
}
return;
}
- Step 4: Add the consumer in
CloudScoutConfigPanel.tsx
const startGmailOAuth = trpc.scout.cloud.startGmailOAuth.useMutation();
const completeGmailOAuth = trpc.scout.cloud.completeGmailOAuth.useMutation();
useEffect(() => {
const off = window.electronAI.on('scout:gmailOAuthCallback', async ({ code, state }) => {
await completeGmailOAuth.mutateAsync({ code, state });
notify('success', 'toast.scout.gmailConnected');
void utils.scout.cloud.list.invalidate();
});
return () => off?.();
}, []);
// In render, when scout.provider === 'gmail' && !scout.oauthTokenEncrypted:
<Button onClick={() => startGmailOAuth.mutate({ scoutId: scout.id })}>
{t('scouts.connectGmail')}
</Button>
(Adjust to actual electronAI API shape — check adiuvAI/src/preload/index.ts for the exact event subscription pattern.)
- Step 5: Add i18n keys
In all 5 translation.json files:
"scouts": {
"connectGmail": "Connect Gmail",
...
},
"toast": {
"scout": {
"gmailConnected": "Gmail connected. Watching for new messages.",
...
}
}
(Translate per language.)
- Step 6: Typecheck + lint + manual test
cd adiuvAI
npx tsc --noEmit
npm run lint
Then manual:
npm run start
Open Settings > Scouts > "Add Cloud Scout" → choose Gmail → click "Connect Gmail" → consent screen opens in browser → choose account, grant scopes → browser redirects to adiuvai://... → app receives callback → toast confirms.
- Step 7: Commit (split across submodules)
cd api
git add app/api/routes/scouts.py app/config/settings.py
git commit -m "feat(scouts): add Gmail OAuth scout-setup routes"
cd ../adiuvAI
git add src/main/router/index.ts src/main/index.ts \
src/renderer/components/settings/CloudScoutConfigPanel.tsx \
src/renderer/locales/
git commit -m "feat(scouts): Gmail OAuth UI flow"
Task 28: Submodule pointer bump for Phase 3
- Step 1: Bump submodule pointers
cd c:\Users\PC-Roby\Documents\_adiuvai_workspace
git add api adiuvAI
git commit -m "chore: bump submodules — Phase 3 Gmail scout end-to-end"
Phase 3 complete. End-to-end Gmail scout: OAuth → watch → push → triage → queue → on reconnect → SQLite scout_suggestions rows with category='unprocessed'.
Final Acceptance Walkthrough (manual, after all tasks)
- Boot app, log in.
- Settings > Scouts: header reads "Scouts" (or localized equivalent).
- Existing local directory monitor still listed and operational.
- "Add scout" → choose Gmail → connect → consent → return to app, scout shows as enabled.
- Send yourself a test email.
- Wait ~30 seconds (Pub/Sub push) or trigger via the cron tick.
- Run from
adiuvAI/:sqlite3 dev.db "select id, scout_id, raw_subject, status, category from scout_suggestions". - Expected: one row with
status='pending',category='unprocessed',raw_subjectmatching the email subject. - Run on the API DB:
psql -d adiuvai -c "select id, scout_id, status, triage_verdict from scout_triage_queue". - Expected: same row,
status='acked'(Electron acked on receipt). - No content (body) anywhere in
scout_triage_queue— verify with\d scout_triage_queue. - Send a spam-looking email (e.g. all-caps subject, "$$$ buy now $$$" body). With
auto_trash_spam=false(default), no row appears in either DB. Toggleauto_trash_spam=trueon the scout, send another spam email — verify it lands in Gmail Trash and no rows appear.
Plan Self-Review
Spec coverage:
- Phase 1 rename — Tasks 1-11. All renamed surfaces from spec table covered.
- Phase 2 connector skeleton — Tasks 12-21. Covers
scout_triage_queuetable,cloud_scout_configsalters,SourceConnectorProtocol, registry,ScoutEngine.trigger_scout+_process_item+deliver_pending+ack_proposal, Drizzlescout_suggestionstable, WS frame schemas, Electron handler. - Phase 3 Gmail end-to-end — Tasks 22-28.
GmailConnectorimpl, registration, real triage LLM, Pub/Sub webhook, cron-fallback + watch renewal scheduler, OAuth setup UI. - Phase 4 explicitly out of scope per spec.
Type/method consistency:
ScoutTriageQueuecolumns match the Alembic migration in Task 12 and the SQLAlchemy model in Task 13.scoutSuggestionsDrizzle columns in Task 17 match theIncomingScoutProposalinterface in Task 20 (snake_case in DB, camelCase in TS object)._triage_llmis a stub in Task 16 and replaced in Task 24 — explicit.ScoutEngine.deliver_pendingsignature(user_id: UUID, ws)is the same in Task 19 (definition) and Task 19 (WS endpoint usage).ScoutProposalPayload(Task 18) matches the dict the engine sends in Task 19.
Placeholder scan: no TBDs or "implement appropriate error handling" stubs. Where the underlying API name (e.g. _extract_body, get_llm) might differ from this codebase's actual symbols, the task gives a grep command to discover the real name and adapt — that's a task-time investigation, not a placeholder.
Scope check: Three phases, each with a clear cut-point and submodule bump. Phase 1 alone is shippable and produces working software (rename, no behavior change). Phase 2 alone is internal-only infra. Phase 3 lands the user-visible feature. All three together fit one implementation track per the approved spec.
Execution Handoff
Plan complete and saved to docs/superpowers/specs/2026-05-15-scouts-refactor-and-gmail-integration-plan.md. Two execution options:
1. Subagent-Driven (recommended) — I dispatch a fresh subagent per task, review between tasks, fast iteration.
2. Inline Execution — Execute tasks in this session using executing-plans, batch execution with checkpoints.
Which approach?