Agent Runner — Batch File Processing

UML Sequence  ·  Electron FE ↔ FastAPI BE ↔ LLM

sequenceDiagram
    autonumber

    participant FE as Electron FE
(AgentScheduler · DrizzleExecutor)
    participant BE as FastAPI BE
(AgentRunner · routes/agents)
    participant LLM as LLM
(gpt-4.1 via LiteLLM)

    note over FE: AgentScheduler cron tick
or manual "Run Now"

    rect rgb(30, 33, 48)
        note right of FE: PHASE 1 — TRIGGER
        FE->>+BE: POST /agents/trigger
(agent config, directories, schedule)
        BE->>BE: billing check · concurrency guard
create AgentRunLog (status=running)
        BE-->>-FE: 202 Accepted { run_id }
        note over FE: store runId in local agentRuns
        note over BE: asyncio.create_task(run_local_agent)
fire-and-forget background task
    end

    rect rgb(25, 37, 35)
        note right of FE: PHASE 2 — DIRECTORY SCAN (via WS → FE filesystem)
        loop For each directory in agent config (max depth=5)
            BE->>FE: WS tool_call: list_directory { path }
            FE->>FE: fs.readdir(path)
            FE-->>BE: WS tool_result: { entries[] }
            BE->>FE: WS tool_call: get_file_metadata { path }
            FE->>FE: fs.stat(file)
            FE-->>BE: WS tool_result: { modifiedAt, size }
            note over BE: Skip if modifiedAt ≤ last_run_at
(first run: last_run_at=null → all pass)
        end
        note over BE: Result: file_list[] — paths passing
extension + date filters (e.g., 22 files)
    end

    BE->>FE: WS tool_call: select { table: "projects" }
    FE-->>BE: WS tool_result: { rows[] }
    note over BE: Cache project list for prompt context

    rect rgb(35, 30, 22)
        note right of FE: PHASE 3+4 — FOR EACH FILE: READ → PREPROCESS → LLM
        loop For each file in file_list (sequential)

            rect rgb(40, 35, 25)
                note over BE: Phase 3 — Read + Preprocess
                BE->>FE: WS tool_call: read_file_content { path }
                FE->>FE: fs.readFile(path)
                FE-->>BE: WS tool_result: { content }
                BE->>BE: detect_content_type(filename, content)
preprocess() → clean text + metadata
(Python only, zero LLM calls)
            end

            rect rgb(30, 28, 42)
                note over BE: Phase 4 — LLM Agent Tool Loop
                BE->>+LLM: system prompt + clean text
+ available tools + project list
                loop Max 12 tool-call iterations
                    LLM-->>BE: tool_call (e.g., list_tasks, create_note)
                    BE->>FE: WS tool_call: select/insert/update
{ table, data }
                    FE->>FE: DrizzleExecutor
local SQLite CRUD
                    FE-->>BE: WS tool_result: { rows }
                    BE->>LLM: tool_result → continue
                end
                LLM-->>-BE: final text response (no more tool_calls)
            end

            note over BE: log: file processed, created=N entities
            opt If create_project was called
                BE->>FE: WS tool_call: select { table: "projects" }
                FE-->>BE: WS tool_result: { rows[] }
                note over BE: Refresh project list cache
            end
        end
    end

    rect rgb(22, 35, 32)
        note right of FE: PHASE 5 — COMPLETION
        BE->>BE: _finalize_run()
update AgentRunLog in PostgreSQL
status = success | partial | error
        BE->>FE: WS run_complete: { run_id, status }
        FE->>FE: update local agentRuns table
{ status, completedAt }
    end

    note over FE,LLM: ⚠ No file-path journal exists. On re-trigger,
BE re-scans all files. Date filter (modifiedAt > last_run_at)
skips unchanged files, but LLM dedup is the only guard
if last_run_at is null or files are unmodified.