diff --git a/AI_REFACTOR_PLAN.md b/AI_REFACTOR_PLAN.md index d49fb26..226f07a 100644 --- a/AI_REFACTOR_PLAN.md +++ b/AI_REFACTOR_PLAN.md @@ -1,6 +1,6 @@ # AI Refactor Plan — Adiuva Electron App -> **Objective:** Transform the Electron app from a single-process AI integration into a local-first multi-agent client with plugin-based batch agents, multi-provider LLM support, E2E encrypted backup, granular permissions, and cloud backend integration. +> **Objective:** Transform the Electron app into a hybrid-first multi-agent client. The user controls where data is stored (local / cloud / sync), which AI provider to use (BYOK multi-provider), and which automations to run — either custom batch agents built with the LLM-powered Batch Builder, or pre-built plugins from the marketplace. All data access is opt-in, transparent, and auditable. > > **Backend:** Lives in a separate repository. See `BACKEND_PLAN.md` for the API contract and backend implementation guide. > @@ -12,7 +12,7 @@ ### Step 0.1 — Define backend API contract types - [ ] Create `src/shared/api-types.ts` with all interfaces the Electron app needs to communicate with the backend: - - `ExecutionPlan`, `PlanStep`, `PlanAction` (action types: `create_record`, `update_record`, `delete_record`, `index_document`, `send_notification`) + - `ExecutionPlan`, `PlanStep`, `PlanAction` (action types: `create_record`, `update_record`, `delete_record`, `index_document`, `send_notification`, `call_agent`) - `ChatRequest` (message, context, execution_mode: `'direct'` | `'plan'`) - `ChatResponse` (response, actions) - `ChatContext` (user_profile, relevant_documents, recent_tasks, conversation_history) @@ -22,9 +22,25 @@ - `BillingTier` enum (`free`, `pro`, `power`, `team`) - `AuthTokens` (access_token, refresh_token, expires_at) - `UserProfile` (id, email, tier) +- [ ] Create `src/shared/batch-types.ts` with all types for the batch builder and storage layer: + - `StorageTarget` — `'local'` | `'cloud'` | `'sync'` | `'none'` + - `ConnectorType` — `'imap'` | `'filesystem'` | `'calendar'` | `'api'` | `'gmail'` | `'gdrive'` | `'outlook'` + - `BatchActionType` — `'create_record'` | `'update_record'` | `'delete_record'` | `'index_document'` | `'send_notification'` | `'call_agent'` + - `BatchSource` — `{ connector: ConnectorType, config: Record }` + - `BatchTrigger` — `{ type: 'cron' | 'event', schedule?: string, timezone?: string }` + - `BatchAnalysis` — `{ prompt: string, model_override?: string, output_schema?: object }` + - `BatchAction` — `{ type: BatchActionType, table?: string, mapping?: Record }` + - `BatchStorage` — `{ records: StorageTarget, vectors: StorageTarget, raw_data: StorageTarget }` + - `BatchConfig` — full config object: `id`, `name`, `description`, `enabled`, `source`, `trigger`, `analysis`, `actions`, `storage`, `permissions` + - `BatchStatus` — `'idle'` | `'running'` | `'error'` | `'disabled'` + - `BatchRunResult` — `{ batchId, runAt, status, itemsProcessed, errors }` + - `PluginListing` — `{ id, name, description, author, version, rating, installs, category, permissions, price }` + - `InstalledPlugin` — `{ listing: PluginListing, installedAt, enabled, storageConfig: BatchStorage }` + - `DataSourceInfo` — `{ type: ConnectorType, label, recordCount, sizeBytes, storageTarget: StorageTarget }` + - `StorageStats` — `{ localUsedBytes, cloudUsedBytes, cloudLimitBytes, sources: DataSourceInfo[] }` - [ ] Update `tsconfig.json` paths if needed to include `src/shared/` -- **Files:** `src/shared/api-types.ts`, `tsconfig.json` -- **Outcome:** Type-safe contracts for all backend communication. Backend repo mirrors these as Pydantic schemas. +- **Files:** `src/shared/api-types.ts`, `src/shared/batch-types.ts`, `tsconfig.json` +- **Outcome:** Type-safe contracts for all backend communication and the batch/storage subsystem. Backend repo mirrors these as Pydantic schemas. --- @@ -279,55 +295,173 @@ ## Phase 6 — Renderer UI Updates -### Step 6.1 — Update Settings page for multi-provider config -- [ ] Add provider management UI to Settings: - - List of configured providers with status (active/inactive/error) - - Add provider form: name dropdown (OpenAI, Anthropic, Google, Mistral, Groq, Ollama), API key input, model selection, endpoint (for Ollama) - - Set primary and fallback providers - - Test connection button per provider -- [ ] Add auth section to Settings: - - Login/register form - - Current tier display with upgrade CTA - - Logout button -- [ ] Add backup section to Settings: - - Create/view recovery passphrase - - Manual backup trigger - - Backup history with restore points - - Auto-backup schedule toggle -- **Files:** `src/renderer/components/settings/` (new), route file -- **Outcome:** Users can manage AI providers, auth, and backups from Settings. +> **Navigation model:** The app has a sidebar with top-level routes matching the pages below. Each page is a full-screen view. Shared hooks live in `src/renderer/hooks/`. All data access goes through tRPC procedures — no direct IPC calls from components. -### Step 6.2 — Add Permission Dialog and Activity Log -- [ ] Create `src/renderer/components/permissions/PermissionDialog.tsx`: - - Modal shown when a plugin requests new permissions - - Lists requested permissions with reasons - - Per-permission approve/deny toggles - - Shows plugin manifest info (name, description, version) -- [ ] Create `src/renderer/components/permissions/ActivityLog.tsx`: - - Filterable table of all plugin activity - - Columns: timestamp, plugin name, action type, resource, status - - Filter by plugin, date range, action type - - Export as CSV -- [ ] Add tRPC procedures for permission management and activity log queries -- **Files:** `src/renderer/components/permissions/PermissionDialog.tsx`, `src/renderer/components/permissions/ActivityLog.tsx`, `src/main/router/index.ts` -- **Outcome:** Transparent permission system with full activity audit trail. +### Step 6.1 — Restructure app shell and routing +- [ ] Update `src/renderer/App.tsx`: + - Define top-level routes: `/chat`, `/batch-builder`, `/plugins`, `/data-manager`, `/settings`, `/activity` + - Add sidebar navigation with icons and labels for each route + - Persist last active route in electron-store +- [ ] Create `src/renderer/hooks/useProvider.ts`: + - `useProvider()` — returns active provider config, `setProvider()`, `testProvider()`, list of configured providers + - Backed by tRPC `provider.*` procedures (to be added in Phase 1) +- [ ] Create `src/renderer/hooks/useStorage.ts`: + - `useStorage()` — returns `StorageStats`, `setStorageTarget(source, target)`, `migrateData(source, from, to)` + - Backed by tRPC `storage.*` procedures (to be added in Phase 2) +- **Files:** `src/renderer/App.tsx`, `src/renderer/hooks/useProvider.ts`, `src/renderer/hooks/useStorage.ts` +- **Outcome:** App shell with all top-level routes and shared data hooks. -### Step 6.3 — Update AIChatPanel for backend-powered chat -- [ ] Update `src/renderer/hooks/useAIChat.ts`: - - Support WebSocket streaming from backend (when online) - - Fall back to IPC streaming (when offline, using local orchestrator) - - Add connection status indicator (online/offline/reconnecting) - - Support execution plan responses: show plan preview, allow user to approve/modify before execution -- [ ] Update `src/renderer/components/ai/AIChatPanel.tsx`: - - Add connection status badge - - Add tier indicator (shows current plan limitations) - - Plan approval UI: expandable plan steps with approve/reject buttons - - Enhanced error states: differentiate between offline, auth expired, rate limited, server error -- [ ] Update `src/renderer/components/ai/FloatingChat.tsx`: - - Same streaming changes as AIChatPanel - - Compact plan approval for inline context -- **Files:** `src/renderer/hooks/useAIChat.ts`, `src/renderer/components/ai/AIChatPanel.tsx`, `src/renderer/components/ai/FloatingChat.tsx` -- **Outcome:** Chat UI seamlessly handles both online (backend) and offline (local) modes. +### Step 6.2 — ChatPage with context panel +- [ ] Create `src/renderer/pages/ChatPage.tsx`: + - Two-column layout: chat area (left/main) + collapsible `ContextPanel` (right) + - Wraps `ChatWindow` and `ContextPanel` components + - Online/offline status bar at top +- [ ] Create `src/renderer/components/chat/ChatWindow.tsx`: + - Message list rendering `MessageBubble` for each entry + - Input bar with send button and attachment support + - Handles streaming tokens from `useChat` hook + - Plan approval UI inline: expandable plan steps with approve/reject per-step + - Error states: offline, auth expired, rate limited, server error (distinct UI for each) +- [ ] Create `src/renderer/components/chat/MessageBubble.tsx`: + - Renders user / assistant / system messages + - Supports markdown rendering for assistant messages + - Shows tool-call indicators when the agent uses a tool + - Timestamp and copy-to-clipboard action +- [ ] Create `src/renderer/components/chat/ContextPanel.tsx`: + - Shows what context the agent used for the last response: matched documents, recent tasks, memory entries + - Each context item links to its source (note, file, batch result) + - Collapsible, persists open/closed state +- [ ] Create `src/renderer/hooks/useChat.ts`: + - `useChat(sessionId)` — message list, `sendMessage()`, streaming state, connection mode (`'backend'` | `'local'`) + - Automatically falls back to local orchestrator when offline + - Exposes `approveStep(stepId)` / `rejectStep(stepId)` for plan execution +- **Files:** `src/renderer/pages/ChatPage.tsx`, `src/renderer/components/chat/ChatWindow.tsx`, `src/renderer/components/chat/MessageBubble.tsx`, `src/renderer/components/chat/ContextPanel.tsx`, `src/renderer/hooks/useChat.ts` +- **Outcome:** Full chat UI with context transparency, plan approval, and seamless online/offline fallback. + +### Step 6.3 — BatchBuilderPage +- [ ] Create `src/renderer/pages/BatchBuilderPage.tsx`: + - Two views: **Active Batches** list (default) and **Create New Batch** wizard + - Active list renders `BatchCard` for each active batch config + - "Create" button opens the wizard +- [ ] Create `src/renderer/components/batch-builder/NaturalLanguageInput.tsx`: + - Textarea where the user describes the batch in plain language + - "Generate" button calls `useBatchBuilder().generate(description)` + - Loading skeleton while the LLM generates the config +- [ ] Create `src/renderer/components/batch-builder/ConfigPreview.tsx`: + - Shows the generated `BatchConfig` as an editable form (not raw JSON) + - Sections: Source, Trigger, Analysis, Actions, Storage — each collapsible + - Inline editing for every field (prompt textarea, cron expression with human-readable label, mapping table) + - "Edit raw JSON" toggle for power users +- [ ] Create `src/renderer/components/batch-builder/ConnectorPicker.tsx`: + - Dropdown of available connector types (IMAP, Filesystem, Gmail, GDrive, Outlook, Calendar, Generic API) + - When selected, shows connector-specific config fields (e.g. IMAP: host, folder, filter_from; Filesystem: path picker) + - OAuth connectors show "Connect account" button that opens the OAuth flow +- [ ] Create `src/renderer/components/batch-builder/StoragePicker.tsx`: + - Three-way toggle per storage dimension: **Local** / **Cloud** / **Sync** / **None** + - Dimensions: Records, Vectors, Raw data + - Shows storage impact estimate per option + - Disabled options grayed out with tier tooltip if current tier doesn't support cloud +- [ ] Create `src/renderer/components/batch-builder/SchedulePicker.tsx`: + - Mode toggle: **Cron** (with human-readable label, e.g. "Every day at 08:00") / **Event** (on new data from connector) + - Timezone selector (defaults to system timezone) + - Visual cron builder for non-technical users (with raw cron input fallback) +- [ ] Create `src/renderer/components/batch-builder/BatchCard.tsx`: + - Shows batch name, connector icon, last run time, next run time, status badge (`idle` / `running` / `error` / `disabled`) + - Actions: Run now, Edit, Disable/Enable, Delete + - Expandable to show last run summary (items processed, errors) +- [ ] Create `src/renderer/components/batch-builder/BatchTestRunner.tsx`: + - "Dry Run" panel: picks one real item from the source, runs the full analysis pipeline, shows output without saving + - Shows LLM output, action mapping preview, what would be stored and where + - Pass/Fail indicator with detailed error on failure +- [ ] Create `src/renderer/hooks/useBatchBuilder.ts`: + - `useBatchBuilder()` — `generate(description): Promise`, `validate(config)`, `save(config)`, `activate(id)`, `deactivate(id)`, `runNow(id)`, `dryRun(id)`, `delete(id)`, list of saved configs with live status + - Backed by tRPC `batch.*` procedures +- **Files:** `src/renderer/pages/BatchBuilderPage.tsx`, `src/renderer/components/batch-builder/{NaturalLanguageInput,ConfigPreview,ConnectorPicker,StoragePicker,SchedulePicker,BatchCard,BatchTestRunner}.tsx`, `src/renderer/hooks/useBatchBuilder.ts` +- **Outcome:** Full Batch Builder UI — users can describe a batch in natural language, review/edit the generated config, dry-run it, and activate it with a single flow. + +### Step 6.4 — PluginStorePage +- [ ] Create `src/renderer/pages/PluginStorePage.tsx`: + - Two tabs: **Marketplace** (browse available plugins) and **Installed** (manage installed plugins) + - Marketplace: search bar, category filter chips, grid of plugin cards sorted by rating/installs + - Installed: list of `InstalledPlugin` entries with enable/disable toggles and settings links +- [ ] Create plugin card component (inline or shared `common/`): + - Shows name, author, description, rating (stars), install count, category badge, price/free badge + - "Install" button → triggers permission request dialog → installs plugin + - "Settings" button (installed) → opens plugin-specific config drawer +- [ ] Plugin install flow: + - On install click: fetch plugin manifest from backend + - Show `PermissionDialog` with the permissions the plugin requires + - On approve: call tRPC `plugins.install(id)`, download and register the plugin worker + - Show `StoragePicker` for the plugin's data (what goes local/cloud/sync) +- **Files:** `src/renderer/pages/PluginStorePage.tsx` +- **Outcome:** Users can discover and install pre-built plugins from the marketplace with full permission visibility. + +### Step 6.5 — DataManagerPage +- [ ] Create `src/renderer/pages/DataManagerPage.tsx`: + - Top section: `StorageOverview` dashboard + - Below: list of `DataSourceCard` for each active data source (one card per connector/plugin) + - "Migrate" button opens `MigrationWizard` +- [ ] Create `src/renderer/components/data-manager/StorageOverview.tsx`: + - Visual breakdown: local disk used vs. cloud used vs. cloud limit + - Per-category breakdown (emails, files, notes, calendar, vectors) + - Tier upgrade CTA if approaching cloud limit +- [ ] Create `src/renderer/components/data-manager/DataSourceCard.tsx`: + - Card per data source (e.g. "Gmail Scanner", "Documenti/Fatture watcher") + - Shows record count, size, last sync time + - Inline `StoragePicker` toggle for that source (where its data lives) + - "Clear local cache" / "Delete all data" actions with confirmation +- [ ] Create `src/renderer/components/data-manager/MigrationWizard.tsx`: + - Step wizard: select source → select direction (local → cloud or cloud → local) → confirm + - Shows estimated data size and time + - Progress indicator during migration + - Rolls back on error +- **Files:** `src/renderer/pages/DataManagerPage.tsx`, `src/renderer/components/data-manager/{StorageOverview,DataSourceCard,MigrationWizard}.tsx` +- **Outcome:** Users have full visibility and control over where every piece of their data lives. + +### Step 6.6 — ActivityLogPage +- [ ] Create `src/renderer/pages/ActivityLogPage.tsx`: + - Full-page filterable table of all batch/plugin activity entries + - Columns: timestamp, source (batch name / plugin name), action type, data accessed, storage destination, status + - Filters: source, date range, action type, status (success/error) + - Row expand: shows full detail — which records were created/updated, which files were read, LLM calls made + - Export as CSV button +- **Files:** `src/renderer/pages/ActivityLogPage.tsx` +- **Outcome:** Complete transparency log so users can audit exactly what each agent did and when. + +### Step 6.7 — SettingsPage (multi-provider, auth, backup, embeddings) +- [ ] Create `src/renderer/pages/SettingsPage.tsx` with tabbed sections: + - **AI Providers** tab: + - List of configured providers with status badge (active / inactive / error) + - Add provider form: name dropdown (OpenAI, Anthropic, Google, Mistral, Groq, Ollama), API key input, model selection, endpoint (for Ollama) + - Set primary provider and fallback chain + - Test connection button per provider + - Separate "Embeddings provider" section: provider + model for embeddings (OpenAI, Cohere, Voyage, Mistral Embed) + - Info callout: "Text sent to the embeddings provider to generate vectors — make sure you trust this provider with your data" + - **Account & Billing** tab: + - Login/register form (when not authenticated) + - Current tier display with feature list and upgrade CTA + - Usage indicators (batch count, cloud storage used) + - Logout button + - **Backup & Sync** tab: + - Recovery passphrase: generate new / view existing (masked, reveal on click) + - Manual backup trigger with last backup timestamp + - Auto-backup schedule toggle + interval picker + - Backup history table (timestamp, size, restore button) + - **Permissions** tab: + - Table of all active permission grants (plugin/batch, permission type, resource, granted date) + - Revoke button per grant + - Links to ActivityLogPage for per-source audit +- [ ] Create `src/renderer/components/common/ProviderSelector.tsx`: + - Reusable dropdown that lists configured LLM providers + - Used in BatchBuilder (model_override field) and Settings +- [ ] Create `src/renderer/components/common/PermissionDialog.tsx`: + - Modal triggered when a plugin/batch requests new permissions + - Lists each requested permission with its reason and resource path + - Per-permission approve/deny toggles (deny is default) + - Shows plugin/batch manifest info (name, description, version) + - "Approve selected" confirms; "Deny all" closes without granting +- **Files:** `src/renderer/pages/SettingsPage.tsx`, `src/renderer/components/common/PermissionDialog.tsx`, `src/renderer/components/common/ProviderSelector.tsx` +- **Outcome:** Centralised settings covering providers, embeddings, auth, backup, and permissions. --- @@ -375,8 +509,10 @@ | `argon2` | Key derivation for E2E backup | | `node-cron` | Batch agent scheduling | | `chokidar` | File watching (FileWatcher plugin) | -| `imapflow` | IMAP client (EmailScanner plugin) | -| `onnxruntime-node` | Local embeddings (optional) | +| `imapflow` | IMAP client (IMAP connector) | +| `googleapis` | Gmail + GDrive OAuth connectors | +| `lancedb` | Local vector store | +| `onnxruntime-node` | Local embeddings (optional, future) | ---