Files
workspace/docs/plan-onboarding-wizard.md
Roberto Musso 8ce3ade8ce feat(onboarding): implement first-run user onboarding wizard for profile setup
- Added a new onboarding wizard that runs on the first app launch post-login.
- Collects user personalization data (job role, industry, primary use case, tone preference, language) and stores it in encrypted core memory.
- Auto-detects and saves formatting preferences (timezone, time format, date format) in local electron-store.
- Normalizes user free-text inputs via a backend LLM call before persisting.
- Introduced new backend routes for memory updates and normalization.
- Updated frontend components to support the onboarding flow with a chat-bubble aesthetic.
- Added settings section for profile editing and re-running the onboarding process.
- Ensured that the onboarding process is skippable and editable in the settings.
- Implemented verification steps to ensure proper functionality and data handling.

chore: update submodules for waitlist and website
2026-04-12 00:36:11 +02:00

313 lines
24 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# First-Run User Onboarding (Profile → Core Memory + Format Prefs)
## Context
Today, after sign-up or login, users land directly on the home chat with no introduction. Signup only collects `name`, `surname`, `email`. The backend AI agents have no idea who they're talking to — generic answers, generic tone, no language match, raw timestamps.
This change adds a one-time wizard that runs the **first time a user opens the app post-login**. It:
1. Seeds the user's **core memory** (encrypted, server-side) with personalization data the AI should reason about (`job_role`, `industry`, `primary_use_case`, `tone_preference`, `language`).
2. Auto-detects and stores **formatting preferences** (`timezone`, `time_format`, `date_format`) **on the FE** as electron-store settings — *not* in core memory, because the LLM should never see raw timestamps or have to reason about format strings. Instead, the FE applies these to tool-result rows before they're sent back to the backend.
3. Optionally normalizes user free-text answers via a single backend LLM call before persisting, so messy inputs like "i build websites" become clean values like "Web Developer".
Because [memory_middleware.py:53-94](api/app/core/memory_middleware.py#L53-L94) already auto-injects `core_memory` into every orchestrator call (see [device_ws.py:213](api/app/api/routes/device_ws.py#L213) and [device_ws.py:282](api/app/api/routes/device_ws.py#L282)), no system-prompt code changes — writing to `MemoryCore` is enough for agents to "see" the data on their next call.
**Decisions made with the user:**
- **UI style**: hybrid chat-styled wizard (looks like AIChatPanel — bubbles, chips — but pre-scripted, no LLM calls per step)
- **Storage split**: AI-relevant fields in encrypted `MemoryCore`; formatting prefs in FE-local electron-store
- **Skippable + editable** in a new Settings → Profile section
- **OS-derived defaults**: language, timezone, time format, date format auto-detected from the OS. Language is shown in the wizard for confirmation; the three formatting prefs are seeded silently and editable in Settings.
- **Avatar**: comes from Google OAuth (already supported via `users.avatar_url`). Not in this wizard. A manual upload control in Settings → Profile is a nice-to-have but **out of scope** for this change.
- **LLM normalization**: yes, but only on free-text answers, in **one batch call at the final 'Done' step**. User sees a "Here's what I saved" review screen and can edit before persisting.
## Fields collected
| Key | Lives in | Source | In wizard? | Editable in Settings? | Used by |
|---------------------|---------------------------|------------------------------|------------------------|----------------------|------------|
| `job_role` | `MemoryCore` (BE, encrypted) | User (chip + free text) | Yes | Yes | LLM |
| `industry` | `MemoryCore` | User (chip + free text) | Yes | Yes | LLM |
| `primary_use_case` | `MemoryCore` | User (chip) | Yes | Yes | LLM |
| `tone_preference` | `MemoryCore` | User (chip) | Yes | Yes | LLM |
| `language` | `MemoryCore` | OS `app.getLocale()` → user confirms | Yes — confirm/change | Yes | LLM (response language) + future UI i18n |
| `timezone` | electron-store (FE) | `Intl.DateTimeFormat().resolvedOptions().timeZone` | No — silent | Yes | FE formatter |
| `time_format` | electron-store (FE) | Derived from locale (12h/24h)| No — silent | Yes | FE formatter |
| `date_format` | electron-store (FE) | Derived from locale (e.g. dd/MM/yyyy) | No — silent | Yes | FE formatter |
The split is the load-bearing decision: **the LLM never sees raw timestamps or format prefs**. Instead, the FE's drizzle executor formats every timestamp column in tool-result rows using the user's preferences before sending the `tool_result` frame back to the backend.
---
## Architecture notes
1. **AI orchestration is fully delegated to the backend** via WebSocket — see [orchestrator.ts:87-117](adiuvAI/src/main/ai/orchestrator.ts#L87-L117). The Electron client never builds a system prompt. So all LLM-relevant personalization must live on the backend (in `MemoryCore`).
2. **Tool calls are FE-executed**: backend sends `WsToolCall` → FE [drizzle-executor.ts](adiuvAI/src/main/api/drizzle-executor.ts) runs the SELECT and returns `{ rows }` → FE [backend-client.ts:652-658](adiuvAI/src/main/api/backend-client.ts#L652-L658) wraps it as a `tool_result` frame. The formatting hook goes **between the executor returning rows and the frame being sent** — this is where raw `dueDate` numbers become `"15/04/2026 14:30"` strings.
3. **Format prefs are per-device**: `timezone` is inherently per-device (your laptop and phone may be in different cities). For consistency we keep all three format prefs FE-local. If the user wants cross-device sync later, this can migrate to `MemoryCore` without breaking the wire format — but that's not v1.
---
## Files to change
### Backend (`api/`)
1. **`api/alembic/versions/<new>_add_onboarded_flag.py`** — new Alembic migration:
- `ALTER TABLE users ADD COLUMN onboarding_completed_at TIMESTAMPTZ NULL`
The five LLM-relevant values live in the existing `memory_core` table — no new columns.
2. **[api/app/models.py:63-94](api/app/models.py#L63-L94)** — add `onboarding_completed_at: Mapped[datetime | None]` to `User`.
3. **[api/app/schemas.py:27-33](api/app/schemas.py#L27-L33)** — extend `UserProfile`:
```python
class UserProfile(BaseModel):
id: str
email: str
name: str | None = None
surname: str | None = None
tier: BillingTier
avatar_url: str | None = None
onboarding_completed_at: int | None = None # epoch ms
memory: dict[str, str] = Field(default_factory=dict)
```
4. **[api/app/api/middleware/auth.py:74-79](api/app/api/middleware/auth.py#L74-L79)** — extend `get_current_user`:
- Read `onboarding_completed_at` from the user row.
- Use `MemoryMiddleware(db).list_core_blocks(user_id)` to load decrypted core blocks → `{label: value}` dict, attach as `memory`.
5. **[api/app/api/routes/auth.py](api/app/api/routes/auth.py)** — add a new route. Do not extend `_UpdateProfileRequest` (keep name/surname separate).
```python
class _UpdateMemoryRequest(BaseModel):
memory: dict[str, str] = Field(default_factory=dict)
mark_onboarded: bool = False
@router.put("/me/memory", response_model=UserProfile)
async def update_memory(
body: _UpdateMemoryRequest,
current_user: UserProfile = Depends(get_current_user),
db: AsyncSession = Depends(get_session),
) -> UserProfile:
memory = MemoryMiddleware(db)
for key, value in body.memory.items():
await memory.update_core(current_user.id, key, value)
if body.mark_onboarded:
result = await db.execute(select(User).where(User.id == current_user.id))
user = result.scalar_one()
user.onboarding_completed_at = datetime.now(timezone.utc)
await db.commit()
# Re-fetch via get_current_user-style logic and return UserProfile.
```
6. **`api/app/api/routes/auth.py`** — new normalization route:
```python
class _NormalizeRequest(BaseModel):
inputs: dict[str, str] # e.g. {"job_role": "i build websites"}
class _NormalizeResponse(BaseModel):
normalized: dict[str, str]
@router.post("/onboarding/normalize", response_model=_NormalizeResponse)
async def normalize_onboarding(
body: _NormalizeRequest,
current_user: UserProfile = Depends(get_current_user),
) -> _NormalizeResponse:
"""One-shot LLM normalization for free-text onboarding answers."""
```
Implementation: build a small system prompt ("You normalize user onboarding answers. Return JSON only. Each key maps to a clean, ≤3-word canonical label."), call `get_llm("gpt-4o-mini", temperature=0)` from [api/app/core/llm.py](api/app/core/llm.py) with `response_format={"type": "json_object"}`, parse, return. Must short-circuit and return the inputs unchanged on any LLM error so the wizard never blocks on a flaky model call. Rate-limited by the existing `TierRateLimiter` middleware.
7. **No orchestrator / prompt changes needed.** `MemoryMiddleware.enrich_context()` already injects `core_memory` into every chat call. **This is the whole point of using `MemoryCore` instead of system-prompt injection.**
### Electron main (`adiuvAI/src/main/`)
8. **[src/shared/api-types.ts:26-34](adiuvAI/src/shared/api-types.ts#L26-L34)** — extend `UserProfileSchema` with `onboardingCompletedAt: z.number().int().nullable().optional()` and `memory: z.record(z.string(), z.string()).default({})`.
9. **[src/main/store.ts:23-38](adiuvAI/src/main/store.ts#L23-L38)** — add a `formatPrefs` block to `AppSettings`:
```ts
formatPrefs: {
timezone: string; // 'Europe/Rome'
dateFormat: string; // 'dd/MM/yyyy' | 'MM/dd/yyyy' | 'yyyy-MM-dd'
timeFormat: '12h' | '24h';
} | null; // null = not yet seeded
```
Default to `null`. Add helpers `getFormatPrefs()` and `setFormatPrefs(prefs)`.
10. **New: `src/main/auth/locale-defaults.ts`** — small helper:
```ts
export function detectFormatPrefs(): FormatPrefs {
const locale = app.getLocale(); // 'it-IT'
const timezone = Intl.DateTimeFormat().resolvedOptions().timeZone;
const timeFormat = Intl.DateTimeFormat(locale, { hour: 'numeric' }).resolvedOptions().hour12 ? '12h' : '24h';
const dateFormat = inferDateFormatFromLocale(locale); // small lookup: 'en-US'→MM/dd/yyyy, 'en-GB'/'it-IT'/...→dd/MM/yyyy, 'ja-JP'/...→yyyy-MM-dd
return { timezone, timeFormat, dateFormat };
}
export function detectLanguage(): string { return app.getLocale(); } // 'it-IT'
```
11. **New: `src/main/api/format-row.ts`** — pure function called by the executor:
```ts
const TIMESTAMP_COLUMNS = new Set([
'createdAt', 'updatedAt', 'dueDate', 'date', 'endDate',
'lastRunAt', 'startedAt', 'completedAt',
]);
export function formatRow<T extends Record<string, unknown>>(row: T, prefs: FormatPrefs): T;
export function formatRows<T extends Record<string, unknown>>(rows: T[], prefs: FormatPrefs): T[];
```
For each known timestamp column whose value is a `number`, replace it with `formatInstant(value, prefs)` where `formatInstant` uses `Intl.DateTimeFormat(locale, { timeZone: prefs.timezone, hour12: prefs.timeFormat === '12h', ... })` and the `dateFormat` setting. Returns a new object — does not mutate.
The set of timestamp columns is hard-coded against the Drizzle schema; if a new timestamp column is added, this set must be updated. (Acceptable for v1 — the schema is small. If it grows, we can derive the set from the Drizzle schema's `integer('...', { mode: 'number' })` columns at startup.)
12. **[src/main/api/drizzle-executor.ts:204-263](adiuvAI/src/main/api/drizzle-executor.ts#L204-L263)** — wrap the executor's `select`/`get`/`insert`/`update` return paths so that `rows`/`row` get passed through `formatRow(s)(..., getFormatPrefs() ?? detectFormatPrefs())`. The `?? detect…` fallback handles the edge case where the executor runs before the first auth.status seed call (e.g. background tool calls during login).
13. **[src/main/auth/auth-manager.ts:170-174](adiuvAI/src/main/auth/auth-manager.ts#L170-L174)** — add two methods:
```ts
async updateMemory(memory: Record<string, string>, markOnboarded = false): Promise<UserProfile>
async normalizeOnboarding(inputs: Record<string, string>): Promise<Record<string, string>>
```
Both call the new backend routes via the existing `put`/`post` helpers.
14. **[src/main/router/index.ts:1059-1098](adiuvAI/src/main/router/index.ts#L1059-L1098)** — extend `authRouter`:
- Add `auth.updateMemory` mutation: input `{ memory, markOnboarded? }`.
- Add `auth.normalizeOnboarding` mutation: input `{ inputs: Record<string, string> }`.
- **Extend `auth.status`** so that immediately after fetching the profile, if `getFormatPrefs()` is `null`, it calls `setFormatPrefs(detectFormatPrefs())` (silent FE seed). If `profile.memory.language` is missing, it also calls `authManager.updateMemory({ language: detectLanguage() })` (silent BE seed). Both run only on first launch — subsequent calls find the values present and short-circuit.
### Electron renderer (`adiuvAI/src/renderer/`)
15. **[src/renderer/components/layout/AppShell.tsx:79-119](adiuvAI/src/renderer/components/layout/AppShell.tsx#L79-L119)** — add the first-run gate. After the `authStatusQuery.data?.authenticated === false` branch:
```tsx
if (authStatusQuery.data?.profile && authStatusQuery.data.profile.onboardingCompletedAt == null) {
return <OnboardingFlow profile={authStatusQuery.data.profile} />;
}
```
16. **New: `src/renderer/components/onboarding/OnboardingFlow.tsx`** — the wizard. Internal state machine:
```ts
type Step = 'welcome' | 'jobRole' | 'industry' | 'useCase' | 'tone' | 'language' | 'reviewing' | 'done';
```
Renders chat-bubble layout matching [AIChatPanel.tsx](adiuvAI/src/renderer/components/ai/AIChatPanel.tsx) — Sparkles icon, `rounded-2xl`, glassmorphism, spring transitions per the design context in `adiuvAI/.claude/CLAUDE.md`. Each step shows an "AI" bubble with the question, 36 chip presets, an optional "type your own" input, and a Skip link.
The `language` step pre-selects the value already in `profile.memory.language` (auto-seeded). User confirms or picks a different one.
**`reviewing` step** (the LLM normalization gate):
- On entry, partition the user's answers into two groups:
- **Chip selections** — already canonical, skip the LLM entirely.
- **Free-text answers** — bundle into a `{key: rawText}` map.
- If the free-text map is non-empty, call `trpc.auth.normalizeOnboarding.useMutation` with it. Show a small inline loader on those fields only ("Tidying up…", ~1-2s).
**Review screen UX** — single card titled "Here's what I'll save", listing all five fields as rows:
| Row appearance | When |
|---------------------------------|---------------------------------------------------|
| Read-only label + value | Chip-selected values (`use_case`, `tone`, etc.) — checkmark icon |
| Read-only label + value + small grey hint `auto-tidied from "i build websites"` | Free-text values that the LLM normalized |
| Read-only label + value | Free-text values that the LLM did NOT change |
Each row has a small **Edit** pencil icon on the right. Clicking it converts that row in-place into a text input (or a Select for chip-based fields like `tone`/`use_case`, populated with the same chip presets plus a free-text "Other" option). The user types the new value, presses Enter or clicks Save → the row goes back to read-only with the **new value as-typed**.
**Edited values are stored verbatim — no re-normalization.** Rationale: the LLM normalization exists to clean up the *initial* messy answer; once the user has seen the suggestion and chosen to override it, re-running the LLM would either no-op (their text is already clean) or fight them. The user is the final arbiter. The "auto-tidied from…" hint disappears once a row is edited (the new value is no longer LLM-derived).
Bottom of the card: a single primary **"Looks good — save"** button → calls `trpc.auth.updateMemory.useMutation` with the final map + `markOnboarded: true` → `utils.auth.status.invalidate()` → AppShell remounts into the normal app. A secondary **"Back to wizard"** link drops the user back to the first wizard step (`jobRole`) with all current values pre-filled — used when the review reveals the answers are wrong enough that re-running the wizard is faster than five inline edits.
**Failure modes**:
- Normalization HTTP call fails → review screen shows raw values with a small banner "Couldn't auto-tidy — review and save". Save still works.
- User clicks "Looks good — save" and `updateMemory` fails → toast error, user stays on the review screen, can retry.
**Skip behaviour**: clicking Skip on any step calls `updateMemory({}, markOnboarded: true)` — empty map, just the flag. We don't re-prompt next launch.
17. **New: `src/renderer/components/onboarding/onboardingOptions.ts`** — preset chip lists:
```ts
export const JOB_ROLES = ['Developer', 'Designer', 'Consultant', 'Founder', 'Project Manager'];
export const INDUSTRIES = ['Tech', 'Design', 'Consulting', 'Legal', 'Marketing', 'Education'];
export const USE_CASES = ['Solo freelancer', 'Client manager', 'Team lead', 'Personal productivity'];
export const TONES = ['Casual', 'Formal', 'Concise', 'Detailed'];
```
18. **[src/renderer/components/settings/types.ts:3-9](adiuvAI/src/renderer/components/settings/types.ts#L3-L9)** — add `'profile'` to `SectionId` and `{ id: 'profile', label: 'Profile' }` to `SECTIONS` (before `'account'`).
19. **New: `src/renderer/components/settings/ProfileSection.tsx`** — Settings → Profile editor. Plain form (no chat aesthetic in Settings). Two cards:
- **"About you"** (writes to `MemoryCore` via `auth.updateMemory`): job_role, industry, primary_use_case, tone_preference, language. "Re-run onboarding" button → small backend route `POST /auth/onboarding/reset` (or just an extension of `update_memory` with `clear_onboarded: true`) that nulls `users.onboarding_completed_at`, then `auth.status.invalidate()` remounts the wizard.
- **"Display preferences"** (writes to electron-store via a new `trpc.settings.setFormatPrefs` mutation): timezone (select populated from `Intl.supportedValuesOf('timeZone')`), date_format (select: dd/MM/yyyy, MM/dd/yyyy, yyyy-MM-dd), time_format (radio: 12h / 24h).
20. **[src/main/router/index.ts](adiuvAI/src/main/router/index.ts)** — `settingsRouter`: add `getFormatPrefs` query and `setFormatPrefs` mutation that read/write to electron-store via `getFormatPrefs()` / `setFormatPrefs()`.
21. **[src/renderer/routes/settings.tsx:55-58](adiuvAI/src/renderer/routes/settings.tsx#L55-L58)** — add `{section === 'profile' && <ProfileSection />}`.
---
## Patterns to reuse (do not duplicate)
- **Stepper state**: `InlineAgentCreationStepper` — `useState<...>` plus conditional rendering.
- **Chat bubble aesthetic**: copy bubble + Sparkles + glass styling from [AIChatPanel.tsx](adiuvAI/src/renderer/components/ai/AIChatPanel.tsx). Do **not** invent a new chat shell.
- **Form components**: shadcn `Field`/`Input`/`Select`/`Button`/`Card` already used in existing settings sections.
- **`MemoryMiddleware.update_core`** ([memory_middleware.py:137-173](api/app/core/memory_middleware.py#L137-L173)) — already used by `deep_agent.py:343`. We just expose it via REST.
- **`get_llm()` from [api/app/core/llm.py](api/app/core/llm.py)** — for the normalization route. Use `gpt-4o-mini` with `temperature=0` and JSON response format.
- **`toCamelCase` / `toSnakeCase`** in `auth-manager` — handles `mark_onboarded` ↔ `markOnboarded` automatically.
- **electron-store helpers** ([store.ts:62-98](adiuvAI/src/main/store.ts#L62-L98)) — same pattern as `getDeviceId` / `getLocalAgents`.
---
## Verification
1. **Backend migration + tests**:
```
cd api && alembic upgrade head
pytest tests/test_auth.py -k "memory or normalize"
```
Manually `curl PUT /api/v1/auth/me/memory` with `{"memory": {"job_role":"Developer"}, "mark_onboarded": true}` and confirm round-trip via `GET /api/v1/auth/me`.
2. **LLM normalization route**:
- `curl POST /api/v1/auth/onboarding/normalize` with `{"inputs": {"job_role": "i build websites", "industry": "tech-ish stuff"}}`.
- Expect `{"normalized": {"job_role": "Web Developer", "industry": "Technology"}}` (or similar — exact phrasing varies).
- Stop the LLM provider (or use an invalid `OPENAI_API_KEY`) and re-run — must return inputs unchanged, never 500.
3. **Locale auto-seed (FE + BE)**:
- Fresh user, fresh electron-store. Log in via Electron.
- `getFormatPrefs()` should now return the detected `{timezone, dateFormat, timeFormat}`.
- `memory_core` should have one row: `language`.
- Reload app → no second seed call (idempotent).
4. **First-run wizard (golden path with chips only)**:
- Reset: backend `UPDATE users SET onboarding_completed_at=NULL; DELETE FROM memory_core WHERE user_id=...`. FE: clear electron-store `formatPrefs`.
- `npm start` → log in → land on `OnboardingFlow`.
- Pick a chip on every step (no free text). Confirm language. Land on review screen — should not show a loading spinner (no normalization needed). Click Confirm.
- `auth.status` invalidates, AppShell mounts the home chat.
- `SELECT key FROM memory_core WHERE user_id=...` → 5 keys (job_role, industry, primary_use_case, tone_preference, language).
- Reload app → does not re-prompt.
5. **First-run wizard (free-text path)**:
- Reset. Walk through wizard typing free text on `job_role` and `industry` (e.g. "i build websites", "tech-ish stuff").
- On final step, see ~1-2s "Tidying up…" spinner, then a review screen showing the normalized values plus the chip-selected use_case/tone/language.
- Edit one normalized value manually. Confirm. The edited value is what lands in `memory_core`.
6. **AI uses the data (proof the wiring works)**:
- With an onboarded user whose `tone_preference="Formal"` and `language="it-IT"`, ask the home chat "draft a quick status email".
- Response should be in Italian and read formal. **If language doesn't match, `enrich_context` is not feeding `core_memory` into the prompt as expected — investigate before declaring done.** This is the single most likely failure point because we don't modify it.
7. **Format prefs reach the LLM as strings**:
- From the home chat, ask "what tasks are due this week?".
- Inspect the network/log of the `tool_result` frame the FE sends back. Every `dueDate` field must be a formatted string like `"15/04/2026 14:30"`, not a numeric timestamp.
- The AI's response must reference dates in the user's preferred format.
- Change `time_format` from 24h to 12h in Settings → Profile. Re-ask. Times should now be `2:30 PM` style.
8. **Skip flow**:
- Reset, log in, click Skip on step 1.
- `users.onboarding_completed_at` set; `memory_core` only has `language` (from auto-seed).
- Reload → no re-prompt.
9. **Re-run onboarding**: Settings → Profile → "Re-run onboarding" → wizard mounts immediately.
10. **Lint**:
```
cd adiuvAI && source ~/.nvm/nvm.sh && npm run lint
cd api && ruff check .
```
---
## Out of scope (deferred)
- **UI internationalisation framework**: storing `language` enables future i18n, but no translation library is added. Wizard copy hardcoded in English for v1.
- **Avatar upload control** in Settings → Profile: avatar already comes from Google OAuth via `users.avatar_url`. A manual upload UI is a nice-to-have follow-up.
- **Working hours**, **top goals** (free-text seeds): same `MemoryCore` pattern — easy to add later.
- **Cross-device sync of format prefs**: v1 stores them per-device in electron-store. Migrating to `MemoryCore` later doesn't break the wire format.
- **Schema-bump re-prompting**: when we add a new wizard question later we'll need a `core_memory["__onboarding_version__"]` key and a guard. Not needed now.
- **Animated typing effect** on AI bubbles.