refactor: remove storage, backup, plugin/marketplace features
- Delete app/storage/ (blob_store, vector_store, encryption) - Delete app/marketplace/ (plugin_registry, plugin_review, revenue_share) - Delete routes: backup.py, plugins.py, storage.py, vectors.py - Relocate embed endpoint to POST /chat/embed - Rewrite migration 001 (remove storage/plugin tables) - Delete migration 002 (seed_plugins) - Remove S3/Pinecone/Qdrant env vars from settings - Remove storage/backup quotas from tier_manager - Remove MinIO and Qdrant from docker-compose - Delete tests: test_backup, test_plugins, test_storage - Update README.md and clean .env.example
This commit is contained in:
300
README.md
300
README.md
@@ -1,8 +1,8 @@
|
||||
# Adiuva Cloud API
|
||||
|
||||
**AI-powered project management backend with E2E encrypted cloud storage, LLM orchestration, and a plugin marketplace.**
|
||||
**AI-powered project management backend with LLM orchestration and subscription billing.**
|
||||
|
||||
Built with FastAPI · Python 3.12 · PostgreSQL · LangChain · Stripe · AWS S3
|
||||
Built with FastAPI · Python 3.12 · PostgreSQL · LangChain · Stripe
|
||||
|
||||
---
|
||||
|
||||
@@ -20,9 +20,7 @@ Built with FastAPI · Python 3.12 · PostgreSQL · LangChain · Stripe · AWS S3
|
||||
- [AI Agent System](#ai-agent-system)
|
||||
- [Orchestration & Execution Plans](#orchestration--execution-plans)
|
||||
- [Middleware](#middleware)
|
||||
- [Storage Layer](#storage-layer)
|
||||
- [Billing & Tiers](#billing--tiers)
|
||||
- [Plugin Marketplace](#plugin-marketplace)
|
||||
- [Testing](#testing)
|
||||
- [Project Structure](#project-structure)
|
||||
- [License](#license)
|
||||
@@ -31,15 +29,13 @@ Built with FastAPI · Python 3.12 · PostgreSQL · LangChain · Stripe · AWS S3
|
||||
|
||||
## Overview
|
||||
|
||||
Adiuva Cloud API is the FastAPI backend that powers the **Adiuva Electron desktop app**. It provides LLM-powered chat orchestration, end-to-end encrypted cloud storage, a vector search engine, an encrypted backup system, a plugin marketplace with revenue sharing, and Stripe-based subscription billing across four tiers.
|
||||
Adiuva Cloud API is the FastAPI backend that powers the **Adiuva Electron desktop app**. It provides LLM-powered chat orchestration, text embedding generation, and Stripe-based subscription billing across four tiers.
|
||||
|
||||
### Design Principles
|
||||
|
||||
1. **Never persist user data in plaintext** — the database stores only auth, billing, storage metadata, and marketplace data. All user content is E2E encrypted by the client before reaching the server.
|
||||
2. **Never expose prompts** — system prompts stay server-side; responses are sanitized to strip any leaked prompt fragments.
|
||||
3. **Never decrypt user blobs** — the backend performs only checksum verification; no decryption keys ever reach the server.
|
||||
4. **Stateless request handling** — all context comes from the client and JWT; no server-side session state.
|
||||
5. **Tier gates enforced server-side** — the server always reads the current tier from the database, never trusting client-reported values.
|
||||
1. **Never expose prompts** — system prompts stay server-side; responses are sanitized to strip any leaked prompt fragments.
|
||||
2. **Stateless request handling** — all context comes from the client and JWT; no server-side session state.
|
||||
3. **Tier gates enforced server-side** — the server always reads the current tier from the database, never trusting client-reported values.
|
||||
|
||||
---
|
||||
|
||||
@@ -54,27 +50,26 @@ Adiuva Cloud API is the FastAPI backend that powers the **Adiuva Electron deskto
|
||||
│ ┌──────────────────┐ ┌────────────────────────────┐ │
|
||||
│ │ Auth Routes │ │ Chat Routes │ │
|
||||
│ │ Billing Routes │ │ ↓ │ │
|
||||
│ │ Storage Routes │ │ Orchestrator (GPT-4o-mini)│ │
|
||||
│ │ Backup Routes │ │ ↓ classify intent │ │
|
||||
│ │ Plugin Routes │ │ Agent Registry │ │
|
||||
│ │ Vector Routes │ │ ↓ │ │
|
||||
│ │ Plans Routes │ │ TaskAgent | ProjectAgent │ │
|
||||
│ └──────────────────┘ │ NoteAgent | CheckptAgent │ │
|
||||
│ │ Agent Routes │ │ Orchestrator (GPT-4o-mini)│ │
|
||||
│ │ Device WS │ │ ↓ classify intent │ │
|
||||
│ └──────────────────┘ │ Agent Registry │ │
|
||||
│ │ ↓ │ │
|
||||
│ │ TaskAgent | ProjectAgent │ │
|
||||
│ │ NoteAgent | CheckptAgent │ │
|
||||
│ │ (GPT-4o + LangChain) │ │
|
||||
│ └────────────────────────────┘ │
|
||||
└────────────────────────────────────────────────────────┘
|
||||
│ │ │
|
||||
┌────────▼───┐ ┌───────▼───────┐ ┌──▼─────────────┐
|
||||
│ PostgreSQL │ │ AWS S3 │ │ Pinecone / │
|
||||
│ (Auth, │ │ (E2E blobs, │ │ Qdrant │
|
||||
│ Billing, │ │ backups) │ │ (Vectors) │
|
||||
│ Metadata) │ └───────────────┘ └────────────────┘
|
||||
│
|
||||
┌────────▼───┐
|
||||
│ PostgreSQL │
|
||||
│ (Auth, │
|
||||
│ Billing, │
|
||||
│ Agents) │
|
||||
└────────────┘
|
||||
│
|
||||
┌────────▼───┐
|
||||
│ Stripe │
|
||||
│ (Billing, │
|
||||
│ Connect) │
|
||||
│ (Billing) │
|
||||
└────────────┘
|
||||
```
|
||||
|
||||
@@ -85,18 +80,14 @@ Adiuva Cloud API is the FastAPI backend that powers the **Adiuva Electron deskto
|
||||
1. **LLM-powered orchestration** — GPT-4o-mini classifies user intent and routes to the appropriate domain agent.
|
||||
2. **4 specialized AI agents** — Tasks (8 tools), Projects (6 tools), Timelines (4 tools), Notes (5 tools), all powered by GPT-4o via LangChain.
|
||||
3. **Execution plans & playbooks** — Server-side prompt template registry; clients receive only opaque template IDs, never raw prompts.
|
||||
4. **E2E encrypted cloud storage** — The backend never decrypts user data; SHA-256 checksum verification uses constant-time comparison to prevent timing attacks.
|
||||
5. **Cloud vector store** — Pinecone or Qdrant with user-isolated namespaces and encrypted blob payloads.
|
||||
6. **Encrypted backup system** — Tiered storage limits with `If-Modified-Since` support for efficient syncing.
|
||||
7. **Plugin marketplace** — Catalog, admin review/approval workflow, security checklist, and 70/30 revenue sharing via Stripe Connect.
|
||||
8. **Stripe billing** — Four-tier subscription model (Free / Pro / Power / Team) with checkout sessions and full webhook lifecycle handling.
|
||||
9. **JWT authentication** — Access + refresh tokens with bcrypt password hashing, SHA-256 token hashing, and automatic rotation.
|
||||
10. **Prompt IP protection** — Sanitizer middleware strips system prompts, reasoning markers, tool schemas, and agent routing metadata from all chat responses.
|
||||
11. **Tier-based rate limiting** — Sliding-window per-user limiter scaling from 20 to 200 requests/min by subscription tier.
|
||||
12. **Zero-trust data model** — User content is never stored in plaintext; the database holds only authentication, billing, and metadata records.
|
||||
13. **WebSocket streaming** — Real-time chat with 30-second heartbeat keep-alive and chunked text delivery.
|
||||
14. **Alembic migrations** — Versioned schema management with seed data for the plugin marketplace.
|
||||
15. **Comprehensive test suite** — In-memory SQLite + moto S3 mocks, per-tier test fixtures, and full API coverage without external dependencies.
|
||||
4. **Text embeddings** — Generates text-embedding-3-small vectors for local client-side note search.
|
||||
5. **Stripe billing** — Four-tier subscription model (Free / Pro / Power / Team) with checkout sessions and full webhook lifecycle handling.
|
||||
6. **JWT authentication** — Access + refresh tokens with bcrypt password hashing, SHA-256 token hashing, and automatic rotation.
|
||||
7. **Prompt IP protection** — Sanitizer middleware strips system prompts, reasoning markers, tool schemas, and agent routing metadata from all chat responses.
|
||||
8. **Tier-based rate limiting** — Sliding-window per-user limiter scaling from 20 to 200 requests/min by subscription tier.
|
||||
9. **WebSocket streaming** — Real-time chat with 30-second heartbeat keep-alive and chunked text delivery.
|
||||
10. **Alembic migrations** — Versioned schema management.
|
||||
11. **Comprehensive test suite** — In-memory SQLite, per-tier test fixtures, and full API coverage without external dependencies.
|
||||
|
||||
---
|
||||
|
||||
@@ -114,7 +105,6 @@ Adiuva Cloud API is the FastAPI backend that powers the **Adiuva Electron deskto
|
||||
| `pydantic-settings` | ≥ 2.7.0 | Environment-based configuration |
|
||||
| `python-jose[cryptography]` | ≥ 3.3.0 | JWT encoding and decoding |
|
||||
| `stripe` | ≥ 11.0.0 | Billing and payment integration |
|
||||
| `boto3` | ≥ 1.35.0 | AWS S3 client |
|
||||
| `slowapi` | ≥ 0.1.9 | Rate limiting utilities |
|
||||
| `sqlalchemy` | ≥ 2.0.0 | Async ORM and query builder |
|
||||
| `asyncpg` | ≥ 0.30.0 | PostgreSQL async driver |
|
||||
@@ -124,12 +114,9 @@ Adiuva Cloud API is the FastAPI backend that powers the **Adiuva Electron deskto
|
||||
| `httpx` | ≥ 0.28.0 | Async HTTP client (used in tests) |
|
||||
| `websockets` | ≥ 14.0 | WebSocket protocol support |
|
||||
| `psycopg2-binary` | ≥ 2.9.0 | Synchronous PostgreSQL driver (Alembic) |
|
||||
| `pinecone` | ≥ 5.0.0 | Pinecone vector store client |
|
||||
| `qdrant-client` | ≥ 1.7.0 | Qdrant vector store client |
|
||||
| `pytest` | ≥ 8.0.0 | Test framework |
|
||||
| `pytest-asyncio` | ≥ 0.24.0 | Async test support |
|
||||
| `aiosqlite` | ≥ 0.20.0 | In-memory SQLite for tests |
|
||||
| `moto[s3]` | ≥ 5.0.0 | AWS S3 mock for tests |
|
||||
| `ruff` | ≥ 0.8.0 | Linter and formatter |
|
||||
|
||||
---
|
||||
@@ -142,7 +129,6 @@ Adiuva Cloud API is the FastAPI backend that powers the **Adiuva Electron deskto
|
||||
- PostgreSQL 16+
|
||||
- An OpenAI API key (for LLM features)
|
||||
- Stripe API keys (optional — billing stubs gracefully when unconfigured)
|
||||
- AWS credentials (optional — needed for S3 storage in production)
|
||||
|
||||
### Installation
|
||||
|
||||
@@ -194,11 +180,6 @@ This starts two services:
|
||||
- **app** — FastAPI server on port `8000`
|
||||
- **db** — PostgreSQL 16 (Alpine) on port `5432` with a persistent volume and health checks
|
||||
|
||||
The compose file also includes optional services for fully local deployments:
|
||||
|
||||
- **minio** — S3-compatible object storage on ports `9000` (API) and `9001` (console)
|
||||
- **qdrant** — Vector search engine on ports `6333` (HTTP) and `6334` (gRPC)
|
||||
|
||||
### Dockerfile Details
|
||||
|
||||
The Dockerfile uses a multi-stage build:
|
||||
@@ -216,7 +197,7 @@ gunicorn app.main:app -k uvicorn.workers.UvicornWorker -w 4 --timeout 120 -b 0.0
|
||||
|
||||
## Homelab / Self-Hosted Deployment
|
||||
|
||||
You can run the entire stack locally on a homelab with **no cloud dependencies except the LLM provider**. The compose file includes MinIO (S3 replacement) and Qdrant (vector store) out of the box.
|
||||
You can run the entire stack locally on a homelab with **no cloud dependencies except the LLM provider**.
|
||||
|
||||
### 1. Start all services
|
||||
|
||||
@@ -224,35 +205,14 @@ You can run the entire stack locally on a homelab with **no cloud dependencies e
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
This starts PostgreSQL, MinIO, and Qdrant alongside the app.
|
||||
This starts PostgreSQL alongside the app.
|
||||
|
||||
### 2. Create the MinIO bucket
|
||||
|
||||
Open the MinIO console at [http://localhost:9001](http://localhost:9001) (login: `minioadmin` / `minioadmin`) and create a bucket named `adiuva`, or use the CLI:
|
||||
|
||||
```bash
|
||||
docker compose exec minio mc alias set local http://localhost:9000 minioadmin minioadmin
|
||||
docker compose exec minio mc mb local/adiuva
|
||||
```
|
||||
|
||||
### 3. Configure your `.env`
|
||||
### 2. Configure your `.env`
|
||||
|
||||
```bash
|
||||
# Database (uses the compose PostgreSQL)
|
||||
DATABASE_URL=postgresql+asyncpg://postgres:postgres@db:5432/adiuva
|
||||
|
||||
# S3 → MinIO
|
||||
S3_BUCKET=adiuva
|
||||
S3_REGION=us-east-1
|
||||
S3_ENDPOINT_URL=http://minio:9000
|
||||
AWS_ACCESS_KEY_ID=minioadmin
|
||||
AWS_SECRET_ACCESS_KEY=minioadmin
|
||||
|
||||
# Vector store → local Qdrant (leave PINECONE_API_KEY empty)
|
||||
QDRANT_URL=http://qdrant:6333
|
||||
QDRANT_API_KEY=
|
||||
PINECONE_API_KEY=
|
||||
|
||||
# Billing — leave empty to stub (no Stripe needed)
|
||||
STRIPE_SECRET_KEY=
|
||||
STRIPE_WEBHOOK_SECRET=
|
||||
@@ -267,7 +227,7 @@ JWT_SECRET=your-secret-here
|
||||
ENV=dev
|
||||
```
|
||||
|
||||
### 4. Run migrations
|
||||
### 3. Run migrations
|
||||
|
||||
```bash
|
||||
docker compose exec app alembic upgrade head
|
||||
@@ -278,9 +238,7 @@ docker compose exec app alembic upgrade head
|
||||
| Service | Runs on | Port | Notes |
|
||||
|---|---|---|---|
|
||||
| FastAPI app | Docker | 8000 | API server |
|
||||
| PostgreSQL | Docker | 5432 | Auth, billing, metadata |
|
||||
| MinIO | Docker | 9000 / 9001 | S3-compatible blob & backup storage |
|
||||
| Qdrant | Docker | 6333 / 6334 | Vector search (replaces Pinecone) |
|
||||
| PostgreSQL | Docker | 5432 | Auth, billing, agents |
|
||||
| Stripe | — | — | Stubbed when keys are empty |
|
||||
| OpenAI / LLM | Cloud | — | Only external dependency |
|
||||
|
||||
@@ -300,17 +258,7 @@ All variables are loaded from a `.env` file via Pydantic Settings. Source: `app/
|
||||
| `JWT_ACCESS_TOKEN_EXPIRE_MINUTES` | `int` | `30` | Access token time-to-live |
|
||||
| `JWT_REFRESH_TOKEN_EXPIRE_DAYS` | `int` | `30` | Refresh token time-to-live |
|
||||
| `STRIPE_SECRET_KEY` | `str` | `""` | Stripe API key (empty = stub mode) |
|
||||
| `STRIPE_WEBHOOK_SECRET` | `str` | `""` | Stripe webhook signature secret |
|
||||
| `S3_BUCKET` | `str` | `""` | S3 bucket for encrypted blobs and backups |
|
||||
| `S3_REGION` | `str` | `us-east-1` | AWS region |
|
||||
| `S3_ENDPOINT_URL` | `str` | `""` | Custom S3 endpoint (e.g. `http://minio:9000` for MinIO). Leave empty for AWS. |
|
||||
| `AWS_ACCESS_KEY_ID` | `str` | `""` | AWS credentials |
|
||||
| `AWS_SECRET_ACCESS_KEY` | `str` | `""` | AWS credentials |
|
||||
| `PINECONE_API_KEY` | `str` | `""` | Pinecone API key (if set, Pinecone is used for vectors) |
|
||||
| `PINECONE_INDEX` | `str` | `adiuva` | Pinecone index name |
|
||||
| `QDRANT_URL` | `str` | `""` | Qdrant URL (used when Pinecone is not configured) |
|
||||
| `QDRANT_API_KEY` | `str` | `""` | Qdrant API key |
|
||||
| `OPENAI_API_KEY` | `str` | `""` | OpenAI key for LLM agent calls |
|
||||
| `STRIPE_WEBHOOK_SECRET` | `str` | `\"\"` | Stripe webhook signature secret |\n| `OPENAI_API_KEY` | `str` | `\"\"` | OpenAI key for LLM agent calls |
|
||||
| `LLM_MODEL` | `str` | `gpt-4o` | LiteLLM model identifier for agents (e.g. `anthropic/claude-3.5-sonnet`, `gemini/gemini-pro`, `ollama/llama3`) |
|
||||
| `LLM_ROUTER_MODEL` | `str` | `gpt-4o-mini` | Lighter model used for intent classification / routing |
|
||||
| `CORS_ORIGINS` | `list[str]` | `["app://.", "http://localhost:3000", "http://localhost:5173"]` | Allowed CORS origins |
|
||||
@@ -342,6 +290,7 @@ All routes are prefixed with `/api/v1`. **27 endpoints** total (25 REST + 1 WebS
|
||||
| Method | Path | Auth | Description |
|
||||
|---|---|---|---|
|
||||
| `POST` | `/api/v1/chat` | JWT | Route message through the orchestrator; returns `ChatResponse` or `ExecutionPlan` depending on execution mode |
|
||||
| `POST` | `/api/v1/chat/embed` | JWT | Generate a 1536-dim text embedding vector (`text-embedding-3-small`). Used by Electron for local note search. |
|
||||
| `WS` | `/api/v1/chat/stream` | JWT (query param `?token=`) | Streaming chat — first frame is a `ChatRequest`, server yields text chunks, final frame is `{"done": true, "response": "...", "actions": [...]}`. 30-second heartbeat ping. |
|
||||
|
||||
### Plans
|
||||
@@ -351,42 +300,6 @@ All routes are prefixed with `/api/v1`. **27 endpoints** total (25 REST + 1 WebS
|
||||
| `GET` | `/api/v1/plans/playbook` | JWT | List all cached execution plan playbooks |
|
||||
| `GET` | `/api/v1/plans/playbook/{plan_id}` | JWT | Retrieve a specific playbook by ID |
|
||||
|
||||
### Storage (Cloud Records)
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|---|---|---|---|
|
||||
| `POST` | `/api/v1/storage/records` | JWT | Upload an E2E encrypted record (verifies checksum, enforces storage quota) |
|
||||
| `GET` | `/api/v1/storage/records` | JWT | List record metadata with pagination (`?table`, `?page`, `?limit`); no blob bytes returned |
|
||||
| `GET` | `/api/v1/storage/records/{id}` | JWT | Download encrypted blob with `X-Checksum` response header |
|
||||
| `PUT` | `/api/v1/storage/records/{id}` | JWT | Replace an existing blob (verifies checksum, enforces quota) |
|
||||
| `DELETE` | `/api/v1/storage/records/{id}` | JWT | Delete a record and its S3 blob |
|
||||
|
||||
### Vectors (Cloud Vector Store)
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|---|---|---|---|
|
||||
| `POST` | `/api/v1/storage/vectors/upsert` | JWT | Verify checksums and upsert encrypted vectors |
|
||||
| `POST` | `/api/v1/storage/vectors/search` | JWT | Search user-scoped vector namespace |
|
||||
| `DELETE` | `/api/v1/storage/vectors` | JWT | Delete vectors by ID list |
|
||||
|
||||
### Backup
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|---|---|---|---|
|
||||
| `PUT` | `/api/v1/backup` | JWT | Upload encrypted backup blob with custom headers (`X-Backup-Version`, `X-Backup-Timestamp`, `X-Backup-Checksum`). Tier quota enforced. |
|
||||
| `GET` | `/api/v1/backup` | JWT | Download latest backup blob. Supports `If-Modified-Since`. |
|
||||
| `GET` | `/api/v1/backup/history` | JWT | List backup metadata (no blob content) |
|
||||
| `DELETE` | `/api/v1/backup/{backup_id}` | JWT | Delete a specific backup |
|
||||
|
||||
### Plugins (Marketplace)
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|---|---|---|---|
|
||||
| `GET` | `/api/v1/plugins` | JWT (Power+) | Browse the marketplace (`?category`, `?q`, `?page`, `?sort=rating\|installs\|newest`) |
|
||||
| `GET` | `/api/v1/plugins/{id}` | JWT (Power+) | Plugin detail with install count and ratings |
|
||||
| `POST` | `/api/v1/plugins/{id}/install` | JWT (Power+) | Install plugin; triggers Stripe Connect revenue split for paid plugins |
|
||||
| `DELETE` | `/api/v1/plugins/{id}/install` | JWT | Uninstall plugin |
|
||||
|
||||
### Billing
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
@@ -400,7 +313,7 @@ All routes are prefixed with `/api/v1`. **27 endpoints** total (25 REST + 1 WebS
|
||||
|
||||
## Data Model
|
||||
|
||||
9 tables managed by Alembic migrations. Source: `app/models.py`
|
||||
3 tables managed by Alembic migrations. Source: `app/models.py`
|
||||
|
||||
### Tables
|
||||
|
||||
@@ -409,27 +322,18 @@ All routes are prefixed with `/api/v1`. **27 endpoints** total (25 REST + 1 WebS
|
||||
| `users` | `id` (UUID) | `email` (unique), `password_hash`, `tier`, `stripe_customer_id`, timestamps | User accounts |
|
||||
| `refresh_tokens` | `id` (UUID) | `user_id` (FK), `token_hash` (SHA-256, unique), `expires_at` | Hashed refresh tokens for rotation |
|
||||
| `subscriptions` | `id` (UUID) | `user_id` (FK, unique), `stripe_subscription_id`, `tier`, `status`, `current_period_end` | Stripe subscription records |
|
||||
| `storage_records` | `id` (UUID) | `user_id` (FK), `table_name`, `s3_key`, `checksum`, `size_bytes`, timestamps | S3 blob metadata (no plaintext content) |
|
||||
| `backup_metadata` | `id` (UUID) | `user_id` (FK), `s3_key`, `version`, `timestamp`, `checksum`, `size_bytes` | Backup manifests |
|
||||
| `plugins` | `id` (String) | `name`, `description`, `version`, `author_id` (FK), `category`, `price_cents`, `permissions` (JSON), `status`, `s3_package_key`, `install_count`, `avg_rating` | Marketplace plugin catalog |
|
||||
| `plugin_installations` | `id` (UUID) | `plugin_id` (FK), `user_id` (FK), unique constraint on (`plugin_id`, `user_id`) | Per-user install tracking |
|
||||
| `plugin_reviews` | `id` (UUID) | `plugin_id` (FK), `reviewer_id` (FK), `decision`, `notes`, `reviewed_at` | Admin review decisions |
|
||||
| `revenue_events` | `id` (UUID) | `plugin_id` (FK), `user_id` (FK), `amount_cents`, `developer_share_cents`, `stripe_transfer_id` | 70/30 revenue split ledger |
|
||||
|
||||
### Enum Types
|
||||
|
||||
| Enum | Values |
|
||||
|---|---|
|
||||
| `billing_tier` | `free`, `pro`, `power`, `team` |
|
||||
| `plugin_status` | `pending_review`, `approved`, `rejected` |
|
||||
| `review_decision` | `approved`, `rejected` |
|
||||
|
||||
### Migrations
|
||||
|
||||
| Version | Description |
|
||||
|---|---|
|
||||
| `001_initial_schema` | Creates all 9 tables with indexes and foreign key constraints |
|
||||
| `002_seed_plugins` | Seeds 3 approved plugins: GitHub Sync (free), Slack Notifier (€4.99), Time Tracker (€9.99) |
|
||||
| `001_initial_schema` | Creates core auth and billing tables with indexes and foreign key constraints |
|
||||
|
||||
---
|
||||
|
||||
@@ -439,7 +343,7 @@ The agent system uses a registry pattern with LangChain tool-calling agents powe
|
||||
|
||||
### Architecture
|
||||
|
||||
- **`BaseAgent`** — Abstract base with `user_id`, `shared_memory`, and `vector_store_context`.
|
||||
- **`BaseAgent`** — Abstract base with `user_id` and `shared_memory`.
|
||||
- **`ChatAgent(BaseAgent)`** — Abstract `handle(query, context)` and `get_tools()` methods, plus a shared `_tool_loop(llm, messages, tools, max_iter=5)` for iterative tool calling.
|
||||
- **`AgentRegistry`** — Singleton registry with `@register` decorator, `get(name)`, `list_agents()`, and `call_agent(name, query, context)`.
|
||||
|
||||
@@ -554,39 +458,6 @@ Source: `app/api/middleware/sanitizer.py`
|
||||
- Scans JSON response bodies and replaces leaked prompt IP fragments with `[REDACTED]`.
|
||||
- Detects: system prompt openers, agent routing metadata, LangChain tool schemas, internal reasoning markers (`<thinking>`, `[INST]`), and known prompt fingerprints.
|
||||
- Logs sanitization events as `WARNING`.
|
||||
- Binary responses (storage, backup) are never touched.
|
||||
|
||||
---
|
||||
|
||||
## Storage Layer
|
||||
|
||||
### Blob Store
|
||||
|
||||
Source: `app/storage/blob_store.py`
|
||||
|
||||
- S3-backed storage for E2E encrypted blobs.
|
||||
- Object keys follow the pattern: `{user_id}/{table}/{record_id}`
|
||||
- Server-side SSE-S3 encryption at rest (additional layer on top of client-side E2E encryption).
|
||||
- Methods: `upload()`, `download()`, `delete()` (idempotent), `list_keys()`
|
||||
- The backend **never inspects or decrypts blob content**.
|
||||
|
||||
### Vector Store
|
||||
|
||||
Source: `app/storage/vector_store.py`
|
||||
|
||||
- Runtime-configurable: **Pinecone** (when `PINECONE_API_KEY` is set) or **Qdrant** (fallback).
|
||||
- User isolation: Pinecone uses `namespace=user_id`; Qdrant filters by `user_id` payload field.
|
||||
- 32-dimensional SHA-256-derived float vectors (deterministic, not semantically meaningful on encrypted data — a documented trade-off for privacy).
|
||||
- Encrypted blobs are stored as base64 in metadata/payload for verbatim retrieval.
|
||||
- Methods: `upsert()`, `search()`, `delete()`
|
||||
|
||||
### Encryption Utilities
|
||||
|
||||
Source: `app/storage/encryption.py`
|
||||
|
||||
- `verify_checksum(blob, checksum)` — SHA-256 hash comparison using `hmac.compare_digest` (constant-time to prevent timing attacks).
|
||||
- `reject_if_tampered(blob, checksum)` — Raises HTTP 400 on checksum mismatch.
|
||||
- **No decryption key ever reaches the backend.**
|
||||
|
||||
---
|
||||
|
||||
@@ -600,11 +471,8 @@ Source: `app/billing/stripe_service.py`, `app/billing/tier_manager.py`
|
||||
|---|---|---|---|---|
|
||||
| AI Agents | 3 | Unlimited | Unlimited | Unlimited |
|
||||
| Batch Active | 2 | 10 | Unlimited | Unlimited |
|
||||
| Cloud Storage | 0 GB | 5 GB | 25 GB | Unlimited |
|
||||
| Backup Storage | 0 GB | 5 GB | 25 GB | Unlimited |
|
||||
| LLM Providers | 1 | Unlimited | Unlimited | Unlimited |
|
||||
| Batch Builder | — | — | ✓ | ✓ |
|
||||
| Plugin Marketplace | — | — | ✓ | ✓ |
|
||||
| SSO | — | — | — | ✓ |
|
||||
| Rate Limit | 20 req/min | 60 req/min | 120 req/min | 200 req/min |
|
||||
|
||||
@@ -620,47 +488,6 @@ Source: `app/billing/stripe_service.py`, `app/billing/tier_manager.py`
|
||||
- `get_tier(user_id)` — Returns the user's current billing tier.
|
||||
- `check_feature(tier, feature)` — Boolean feature gate check.
|
||||
- `require_feature(tier, feature)` — Raises HTTP 403 if the feature is not available.
|
||||
- `enforce_quota(user_id, tier)` / `enforce_backup_quota(user_id, tier)` — Raises HTTP 402 if storage limits are exceeded.
|
||||
|
||||
---
|
||||
|
||||
## Plugin Marketplace
|
||||
|
||||
Source: `app/marketplace/`
|
||||
|
||||
### Plugin Registry
|
||||
|
||||
- PostgreSQL-backed catalog of submitted and approved plugins.
|
||||
- `list_plugins(db, category, query, page, sort)` — Paginated listing (page size: 20) with optional filtering by category, text search, and sorting by `rating`, `installs`, or `newest`.
|
||||
- `get_plugin(db, plugin_id)` — Full manifest with install count and ratings.
|
||||
- `submit_plugin(db, manifest, s3_key)` — Submits a plugin with `pending_review` status.
|
||||
- `approve_plugin()` / `reject_plugin(reason)` — Admin workflow for plugin approval.
|
||||
- `record_install()` / `record_uninstall()` — Tracks per-user installations and updates install counts.
|
||||
|
||||
### Review Queue
|
||||
|
||||
- Automated security checklist before human review:
|
||||
- Plugin ID must match `^[a-z0-9-]+$`
|
||||
- Permissions must be from the allowed set only
|
||||
- No binary blobs in the manifest
|
||||
- **Allowed permissions:** `read:tasks`, `write:tasks`, `read:projects`, `write:projects`, `read:notes`, `write:notes`, `read:timelines`, `write:timelines`, `read:calendar`, `write:calendar`
|
||||
- `get_pending(db)` — Lists plugins awaiting review.
|
||||
- `submit_review(db, plugin_id, reviewer_id, decision, notes)` — Records the review decision.
|
||||
|
||||
### Revenue Sharing
|
||||
|
||||
- **70% developer / 30% platform** split on all paid plugin sales.
|
||||
- `record_install(db, plugin_id, user_id, amount_cents)` — Records the revenue event and triggers a Stripe Connect transfer for the developer share.
|
||||
- `get_earnings(db, developer_id, period)` — Aggregated earnings report for plugin developers.
|
||||
- Gracefully stubs transfers when Stripe is not configured.
|
||||
|
||||
### Seed Plugins
|
||||
|
||||
| Plugin | Category | Price |
|
||||
|---|---|---|
|
||||
| GitHub Sync | Productivity | Free |
|
||||
| Slack Notifier | Communication | €4.99 |
|
||||
| Time Tracker | Productivity | €9.99 |
|
||||
|
||||
---
|
||||
|
||||
@@ -682,10 +509,8 @@ pytest -v
|
||||
### Test Infrastructure
|
||||
|
||||
- **Database:** Async SQLite in-memory via `aiosqlite` + `StaticPool` — fast, no PostgreSQL needed.
|
||||
- **S3 mock:** `moto[s3]` with a fixture that patches `BlobStore` settings.
|
||||
- **Auth helpers:** `make_jwt(tier)` and `auth_header(tier)` generate per-tier test tokens.
|
||||
- **Seed data:** Auto-creates one `User` + `Subscription` per tier (free/pro/power/team) before each test.
|
||||
- **Plugin seeds:** Fixture adds 3 approved plugins for marketplace tests.
|
||||
- **FK enforcement:** SQLite `PRAGMA foreign_keys=ON`.
|
||||
- **No external dependencies** — all tests run fully offline.
|
||||
|
||||
@@ -694,13 +519,6 @@ pytest -v
|
||||
| File | Coverage |
|
||||
|---|---|
|
||||
| `test_auth.py` | Register, login, token access, refresh, expiration |
|
||||
| `test_orchestrator.py` | Intent classification, single agent routing, pipeline, plan mode |
|
||||
| `test_agents.py` | Each agent with mocked LLM: registration, tools, handle method |
|
||||
| `test_storage.py` | Create, list, download, update, delete records; checksum rejection; quota enforcement |
|
||||
| `test_backup.py` | Upload, download, history, delete; tier-based storage limits |
|
||||
| `test_plugins.py` | List, install, uninstall, revenue events, tier gate enforcement |
|
||||
| `test_agent_registry.py` | Registry singleton, registration, lookup, listing |
|
||||
| `test_execution_plan.py` | Plan builder, template registry, plan cache |
|
||||
| `test_middleware.py` | Rate limiting by tier, sanitizer prompt leak detection |
|
||||
|
||||
---
|
||||
@@ -710,7 +528,6 @@ pytest -v
|
||||
```
|
||||
adiuva-api/
|
||||
├── alembic.ini # Alembic configuration
|
||||
├── BACKEND_PLAN.md # Architecture & design decisions
|
||||
├── docker-compose.yml # Docker Compose (app + PostgreSQL)
|
||||
├── Dockerfile # Multi-stage production build
|
||||
├── requirements.txt # Python dependencies
|
||||
@@ -719,13 +536,12 @@ adiuva-api/
|
||||
│ ├── env.py # Alembic environment config
|
||||
│ ├── script.py.mako # Migration template
|
||||
│ └── versions/
|
||||
│ ├── 001_initial_schema.py # Tables, indexes, FKs
|
||||
│ └── 002_seed_plugins.py # Seed marketplace plugins
|
||||
│ └── 001_initial_schema.py # Tables, indexes, FKs
|
||||
│
|
||||
├── app/ # Application source
|
||||
│ ├── main.py # FastAPI app factory, middleware, routes
|
||||
│ ├── db.py # Async SQLAlchemy engine & session
|
||||
│ ├── models.py # SQLAlchemy ORM models (9 tables)
|
||||
│ ├── models.py # SQLAlchemy ORM models
|
||||
│ ├── schemas.py # Pydantic request/response schemas
|
||||
│ │
|
||||
│ ├── config/
|
||||
@@ -734,53 +550,35 @@ adiuva-api/
|
||||
│ ├── agents/ # LLM-powered domain agents
|
||||
│ │ ├── task_agent.py # Task & comment CRUD (8 tools)
|
||||
│ │ ├── project_agent.py # Project lifecycle (6 tools)
|
||||
│ │ ├── timeline_agent.py # Milestones (4 tools)
|
||||
│ │ ├── timeline_agent.py # Milestones (4 tools)
|
||||
│ │ └── note_agent.py # Markdown notes (5 tools)
|
||||
│ │
|
||||
│ ├── core/ # Orchestration engine
|
||||
│ │ ├── agent_registry.py # BaseAgent, ChatAgent, AgentRegistry
|
||||
│ │ ├── llm.py # LiteLLM factory (get_llm, get_router_llm)
|
||||
│ │ ├── orchestrator.py # Intent classification & routing
|
||||
│ │ └── execution_plan.py # Plan builder, templates, cache
|
||||
│ │ └── deep_agent.py # Deep agent orchestration
|
||||
│ │
|
||||
│ ├── api/ # HTTP layer
|
||||
│ │ ├── deps.py # Shared FastAPI dependencies
|
||||
│ │ ├── middleware/
|
||||
│ │ │ ├── auth.py # JWT validation, live tier lookup
|
||||
│ │ │ ├── rate_limit.py # Sliding-window tier rate limiter
|
||||
│ │ │ └── sanitizer.py # Prompt IP leak protection
|
||||
│ │ └── routes/
|
||||
│ │ ├── auth.py # Register, login, refresh, me
|
||||
│ │ ├── chat.py # Chat + WebSocket streaming
|
||||
│ │ ├── plans.py # Execution plan playbooks
|
||||
│ │ ├── storage.py # E2E encrypted record CRUD
|
||||
│ │ ├── vectors.py # Vector upsert, search, delete
|
||||
│ │ ├── backup.py # Encrypted backup management
|
||||
│ │ ├── plugins.py # Marketplace browse & install
|
||||
│ │ └── billing.py # Stripe checkout & webhooks
|
||||
│ │ ├── chat.py # Chat + embed endpoint
|
||||
│ │ ├── billing.py # Stripe checkout, webhooks, subscription
|
||||
│ │ ├── agents.py # Agent catalog, config, runs
|
||||
│ │ └── device_ws.py # Persistent device WebSocket
|
||||
│ │
|
||||
│ ├── storage/ # Storage backends
|
||||
│ │ ├── blob_store.py # S3 blob storage
|
||||
│ │ ├── vector_store.py # Pinecone / Qdrant vector store
|
||||
│ │ └── encryption.py # Checksum verification utilities
|
||||
│ │
|
||||
│ ├── billing/ # Subscription management
|
||||
│ │ ├── stripe_service.py # Stripe API integration
|
||||
│ │ └── tier_manager.py # Feature matrix & quota enforcement
|
||||
│ │
|
||||
│ └── marketplace/ # Plugin ecosystem
|
||||
│ ├── plugin_registry.py # Catalog CRUD & search
|
||||
│ ├── plugin_review.py # Security checklist & review queue
|
||||
│ └── revenue_share.py # 70/30 split & Stripe Connect
|
||||
│ └── billing/
|
||||
│ ├── stripe_service.py # Stripe API wrapper
|
||||
│ └── tier_manager.py # Feature matrix, rate limits
|
||||
│
|
||||
└── tests/ # Test suite
|
||||
├── conftest.py # Fixtures: DB, S3, auth, seeds
|
||||
├── conftest.py # Fixtures: DB, auth, seeds
|
||||
├── test_auth.py
|
||||
├── test_orchestrator.py
|
||||
├── test_agents.py
|
||||
├── test_storage.py
|
||||
├── test_backup.py
|
||||
├── test_plugins.py
|
||||
├── test_agent_registry.py
|
||||
├── test_execution_plan.py
|
||||
└── test_middleware.py
|
||||
|
||||
Reference in New Issue
Block a user