Blame
|
1 | This page is part of the **wikibot.io PRD** (Product Requirements Document). See also: [[Design/Data_Model]], [[Design/Auth]], [[Design/Implementation_Phases]], [[Design/Operations]]. |
||||||
|
2 | |||||||
| 3 | --- |
|||||||
| 4 | ||||||||
| 5 | # PRD/TRD: Serverless Wiki-as-a-Service Platform |
|||||||
| 6 | ||||||||
|
7 | > **Superseded.** This page describes the AWS serverless architecture for wikibot.io. The project has pivoted to a VPS deployment as robot.wtf with ATProto auth. See [[Design/VPS_Architecture]] for the current plan. This page is preserved as an archive — the application logic (ACL model, middleware, MCP tools) carries forward; the AWS infrastructure design does not. |
||||||
| 8 | ||||||||
|
9 | ## Overview |
||||||
| 10 | ||||||||
| 11 | A multi-tenant, serverless platform that lets users create and manage private research wikis — each backed by its own Git repo and exposed via its own MCP endpoint. Built on AWS (Lambda + EFS + DynamoDB + API Gateway) with near-zero cost at rest. Based on the Otterwiki + semantic search + MCP stack already built for the Third Gulf War project. Product name: **wikibot.io**. |
|||||||
| 12 | ||||||||
| 13 | The platform is a freemium service: |
|||||||
| 14 | ||||||||
|
15 | - **Free tier**: 1 wiki, 1 author, full read/write/MCP access, semantic search included |
||||||
| 16 | - **Premium tier**: Multiple wikis (up to 12), multiple authors (up to 25 per wiki), external Git sync |
|||||||
|
17 | |||||||
| 18 | --- |
|||||||
| 19 | ||||||||
| 20 | ## Context |
|||||||
| 21 | ||||||||
| 22 | ### Why this exists |
|||||||
| 23 | ||||||||
| 24 | Setting up a research wiki with MCP integration currently requires: forking Otterwiki, writing API and semantic search plugins, deploying an MCP server, configuring reverse proxy + TLS + auth, and wiring it all together. This took weeks of work. The resulting system is useful enough that it should be productized — but the per-instance infrastructure cost and setup complexity make it impractical to self-host for casual use. |
|||||||
| 25 | ||||||||
| 26 | A serverless architecture solves both problems: zero cost when idle (no VPS bill for abandoned wikis), and the setup complexity is absorbed by the platform. |
|||||||
| 27 | ||||||||
| 28 | ### Key users |
|||||||
| 29 | ||||||||
| 30 | - **Researcher** — creates wikis, writes/reads notes via MCP from Claude.ai or Claude Code, browses via web UI |
|||||||
| 31 | - **Admin** (initially: just us) — manages user accounts, monitors usage, handles billing tier changes |
|||||||
| 32 | ||||||||
| 33 | ### Prior art |
|||||||
| 34 | ||||||||
| 35 | The existing single-tenant system (documented in `otterwiki-research-wiki-prd.md`) proves the core workflow. This PRD describes the multi-tenant wrapper around it. |
|||||||
| 36 | ||||||||
| 37 | --- |
|||||||
| 38 | ||||||||
| 39 | ## Architecture |
|||||||
| 40 | ||||||||
| 41 | AWS Lambda + EFS + DynamoDB + API Gateway + CloudFront. EFS eliminates the git-on-S3 clone/push cycle — git repos live on a persistent NFS filesystem that Lambda mounts directly. |
|||||||
| 42 | ||||||||
| 43 | ``` |
|||||||
| 44 | ┌──────────────────────────────────────────────────────────────────────┐ |
|||||||
| 45 | │ Clients │ |
|||||||
| 46 | │ │ |
|||||||
| 47 | │ Claude.ai ──── MCP (Streamable HTTP) ──────┐ │ |
|||||||
| 48 | │ Claude Code ── MCP (Streamable HTTP) ──────┤ │ |
|||||||
| 49 | │ Any MCP client ─ MCP (Streamable HTTP) ────┤ │ |
|||||||
| 50 | │ Browser ────── HTTPS ──────────────────────┤ │ |
|||||||
| 51 | │ Git CLI ────── HTTPS (smart HTTP) ─────────┤ │ |
|||||||
| 52 | │ CLI ────────── HTTPS ──────────────────────┤ │ |
|||||||
| 53 | └─────────────────────────────────────────────┼────────────────────────┘ |
|||||||
| 54 | │ |
|||||||
| 55 | ┌─────────────────────────────────────────────▼────────────────────────┐ |
|||||||
| 56 | │ Edge / Routing │ |
|||||||
| 57 | │ │ |
|||||||
| 58 | │ {user}.wikibot.io/{wiki}/mcp → Compute (MCP handler) │ |
|||||||
| 59 | │ {user}.wikibot.io/{wiki}/api/* → Compute (REST API handler) │ |
|||||||
| 60 | │ {user}.wikibot.io/{wiki}/ → Compute (Otterwiki Flask) │ |
|||||||
| 61 | │ {user}.wikibot.io/{wiki}.git/* → Compute (Git smart HTTP) │ |
|||||||
| 62 | │ wikibot.io/auth/* → Compute (OAuth flows) │ |
|||||||
| 63 | │ wikibot.io/admin/* → Compute (management API) │ |
|||||||
| 64 | │ wikibot.io/ → Static (SPA) │ |
|||||||
| 65 | │ │ |
|||||||
| 66 | │ OAuth JWT validation on all routes except /auth/* and public wikis │ |
|||||||
| 67 | └─────────────────────────────────────────────┬────────────────────────┘ |
|||||||
| 68 | │ |
|||||||
| 69 | ┌─────────────────────────────────────────────▼────────────────────────┐ |
|||||||
| 70 | │ Compute (scale-to-zero) │ |
|||||||
| 71 | │ │ |
|||||||
| 72 | │ 1. Resolve user + wiki from hostname + path │ |
|||||||
| 73 | │ 2. Check ACL in DynamoDB │ |
|||||||
| 74 | │ 3. Access git repo on EFS (/mnt/efs/{user}/{wiki}/repo.git) │ |
|||||||
| 75 | │ 4. Route to appropriate handler: │ |
|||||||
| 76 | │ - MCP: FastMCP Streamable HTTP handler │ |
|||||||
| 77 | │ - API: REST endpoints (same as existing otterwiki-api) │ |
|||||||
| 78 | │ - Web: Otterwiki Flask app │ |
|||||||
| 79 | │ - Git: Smart HTTP protocol (dulwich) │ |
|||||||
|
80 | │ 5. On write: persist repo, write reindex record to DynamoDB │ |
||||||
| 81 | └──────────┬──────────────┬────────────────────────────────────────────┘ |
|||||||
| 82 | │ │ |
|||||||
| 83 | ┌─────▼─────┐ ┌─────▼──────┐ |
|||||||
| 84 | │ EFS │ │ DynamoDB │ |
|||||||
| 85 | │ │ │ │ |
|||||||
| 86 | │ Git repos│ │ Users │───── DynamoDB Streams |
|||||||
| 87 | │ FAISS │ │ Wikis │ │ |
|||||||
| 88 | │ indexes │ │ ACLs │ ▼ |
|||||||
| 89 | └───────────┘ │ Reindex Q │ Embedding Lambda (VPC+EFS) |
|||||||
| 90 | └────────────┘ MiniLM (local, no Bedrock) |
|||||||
|
91 | ``` |
||||||
| 92 | ||||||||
| 93 | ### Component inventory |
|||||||
| 94 | ||||||||
| 95 | | Component | Purpose | AWS Service | |
|||||||
| 96 | |-----------|---------|-------------| |
|||||||
| 97 | | Routing / TLS | Request routing, cert management | API Gateway + CloudFront + ACM | |
|||||||
| 98 | | Edge security (post-launch) | Rate limiting, bot control, OWASP rules | WAF (on API Gateway + CloudFront) | |
|||||||
| 99 | | Auth | OAuth (Google, GitHub, Microsoft, Apple) | WorkOS AuthKit (recommended) | |
|||||||
| 100 | | Compute | Application logic, scale-to-zero | Lambda (VPC, Mangum adapter) | |
|||||||
| 101 | | Git storage | Bare git repos | EFS (mounted by Lambda) | |
|||||||
|
102 | | Vector indexes | FAISS files | EFS | |
||||||
| 103 | | Metadata | Users, wikis, ACLs, reindex queue | DynamoDB | |
|||||||
|
104 | | Secrets | API keys, auth provider secrets, Git creds | Env vars (dev), Secrets Manager (prod) | |
||||||
|
105 | | Embeddings | Semantic search | all-MiniLM-L6-v2 (runs locally on Lambda) | |
||||||
|
106 | | Frontend | Static SPA | S3 + CloudFront | |
||||||
|
107 | | Async jobs | Reindex on write | DynamoDB Streams → Lambda | |
||||||
|
108 | | Scheduled tasks | Git sync, cleanup jobs | EventBridge Scheduler | |
||||||
| 109 | | Tracing | Request latency, per-service breakdown | X-Ray | |
|||||||
| 110 | ||||||||
| 111 | For a user with one 200-page wiki that's idle: effectively $0/month (a few KB of DynamoDB, a few MB on EFS Infrequent Access at $0.016/GB/month). |
|||||||