Blame
|
1 | --- |
||||||
|
2 | ## How to read this document |
||||||
| 3 | ||||||||
| 4 | - **Dependencies** list task IDs that must be complete before this task starts |
|||||||
| 5 | - **Parallel group** identifies tasks that can run simultaneously within a phase |
|||||||
| 6 | - **Target** identifies which repo and branch the work goes into |
|||||||
| 7 | - Tasks are numbered `P{phase}-{sequence}` (e.g., P0-3) |
|||||||
| 8 | - Acceptance criteria are binary — pass or fail, no judgment calls |
|||||||
| 9 | ||||||||
| 10 | --- |
|||||||
| 11 | ||||||||
| 12 | ## Phase 0: Proof of Concept |
|||||||
| 13 | ||||||||
| 14 | **Goal:** Validate the two core technical risks: git on EFS via Lambda, and MCP OAuth via WorkOS. |
|||||||
| 15 | ||||||||
| 16 | **Two independent tracks** that can run with separate managers: |
|||||||
| 17 | - Track A: EFS + Lambda (P0-1 → P0-5) |
|||||||
| 18 | - Track B: WorkOS + MCP auth (P0-6 → P0-8) |
|||||||
| 19 | ||||||||
| 20 | ### P0-1: Pulumi Scaffold |
|||||||
| 21 | ||||||||
| 22 | **Parallel group:** Track A start |
|||||||
| 23 | **Dependencies:** None |
|||||||
| 24 | **Target:** `wikibot-io` repo, `feat/P0-1-pulumi-scaffold` |
|||||||
| 25 | ||||||||
| 26 | **Description:** |
|||||||
| 27 | Create the Pulumi project with foundational AWS infrastructure: VPC with 1 public and 1 private subnet (single AZ for dev), security groups, route tables, and gateway endpoints for DynamoDB and S3. |
|||||||
| 28 | ||||||||
| 29 | **Deliverables:** |
|||||||
| 30 | - `infra/__main__.py` with top-level composition |
|||||||
| 31 | - `infra/components/vpc.py` — VPC, subnets, security groups, route tables, gateway endpoints |
|||||||
| 32 | - Unit tests verifying resource creation and security group rules |
|||||||
| 33 | - `pulumi up` succeeds against the dev stack |
|||||||
| 34 | ||||||||
| 35 | **Acceptance criteria:** |
|||||||
| 36 | - [ ] VPC created with correct CIDR, 1 public + 1 private subnet |
|||||||
| 37 | - [ ] Gateway endpoints for DynamoDB and S3 attached to private subnet route table |
|||||||
| 38 | - [ ] Security group allows Lambda → EFS (NFS port 2049) and Lambda → internet (egress) |
|||||||
| 39 | - [ ] All resources tagged with `project: wikibot-io`, `environment: dev` |
|||||||
| 40 | - [ ] Unit tests pass with `pulumi.runtime.set_mocks()` |
|||||||
| 41 | - [ ] `pulumi up` succeeds |
|||||||
| 42 | ||||||||
| 43 | --- |
|||||||
| 44 | ||||||||
| 45 | ### P0-2: EFS + Lambda Basic |
|||||||
| 46 | ||||||||
| 47 | **Parallel group:** Track A |
|||||||
| 48 | **Dependencies:** P0-1 |
|||||||
| 49 | **Target:** `wikibot-io` repo, `feat/P0-2-efs-lambda` |
|||||||
| 50 | ||||||||
| 51 | **Description:** |
|||||||
| 52 | Add EFS filesystem with mount target in the private subnet. Create a Lambda function (Python 3.12, VPC-attached) that mounts EFS and performs basic file read/write. This validates the core infrastructure pattern. |
|||||||
| 53 | ||||||||
| 54 | **Deliverables:** |
|||||||
| 55 | - `infra/components/efs.py` — EFS filesystem, mount target, access point |
|||||||
| 56 | - `infra/components/lambda_functions.py` — Lambda function with VPC config and EFS mount |
|||||||
| 57 | - `app/poc/efs_test.py` — Lambda handler that writes a file, reads it back, returns timing |
|||||||
| 58 | - IAM role with EFS and VPC permissions |
|||||||
| 59 | - Integration test that invokes the Lambda and verifies file persistence |
|||||||
| 60 | ||||||||
| 61 | **Acceptance criteria:** |
|||||||
| 62 | - [ ] Lambda can write a file to EFS at `/mnt/efs/test.txt` |
|||||||
| 63 | - [ ] Lambda can read the file back and content matches |
|||||||
| 64 | - [ ] File persists across Lambda invocations (cold and warm) |
|||||||
| 65 | - [ ] Integration test passes |
|||||||
| 66 | - [ ] Lambda execution time logged |
|||||||
| 67 | ||||||||
| 68 | --- |
|||||||
| 69 | ||||||||
| 70 | ### P0-3: Git on EFS |
|||||||
| 71 | ||||||||
| 72 | **Parallel group:** Track A |
|||||||
| 73 | **Dependencies:** P0-2 |
|||||||
| 74 | **Target:** `wikibot-io` repo, `feat/P0-3-git-efs` |
|||||||
| 75 | ||||||||
| 76 | **Description:** |
|||||||
| 77 | Lambda function that initializes a bare git repo on EFS, commits a markdown file, reads it back, and lists commits. Validates that git operations work correctly on NFS-mounted storage. |
|||||||
| 78 | ||||||||
| 79 | Investigate git library choice: gitpython (shells out to `git` binary — verify availability in Lambda runtime) vs. dulwich (pure Python, no binary dependency). If `git` is not available in the Lambda runtime, use dulwich. Document the decision. |
|||||||
| 80 | ||||||||
| 81 | **Deliverables:** |
|||||||
| 82 | - `app/poc/git_test.py` — Lambda handler: init bare repo, commit file, read file, list history |
|||||||
| 83 | - Decision documented: gitpython vs. dulwich, with rationale |
|||||||
| 84 | - Integration test that exercises the full git lifecycle |
|||||||
| 85 | ||||||||
| 86 | **Acceptance criteria:** |
|||||||
| 87 | - [ ] Bare git repo created on EFS at a specified path |
|||||||
| 88 | - [ ] Markdown file committed with author, message, and timestamp |
|||||||
| 89 | - [ ] File content readable from the repo |
|||||||
| 90 | - [ ] Commit history retrievable |
|||||||
| 91 | - [ ] Repo persists across Lambda invocations |
|||||||
| 92 | - [ ] Concurrent read test: 3 simultaneous Lambda invocations reading the same repo |
|||||||
| 93 | - [ ] Git library decision documented with rationale |
|||||||
| 94 | ||||||||
| 95 | --- |
|||||||
| 96 | ||||||||
| 97 | ### P0-4: X-Ray Tracing |
|||||||
| 98 | ||||||||
| 99 | **Parallel group:** Track A (can run parallel with P0-3) |
|||||||
| 100 | **Dependencies:** P0-2 |
|||||||
| 101 | **Target:** `wikibot-io` repo, `feat/P0-4-xray` |
|||||||
| 102 | ||||||||
| 103 | **Description:** |
|||||||
| 104 | Enable AWS X-Ray tracing on Lambda and API Gateway. Add custom subsegments for git operations (init, commit, read) so that Phase 0 benchmarks can break down latency by operation. |
|||||||
| 105 | ||||||||
| 106 | **Deliverables:** |
|||||||
| 107 | - X-Ray tracing enabled on Lambda function(s) via Pulumi |
|||||||
| 108 | - Custom subsegment instrumentation in git test Lambda |
|||||||
| 109 | - API Gateway stage with X-Ray enabled (if API Gateway exists at this point; otherwise, just Lambda tracing) |
|||||||
| 110 | ||||||||
| 111 | **Acceptance criteria:** |
|||||||
| 112 | - [ ] X-Ray traces visible in AWS console after Lambda invocation |
|||||||
| 113 | - [ ] Custom subsegments for git operations appear in trace timeline |
|||||||
| 114 | - [ ] Cold start vs. warm start distinguishable in traces |
|||||||
| 115 | ||||||||
| 116 | --- |
|||||||
| 117 | ||||||||
| 118 | ### P0-5: Performance Benchmarks |
|||||||
| 119 | ||||||||
| 120 | **Parallel group:** Track A (final) |
|||||||
| 121 | **Dependencies:** P0-3, P0-4 |
|||||||
| 122 | **Target:** `wikibot-io` repo, `feat/P0-5-benchmarks` |
|||||||
| 123 | ||||||||
| 124 | **Description:** |
|||||||
| 125 | Benchmark script that invokes the git Lambda repeatedly and collects timing data. Measures cold start latency, warm read latency, warm write latency, and concurrent access behavior. Results compared against Phase 0 exit criteria. |
|||||||
| 126 | ||||||||
| 127 | **Deliverables:** |
|||||||
| 128 | - `scripts/benchmark_efs.py` — invokes Lambda N times, collects X-Ray data or Lambda response times |
|||||||
| 129 | - Results written to `Dev/Phase 0 — EFS Benchmarks` wiki note |
|||||||
| 130 | - Concurrent write test: 5 simultaneous Lambda invocations writing to the same repo |
|||||||
| 131 | ||||||||
| 132 | **Acceptance criteria:** |
|||||||
| 133 | - [ ] Warm page read < 500ms (measured over 20+ invocations) |
|||||||
| 134 | - [ ] Warm page write < 1s (measured over 20+ invocations) |
|||||||
| 135 | - [ ] Cold start < 5s total (measured over 5+ cold starts) |
|||||||
| 136 | - [ ] Concurrent reads succeed without errors |
|||||||
| 137 | - [ ] Concurrent writes succeed (git locking handles serialization) |
|||||||
|
138 | - [ ] Results written to Dev/Phase 0 Summary per Agent Conventions documentation loop |
||||||
|
139 | |||||||
| 140 | --- |
|||||||
| 141 | ||||||||
| 142 | ### P0-6: WorkOS AuthKit Setup |
|||||||
| 143 | ||||||||
| 144 | **Parallel group:** Track B start (independent of Track A) |
|||||||
| 145 | **Dependencies:** None |
|||||||
| 146 | **Target:** `wikibot-io` repo, `feat/P0-6-workos-setup` |
|||||||
| 147 | ||||||||
| 148 | **Description:** |
|||||||
| 149 | Set up WorkOS AuthKit with Google OAuth provider. Configure the WorkOS dashboard, obtain API keys, and write a minimal test that authenticates a user and retrieves their profile including raw OAuth provider `sub` claim. |
|||||||
| 150 | ||||||||
| 151 | **Deliverables:** |
|||||||
| 152 | - WorkOS AuthKit configuration (documented, not in code — dashboard setup) |
|||||||
| 153 | - `app/poc/workos_test.py` — script that initiates OAuth flow and prints user profile |
|||||||
| 154 | - WorkOS API key stored in Pulumi config (`pulumi config set --secret workos_api_key`) |
|||||||
| 155 | - Documentation of provider sub retrieval (especially Apple, if testable) |
|||||||
| 156 | ||||||||
| 157 | **Acceptance criteria:** |
|||||||
| 158 | - [ ] Google OAuth login flow completes successfully |
|||||||
| 159 | - [ ] User profile retrieved with email, name, and raw Google `sub` claim |
|||||||
| 160 | - [ ] API key securely stored in Pulumi config |
|||||||
| 161 | - [ ] Apple provider sub retrieval status documented (verified or flagged as unavailable) |
|||||||
| 162 | ||||||||
| 163 | --- |
|||||||
| 164 | ||||||||
| 165 | ### P0-7: FastMCP + WorkOS on Lambda |
|||||||
| 166 | ||||||||
| 167 | **Parallel group:** Track B |
|||||||
| 168 | **Dependencies:** P0-2, P0-6 |
|||||||
| 169 | **Target:** `wikibot-io` repo, `feat/P0-7-mcp-workos-lambda` |
|||||||
| 170 | ||||||||
| 171 | **Description:** |
|||||||
| 172 | Deploy a minimal FastMCP server on Lambda with WorkOS OAuth 2.1 authentication. The MCP server exposes a single test tool (e.g., `echo`) that returns its input. Uses Streamable HTTP transport (not SSE). Validates that the FastMCP + WorkOS integration works on Lambda behind API Gateway. |
|||||||
| 173 | ||||||||
| 174 | Reference the existing `otterwiki-mcp` auth implementation for patterns (MultiAuth, InMemoryOAuthProvider, StaticTokenVerifier). |
|||||||
| 175 | ||||||||
| 176 | **Deliverables:** |
|||||||
| 177 | - `app/poc/mcp_server.py` — minimal FastMCP server with WorkOS auth and one test tool |
|||||||
| 178 | - Mangum adapter wrapping the MCP server for Lambda |
|||||||
| 179 | - API Gateway route for MCP endpoint |
|||||||
| 180 | - Unit tests for auth setup (mock WorkOS) |
|||||||
| 181 | - Integration test that calls the echo tool with a valid token |
|||||||
| 182 | ||||||||
| 183 | **Acceptance criteria:** |
|||||||
| 184 | - [ ] MCP server deploys to Lambda successfully |
|||||||
| 185 | - [ ] API Gateway routes to MCP endpoint |
|||||||
| 186 | - [ ] OAuth 2.1 flow completes (WorkOS issues token, MCP server validates) |
|||||||
| 187 | - [ ] Test tool callable with valid auth token |
|||||||
| 188 | - [ ] Invalid/missing tokens rejected |
|||||||
| 189 | - [ ] Bearer token auth works alongside OAuth (if MCP_AUTH_TOKEN configured) |
|||||||
| 190 | ||||||||
| 191 | --- |
|||||||
| 192 | ||||||||
| 193 | ### P0-8: Claude.ai MCP End-to-End |
|||||||
| 194 | ||||||||
| 195 | **Parallel group:** Track B (final) |
|||||||
| 196 | **Dependencies:** P0-7 |
|||||||
| 197 | **Target:** Manual testing (no code deliverable) |
|||||||
| 198 | ||||||||
| 199 | **Description:** |
|||||||
| 200 | Connect Claude.ai to the MCP endpoint deployed in P0-7. Verify that Claude.ai can authenticate via WorkOS OAuth and call the test tool. This is a manual test performed by the human or the manager. |
|||||||
| 201 | ||||||||
| 202 | **Acceptance criteria:** |
|||||||
| 203 | - [ ] Claude.ai MCP connection configured with the endpoint URL |
|||||||
| 204 | - [ ] OAuth flow completes in Claude.ai |
|||||||
| 205 | - [ ] Claude.ai can call the echo tool and receive a response |
|||||||
|
206 | - [ ] Results written to Dev/Phase 0 Summary per Agent Conventions documentation loop |
||||||
|
207 | |||||||
| 208 | --- |
|||||||
| 209 | ||||||||
| 210 | ### P0-9: Billing Alarm |
|||||||
| 211 | ||||||||
| 212 | **Parallel group:** Can run anytime after P0-1 |
|||||||
| 213 | **Dependencies:** P0-1 |
|||||||
| 214 | **Target:** `wikibot-io` repo, `feat/P0-9-billing-alarm` |
|||||||
| 215 | ||||||||
| 216 | **Description:** |
|||||||
| 217 | Set up AWS Budgets billing alarm with a $50/month threshold. Email notification to the project owner. |
|||||||
| 218 | ||||||||
| 219 | **Deliverables:** |
|||||||
| 220 | - Pulumi resource for AWS Budget with email alert |
|||||||
| 221 | - Alert threshold: $50/month |
|||||||
| 222 | ||||||||
| 223 | **Acceptance criteria:** |
|||||||
| 224 | - [ ] Budget alarm created |
|||||||
| 225 | - [ ] Email notification configured |
|||||||
| 226 | - [ ] `pulumi up` succeeds |
|||||||