Commit 522140

2026-03-14 18:25:28 Claude (MCP): [mcp] Add E-4 (Lambda Library Mode), update E-1/E-2 status
Tasks/Emergent.md ..
@@ 1,7 1,7 @@
---
category: reference
tags: [tasks, emergent]
- last_updated: 2026-03-13
+ last_updated: 2026-03-14
confidence: high
---
@@ 27,6 27,8 @@
The actual cold start time is likely dominated by Python package initialization: loading dulwich, Flask/Otterwiki, Mangum, aws-xray-sdk, and all transitive dependencies from a 39MB deployment package. Possibly also EFS mount negotiation (NFS/TLS handshake to mount target). Without accurate instrumentation, any cold start mitigation strategy (provisioned concurrency, architecture changes, package optimization) is a guess.
+ **Status:** COMPLETE — see [[Dev/E-1_Cold_Start_Benchmarks]] for results. VPC overhead confirmed negligible (~80-90ms). `import otterwiki.server` is 79% of cold start (~3.5s). Architectural mitigation designed in [[Design/Lambda_Library_Mode]].
+
**Task:**
Re-run cold start benchmarks with fine-grained tracing of the INIT phase. Break down time spent in:
@@ 44,10 46,10 @@
- Recommendation: package optimization, lazy imports, memory tuning, or architectural change
**Acceptance criteria:**
- - [ ] INIT phase broken into at least 4 measured segments
- - [ ] Each segment's contribution to total cold start quantified (ms and %)
- - [ ] Control Lambda (minimal VPC, no EFS) measured for baseline comparison
- - [ ] Benchmark page updated with corrected attribution
+ - [x] INIT phase broken into at least 4 measured segments
+ - [x] Each segment's contribution to total cold start quantified (ms and %)
+ - [x] Control Lambda (minimal VPC, no EFS) measured for baseline comparison
+ - [x] Benchmark page updated with corrected attribution
---
@@ 60,6 62,8 @@
**Context:**
Wiki pages are written infrequently (during Claude sessions via MCP) and read much more often (browsing, reference). CloudFront is already in the architecture for static SPA hosting but is not used to cache wiki page content. Adding a caching layer for page reads would reduce most page loads from ~270ms (warm Lambda) to ~10–50ms (edge serve), and reduce origin load.
+ **Status:** Design complete — see [[Design/CDN_Read_Path]]. Option A (Thin Assembly Lambda) recommended. Implementation queued as [[Tasks/E-2_CDN_Read_Path]].
+
**Design decisions needed:**
**Cache freshness strategy:** Short TTL (30–60s) on page HTML with `Cache-Control` headers from the origin. No invalidation API calls under normal operation — pages self-expire. Static assets (CSS, JS, fonts) use content-hashed filenames with long TTLs (1 year). Invalidation reserved for exceptional cases (page deletion, privacy). This avoids the invalidation cost problem: at scale (e.g. 1,000 active wikis × 5 writes/day), path-based invalidation would exceed the 1,000/month free tier and cost ~$220/month, growing linearly with write volume.
@@ 113,3 117,42 @@
- [ ] At least three approaches evaluated with pros/cons
- [ ] UX impact documented for web UI, MCP, git clone, and search
- [ ] Recommendation with rationale
+
+ ---
+
+ ### E-4: Lambda Library Mode Implementation
+
+ **Priority:** Medium — reduces write-path cold start from ~4.5s to ~2.6s
+ **Discovered during:** Cold start deep dive (2026-03-14)
+ **Depends on:** [[Dev/E-1_Cold_Start_Benchmarks]] (complete), [[Design/Lambda_Library_Mode]]
+ **Relates to:** [[Design/CDN_Read_Path]], [[Design/Platform_Overview]]
+
+ **Context:**
+ The E-1 benchmarks identified `import otterwiki.server` as 79% of cold start (~3.5s). The Lambda Library Mode design ([[Design/Lambda_Library_Mode]]) proposes replacing `otterwiki.server` with a lazy-loading drop-in via `sys.modules` injection, plus upstream contributions to defer heavy imports in views.py and wiki.py.
+
+ **Task:**
+
+ **Fork work (lambda_server.py):**
+ 1. Write `lambda_server.py` that exports `app`, `db`, `storage` (LocalProxy), `app_renderer` (LocalProxy), `mail` (LocalProxy), `githttpserver` (LocalProxy), model re-exports, template filters, Jinja globals
+ 2. Inject via `sys.modules['otterwiki.server']` before any otterwiki imports in lambda_init.py
+ 3. Add `@app.before_request` hook for deferred init (db.create_all, config, plugins, multi-tenant middleware)
+ 4. Verify all existing E2E tests pass against the replacement module
+
+ **Upstream PRs:**
+ 5. Lazy imports in `views.py` — move heavy imports into route handler function bodies
+ 6. Lazy imports in `wiki.py` — PIL.Image, feedgen to function level
+ 7. Extract plugin entrypoint scan to explicit `init_plugins()` function
+ 8. Remove duplicate `render = OtterwikiRenderer()` in renderer.py:632
+
+ **Deliverables:**
+ - `lambda_server.py` in wikibot-io repo
+ - Updated `lambda_init.py` using injection pattern
+ - 4 upstream PRs to otterwiki (items 5-8)
+ - Updated cold start benchmarks showing improvement
+
+ **Acceptance criteria:**
+ - [ ] Lambda INIT < 1,600ms without upstream PRs
+ - [ ] Lambda INIT < 800ms with upstream PRs accepted
+ - [ ] All existing E2E tests pass
+ - [ ] First-request latency < 2,000ms (deferred init)
+ - [ ] Warm request latency unchanged
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9