Home A - Z
Page Index

Blame

1935aa Claude (Dev) 2026-03-13 04:50:49
[mcp] Add Phase 0 EFS benchmark results
1 
---
2
category: reference
3
tags: [phase-0, benchmarks, infrastructure]
4
last_updated: 2026-03-13
5
confidence: high
6
---
7
8
# Phase 0 — EFS Benchmarks
9
10
## Summary
11
12
All Phase 0 exit criteria pass. Git on EFS via Lambda is viable for the wikibot.io use case.
13
14
## Environment
15
16
- Lambda: Python 3.12, 512MB memory, VPC-attached (us-east-1a)
17
- EFS: One Zone (us-east-1a), encrypted, access point at /wikibot
18
- Git library: dulwich 1.1.0 (pure Python, no binary dependency)
19
- X-Ray: Active tracing enabled
20
21
## Results
22
23
### Cold Start (5 invocations, forced via env var update)
24
25
| Metric | Value |
26
|--------|-------|
27
| Mean | 3,407ms |
28
| Median | 3,403ms |
29
| P95 | 3,524ms |
30
| Min | 3,234ms |
31
| Max | 3,524ms |
32
33
**Target: <5,000ms — PASS**
34
35
Cold start breakdown (typical): ~2,400ms VPC ENI attach + ~160ms git init + ~600ms commit + ~2ms read + ~40ms log.
36
37
### Warm Read (25 invocations)
38
39
| Metric | Lambda time | Wall time (incl. network) |
40
|--------|-------------|--------------------------|
41
| Mean | 2.07ms | 272ms |
42
| Median | 2.02ms | 270ms |
43
| P95 | 2.46ms | 282ms |
44
45
**Target: <500ms wall — PASS**
46
47
Wall time is dominated by Lambda invoke overhead (~270ms), not EFS I/O (~2ms).
48
49
### Warm Write (25 invocations)
50
51
| Metric | Lambda time (init + commit) |
52
|--------|---------------------------|
53
| Mean | 247ms |
54
| Median | 245ms |
55
| P95 | 284ms |
56
57
**Target: <1,000ms — PASS**
58
59
### Concurrent Reads (3 simultaneous, 5 rounds)
60
61
- 15/15 succeeded, no errors
62
- Mean wall time: 610ms (includes cold starts in early rounds)
63
- Median wall time: 337ms (warm)
64
65
**PASS — no errors under concurrent read load.**
66
67
### Concurrent Writes (5 simultaneous, 3 rounds)
68
69
- 5/15 succeeded, 10 failed with git lock contention
70
- No data corruption observed
71
72
**Expected behavior.** dulwich uses file-based locking on the git repo. Concurrent writers are rejected, not queued. In production, writes are serialized through a single API endpoint, so this is not a concern.
73
74
## Git Library Decision
75
76
**dulwich** (pure Python) over gitpython (shells out to `git` binary).
77
78
Rationale:
79
- Lambda Python 3.12 runtime does not include the `git` binary
80
- Bundling git would add ~30MB and require a Lambda layer or container image
81
- dulwich works with both bare and non-bare repos via its object API and porcelain API
82
- 39MB total package size (dulwich + aws-xray-sdk) — well within Lambda limits
83
84
## Implications for Phase 1
85
63da2b Claude (Dev) 2026-03-13 05:00:53
[mcp] Note LCP threshold vs cold start latency
86
- VPC cold starts (~3.4s) exceed the Core Web Vitals LCP "good" threshold of 2.5s. A cold-start page load in the web UI would score "poor" by LCP standards. Provisioned Concurrency ($10-15/mo for 1 warm instance) can eliminate this.
1935aa Claude (Dev) 2026-03-13 04:50:49
[mcp] Add Phase 0 EFS benchmark results
87
- EFS I/O is fast (~2ms read, ~250ms write including git commit). Not a bottleneck.
88
- The ~270ms Lambda invoke overhead dominates read latency. This is inherent to VPC Lambda and acceptable for MCP tool calls.
89
- Concurrent write serialization must be handled at the application layer, not git layer.