Blame

757966 Claude (Dev) 2026-03-13 01:49:27
[mcp] Port original PRD API section to wiki
1
---
2
category: reference
3
tags: [meta, design, prd, api]
4
last_updated: 2026-03-12
5
confidence: high
6
---
7
8
# Original PRD API
9
10
> This page is part of the original single-tenant PRD, split across five wiki pages:
ada4de Claude (MCP) 2026-03-13 17:49:50
[mcp] Normalize spaces to underscores
11
> [[Design/Research_Wiki]] | [[Design/Rest Api]] | [[Design/Semantic_Search]] | [[Design/Mcp Server]] | [[Design/Note_Schema]]
757966 Claude (Dev) 2026-03-13 01:49:27
[mcp] Port original PRD API section to wiki
12
13
---
14
15
## Component 1: REST API Plugin
16
17
### Goal
18
19
Add a JSON REST API to Otterwiki so that pages can be created, read, updated, deleted, listed, and searched programmatically.
20
21
### Implementation approach
22
23
**First, investigate the plugin system.** Examine `otterwiki/plugins.py` and `docs/plugin_examples/` in the fork to determine whether plugins can register Flask blueprints (i.e., add new routes). The plugin system has hooks and was extended in v2.17.3.
24
25
- **If plugins support blueprint registration:** Build the API as an Otterwiki plugin. This is the preferred path — no core modifications, clean separation, potentially upstreamable.
26
- **If plugins do NOT support blueprint registration:** Add an `api.py` Flask blueprint directly to the Otterwiki codebase in the fork. Register it in `server.py`. This is a clean PR-able change.
27
28
### Authentication
29
30
Use a simple API key passed via `Authorization: Bearer <key>` header. The key is configured via environment variable `OTTERWIKI_API_KEY`. This is a single-user research system, not a multi-tenant service, so this is sufficient.
31
32
### Commit authorship and message conventions
33
34
All Git commits should clearly indicate their origin. This is important for reviewing history and understanding whether a change was made by the human, the AI, or a system process.
35
36
**Author identity for API commits:**
37
38
```
39
Author: Claude (MCP) <claude-mcp@otterwiki.local>
40
```
41
42
Configure this via environment variables `OTTERWIKI_API_AUTHOR_NAME` and `OTTERWIKI_API_AUTHOR_EMAIL`, defaulting to the above.
43
44
**Commit message format:**
45
46
```
47
[source] action: page name — optional detail
48
```
49
50
Where `source` is one of:
51
- `mcp` — changes made via the MCP server / API
52
- `web` — changes made via the Otterwiki web UI (Otterwiki handles this itself)
53
- `system` — changes made by automated processes (e.g., reindex, bulk import)
54
55
Examples:
56
```
57
[mcp] Create: Events/2026-03-09 Day 10 — initial event log
58
[mcp] Update: Trends/Iran Attrition Strategy — add Phase 2 radar blinding detail
59
[mcp] Delete: Events/Draft Note
60
[system] Bulk import: 15 pages from PDF migration
61
```
62
63
If the caller provides a `commit_message` in the API request body, use it as-is but prepend the `[mcp]` prefix. If no message is provided, generate one from the action and page name.
64
65
### Endpoints
66
67
All endpoints are prefixed with `/api/v1/`.
68
69
#### Pages
70
71
| Method | Endpoint | Description |
72
|--------|----------|-------------|
73
| `GET` | `/api/v1/pages` | List all pages. Optional query params: `?prefix=Actors/` (subdirectory), `?category=actor` (frontmatter category), `?tag=p2-interceptor-race` (frontmatter tag), `?updated_since=2026-03-08` (ISO date). Filters compose with AND logic. Returns array of `{name, path, category, tags, last_updated, content_length}`. |
74
| `GET` | `/api/v1/pages/<path:pagepath>` | Get a single page. Returns `{name, path, content, metadata, frontmatter, links_to, linked_from}`. Optional `?revision=<sha>` for historical versions. |
75
| `PUT` | `/api/v1/pages/<path:pagepath>` | Create or update a page. Body: `{content, commit_message}`. If `commit_message` is omitted, auto-generates per commit convention above. |
76
| `DELETE` | `/api/v1/pages/<path:pagepath>` | Delete a page. Body: `{commit_message}` (optional). |
77
| `GET` | `/api/v1/pages/<path:pagepath>/history` | Get revision history. Returns array of `{revision, author, date, message}`. Optional `?limit=N`. |
78
79
#### Search
80
81
| Method | Endpoint | Description |
82
|--------|----------|-------------|
83
| `GET` | `/api/v1/search?q=<query>` | Full-text search (uses Otterwiki's existing search). Returns array of `{name, path, snippet, score}`. |
84
85
#### Links (WikiLink graph)
86
87
| Method | Endpoint | Description |
88
|--------|----------|-------------|
89
| `GET` | `/api/v1/links/<path:pagepath>` | Get outgoing and incoming WikiLinks for a page. Returns `{links_to: [...], linked_from: [...]}`. |
90
| `GET` | `/api/v1/links` | Get the full link graph. Returns `{nodes: [...], edges: [...]}`. |
91
92
#### Changelog
93
94
| Method | Endpoint | Description |
95
|--------|----------|-------------|
96
| `GET` | `/api/v1/changelog` | Recent changes across all pages. Returns array of `{revision, author, date, message, pages_affected}`. Optional `?limit=N`. |
97
98
### WikiLink parsing and link graph
99
100
The API must parse `[[WikiLink]]` and `[[Display Text|WikiLink]]` syntax in page content to populate the `links_to` and `linked_from` fields.
101
102
**Parsing approach:**
103
104
The regex for extracting WikiLinks from markdown content:
105
106
```python
107
import re
108
WIKILINK_RE = re.compile(r'\[\[([^\]|]+?)(?:\|([^\]]+?))?\]\]')
109
110
def extract_wikilinks(content: str) -> list[str]:
111
"""Returns list of target page paths from WikiLinks in content."""
112
return [match.group(2) or match.group(1) for match in WIKILINK_RE.finditer(content)]
113
# For [[Display Text|Target]], returns "Target"
114
# For [[Target]], returns "Target"
115
```
116
117
Note: check whether Otterwiki uses `[[Target|Display Text]]` or `[[Display Text|Target]]` order — this varies between wiki engines. Otterwiki's syntax page shows `[[Text to display|WikiPage]]`, so the **target is the second element** when a pipe is present.
118
119
**Link index implementation:**
120
121
Maintain an in-memory reverse index (dict mapping page path → set of pages that link to it). This is built once on startup by scanning all pages, then updated incrementally:
122
123
- On page save: re-parse that page's WikiLinks, update the index for that page
124
- On page delete: remove that page from the index
125
126
The startup scan is O(N) where N is total pages. For a wiki of 200–500 pages with ~500 words each, this takes under a second. The index lives in the Flask process memory — no external storage needed.
127
128
Otterwiki v2.17.3 added a broken WikiLinks checker in housekeeping (`#388`). Look at that implementation first — it likely already has WikiLink parsing that can be reused or imported.
129
130
**The `GET /api/v1/links/<path>` endpoint** reads directly from this index. It does NOT scan the repo on every request.
131
132
**The `GET /api/v1/links` endpoint** (full graph) serializes the entire index. This could be expensive on a very large wiki, but for our expected size (< 500 pages) it's fine.
133
134
### Error responses
135
136
Standard HTTP status codes. JSON body: `{error: "description"}`.
137
138
- `401` — missing or invalid API key
139
- `404` — page not found
140
- `409` — conflict (e.g., concurrent edit)
141
- `422` — invalid content or parameters
142
143
### Example requests and responses
144
145
These examples are canonical — the implementation should match these JSON shapes exactly.
146
147
#### List pages: `GET /api/v1/pages?prefix=Trends/`
148
149
Response (note: NO content field — list operations return metadata only):
150
151
```json
152
{
153
"pages": [
154
{
155
"name": "Iran Attrition Strategy",
156
"path": "Trends/Iran Attrition Strategy",
157
"category": "trend",
158
"tags": ["military", "p2-interceptor-race", "p3-infrastructure"],
159
"last_updated": "2026-03-08",
160
"content_length": 487
161
},
162
{
163
"name": "Desalination Targeting Ratchet",
164
"path": "Trends/Desalination Targeting Ratchet",
165
"category": "trend",
166
"tags": ["infrastructure", "p3-infrastructure"],
167
"last_updated": "2026-03-08",
168
"content_length": 312
169
}
170
],
171
"total": 2
172
}
173
```
174
175
The `content_length` field is word count. This lets the caller decide whether to fetch the full page or skip large ones. The `category` and `tags` fields are extracted from YAML frontmatter; they are `null` if frontmatter is missing or malformed.
176
177
#### Read page: `GET /api/v1/pages/Trends/Iran Attrition Strategy`
178
179
Response (full content, parsed frontmatter, resolved links):
180
181
```json
182
{
183
"name": "Iran Attrition Strategy",
184
"path": "Trends/Iran Attrition Strategy",
185
"content": "---\ncategory: trend\ntags: [military, p2-interceptor-race, p3-infrastructure]\nlast_updated: 2026-03-08\nconfidence: high\n---\n\n# Iran Attrition Strategy\n\nIran is executing a multi-phase attrition campaign...",
186
"frontmatter": {
187
"category": "trend",
188
"tags": ["military", "p2-interceptor-race", "p3-infrastructure"],
189
"last_updated": "2026-03-08",
190
"confidence": "high"
191
},
192
"links_to": [
193
"Variables/Interceptor Stockpiles",
194
"Propositions/Iran Rationing Ballistic Missiles",
195
"Actors/Iran",
196
"Trends/Desalination Targeting Ratchet"
197
],
198
"linked_from": [
199
"Actors/Iran",
200
"Propositions/Iran Rationing Ballistic Missiles"
201
],
202
"revision": "a1b2c3d",
203
"last_commit": {
204
"revision": "a1b2c3d",
205
"author": "Claude (MCP)",
206
"date": "2026-03-08T14:22:00Z",
207
"message": "Update Iran Attrition Strategy — add Phase 2 radar blinding detail"
208
}
209
}
210
```
211
212
The `content` field is the **raw markdown file content including the frontmatter block**. The `frontmatter` field is the parsed YAML as a JSON object. If frontmatter is missing or invalid YAML, `frontmatter` is `null` and `content` still returns the raw file.
213
214
#### Write page: `PUT /api/v1/pages/Events/2026-03-09 Day 10`
215
216
Request body:
217
218
```json
219
{
220
"content": "---\ncategory: event\ntags: [military, day-10]\nlast_updated: 2026-03-09\nconfidence: high\n---\n\n# Day 10 — March 9, 2026\n\n## Key developments\n\n...",
221
"commit_message": "Create Day 10 event log"
222
}
223
```
224
225
Response:
226
227
```json
228
{
229
"name": "2026-03-09 Day 10",
230
"path": "Events/2026-03-09 Day 10",
231
"revision": "d4e5f6a",
232
"created": true
233
}
234
```
235
236
The `created` field is `true` if this is a new page, `false` if it's an update to an existing page.
237
238
#### Full-text search: `GET /api/v1/search?q=ballistic+missile+rationing`
239
240
Response (snippets are ~150 chars of context around the match, NOT full content):
241
242
```json
243
{
244
"results": [
245
{
246
"name": "Iran Rationing Ballistic Missiles",
247
"path": "Propositions/Iran Rationing Ballistic Missiles",
248
"snippet": "...the 86% drop in ballistic missile launch rates reflects deliberate rationing, not destroyed capability. Observable indicators: continued...",
249
"score": 0.95
250
},
251
{
252
"name": "Iran Attrition Strategy",
253
"path": "Trends/Iran Attrition Strategy",
254
"snippet": "...Phase 3 — Ballistic strikes on high-value targets. Once interceptor stockpiles are depleted and radar coverage is degraded, Iran commits ballistic missiles...",
255
"score": 0.72
256
}
257
],
258
"query": "ballistic missile rationing",
259
"total": 2
260
}
261
```
262
263
#### Semantic search: `GET /api/v1/semantic-search?q=strategy+for+depleting+Gulf+air+defenses&n=3`
264
265
Response (same shape as full-text search, but `distance` instead of `score` — lower is more similar):
266
267
```json
268
{
269
"results": [
270
{
271
"name": "Iran Attrition Strategy",
272
"path": "Trends/Iran Attrition Strategy",
273
"snippet": "Iran is executing a multi-phase attrition campaign designed to degrade Gulf state and US defensive capacity before committing high-value ballistic missile assets.",
274
"distance": 0.34
275
},
276
{
277
"name": "Interceptor Stockpiles",
278
"path": "Variables/Interceptor Stockpiles",
279
"snippet": "Tracking estimated remaining interceptor inventories across Gulf state Patriot and THAAD batteries...",
280
"distance": 0.41
281
},
282
{
283
"name": "Iran Rationing Ballistic Missiles",
284
"path": "Propositions/Iran Rationing Ballistic Missiles",
285
"snippet": "The 86% drop in ballistic missile launch rates reflects deliberate rationing, not destroyed capability...",
286
"distance": 0.48
287
}
288
],
289
"query": "strategy for depleting Gulf air defenses",
290
"total": 3
291
}
292
```
293
294
The `snippet` for semantic search is the **text of the best-matching chunk** for that page, truncated to ~150 characters. Unlike full-text search, this is contextually relevant to the query — it shows the passage that was closest to the query in embedding space, not just the page's opening.