Home A - Z

Commit 757966

2026-03-13 01:49:27 Claude (Dev): [mcp] Port original PRD API section to wiki
/dev/null .. design/original prd api.md
@@ 0,0 1,294 @@
+ ---
+ category: reference
+ tags: [meta, design, prd, api]
+ last_updated: 2026-03-12
+ confidence: high
+ ---
+
+ # Original PRD API
+
+ > This page is part of the original single-tenant PRD, split across five wiki pages:
+ > [[Design/Original PRD Overview]] | [[Design/Original PRD API]] | [[Design/Original PRD Semantic Search]] | [[Design/Original PRD MCP]] | [[Design/Original PRD Note Schema]]
+
+ ---
+
+ ## Component 1: REST API Plugin
+
+ ### Goal
+
+ Add a JSON REST API to Otterwiki so that pages can be created, read, updated, deleted, listed, and searched programmatically.
+
+ ### Implementation approach
+
+ **First, investigate the plugin system.** Examine `otterwiki/plugins.py` and `docs/plugin_examples/` in the fork to determine whether plugins can register Flask blueprints (i.e., add new routes). The plugin system has hooks and was extended in v2.17.3.
+
+ - **If plugins support blueprint registration:** Build the API as an Otterwiki plugin. This is the preferred path — no core modifications, clean separation, potentially upstreamable.
+ - **If plugins do NOT support blueprint registration:** Add an `api.py` Flask blueprint directly to the Otterwiki codebase in the fork. Register it in `server.py`. This is a clean PR-able change.
+
+ ### Authentication
+
+ Use a simple API key passed via `Authorization: Bearer <key>` header. The key is configured via environment variable `OTTERWIKI_API_KEY`. This is a single-user research system, not a multi-tenant service, so this is sufficient.
+
+ ### Commit authorship and message conventions
+
+ All Git commits should clearly indicate their origin. This is important for reviewing history and understanding whether a change was made by the human, the AI, or a system process.
+
+ **Author identity for API commits:**
+
+ ```
+ Author: Claude (MCP) <claude-mcp@otterwiki.local>
+ ```
+
+ Configure this via environment variables `OTTERWIKI_API_AUTHOR_NAME` and `OTTERWIKI_API_AUTHOR_EMAIL`, defaulting to the above.
+
+ **Commit message format:**
+
+ ```
+ [source] action: page name — optional detail
+ ```
+
+ Where `source` is one of:
+ - `mcp` — changes made via the MCP server / API
+ - `web` — changes made via the Otterwiki web UI (Otterwiki handles this itself)
+ - `system` — changes made by automated processes (e.g., reindex, bulk import)
+
+ Examples:
+ ```
+ [mcp] Create: Events/2026-03-09 Day 10 — initial event log
+ [mcp] Update: Trends/Iran Attrition Strategy — add Phase 2 radar blinding detail
+ [mcp] Delete: Events/Draft Note
+ [system] Bulk import: 15 pages from PDF migration
+ ```
+
+ If the caller provides a `commit_message` in the API request body, use it as-is but prepend the `[mcp]` prefix. If no message is provided, generate one from the action and page name.
+
+ ### Endpoints
+
+ All endpoints are prefixed with `/api/v1/`.
+
+ #### Pages
+
+ | Method | Endpoint | Description |
+ |--------|----------|-------------|
+ | `GET` | `/api/v1/pages` | List all pages. Optional query params: `?prefix=Actors/` (subdirectory), `?category=actor` (frontmatter category), `?tag=p2-interceptor-race` (frontmatter tag), `?updated_since=2026-03-08` (ISO date). Filters compose with AND logic. Returns array of `{name, path, category, tags, last_updated, content_length}`. |
+ | `GET` | `/api/v1/pages/<path:pagepath>` | Get a single page. Returns `{name, path, content, metadata, frontmatter, links_to, linked_from}`. Optional `?revision=<sha>` for historical versions. |
+ | `PUT` | `/api/v1/pages/<path:pagepath>` | Create or update a page. Body: `{content, commit_message}`. If `commit_message` is omitted, auto-generates per commit convention above. |
+ | `DELETE` | `/api/v1/pages/<path:pagepath>` | Delete a page. Body: `{commit_message}` (optional). |
+ | `GET` | `/api/v1/pages/<path:pagepath>/history` | Get revision history. Returns array of `{revision, author, date, message}`. Optional `?limit=N`. |
+
+ #### Search
+
+ | Method | Endpoint | Description |
+ |--------|----------|-------------|
+ | `GET` | `/api/v1/search?q=<query>` | Full-text search (uses Otterwiki's existing search). Returns array of `{name, path, snippet, score}`. |
+
+ #### Links (WikiLink graph)
+
+ | Method | Endpoint | Description |
+ |--------|----------|-------------|
+ | `GET` | `/api/v1/links/<path:pagepath>` | Get outgoing and incoming WikiLinks for a page. Returns `{links_to: [...], linked_from: [...]}`. |
+ | `GET` | `/api/v1/links` | Get the full link graph. Returns `{nodes: [...], edges: [...]}`. |
+
+ #### Changelog
+
+ | Method | Endpoint | Description |
+ |--------|----------|-------------|
+ | `GET` | `/api/v1/changelog` | Recent changes across all pages. Returns array of `{revision, author, date, message, pages_affected}`. Optional `?limit=N`. |
+
+ ### WikiLink parsing and link graph
+
+ The API must parse `[[WikiLink]]` and `[[Display Text|WikiLink]]` syntax in page content to populate the `links_to` and `linked_from` fields.
+
+ **Parsing approach:**
+
+ The regex for extracting WikiLinks from markdown content:
+
+ ```python
+ import re
+ WIKILINK_RE = re.compile(r'\[\[([^\]|]+?)(?:\|([^\]]+?))?\]\]')
+
+ def extract_wikilinks(content: str) -> list[str]:
+ """Returns list of target page paths from WikiLinks in content."""
+ return [match.group(2) or match.group(1) for match in WIKILINK_RE.finditer(content)]
+ # For [[Display Text|Target]], returns "Target"
+ # For [[Target]], returns "Target"
+ ```
+
+ Note: check whether Otterwiki uses `[[Target|Display Text]]` or `[[Display Text|Target]]` order — this varies between wiki engines. Otterwiki's syntax page shows `[[Text to display|WikiPage]]`, so the **target is the second element** when a pipe is present.
+
+ **Link index implementation:**
+
+ Maintain an in-memory reverse index (dict mapping page path → set of pages that link to it). This is built once on startup by scanning all pages, then updated incrementally:
+
+ - On page save: re-parse that page's WikiLinks, update the index for that page
+ - On page delete: remove that page from the index
+
+ The startup scan is O(N) where N is total pages. For a wiki of 200–500 pages with ~500 words each, this takes under a second. The index lives in the Flask process memory — no external storage needed.
+
+ Otterwiki v2.17.3 added a broken WikiLinks checker in housekeeping (`#388`). Look at that implementation first — it likely already has WikiLink parsing that can be reused or imported.
+
+ **The `GET /api/v1/links/<path>` endpoint** reads directly from this index. It does NOT scan the repo on every request.
+
+ **The `GET /api/v1/links` endpoint** (full graph) serializes the entire index. This could be expensive on a very large wiki, but for our expected size (< 500 pages) it's fine.
+
+ ### Error responses
+
+ Standard HTTP status codes. JSON body: `{error: "description"}`.
+
+ - `401` — missing or invalid API key
+ - `404` — page not found
+ - `409` — conflict (e.g., concurrent edit)
+ - `422` — invalid content or parameters
+
+ ### Example requests and responses
+
+ These examples are canonical — the implementation should match these JSON shapes exactly.
+
+ #### List pages: `GET /api/v1/pages?prefix=Trends/`
+
+ Response (note: NO content field — list operations return metadata only):
+
+ ```json
+ {
+ "pages": [
+ {
+ "name": "Iran Attrition Strategy",
+ "path": "Trends/Iran Attrition Strategy",
+ "category": "trend",
+ "tags": ["military", "p2-interceptor-race", "p3-infrastructure"],
+ "last_updated": "2026-03-08",
+ "content_length": 487
+ },
+ {
+ "name": "Desalination Targeting Ratchet",
+ "path": "Trends/Desalination Targeting Ratchet",
+ "category": "trend",
+ "tags": ["infrastructure", "p3-infrastructure"],
+ "last_updated": "2026-03-08",
+ "content_length": 312
+ }
+ ],
+ "total": 2
+ }
+ ```
+
+ The `content_length` field is word count. This lets the caller decide whether to fetch the full page or skip large ones. The `category` and `tags` fields are extracted from YAML frontmatter; they are `null` if frontmatter is missing or malformed.
+
+ #### Read page: `GET /api/v1/pages/Trends/Iran Attrition Strategy`
+
+ Response (full content, parsed frontmatter, resolved links):
+
+ ```json
+ {
+ "name": "Iran Attrition Strategy",
+ "path": "Trends/Iran Attrition Strategy",
+ "content": "---\ncategory: trend\ntags: [military, p2-interceptor-race, p3-infrastructure]\nlast_updated: 2026-03-08\nconfidence: high\n---\n\n# Iran Attrition Strategy\n\nIran is executing a multi-phase attrition campaign...",
+ "frontmatter": {
+ "category": "trend",
+ "tags": ["military", "p2-interceptor-race", "p3-infrastructure"],
+ "last_updated": "2026-03-08",
+ "confidence": "high"
+ },
+ "links_to": [
+ "Variables/Interceptor Stockpiles",
+ "Propositions/Iran Rationing Ballistic Missiles",
+ "Actors/Iran",
+ "Trends/Desalination Targeting Ratchet"
+ ],
+ "linked_from": [
+ "Actors/Iran",
+ "Propositions/Iran Rationing Ballistic Missiles"
+ ],
+ "revision": "a1b2c3d",
+ "last_commit": {
+ "revision": "a1b2c3d",
+ "author": "Claude (MCP)",
+ "date": "2026-03-08T14:22:00Z",
+ "message": "Update Iran Attrition Strategy — add Phase 2 radar blinding detail"
+ }
+ }
+ ```
+
+ The `content` field is the **raw markdown file content including the frontmatter block**. The `frontmatter` field is the parsed YAML as a JSON object. If frontmatter is missing or invalid YAML, `frontmatter` is `null` and `content` still returns the raw file.
+
+ #### Write page: `PUT /api/v1/pages/Events/2026-03-09 Day 10`
+
+ Request body:
+
+ ```json
+ {
+ "content": "---\ncategory: event\ntags: [military, day-10]\nlast_updated: 2026-03-09\nconfidence: high\n---\n\n# Day 10 — March 9, 2026\n\n## Key developments\n\n...",
+ "commit_message": "Create Day 10 event log"
+ }
+ ```
+
+ Response:
+
+ ```json
+ {
+ "name": "2026-03-09 Day 10",
+ "path": "Events/2026-03-09 Day 10",
+ "revision": "d4e5f6a",
+ "created": true
+ }
+ ```
+
+ The `created` field is `true` if this is a new page, `false` if it's an update to an existing page.
+
+ #### Full-text search: `GET /api/v1/search?q=ballistic+missile+rationing`
+
+ Response (snippets are ~150 chars of context around the match, NOT full content):
+
+ ```json
+ {
+ "results": [
+ {
+ "name": "Iran Rationing Ballistic Missiles",
+ "path": "Propositions/Iran Rationing Ballistic Missiles",
+ "snippet": "...the 86% drop in ballistic missile launch rates reflects deliberate rationing, not destroyed capability. Observable indicators: continued...",
+ "score": 0.95
+ },
+ {
+ "name": "Iran Attrition Strategy",
+ "path": "Trends/Iran Attrition Strategy",
+ "snippet": "...Phase 3 — Ballistic strikes on high-value targets. Once interceptor stockpiles are depleted and radar coverage is degraded, Iran commits ballistic missiles...",
+ "score": 0.72
+ }
+ ],
+ "query": "ballistic missile rationing",
+ "total": 2
+ }
+ ```
+
+ #### Semantic search: `GET /api/v1/semantic-search?q=strategy+for+depleting+Gulf+air+defenses&n=3`
+
+ Response (same shape as full-text search, but `distance` instead of `score` — lower is more similar):
+
+ ```json
+ {
+ "results": [
+ {
+ "name": "Iran Attrition Strategy",
+ "path": "Trends/Iran Attrition Strategy",
+ "snippet": "Iran is executing a multi-phase attrition campaign designed to degrade Gulf state and US defensive capacity before committing high-value ballistic missile assets.",
+ "distance": 0.34
+ },
+ {
+ "name": "Interceptor Stockpiles",
+ "path": "Variables/Interceptor Stockpiles",
+ "snippet": "Tracking estimated remaining interceptor inventories across Gulf state Patriot and THAAD batteries...",
+ "distance": 0.41
+ },
+ {
+ "name": "Iran Rationing Ballistic Missiles",
+ "path": "Propositions/Iran Rationing Ballistic Missiles",
+ "snippet": "The 86% drop in ballistic missile launch rates reflects deliberate rationing, not destroyed capability...",
+ "distance": 0.48
+ }
+ ],
+ "query": "strategy for depleting Gulf air defenses",
+ "total": 3
+ }
+ ```
+
+ The `snippet` for semantic search is the **text of the best-matching chunk** for that page, truncated to ~150 characters. Unlike full-text search, this is contextually relevant to the query — it shows the passage that was closest to the query in embedding space, not just the page's opening.
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9