Properties

category: reference
tags: [design, tasks, phases, vps]
last_updated: 2026-03-14
confidence: medium

VPS Implementation Phases

Implementation sequence for robot.wtf on the Debian 12 / Proxmox VM. Phase designations prefixed with V to distinguish from the archived AWS phases (P0–P4).

See Design/VPS_Architecture for the architecture these phases build toward. See Dev/V3_V5_Risk_Research for the auth risk assessment informing the spike phase.

What we're starting with

Phases 1 and 2 of the AWS build produced working, tested components:

TenantResolver WSGI middleware — resolves slug from Host header, authenticates (JWT or bearer token), checks ACL, sets Otterwiki proxy headers, swaps per-wiki config. Currently wraps DynamoDB calls. (P2-5b-7)
ManagementMiddleware — wiki CRUD, ACL management, bearer token generation. WSGI middleware over DynamoDB. (P2-4)
Auth middleware — platform JWT (RS256) validation and user resolution. (P2-2)
ACL enforcement — role-to-permission mapping, public wiki handling. (P2-3)
Wiki bootstrap template — parameterized starter pages. (P2-6)
Admin panel hiding — PLATFORM_MODE flag hides conflicting Otterwiki admin sections. (P2-8)
MCP server — 12 tools, Streamable HTTP, bearer token auth. Working on Lambda.
REST API plugin — full CRUD, search, semantic search, history, links.
Semantic search — FAISS + MiniLM, chunking, embedding, deduplication.
Otterwiki fork — PROXY_HEADER auth mode, all plugins installed.
E2E test suite — 17 tests covering cross-path (API↔MCP) operations.

The port is primarily: DynamoDB → SQLite, EFS paths → local disk paths, Mangum wrapper → Gunicorn, WorkOS → ATProto OAuth.

V0: VM Infrastructure

Goal: Ansible-provisioned Debian 12 VM with Caddy serving valid TLS on bare domain and wildcard.

Tasks:

V0-1: Ansible playbook for base VM provisioning (Python 3.11+, git, build-essential, common tools)
V0-2: Install Caddy with DNS challenge plugin (Cloudflare, Route 53, or OVHcloud — depends on where robot.wtf zone is hosted)
V0-3: Configure DNS: robot.wtf A record + *.robot.wtf A record → VM static IP
V0-4: Caddyfile with wildcard TLS via DNS challenge. Bare domain serves a static placeholder page. Wildcard subdomains serve a placeholder.
V0-5: Generate RS256 signing keypair for platform JWTs (/srv/data/signing_key.pem, /srv/data/signing_key.pub)
V0-6: Generate ATProto OAuth confidential client JWK (/srv/data/client_jwk.json)
V0-7: Create /srv directory structure per Design/VPS_Architecture storage layout
V0-8: Initialize SQLite database with schema from Design/VPS_Architecture

Exit criteria:

https://robot.wtf serves HTTPS with valid cert
https://anything.robot.wtf serves HTTPS with valid wildcard cert
Ansible playbook is idempotent (can re-run without breaking things)
SQLite database exists with empty tables

VS: Auth Spikes

Goal: Validate both auth flows empirically before building the real thing. Kill unknowns early.

These are throwaway prototypes, not production code. The point is to prove the protocol flows work against real counterparts (bsky.social PDS, Claude.ai MCP client) so that V3 and V5 are pure implementation, not discovery.

VS-1: ATProto OAuth Spike

Deploy the Bluesky cookbook Flask demo (bluesky-social/cookbook/python-oauth-web-app) directly on the VM behind Caddy. Minimal adaptation — just enough to run it at robot.wtf/auth/ with the robot.wtf client JWK.

Tasks:

VS-1a: Clone the cookbook demo, install deps, configure Caddy to proxy robot.wtf/auth/* to the demo on a local port.
VS-1b: Publish client metadata at https://robot.wtf/auth/client-metadata.json with scope "atproto" (identity-only) and redirect URI https://robot.wtf/auth/oauth/callback.
VS-1c: Log in with a real Bluesky account. Walk through the full flow: handle entry → PDS redirect → approve → callback → token → DID.
VS-1d: Test with a non-bsky.social PDS if one is accessible (e.g., a self-hosted PDS on the ATProto community server). This validates that the flow isn't accidentally Bluesky-specific.
VS-1e: Document what worked, what didn't, any PDS quirks encountered.

Exit criteria:

A real Bluesky login completes end-to-end on robot.wtf
The DID and handle are retrieved from the token response
DPoP nonce handling works without manual intervention
Findings documented in a Dev summary page

Expected time: Half a day. The demo is ready to run.

VS-2: MCP OAuth AS Stub

Build a minimal, hard-coded OAuth 2.1 AS that implements the five endpoints Claude.ai needs. No real auth, no database, no ATProto — just the protocol surface with canned responses. Deploy behind Caddy and point Claude.ai at it.

Tasks:

VS-2a: Implement the stub AS as a single Flask file (~150 lines):
- GET /.well-known/oauth-authorization-server → static JSON metadata
- POST /auth/oauth/register → accept any DCR request, return a client_id + client_secret
- GET /auth/oauth/authorize → auto-approve (no consent UI), redirect with auth code
- POST /auth/oauth/token → exchange code for a JWT access token (hard-coded claims)
- GET /.well-known/jwks.json → serve the RS256 public key
VS-2b: Implement the protected resource metadata on a wiki subdomain:
- GET https://{slug}.robot.wtf/.well-known/oauth-protected-resource → point to robot.wtf AS
VS-2c: Deploy a minimal MCP endpoint (FastMCP with one dummy tool, e.g., echo) behind the protected resource metadata. Wire it to return 401 with WWW-Authenticate header on unauthenticated requests, and accept the stub JWT on authenticated requests.
VS-2d: Add the MCP URL in Claude.ai settings. Walk through the flow: Claude.ai discovers AS → registers → redirects to authorize → gets token → calls the echo tool.
VS-2e: If it fails, inspect what Claude.ai actually sent (log all requests) and iterate. Document the exact request shapes, headers, and timing.
VS-2f: Test the same flow with Claude Code (claude mcp add --transport http) to verify OAuth discovery works there too.

Exit criteria:

Claude.ai completes the MCP OAuth flow against the stub and successfully calls a tool
OR: documented exactly where/why Claude.ai's client diverges from the spec, with a plan to accommodate
Claude Code OAuth discovery tested
Findings documented in a Dev summary page

Expected time: 1–2 days, mostly debugging Claude.ai's behavior.

Why spike before building

The research (Dev/V3_V5_Risk_Research) assessed V3 as low risk and V5 as medium risk, but both assessments are based on documentation and code reading, not empirical testing. The spikes turn "should work" into "does work" for a cost of ~2 days, before committing to the full implementation in V3 and V5. If either spike reveals a fundamental incompatibility, we find out before building the real system, not after.

V1: Otterwiki on Caddy

Goal: Single Otterwiki instance running behind Gunicorn behind Caddy, with multi-tenant slug-based routing on wildcard subdomains. No auth yet — test users hardcoded in SQLite.

Tasks:

V1-1: Port TenantResolver from DynamoDB to SQLite. Replace UserModel.get(), WikiModel.get(), AclModel.query() with SQLite queries. The WSGI middleware structure, proxy header injection, and config-swapping logic stay unchanged.
V1-2: Port data access layer. Create a thin db.py module that wraps SQLite (or SQLAlchemy if preferred). Same interface as the DynamoDB models but backed by SQLite.
V1-3: Gunicorn configuration. Otterwiki Flask app wrapped in TenantResolver, served by Gunicorn on port 8000. Multiple workers (sized to VM CPU cores).
V1-4: Caddy routing for wiki subdomains. *.robot.wtf routes to Gunicorn on port 8000. Caddy handles TLS, Gunicorn handles WSGI.
V1-5: Manual smoke test. Insert a test user and wiki directly into SQLite. git init --bare /srv/wikis/testuser/repo.git. Commit a Home page. Verify https://testuser.robot.wtf serves the wiki.

Exit criteria:

Browsing, editing, and saving wiki pages works at {slug}.robot.wtf
Gunicorn stays up, serves multiple requests, workers don't crash
Config-swap per request works (if multiple test wikis exist, each resolves correctly)

V2: Migrate dev.wikibot.io → dev.robot.wtf

Goal: The existing development wiki (Third Gulf War research + this dev wiki) running on the VPS under robot.wtf, proving the full Otterwiki stack works in the new environment with real data.

Tasks:

V2-1: Export git repos from the existing AWS deployment. git clone --bare from the EFS-backed repos (or from the dev.wikibot.io git remote if accessible). Copy to /srv/wikis/ on the VM.
V2-2: Create corresponding user and wiki records in SQLite. The existing deployment uses UUIDs and DynamoDB; the VPS uses DIDs and SQLite. For the dev migration, create placeholder user records (the owner's DID can be updated once ATProto auth is wired in V3).
V2-3: Rebuild FAISS indexes from the migrated repos. Run the existing embedding code against the local repos with MiniLM.
V2-4: Wire up the MCP server (FastMCP sidecar on port 8001) with bearer token auth against SQLite. Generate new bearer tokens for the migrated wikis.
V2-5: Configure Caddy to route MCP traffic ({slug}.robot.wtf/mcp) to port 8001.
V2-6: Verify end-to-end: browse the wiki in a browser, connect Claude Code via MCP bearer token, run a few tool calls.
V2-7: Update MCP connection config in Claude.ai / Claude Code to point at dev.robot.wtf instead of dev.wikibot.io.

Exit criteria:

The Third Gulf War wiki is browsable at its new robot.wtf subdomain
This dev wiki is browsable at its new robot.wtf subdomain
MCP tools work against the migrated wikis via Claude Code (bearer token)
Semantic search returns results from the migrated FAISS indexes
The old AWS deployment can be left running in parallel until confidence is high, then decommissioned

Note: Auth at this point is still placeholder/hardcoded. The migrated wikis use bearer tokens for MCP and don't require browser login (the wiki owner can browse without auth during this phase, or auth can be faked via a test JWT). Real ATProto auth comes in V3.

Goal: Real users can sign in with their Bluesky handle and get a platform JWT.

VS-1 proved the flow works. This phase replaces the spike with production code: proper error handling, session management, signup flow, and integration with the TenantResolver.

Tasks:

V3-1: Auth service scaffold. Flask app on port 8003. Caddy routes robot.wtf/auth/* to it. (May already exist from VS-1 — evolve the spike or rewrite clean.)
V3-2: Publish ATProto OAuth client metadata at https://robot.wtf/auth/client-metadata.json. This URL becomes the client_id in the ATProto OAuth protocol — it must be stable. (May already exist from VS-1.)
V3-3: Implement the ATProto OAuth flow. Adapt the Bluesky cookbook demo: handle input → DID resolution → PDS discovery → PAR → redirect → callback → token exchange → profile fetch. Store ATProto tokens in the oauth_sessions table. Incorporate lessons from VS-1.
V3-4: Platform JWT issuance. On successful ATProto auth, mint a platform JWT (RS256, same signing key from V0-5), set as HttpOnly cookie on .robot.wtf.
V3-5: Signup flow. First-time users: after ATProto auth, prompt for platform username. Default to ATProto handle prefix (e.g., sderle from sderle.bsky.social, or domain prefix from a custom handle). Validate against reserved names and existing slugs. Create user record in SQLite.
V3-6: Wire TenantResolver to validate platform JWTs from the cookie. Replace the hardcoded test auth from V1.
V3-7: Handle refresh on login. Update handle and display_name from the PDS profile on each login (handles can change).
V3-8: Login/logout UI. Minimal pages served by the auth service: login page (handle input field), logout endpoint (clear cookie). The management SPA comes later in V6; this is just the bare auth flow.

Exit criteria:

A real Bluesky user can visit robot.wtf/auth/login, enter their handle, authorize on their PDS, and land back at robot.wtf with a valid platform JWT
The platform JWT cookie works across subdomains (the user's wiki at {slug}.robot.wtf recognizes them)
First-time signup creates a user record with DID, handle, and chosen username
Returning users are recognized by DID; handle updates are reflected

V4: Management API + Wiki Lifecycle

Goal: Authenticated users can create wikis, manage collaborators, and get MCP bearer tokens via API.

Tasks:

V4-1: Port ManagementMiddleware from DynamoDB to SQLite. Same endpoints, same WSGI middleware pattern. Replace DynamoDB calls with SQLite queries.
V4-2: Wire management API at robot.wtf/api/*. Caddy routes to the platform API service on port 8002.
V4-3: Wiki creation flow. POST /api/wikis → create SQLite records + git init --bare + bootstrap template + generate bearer token. Returns token (shown once).
V4-4: Wiki deletion, ACL management, token regeneration — port remaining endpoints.
V4-5: Tier limit enforcement. 1 wiki per user, 500 pages, 3 collaborators. Enforced in middleware on write operations.
V4-6: Git smart HTTP (read-only). Caddy routes {slug}.robot.wtf/repo.git/* to the platform API. dulwich serves git-upload-pack from the bare repo. Users can git clone https://{slug}.robot.wtf/repo.git.
V4-7: Integration test. Create a user (via V3 auth flow), create a wiki (via API), browse it, connect MCP, clone via git.

Exit criteria:

Authenticated user can create a wiki and get a bearer token
Wiki appears at {slug}.robot.wtf with bootstrap template pages
ACL grants/revocations work
git clone https://{slug}.robot.wtf/repo.git works
Tier limits enforced (cannot create second wiki)

V5: MCP OAuth AS (Claude.ai)

Goal: Claude.ai can connect to a robot.wtf wiki via its standard MCP OAuth flow.

VS-2 proved (or debugged) the protocol surface. This phase replaces the stub with production code: authlib's AuthorizationServer, real DCR persistence, ATProto-backed consent UI, proper token lifecycle.

Tasks:

V5-1: Implement OAuth 2.1 AS using authlib's AuthorizationServer + AuthorizationCodeGrant (PKCE) + ClientRegistrationEndpoint (RFC 7591). Wire model callbacks against SQLite (mcp_oauth_clients table for DCR, authorization codes in a transient table or in-memory store). Incorporate findings from VS-2 about Claude.ai's specific requirements.
V5-2: MCP protected resource metadata. Each wiki's MCP endpoint serves /.well-known/oauth-protected-resource pointing to robot.wtf's AS. Caddy routes this from the wiki subdomain to the MCP sidecar (or a small handler).
V5-3: Authorization UI. The /auth/oauth/authorize endpoint renders a consent page. If the user is already logged in (platform JWT cookie): show "Authorize Claude to access {wiki}?". If not logged in: show "Sign in with Bluesky" which triggers the ATProto flow (V3) and then returns to consent.
V5-4: Token issuance. The AS issues JWTs containing the user's DID and authorized wiki slug, signed with the platform RS256 key. MCP sidecar validates these the same way it validates platform JWTs.
V5-5: Test against Claude.ai. Full end-to-end: add wiki MCP URL in Claude.ai, complete OAuth flow with real ATProto login, call wiki tools.
V5-6: Test against Claude Code. Verify claude mcp add --transport http with OAuth discovery works (in addition to the existing bearer token path).

Exit criteria:

Claude.ai can discover, authenticate, and use MCP tools on a robot.wtf wiki
Claude Code can connect via OAuth (in addition to the existing bearer token path)
Token refresh works (Claude.ai doesn't lose connection after token expiry)

V6: Frontend + Landing Page

Goal: Web UI for account management and a landing page that explains what robot.wtf is.

Tasks:

V6-1: Landing page. Static HTML/CSS at robot.wtf/. Explains what robot.wtf is, shows how to get started, links to sign in. Tone and structure adapted from Design/Landing_Page but reframed as a free volunteer project, not a product. Served by Caddy's file_server.
V6-2: Management SPA. Svelte app at robot.wtf/app/*. Served by Caddy (try_files fallback to index.html for client-side routing). Screens from the archived Design/Frontend, minus billing:
- Dashboard (wiki list — for now just one wiki, but the UI should handle the general case)
- Wiki settings (display name, public/private toggle, link to Otterwiki admin panel)
- Collaborators (list, invite by handle/email, change role, revoke)
- MCP connection instructions (bearer token display/regen, Claude Code command, Claude.ai setup walkthrough)
- Account settings (username, connected ATProto handle, delete account)
V6-3: /api/me endpoint. Returns current user info from platform JWT. The SPA uses this to determine login state and populate the UI.
V6-4: "Logged in" detection on landing page. Small inline script checks for a companion cookie or probes /api/me to swap "Sign in" for "Dashboard" link.

Exit criteria:

New user can visit robot.wtf, understand the service, sign in, create a wiki, and connect MCP — entirely through the web UI
Returning user can manage collaborators, regenerate tokens, toggle public access
Landing page is clear, honest, and matches the documentation style (write like a README, not a pitch deck)

V7: Semantic Search + Operational Hardening

Goal: Semantic search works. Backups run. The service is ready for real users.

Tasks:

V7-1: Embedding background worker. In-process thread (or dedicated worker process) that polls the reindex_queue table in SQLite. On dequeue: read page from git repo, chunk, embed with MiniLM, update FAISS index on disk.
V7-2: Wire reindex queue. Otterwiki write operations (via MCP or REST API) write {wiki_slug, page_path, action} to reindex_queue on page save/delete.
V7-3: Wire semantic search into MCP sidecar and REST API. Load FAISS index from disk, embed query with MiniLM, search, deduplicate, return results.
V7-4: Backup cron. Daily: SQLite .backup + rsync of /srv/wikis and /srv/data to offsite storage. Alert on failure.
V7-5: Health checks. Each service (Otterwiki, MCP, platform API, auth) exposes /health. External monitor (UptimeRobot free tier) pings them.
V7-6: Log rotation. systemd journal or logrotate configuration so logs don't fill the disk.
V7-7: Disk space monitoring. Cron job alerts when usage exceeds 80%.
V7-8: Account deletion. User can delete their account from the SPA. Deletes wiki (git repo, FAISS index), SQLite records, ACL grants. Requires typing username to confirm.
V7-9: Announce to the ATProto community. Post on Bluesky, add to ATProto app directories if any exist.

Exit criteria:

Semantic search returns relevant results for queries against indexed wikis
Backups run daily and succeed
Health checks pass and alert on failure
A real user who is not the operator can sign up, create a wiki, connect MCP, and use it without assistance

Dependency graph

V0 (VM + Caddy + TLS)
 ├─ VS-1 (ATProto OAuth spike)
 ├─ VS-2 (MCP OAuth AS stub)
 │
 └─ V1 (Otterwiki on Caddy)
     ├─ V2 (Migrate dev wikis)
     └─ V3 (ATProto OAuth — informed by VS-1)
         └─ V4 (Management API)
             ├─ V5 (MCP OAuth AS — informed by VS-2)
             └─ V6 (Frontend + Landing)
                 └─ V7 (Semantic search + Ops)

VS-1, VS-2, and V1 can all run in parallel after V0. The spikes don't block V1 — they're independent experiments on the same VM. V2 and V3 can run in parallel after V1. V4 depends on V3 (needs real auth). V5 and V6 can run in parallel after V4. V7 is the final polish.

Minimum viable launch: After V4, the service is usable — users can sign in with Bluesky, create a wiki, connect via Claude Code (bearer token), and browse the web UI. Claude.ai MCP OAuth (V5), the polished frontend (V6), and semantic search (V7) are improvements on a working service.

What's not in scope

These items were part of the commercial wikibot.io design and are deliberately excluded from robot.wtf:

Stripe integration, billing, premium tiers, upgrade/downgrade flows, lapse enforcement
Multiple wikis per user (could be added later as a free feature if resource limits allow)
Read/write git push (only read-only clone)
External git sync (bidirectional GitHub/GitLab)
Custom domains
CDN read path / fragment caching (not needed — no Lambda cold starts)
Lambda library mode / lazy loading (not needed — persistent processes)
WAF (not needed at this scale — Caddy + rate limiting if abuse appears)
Pulumi / IaC (replaced by Ansible for VM setup, systemd for services)
CI/CD pipeline (deploy is git pull + restart; add GitHub Actions later if it matters)

VPS Implementation Phases

What we're starting with

V0: VM Infrastructure

VS: Auth Spikes

VS-1: ATProto OAuth Spike

VS-2: MCP OAuth AS Stub

Why spike before building

V1: Otterwiki on Caddy

V2: Migrate dev.wikibot.io → dev.robot.wtf

V3: ATProto OAuth (Browser Login)

V4: Management API + Wiki Lifecycle

V5: MCP OAuth AS (Claude.ai)

V6: Frontend + Landing Page

V7: Semantic Search + Operational Hardening

Dependency graph

What's not in scope