Properties

category: reference
tags: [dev, phase-1, testing]
last_updated: 2026-03-13
confidence: high

P1-8 E2E Integration Test

Status: Complete (with known gaps)

13/15 tests pass. 2 tests are expected failures (xfail) pending P1-5 MCP deployment.

Test Results

Component	Tests	Status
Otterwiki Web UI	2	PASS
REST API CRUD	5	PASS
Semantic Search	3	PASS
MCP Server (auth required)	1	PASS
MCP Server (initialize, tools/list)	2	XFAIL (bearer token not deployed)
Custom Domain + TLS	2	PASS

Findings

Working

Web UI: Homepage returns 200 with HTML at https://dev.wikibot.io/
Health endpoint: /api/v1/health responds without auth
REST API full CRUD: Create (201) -> Read (200) -> Update (200, with optimistic locking via revision) -> Read (verify update) -> History (2+ entries) -> Delete (200) -> Verify 404
REST API auth: Unauthenticated requests correctly rejected with 401
REST API list/search/changelog: All return correct response shapes
Semantic search: /api/v1/semantic-search and /api/v1/reindex both respond (200)
Chroma status: /api/v1/chroma-status reports backend info
MCP auth enforcement: Unauthenticated POST to /mcp returns 401
TLS: Valid certificate on dev.wikibot.io, httpx connects without errors

Known Gaps

MCP bearer token auth (P1-5 dependency): The deployed MCP Lambda (P0-7 PoC) only has WorkOS OAuth configured. No MCP_AUTH_TOKEN env var is set. Bearer token tests are marked xfail. Once P1-5 deploys with MCP_AUTH_TOKEN, remove the xfail markers.
Otterwiki title-casing: Page paths are title-cased by Otterwiki (e.g., E2ETest/6bc504da becomes E2Etest/6Bc504Da). Tests use case-insensitive comparison.

Files

Test script: tests/test_e2e.py
Conftest (marker registration): tests/conftest.py

How to Run

# From repo root with Pulumi configured:
pytest tests/test_e2e.py -v -m e2e

# Or with explicit credentials:
E2E_API_KEY=<key> E2E_MCP_TOKEN=<token> pytest tests/test_e2e.py -v -m e2e

# Override base URL:
E2E_BASE_URL=https://other.example.com pytest tests/test_e2e.py -v -m e2e

Architecture Decisions

httpx over requests: Consistent with the MCP API client (mcp_api_client.py) which uses httpx. Avoids adding another HTTP library dependency.
Pulumi config fallback: Tests read API key and MCP token from env vars first, falling back to ./pulumi.sh config get. This allows running from CI without Pulumi.
UUID test page paths: Each test run creates pages under E2ETest/<uuid> to avoid collisions with concurrent runs.
Idempotent cleanup: CRUD test uses try/finally to delete the test page even on assertion failure.
xfail over skip for MCP: Using xfail instead of skip so the tests still run and will surface when the issue is fixed (tests will unexpectedly pass, prompting removal of the marker).