Commit 66fbd9
2026-03-13 06:30:55 Claude (Dev): [mcp] Add P1-8 E2E test results| /dev/null .. dev/p1-8 e2e test.md | |
| @@ 0,0 1,66 @@ | |
| + | --- |
| + | category: reference |
| + | tags: [dev, phase-1, testing] |
| + | last_updated: 2026-03-13 |
| + | confidence: high |
| + | --- |
| + | |
| + | # P1-8 E2E Integration Test |
| + | |
| + | ## Status: Complete (with known gaps) |
| + | |
| + | 13/15 tests pass. 2 tests are expected failures (xfail) pending P1-5 MCP deployment. |
| + | |
| + | ## Test Results |
| + | |
| + | | Component | Tests | Status | |
| + | |-----------|-------|--------| |
| + | | Otterwiki Web UI | 2 | PASS | |
| + | | REST API CRUD | 5 | PASS | |
| + | | Semantic Search | 3 | PASS | |
| + | | MCP Server (auth required) | 1 | PASS | |
| + | | MCP Server (initialize, tools/list) | 2 | XFAIL (bearer token not deployed) | |
| + | | Custom Domain + TLS | 2 | PASS | |
| + | |
| + | ## Findings |
| + | |
| + | ### Working |
| + | - **Web UI**: Homepage returns 200 with HTML at `https://dev.wikibot.io/` |
| + | - **Health endpoint**: `/api/v1/health` responds without auth |
| + | - **REST API full CRUD**: Create (201) -> Read (200) -> Update (200, with optimistic locking via revision) -> Read (verify update) -> History (2+ entries) -> Delete (200) -> Verify 404 |
| + | - **REST API auth**: Unauthenticated requests correctly rejected with 401 |
| + | - **REST API list/search/changelog**: All return correct response shapes |
| + | - **Semantic search**: `/api/v1/semantic-search` and `/api/v1/reindex` both respond (200) |
| + | - **Chroma status**: `/api/v1/chroma-status` reports backend info |
| + | - **MCP auth enforcement**: Unauthenticated POST to `/mcp` returns 401 |
| + | - **TLS**: Valid certificate on `dev.wikibot.io`, httpx connects without errors |
| + | |
| + | ### Known Gaps |
| + | - **MCP bearer token auth (P1-5 dependency)**: The deployed MCP Lambda (P0-7 PoC) only has WorkOS OAuth configured. No `MCP_AUTH_TOKEN` env var is set. Bearer token tests are marked `xfail`. Once P1-5 deploys with `MCP_AUTH_TOKEN`, remove the `xfail` markers. |
| + | - **Otterwiki title-casing**: Page paths are title-cased by Otterwiki (e.g., `E2ETest/6bc504da` becomes `E2Etest/6Bc504Da`). Tests use case-insensitive comparison. |
| + | |
| + | ## Files |
| + | |
| + | - Test script: `tests/test_e2e.py` |
| + | - Conftest (marker registration): `tests/conftest.py` |
| + | |
| + | ## How to Run |
| + | |
| + | ```bash |
| + | # From repo root with Pulumi configured: |
| + | pytest tests/test_e2e.py -v -m e2e |
| + | |
| + | # Or with explicit credentials: |
| + | E2E_API_KEY=<key> E2E_MCP_TOKEN=<token> pytest tests/test_e2e.py -v -m e2e |
| + | |
| + | # Override base URL: |
| + | E2E_BASE_URL=https://other.example.com pytest tests/test_e2e.py -v -m e2e |
| + | ``` |
| + | |
| + | ## Architecture Decisions |
| + | |
| + | - **httpx over requests**: Consistent with the MCP API client (`mcp_api_client.py`) which uses httpx. Avoids adding another HTTP library dependency. |
| + | - **Pulumi config fallback**: Tests read API key and MCP token from env vars first, falling back to `./pulumi.sh config get`. This allows running from CI without Pulumi. |
| + | - **UUID test page paths**: Each test run creates pages under `E2ETest/<uuid>` to avoid collisions with concurrent runs. |
| + | - **Idempotent cleanup**: CRUD test uses try/finally to delete the test page even on assertion failure. |
| + | - **xfail over skip for MCP**: Using `xfail` instead of `skip` so the tests still run and will surface when the issue is fixed (tests will unexpectedly pass, prompting removal of the marker). |