Blame
|
1 | --- |
||||||
| 2 | category: reference |
|||||||
| 3 | tags: [architecture, database, multi-tenancy] |
|||||||
|
4 | last_updated: 2026-03-17 |
||||||
|
5 | confidence: high |
||||||
| 6 | --- |
|||||||
| 7 | ||||||||
| 8 | # Per-Wiki Database Design |
|||||||
| 9 | ||||||||
|
10 | ## Decision (2026-03-17) |
||||||
| 11 | ||||||||
| 12 | **Implementing Option A (per-request DB swap via monkey-patching) now.** A cleaner upstream refactor (contextvars-based dependency injection) is sketched below and will be proposed to redimp as a separate PR. Option A is throwaway work if the upstream refactor lands, but unblocks per-wiki permissions immediately. |
|||||||
| 13 | ||||||||
|
14 | ## Current State |
||||||
| 15 | ||||||||
|
16 | Otterwiki uses a single shared SQLite DB (`/tmp/otterwiki_default.db`) for all tenants. The `Preferences`, `User`, `Drafts`, and `Cache` tables are global. The `TenantResolver` swaps `GitStorage` (the git repo path) per-request but never swaps the SQLAlchemy DB binding. |
||||||
|
17 | |||||||
| 18 | This hasn't caused problems because: |
|||||||
| 19 | - `ProxyHeaderAuth.has_permission()` ignores the Preferences table entirely — permissions come from proxy headers |
|||||||
| 20 | - User Management and Permissions admin panels are disabled via `PLATFORM_MODE` |
|||||||
| 21 | - The `User` table is effectively dead code under `ProxyHeaderAuth` |
|||||||
| 22 | ||||||||
|
23 | ### What the resolver swaps today |
||||||
| 24 | ||||||||
| 25 | | Component | Swapped? | How | |
|||||||
| 26 | |-----------|----------|-----| |
|||||||
| 27 | | GitStorage | Yes | Monkey-patches `storage` across 7 modules | |
|||||||
| 28 | | GitHttpServer | Yes | Recreated per-request | |
|||||||
| 29 | | `app.config["REPOSITORY"]` | Yes | Direct assignment | |
|||||||
| 30 | | SQLAlchemy `db` | **No** | All wikis share one DB | |
|||||||
| 31 | | `app_renderer` | No | Created once at import time | |
|||||||
| 32 | | Flask-Mail | No | Never swapped | |
|||||||
| 33 | | `app.config` (other keys) | No | SITE_NAME, permissions, etc. are global | |
|||||||
| 34 | ||||||||
| 35 | ## Option A: Per-Request DB Swap (implementing now) |
|||||||
| 36 | ||||||||
| 37 | ### Storage layout |
|||||||
| 38 | ||||||||
| 39 | ``` |
|||||||
| 40 | /srv/data/ |
|||||||
| 41 | ├── robot.db (platform: users, acls, wikis, oauth) |
|||||||
| 42 | └── wikis/{slug}/ |
|||||||
| 43 | ├── repo/ (git repo — already exists) |
|||||||
| 44 | └── wiki.db (otterwiki: preferences, drafts, user, cache) |
|||||||
| 45 | ``` |
|||||||
| 46 | ||||||||
| 47 | ### Mechanism |
|||||||
| 48 | ||||||||
| 49 | 1. Resolver calls `_swap_database(wiki_slug)` after `_swap_storage(repo_path)` |
|||||||
| 50 | 2. Updates `app.config["SQLALCHEMY_DATABASE_URI"]` to `sqlite:////srv/data/wikis/{slug}/wiki.db` |
|||||||
| 51 | 3. Calls `db.engine.dispose()` to close connections to previous DB |
|||||||
| 52 | 4. Lazy-creates `wiki.db` with schema if it doesn't exist (first request) |
|||||||
| 53 | 5. Calls `update_app_config()` to reload Preferences into `app.config` |
|||||||
| 54 | ||||||||
| 55 | ### Schema initialization |
|||||||
| 56 | ||||||||
| 57 | Raw SQL in management API (avoids importing Otterwiki's server.py). Creates 4 tables matching `otterwiki/models.py`: `preferences`, `drafts`, `user`, `cache`. Seeds `SITE_NAME`. |
|||||||
| 58 | ||||||||
| 59 | ### Permission model |
|||||||
| 60 | ||||||||
| 61 | **Platform ACLs remain authoritative.** ProxyHeaderAuth trusts proxy headers, ignores READ_ACCESS/WRITE_ACCESS. No change to auth flow. Per-wiki Preferences used only for wiki appearance/behavior settings. |
|||||||
| 62 | ||||||||
| 63 | ### DID-as-email |
|||||||
| 64 | ||||||||
| 65 | Already handled — resolver injects `@handle` in the email header field. No User table rows needed under ProxyHeaderAuth. |
|||||||
| 66 | ||||||||
| 67 | ### Migration |
|||||||
| 68 | ||||||||
| 69 | Lazy: if `wiki.db` doesn't exist when resolver hits it, create and seed on the spot. No data migration from shared default DB. |
|||||||
| 70 | ||||||||
| 71 | ### Risks |
|||||||
| 72 | ||||||||
| 73 | - **Flask-SQLAlchemy engine caching**: `dispose()` + config update should work with FSA 3.x lazy engine creation. Fallback: access `db._engines` internals. |
|||||||
| 74 | - **Worker model**: Assumes gunicorn sync workers. Async/threaded workers would race on global state. |
|||||||
| 75 | - **`update_app_config()` side effects**: Recreates Flask-Mail per-call. Harmless. |
|||||||
| 76 | - **Renderer**: Holds reference to `app.config` dict, so Preferences updates propagate. But `RENDERER_HTML_WHITELIST` parsed at init time won't update per-wiki. |
|||||||
|
77 | |||||||
|
78 | ## Option B: Upstream Contextvars Refactor (future PR to redimp) |
||||||
|
79 | |||||||
|
80 | ### Core idea |
||||||
|
81 | |||||||
|
82 | New `otterwiki/context.py` provides getter functions backed by `contextvars`: |
||||||
| 83 | - `get_storage()`, `get_db()`, `get_renderer()`, `get_mail()`, `get_githttpserver()` |
|||||||
| 84 | - Each falls back to the current module-level singleton when no context override is set |
|||||||
| 85 | - `wiki_context()` context manager sets overrides for a block |
|||||||
|
86 | |||||||
|
87 | ### What stays the same |
||||||
|
88 | |||||||
|
89 | - `app` (Flask) stays as module-level singleton — Flask has its own `current_app` |
||||||
| 90 | - `db` stays importable for model definitions (`db.Model`, `db.Column`) |
|||||||
| 91 | - All `@app.route` decorators unchanged |
|||||||
| 92 | - `models.py` unchanged |
|||||||
|
93 | |||||||
|
94 | ### Migration path |
||||||
|
95 | |||||||
|
96 | ~280 mechanical substitutions across 14 files: |
||||||
| 97 | - `storage` → `get_storage()` |
|||||||
| 98 | - `app.config[...]` → `current_app.config[...]` |
|||||||
| 99 | - `db.session.X()` → `get_db().session.X()` |
|||||||
| 100 | - One lazy-init fix for `_serializer` in `helper.py` |
|||||||
|
101 | |||||||
|
102 | ### Phasing |
||||||
|
103 | |||||||
|
104 | **PR 1** (small): Add `context.py`, `init_defaults()` call, migrate 2-3 leaf modules. Purely additive. |
||||||
| 105 | **PR 2** (wide, mechanical): Migrate remaining modules. |
|||||||
|
106 | |||||||
|
107 | ### Upstream pitch |
||||||
|
108 | |||||||
|
109 | - Testability: inject mocks via `wiki_context()` without real git repos or SQLite |
||||||
| 110 | - Flask best practice: `current_app` over direct `app` import |
|||||||
| 111 | - Plugin flexibility: temporary renderer/storage overrides |
|||||||
| 112 | - Backward compatible: getters fall back to singletons |
|||||||
|
113 | |||||||
|
114 | ### Constraint |
||||||
|
115 | |||||||
|
116 | `models.py` uses `db.Model` at class definition time — `db` can never fully move behind a getter. The contextvar is for session operations only. |
||||||
|
117 | |||||||
|
118 | ## SQLite Multi-Tenant: Why It's Fine |
||||||
|
119 | |||||||
|
120 | Per-tenant SQLite is a well-established pattern (37signals, Turso, Laravel Tenancy, Rails Shardines). For our use case: |
||||||
|
121 | |||||||
|
122 | - Small per-tenant data (prefs, drafts, accounts) |
||||||
| 123 | - Read-heavy workload |
|||||||
| 124 | - Single VPS, tens-to-hundreds of tenants |
|||||||
| 125 | - Already using SQLite with per-wiki git repos |
|||||||
|
126 | |||||||
|
127 | Postgres would add a daemon, configuration, and maintenance for no benefit. The pattern breaks down at tens of thousands of tenants — not a concern here. |
||||||
|
128 | |||||||
|
129 | Operational concern is **schema migrations** — lazy migration on connection open (`PRAGMA user_version`) is the simplest approach. |
||||||
