Search Architecture
Meilisearch v1.48 self-hosted. Permission pre-filter (ACL snapshot at query time) plus mandatory authoritative post-recheck using the same capability calculator. Hybrid BM25 + vector search. Platform-wide synonym dictionary v1; per-server fast-follow. Smart Vault lesson/transcript retrieval.
Indexing & Query Pipeline
Two distinct paths: the write path indexes content with pre-filtered permissions; the read path enforces an authoritative Postgres recheck on every result set before returning data to the client.
Security Rule
A search engine hit is a candidate, never authorization evidence and never a client response. Every result is reloaded or version-checked against authoritative Postgres state, checked with the same capability-row calculator and entitlement rules as direct reads. No shortcut exists.
TRW Bug Lesson
TRW's /search/global permitted both server and channel scope to be omitted.
When omitted, channelsToSearch was empty and no permission filter was applied.
The subsequent listBulkMessages explicitly trusted its caller to pre-filter.
Boja must fail closed on an empty ACL snapshot. An empty or missing scope is not "search everything" — it is an error that must be rejected before any search query reaches Meilisearch.
Engine Comparison
| Feature | Meilisearch v1.48 | ParadeDB (challenger) | Typesense v30 (fallback) |
|---|---|---|---|
| BM25/lexical | Custom ranker (near-BM25) | True BM25 (Tantivy) | Custom |
| Vector/hybrid | Native | pgvector | Native |
| Native synonyms | Yes — per-index, runtime API | No — app-owned expansion only | Yes — synonym sets (v30) |
| Reindex on synonym change | Full reindex required | N/A (query-time expansion) | Not required |
| OSS HA/replication | Enterprise only | Postgres native | 3-node OSS |
| Operational complexity | Separate service | Co-located with Postgres | Separate service |
| Recommendation | V1 primary | Spike S7 | Fallback if HA needed |
Synonym Management Design
PostgreSQL is authoritative for synonyms. The platform-wide dictionary is managed through authenticated admin API operations. Synonym changes are auditable, versioned, and follow an explicit lifecycle.
synonym_sets (id, logical_key, version, scope, server_id, status, content_hash, ...)
synonym_entries (id, synonym_set_id, mode, canonical_term, terms[], locale, position)
-- scope: 'platform' | 'server'
-- mode: 'one_way' | 'multi_way'- One-way: jargon term maps to a canonical term (e.g., "ML" → "machine learning")
- Multi-way: equivalent terms in both directions (e.g., "sofa" ↔ "couch")
Every synonym publish triggers a full async reindex in Meilisearch. Treat synonym publishes as deployments: monitor progress, have a rollback plan. The lifecycle states are explicit: Draft → Validate → Publish → Retire → Rollback.
Hot vs Archive Search
| Source | Storage | Query path | Landing |
|---|---|---|---|
| Hot message | Zero-published Postgres (≤90 days) | around(channelId, messageId) |
Jump to message in Zero window |
| Archive message | Postgres archive table (>90 days) | Authorized read-only history API | Transient HistoryWindow — never a fake Zero row |
| Course lesson | Postgres (always available) | Meilisearch transcripts index | Open lesson page |
| Entity | Zero-synced | Local typeahead + API fallback | Navigate directly to user/channel |
Smart Vault — Lesson Retrieval
The Smart Vault indexes course transcripts for semantic search. Retrieval is lesson-level in V1; chunk-level timestamp jumps are a fast-follow.
Transcript Pipeline
- pg-boss worker scans for new/updated lessons
- Media download from R2 storage
- ffmpeg + Whisper transcription
- Embedding via
text-embedding-3-large(3072 dimensions) - Index into Meilisearch
transcriptsindex with hybrid BM25 + vector
Search results point to the lesson, not a specific chunk or timestamp within the video. Chunk-level jumps to an exact transcript segment are scoped to fast-follow after V1 launch.