TRW Chat System — Evidence for Boja

Source: /Users/raynor/dev/trw (Bun/TypeScript monorepo). Investigated 2026-07-16. Primary references: ARCHITECTURE.md, docs/CHAT-CONTEXT-PACK.md, app/AGENTS.md, plus direct code reads cited inline.

0. System Topology

Chat spans six runtimes:

Key architectural invariant: write handlers never emit realtime events directly — writes go through models/context/* helpers, and fanout is either explicit pubsub from serf workers (chat/EventV1) or derived from Mongo change streams in sink (EventV2). Chat uses EventV1 only; EventV2 is job-portal/video-calls.

The frontend chat data plane is a four-layer sandwich: TanStack Query infinite cache → MobX entity maps (appClient) → Jotai mirror (app/src/utils/atomWithQueryData.ts) → derived Jotai selectors (messagesState.ts) → an imperative runtime (chatRuntime.ts). This layering is the single biggest source of complexity.

1. Message Data Model

Collections (models/)

• messages (models/Message.ts): _id (ULID), channel, author, sort_id (ULID, the pagination key), content, attachments, embeds, mentions, role_mentions, replies, links, timestamp, edited, reaction_counts (denormalized Map), poll, actionBlock, system, nonce (client idempotency key).
• message_reactions (models/MessageReaction.ts): composite _id: { emoji, user, message }; indexes on _id.message + _id.user and _id.message + _id.emoji. Counts are denormalized onto message.reaction_counts and reconciled via MessageReactionCount delta events.
• channels (models/Channel.ts): union of 5 types — SavedMessages, DirectMessage, Group, TextChannel, JobWorkspace. Channels denormalize last_message_id, last_message_author, first_message_id, active — updated by serf/tasks/messagePublishTask.ts after each send.
• user_unreads (models/UserUnreads.ts): read-state store, sharded (see §5).
• attachments (models/Attachment.ts): separate collection; messages reference them, attachment.message_id back-links.

Indexes

• messages: { channel: 1 } — backs every channel-scoped query. Without this index they all collection-scan. Prod indexes: channel + sort_id, channel + timestamp, author/channel composites.
• Notably commented out in models/Message.ts:222-229: a unique partial index { channel: 1, nonce: 1 } for send idempotency — dedupe is currently enforced client-side + soft server-side, not by a DB constraint. A rewrite should make this a real unique index.

Denormalization inventory

2. Infinite-Scroll Pagination

Server side (eden/src/routers/messagesRouter.ts → models/Message.ts queryMessages)

Single bulk endpoint POST /messages/query supports: before, after, before_sorted, after_sorted, nearby, ids, sort (Latest/Oldest), limit (capped 100), throw_on_not_found, has_attachment_types, has_link.

• Cursors are sort_id (ULID) range scans: filter.sort_id = { $lt: beforeCursor }. Clients may pass either a message _id (resolved to its sort_id via resolveSortCursor) or the sort_id directly (*_sorted params).
• nearby: fetches the anchor message, then runs two parallel range queries (sort_id < anchor and sort_id > anchor) and stitches [...before, anchor, ...after]. If the anchor doesn't exist (deleted message, or synthetic ULID from an unread ack), it falls back to raw sort_id bounds — this is what makes "jump to last-read" work with an ack cursor that was never a real message id.
• Response (MessagesResponse) is denormalized: messages, users, members, related_messages (reply parents), my_reactions, related_threads, my_thread_memberships — one round trip hydrates everything the UI needs.

Client side (app/src/modules/Chat/ChatMessageList/messagesApi.ts)

TanStack useInfiniteQuery with a typed page-param discriminated union MessageFetchAction = {type:"top"|"bottom"|"nearby", target?, target_sorted?}:

• queryKey: [MESSAGES_QUERY_KEY_PREFIX, channelId]; staleTime: Infinity, gcTime: 60min.
• Page sizes (LIST_PAGE_SIZES): initial=60, top=20, bottom=15, nearby=60 desktop / 30 touch.
• Window cap + trim: LIST_SIZE_LIMIT = 150 desktop / 100 touch; after every fetch and realtime append, trimChannelMessages evicts pages from the opposite side. So the in-memory window slides bidirectionally.
• presentTailMessageId is stored on the InfiniteData itself — the newest message id known to be "present". getRealtimeAppendPlan uses it to decide whether a WS message may append to the cache (only when the loaded window actually reaches present) vs. be dropped (user scrolled deep into history). This prevents realtime appends from teleporting messages into a historical window.

Bidirectional scroll + anchoring (ChatV4.tsx)

• Virtualizer: virtua <VList reverse shift={isPrependRef.current} overscan={20}>. reverse mode; on top-prepend, isPrependRef flips true for one render so virtua uses shift (offset compensation).
• Settle loop (ChatV3/settleAtBottom.ts): rAF loop keeps calling scrollTo(bottom) until scrollSize is stable for 3 frames and within BOTTOM_THRESHOLD_PX = 5, with a 1.5s safety timeout.
• Follow-present hysteresis (resolveFollowPresentOnScroll.ts): content growing below a bottom-pinned list would un-pin and show a "See present" bar even though the user never scrolled. Fixed with a drift buffer + "if offset didn't move up, re-pin" heuristic.

3. Replies and Threading

Flat replies (Discord-style, primary)

• message.replies: string[] — but only the first reply is processed. So in practice it's a single reply_to pointer stored as an array (schema legacy).
• Reply parents come back as related_messages in query responses so MessageReplyBar can render inline previews without extra fetches.
• Full chain: GET /messages/:messageId/ancestry returns ancestors + children.

Threads (newer, channel-backed)

• Backend on eden/src/routers/channelRouter.ts: POST /:channelId/messages/:messageId/thread, GET /:channelId/threads.
• Threads are channels (query responses include related_threads: Channel[] and my_thread_memberships: ThreadMember[]).

4. Jump-to-Message (Permalinks, Old History, Window Re-establishment)

• Routes: permalink routes just redirect to the base route with ?messageId=.
• Entry plan (chatRouteEntryPlan.ts): route loader computes ChatEntryPlan with 3 intents — selected-message (permalink → nearby fetch on messageId), last-read (nearby fetch on lastReadId), latest (top fetch, land at bottom).
• Bootstrap: useChatRouteBootstrap.ts writes chatEntryPlanAtomFamily(channelId) + selectedMessageIdByChannelAtom(channelId) into Jotai.
• Jump while mounted (ChatViewRuntime.jumpToMessage):
  • Target already in loaded slice: look up index in messageItemIndexMapAtomFamily, scroll across several frames, flash highlight.
  • Target not loaded: fetch nearby → reset Jotai state + destroy query cache → seed cache with the new slice. The context pack is blunt: "This is why permalink jumping outside the current slice feels like a local reload — it is one".
• Return to present (restartAtPresent): if the slice already reaches channel.last_message_id, just settle to bottom; otherwise phase → recovering-present, cancel in-flight queries, fresh top fetch, replace pages, settle.

5. Unread Tracking / Read States

Storage: models/UserUnreads.ts — sharded

• Per-user read state: channels: Record<channelId, { last_id, mentions[] }>.
• Sharded across 8 docs (NUM_SHARDS = 8, doc _id = ${userId}_${shardIndex}), shard chosen by hashing the last 4 chars of channelId. Feature-flagged, legacy single-doc auto-migrates lazily on write. Reason: a single unbounded channels map per user hit Mongo document-size/write-contention limits.

Write path

• Acks: POST /:channelId/ack/:lastId, POST /:channelId/ackMention/:mentionId, POST /:serverId/ack (ack-all).
• All unread mutations funnel through serf/tasks/unreadsTask.ts — deduped/batched via Redis (10k max, 500 min, 5s timeout, grouped per channel, concurrency 1).
• Mentions/DM messages add unreads in notifyMessageTask per-recipient.

Read path / client

• Client seeds unreads at bootstrap (Ready packet), listens to UserUnreads events, keeps appClient.unreads in MobX.
• Unread rendering is derived: UnreadIndicator item inserted by mapMessagesListItems before the first unread incoming message.
• "Mark unread" in the message popup menu writes a synthetic ack cursor — which is why the server's nearby-fallback for non-message ULIDs exists.

6. Frontend Resumability & Caching

Caching layers

1. TanStack Query infinite cache per channel (staleTime: Infinity, gcTime: 60min) — messages never go stale by time.
2. MobX entity maps (app/src/client.js/maps/Messages.ts, Users.ts, Channels.ts, Members.ts) — the legacy entity cache shared with the websocket layer. Every fetched page is hydrated into MobX and stored in TanStack pages (dual write).
3. Jotai mirror (app/src/utils/atomWithQueryData.ts) subscribes to TanStack cache events and mirrors into messagesAtomFamily → derived messageItemsAtomFamily (renderable items).
4. Drafts persisted to IndexedDB via persistStore (chatState.ts messageDraftByChannelFamily — migrated out of localStorage for space).

React <Activity/> offscreen rendering

• app/src/modules/Chat/ChatChannelActivityPool.tsx renders every recently-visited channel inside <Activity mode={id === selectedChannelId ? "visible" : "hidden"}>. Pool: LRU with maxEntriesPerGroup: 30, maxEntriesTotal: 60, maxAgeMs: 30min.
• Channel switching is therefore mostly an Activity visibility flip, not a remount.

Runtime suspension (channel switch)

• One ChatViewRuntime per channel (getOrCreateChatRuntime, chatRuntime.ts) that survives unmount as suspended once initialized.
• ChatV4 resumes it if hasInitializedOnce && selected message (if any) is still in the loaded slice; otherwise a full reset: resetAllMessagesState(channelId) + messagesApi.destroyMessagesCache(channelId) + runtime.resetForFreshLoad().

Reconnect / tab switch

• Cursors event_v1_last_seq / event_v2_last_seq persisted in localStorage; on reconnect the client sends the cursor and nile replays from a Redis stream; expired cursor → connectFresh().
• On resync (replay window missed): all hidden pooled channels are evicted and the active channel emits restart_at_present. Major query keys invalidate.
• Boot watchdog: 5s timer; first timeout destroys cache + retries, second → error phase.

7. Optimistic Sends, Retries, Dedupe

Send flow (app/src/modules/Chat/ChatFooter/useSendMessage.ts + chatOutbox/):

1. Draft merged; client generates _id = clockDriftedUlid() and a separate nonce ULID.
2. Optimistic message created via appClient.messages.createObj and appended to the query cache immediately; uploads awaited first.
3. Entry pushed into the outbox (chatState.ts: outboxAtom), processed by chatOutbox/outboxProcessor.ts:
   • In-flight set prevents a WS disconnect/reconnect cycle from firing a duplicate HTTP send for the same optimistic id.
   • Offline pause: entries move to paused when WS is disconnected, rescheduled on connected.
   • Error classification (outboxErrorClassifier.ts): retryable = network failure / timeout / 502-503-504 → infinite retry with exponential backoff; terminal = SlowMode, TimedOut, 4xx/429, and unknown errors default terminal ("avoids infinite loops on logic bugs") → failed state + toast.
   • Retry id refresh: on retry, the optimistic message is re-registered under a fresh ULID so its timestamp stays current — display-only, because "idempotency on the server is protected by nonce."
4. Server: POST /:channelId/messages → messagesContext.sendMessage (validation, cooldown, anti-spam 30s slowmode >60msg/min, attachments, mentions caps) → insert → messagePublishTask → MessageWithUser fanout.
5. Dedupe on echo: messagesApi.appendChannelMessage matches by nonce first, _id second and replaces the optimistic entry in-place rather than double-appending.

Caveat: the server-side channel + nonce unique index is commented out, so true server-side idempotency depends on soft checks; a rapid retry racing a slow success could theoretically double-insert.

8. Attachments / Media

• Separate service ark/: multipart upload → validation/tag constraints → S3-compatible storage. Attachment metadata: _id, tag, filename, content_type, size, metadata (Audio/Video/Image/File/Text), message_id?, deleted?, saved_sizes[].
• Upload-before-send: UploadDrafts.tsx queue with per-file progress; useSendMessage awaits all uploads, then sends message referencing attachment ids. Limits: 5 per message (100 in JobWorkspace).
• Post-processing in serf: attachmentPlaceholderTask (blurhash), attachmentResizeTask (per-maxSide cached in saved_sizes), videoThumbnailBackfillTask, attachmentCleanupTask (orphans).
• Rendering: Attachment/Attachment.tsx dispatcher (Image with blurhash placeholder + fullscreen, Video with server thumbnail, Audio inline player, Text preview, File download, Spoiler overlay), message-level AttachmentGrid.tsx via optimizeGridLayout.ts.
• Link embeds are separate from attachments: processEmbedsTask unfurls links post-send (idempotency key process-embeds-${messageId}-${contentHash}; replaces rather than appends embeds to prevent accumulation) and fans out MessageAppend.

9. Known Pain Points

1. Five-layer state plumbing. TanStack Query pages + MobX entity maps + Jotai mirror + derived atom families + imperative ChatViewRuntime. The context pack warns: "Cache reset and atom reset must stay aligned... Always pair messagesApi.destroyMessagesCache + resetAllMessagesState". MobX is officially a legacy boundary no new feature may extend.
2. Jump = local reload. Permalink outside the loaded slice destroys and rebuilds the entire window. Called "Expensive but correct" internally.
3. Scroll jank management is a subsystem of its own. settleAtBottom.ts rAF loop (3-stable-frame heuristic + 1.5s safety timeout), isPrependRef/shift dance, BOTTOM_THRESHOLD_PX=5, POST_PRESENT_SCROLL_DRIFT_BUFFER_PX hysteresis, and the APP-3067 fix in resolveFollowPresentOnScroll.ts.
4. Boot watchdog as loading-screen band-aid. 5s timeout → destroy cache and retry → second timeout → error phase. The loading curtain needs an explicit shouldBypassResumeLoading escape hatch.
5. Resync = refetch storm. On missed replay window: evict all pooled hidden channels + restart_at_present + invalidate "major query keys". Every warm channel becomes a cold boot at once.
6. Missing DB idempotency index — channel + nonce unique index commented out.
7. Read-path repairs: NaN-timestamp repair on every read; replies[] array where only element 0 counts; legacy ChannelUnread beside sharded UserUnreads.
8. 60s Redis hydration cache for users/members in message responses can serve stale author data after deployments.
9. DOM node contention — hover focus, popup targeting, swipe transforms, reply-arrow portal, and highlight styling all target the same message row DOM; changing DOM structure in ChatMessage.tsx can break multiple interaction subsystems at once.
10. Multi-hop realtime chain fragility — models write → serf/sink → Redis pubsub → nile → WS → client; a failure in any link breaks live updates without necessarily failing the originating request.
11. Unreads eventual consistency + shard hack — batching pipeline exists purely to survive write volume of a chatty schema.
12. Assorted in-code TODOs: menu-flicker setTimeout(..., 1) hack, embed addon needing "a better way to get server & channel", kick error swallowed in finally.
13. In-memory outbox — queued/failed sends don't survive refresh (drafts do, via IndexedDB).

10. What to Keep / What to Avoid in a Rewrite

Keep (proven, well-shaped ideas)

• Client-generated ULID identity plus a separate authoritative sort_id cursor; preserve before/after/nearby, parallel two-sided nearby stitching, and the synthetic-cursor fallback for deleted/never-existed anchors. Identity and ordering remain conceptually separate even when both encodings are lexically sortable.
• Short-page boundary detection + first_message_id/last_message_id channel pointers for pagination termination and present detection.
• presentTailMessageId / append-plan gating — realtime appends only land when the loaded window reaches present. A sync engine needs the same concept (live tail vs historical window).
• The outbox: per-entry state machine, retryable/terminal error classification (with unknown-defaults-terminal), exponential backoff, offline pause, nonce-based dedupe, in-flight guard. Make it durable and DB-enforced.
• <Activity/> channel pool + suspended per-channel runtimes — instant channel switching. In a sync-engine world this becomes even cheaper because state lives in the local store, not component trees.
• Denormalized single-round-trip page responses (messages + users + members + reply parents + threads + my_reactions) — as a wire format; normalize into the local store on arrival.
• Threads as channels — reuses pagination, permissions, unreads.
• Attachment pipeline: upload-before-send with progress, entity-per-attachment, blurhash, server-side variants.
• Write-path discipline: all writes through models/context/*; fanout owned by one pipeline. Retain EventV2/CDC lessons for server-side search, audit, notifications and jobs.
• Entry-plan concept: explicit latest | last-read | selected-message landing intents computed at route time.

Avoid (the scar tissue)

• Layered dual/triple state stores. No MobX-entity-map + query-cache + atom-mirror sandwich.
• The query cache as the message window. Storing the visible window as TanStack infinite pages forces destructive resets on jump, paired cache/atom resets, trim bookkeeping, and append-plan gymnastics.
• Replay-or-nuke reconnects. Let Zero reconcile authoritative row changes; do not build a second per-channel version/checkpoint protocol.
• Imperative phase machine + event emitter as the control plane (chatRuntime.ts token cancellation, chatEmitter, watchdog).
• Reverse-virtualization fights: budget for scroll anchoring as a first-class feature (height reservation from attachment/embed metadata, anchor-based scroll restoration).
• Async batched read-state pipeline (Redis batch → BullMQ → sharded Mongo docs → pubsub).
• Client-only idempotency — enforce (channel, nonce) uniqueness in the DB from day one.
• Post-hoc mutation events that grow content (MessageAppend embeds) without reserved layout.
• Read-path repair code as permanent fixtures (NaN timestamps, replies[0], legacy unread model).
• Six-runtime fanout for a single message send. Fewer hops = fewer partial-failure modes; a sync engine collapses "write" and "notify" into one log.

Files most worth reading before designing the replacement: docs/CHAT-CONTEXT-PACK.md (1,434 lines, excellent), app/src/modules/Chat/ChatMessageList/messagesApi.ts, app/src/modules/Chat/ChatV3/chatRuntime.ts, app/src/modules/Chat/chatOutbox/outboxProcessor.ts, models/Message.ts, models/UserUnreads.ts, app/src/modules/Chat/ChatChannelActivityPool.tsx.