Infrastructure Services

Executive Recommendation

Part A — Durable Jobs

A1. Decision

Recommendation: pg-boss on the existing PostgreSQL cluster. Use PostgreSQL as both the transaction boundary and the durable execution queue. A domain transaction writes the mutation and canonical outbox event. The dispatcher then inserts the pg-boss jobs for that event and marks their delivery rows dispatched in one PostgreSQL transaction; a direct domain-transaction enqueue is also valid for single-purpose commands that have no fan-out or audit value. The worker validates the payload with the same kind of Zod contract TRW used in defineTask, performs an idempotent effect, and records the effect receipt before acknowledging the job.

pg-boss uses PostgreSQL SKIP LOCKED, supports retries, priorities, cron, delayed jobs, queue policies, job dependencies, and dead-letter behavior. Its current package also includes ORM transaction adapters and an optional dashboard and proxy. The project currently documents Node 22.12 or newer, so Bun worker compatibility is a required spike rather than an assumption.

A2. What the Queue Must Do

The serf successor needs:

• a typed task registry with versioned, schema-validated payloads;
• immediate, delayed, scheduled, and recurring jobs;
• retries with bounded exponential backoff and explicit terminal failure;
• queue-wide and task-specific concurrency limits;
• deterministic idempotency keys;
• graceful worker shutdown and safe lease recovery;
• inspection, replay, cancellation, and dead-letter operations;
• metrics for wait time, execution time, retries, failures, and stale jobs;
• atomic enqueue with the domain transaction where correctness requires it;
• Bun compatibility, or a deliberately isolated Node/Go worker runtime;
• no non-Cloudflare production SaaS dependency.

These bounded jobs run as short, idempotent handlers over explicit PostgreSQL state machines.

A3. The Transactional Boundary

Same-database enqueue: outbox plus pg-boss

The preferred two-transaction sequence is:

1. the domain transaction updates domain rows and inserts one immutable outbox event;
2. a dispatcher claims that event;
3. in one second PostgreSQL transaction, it inserts one stable-ID pg-boss job per handler and records the corresponding delivery rows as dispatched;
4. a retry repeats that transaction without losing or multiplying a logical delivery.

pg-boss documents transaction adapters for Kysely, Prisma, and other common clients; its db option lets the enqueue participate in the caller's transaction. For a one-off command such as "expire this reservation at T," the domain transaction may insert the pg-boss job directly through the same adapter.

The rows have different responsibilities:

• the outbox is the durable fact that something happened, suitable for multiple consumers, audit, replay, Zero projections, search, notification fan-out, and integrations;
• the job row is an execution lease with attempts, scheduling, concurrency, and failure state.

Derive a stable pg-boss UUID from (outbox_event_id, handler_name) because pg-boss custom job IDs are UUIDs. The job payload should normally be { eventId, taskVersion }, not a second copy of a large mutable domain object. The worker reloads the authoritative state and refuses stale or unauthorized work.

Same-database enqueue: outbox plus Graphile Worker

Graphile Worker exposes graphile_worker.add_job(...) as a PostgreSQL function. Calling it inside the domain transaction gives the same atomicity without an application-level relay. It supports delayed execution, priorities, attempts, flags, named queues, and job keys. Its job-key modes are useful for debounce and replacement, but completed jobs are deleted and the key is not a permanent execution receipt. Under high contention there is also a documented race for a particular job key that callers must detect and retry.

A4. pg-boss Operating Requirements and Graphile Fallback

pg-boss strengths

• one existing durable dependency;
• atomic insertion using the caller's transaction;
• broad queue policy vocabulary: singleton, debounce, throttle, stateful, exclusive, and strict keyed-FIFO controls;
• cron, deferred jobs, retries, dependencies, priorities, and dead-letter patterns;
• current dashboard and proxy packages;
• typed JavaScript/TypeScript integration.

pg-boss risks

• official runtime requirements name Node, not Bun;
• queue load shares PostgreSQL I/O, vacuum, connections, and failure domain with product data;
• the team must set capacity thresholds rather than assuming PostgreSQL is infinite;
• its "exactly once" phrasing describes delivery/locking, not arbitrary external effects.

For correctness-sensitive per-entity serialization, prefer key_strict_fifo over a best-effort group-concurrency setting. Mitigations:

• use a dedicated schema, connection pool, metrics, and autovacuum review;
• keep payloads small and retention bounded;
• use separate worker deployments and resource limits;
• archive business audit data in the outbox, not completed queue rows;
• establish measured PostgreSQL/WAL/autovacuum capacity thresholds and isolate worker resource use.

Graphile Worker fallback

Graphile Worker is the documented fallback. It is mature, MIT-licensed, and PostgreSQL-native, using LISTEN/NOTIFY for wakeups and SKIP LOCKED for safe concurrent claims. It provides SQL enqueue and cron backfill after downtime. Named queues serialize jobs such as one reconciliation per stream and require bounded cardinality. Its OSS crash-recovery contract permits a crashed worker's active jobs to remain locked for up to four hours, which the fallback harness must account for.

A5. Operating Shape

• Package task contracts separately from worker implementations.
• Give every payload an explicit schema version and Zod validator.
• Use task names such as attachments.scan.v1, not implementation class names.
• Insert { eventId, taskVersion } in pg-boss inside the outbox-delivery transaction; use a direct domain transaction only for non-event commands.
• Use an effect-receipt unique key such as (event_id, effect_kind, destination).
• Use stable vendor idempotency keys wherever the destination supports them.
• Set attempts, backoff, timeout, retention, and concurrency per task definition.
• Quarantine terminal failures; do not silently discard them.
• Export queue depth, oldest-ready age, attempts, execution latency, and terminal failures.
• Run jobs in a separately scalable worker deployment self-hosted.
• Keep presence Redis ephemeral and independently configured.

Part B — Live Streams and Video

B1. Decision

Recommendation: Cloudflare Stream remains the video backbone. Use Stream Live's production RTMPS/SRT-to-HLS/DASH path for one-to-many broadcasts and automatic recording, Stream VOD for course lessons and message video attachments, and a LiveKit SFU self-hosted for interactive rooms. When an interactive room becomes a broadcast, a self-hosted LiveKit Egress deployment composites the room or stage and sends RTMPS to a Cloudflare Stream Live Input. The audience watches Stream; only speakers and active participants remain in LiveKit.

B2. Product Boundary Map

B3. Important 2026 Correction: WHIP is a Separate Beta Lane

• Production recorded broadcast: RTMPS or SRT → Stream → HLS/DASH → VOD
• Experimental sub-second broadcast: WHIP → Stream beta → WHEP, with no recording promise
• Interactive conversation: LiveKit, not Stream HLS

This distinction should appear in product and infrastructure schemas so a future beta upgrade does not silently change recording or entitlement guarantees.

B4. Interactive-Room Handoff

LiveKit remains responsible for rooms where participants need to hear and respond to each other in under roughly 500 ms: calls, office hours, stage speakers, classrooms, and green rooms. For a large event:

1. hosts and invited speakers join a LiveKit room;
2. the application selects a composite layout;
3. LiveKit RoomComposite Egress renders that layout;
4. Egress publishes RTMPS to the Stream Live Input;
5. Stream provides ABR delivery to the large view-only audience;
6. chat, reactions, entitlement, and viewer presence remain application features;
7. the resulting Stream recording becomes the replay asset.

LiveKit Egress officially supports RTMP/RTMPS and SRT stream outputs. Budget self-hosted infrastructure CPU for the headless-browser composite and encoder; it is not free SFU forwarding. LiveKit's self-hosting guidance starts Egress around four CPU cores and 4 GB per instance and defaults to one room-composite job per instance, so scale and alert it independently.

B5. Signed Playback and Entitlement

All authorization lives in the application database. Stream tokens are short-lived delivery capabilities, not the entitlement source of truth. The playback endpoint should:

1. authenticate the Better Auth session;
2. load course, group, purchase, moderation, and territorial entitlement;
3. select the Stream video or live input UID;
4. mint a short-lived token with the minimum claims needed;
5. return a player URL or HLS/DASH manifest URL;
6. avoid persisting that URL in Zero, messages, or durable API caches.

Set requireSignedURLs on private videos and live inputs. Current custom-token controls include: exp (documented maximum 24 hours), optional nbf, downloadable when downloads are explicitly allowed, up to five accessRules, and allowed origins as a separate defense-in-depth control.

B6. Webhooks, Reconciliation, and the Outbox

Cloudflare has two relevant webhook families:

• Stream Live Input webhooks with exact event types live_input.connected, live_input.disconnected, and live_input.errored;
• video lifecycle webhooks reporting upload/processing readiness and errors.

Cloudflare permits one account-level processing webhook subscription, so route it through one verified ingress and fan out internally through the outbox. Normalize provider payloads into internal events:

• stream.ingest_connected.v1
• stream.ingest_disconnected.v1
• stream.ingest_error.v1
• video.processing_ready.v1
• video.processing_failed.v1

Webhook receipt flow:

1. verify the exact raw body (processing webhooks use the Webhook-Signature HMAC, while Live Notifications can carry the configured cf-webhook-auth secret);
2. store the raw receipt hash and provider identifiers idempotently;
3. update video_assets or stream_sessions state;
4. insert an internal outbox event in the same PostgreSQL transaction;
5. let pg-boss perform captions, notifications, replay publication, and reconciliation;
6. return quickly and never perform fan-out inline.

Webhooks are the fast path, not the only truth. Run a scheduled reconciler for uploads stuck in processing, missed disconnects, and live inputs whose provider state disagrees with the database.

B7. Upload Paths

Course lessons and message video attachments should upload directly to Stream through a one-time Direct Creator Upload URL. The application reserves an internal asset ID first and stores the Stream UID against that row. Cloudflare requires tus for uploads larger than 200 MB and recommends it for unreliable connections; the application can cap duration in the reservation. The resumable protocol requires chunks of at least 5 MiB except for the final chunk.

Do not route video bytes through the self-hosted infrastructure app or through R2 merely to hand them to Stream later. That adds failure states and bandwidth without adding value. Stream does not return the exact source upload; it provides playable renditions and can produce an encoded downloadable MP4. Duplicate the original into R2 only if "download the exact uploaded file" is a real requirement.

B8. Playback on Web and React Native

Do not proxy or cache manifests in the application. Let Stream control segment URLs, token handling, and rendition changes.

B9. Limitations and Capacity Boundaries

Latency

Standard HLS is a broadcast path, not an interactive path. Design to a several-second glass-to-glass budget; a 3–10 second product budget is reasonable for LL-HLS validation. Cloudflare notes that a newly connected input may take up to roughly 30 seconds to become playable, which is startup time rather than steady-state glass-to-glass latency. The WebRTC beta offers sub-second playback but has the no-recording boundary described above.

Recording and file limits

• A continuous live recording is truncated after the first seven days.
• MP4 downloads for live recordings are limited to recordings shorter than four hours.
• The documented default maximum upload is 30 GB and the default concurrent upload/encoding queue is 120 videos; enterprise support can clarify or raise relevant limits.
• Stream currently produces adaptive H.264 playback up to 1080p.

Long events need deliberate segmentation and replay assembly UX.

Economics

Public Stream pricing is based on minutes stored and minutes delivered. The enterprise contract, not the public page, governs this company. The capacity model still needs: monthly uploaded and recorded minutes; retained minutes by course and creator cohort; live viewer-minutes at p50, p95, and headline-event scale; VOD viewer-minutes and completion rates; replay retention and deletion policy; player preload behavior; and negotiated storage, delivery, and commit tiers.

B10. Could Cloudflare Realtime Replace LiveKit?

The answer is possibly, but not yet by paper decision. Cloudflare's current Realtime suite has two materially different surfaces:

• the raw Realtime SFU, which exposes sessions and tracks but deliberately has no room abstraction or client SDK; the application owns signaling, membership, authorization, presence, and track discovery;
• RealtimeKit, a higher-level beta with meetings, participants, presets, stage concepts, prebuilt UI/SDKs, recording, chat, polls, breakouts, and transcription.

Run a LiveKit-vs-Cloudflare-Realtime spike only if the team actively considers the operational trade-off and has a concrete feasibility plan for the raw SFU's missing room abstraction. Do not decide by paper analysis alone.

Part C — Files Plane

C1. Decision

Recommendation: private R2 as canonical object storage with Cloudflare Images and Workers for transformation and delivery. The files plane has three concerns: durable canonical storage, privacy-controlled short-lived access, and efficient media transformation and delivery. Cloudflare R2 is the right canonical store because it has no egress fee, sits within the Cloudflare enterprise DPA, and has proven path from storage to Workers/Images.

C2. Private R2

All R2 buckets are private. No client ever receives a pre-signed R2 URL directly; all access goes through either:

• a Cloudflare Images delivery URL (for image/attachment display), or
• a short-lived Worker-signed URL (for binary downloads, video, audio, and documents).

The R2 r2_object_key column is server-only and excluded from the Zero publication. Never log, expose in errors, or return it in API responses. Use consistent key naming: {server_id}/{year}/{month}/{attachment_id}/{filename}.

Retention and deletion policy per R2 bucket:

• boja-media: permanent storage for course and live content;
• boja-attachments: retention-class-aware; hard-deleted after deleted_at + retention_days;
• boja-uploads-tmp: 48-hour TTL; reservation-gated; swept by lifecycle rule;
• boja-exports: 7-day TTL; user-initiated data exports.

C3. Cloudflare Images

Cloudflare Images handles upload, storage, and transformation for user avatars, server icons, banners, thumbnails, attachment image previews, and emoji. Key product facts:

• images are stored inside Cloudflare Images, not in R2 (separate store);
• image variants (size, quality, format) are declared on the account and named for the URL;
• delivery URLs are public and content-addressed by default; use signed delivery URLs for private content;
• the Direct Creator Upload path gives a one-time URL; PAN/CVV/PHI must never be in image metadata.

Declared variants for v1: avatar_32, avatar_64, avatar_128, avatar_256, icon_32, icon_64, icon_128, banner_640, banner_1280, thumbnail_200, thumbnail_400. Avoid dynamic transforms on hot paths; lock transforms to declared variants.

C4. File State Machine

Every attachments row progresses through a typed lifecycle:

reserved
  -> upload_pending   (client has a signed upload URL)
  -> uploaded         (R2/Images confirmed receipt)
  -> processing       (scan, transform, transcode workers running)
  -> ready            (all required variants confirmed)
  -> failed           (terminal; manual review)
  -> deleted          (soft-delete; blobs eligible for sweep)

Transitions are driven by webhook receipts from R2/Images or by self-hosted processing workers, both of which write into the same outbox. The controlling invariant: a message cannot display an attachment that has not reached ready. Workers that scan for malware or content policy may insert an intermediate quarantine state before ready.

C5. Upload Flow

All attachment uploads (non-video) use a reservation-first pattern:

1. client calls POST /api/v1/attachments/reserve with media type, byte size, and checksum;
2. API validates actor capability, quota, rate limit, and type policy;
3. API inserts attachments row in state reserved and returns a short-lived signed upload URL;
4. client PUTs the file directly to the signed URL (R2 or Cloudflare Images direct upload);
5. the provider confirms receipt via webhook or polling; the row advances to uploaded;
6. processing workers run scan and transform jobs; the row advances to ready or failed;
7. the client includes the attachment_id in the message send command.

The server must refuse to send a message that references an attachment not in ready state for the same actor. Upload URLs are single-use and expire in 15 minutes. Preserve local file references (device URIs) in the outbox until the attachment is confirmed ready; do not assume a successful upload before the webhook confirms it.

C6. Serving, Access Control, and Sizing

Image attachments serve through Cloudflare Images delivery URLs. Documents, audio, and non-image files serve through short-lived signed download URLs generated by a Worker at request time. These URLs are never stored in the database, messages, notifications, or Zero rows. They are resolved on the client at render/download time through a narrow GET /api/v1/attachments/{id}/url endpoint that checks current authorization and moderation state.

Sizing quotas enforced at reservation:

• image attachment: 50 MB maximum;
• document attachment: 100 MB maximum;
• audio attachment: 500 MB maximum;
• video attachment: 2 GB maximum (tus resumable required above 200 MB).

Per-server and per-user daily upload quotas are enforced through atomic counter rows on the attachments table, reset by a scheduled job. Enterprise server quotas are configurable through the admin API.