Infrastructure

What runs, why it runs, and what it would take to remove it. Local is Docker Compose; production is Kubernetes with the same images.

The pieces

The MinIO image (#308)

No public registry serves the MinIO image prod runs (quay.io/minio/minio:RELEASE.2025-09-07T16-13-09Z): Docker Hub answers 401 and quay.io refuses anonymous pulls. The prod droplet's cached copy was copied, layer for layer, by .github/workflows/mirror-minio.yml to:

ghcr.io/verjson/marketing-studio/minio@sha256:a1a8bd4ac40ad7881a245bab97323e18f971e4d4cba2c2007ec1bedd21cbaba2

The package is private and owned by this repository, like marketing-studio/api, so a droplet logged in to GHCR can pull it. docker-compose.prod.yml reads MINIO_IMAGE. A droplet with no cached copy (nonprod, a rebuilt prod) sets it to the reference above. Unset, it falls back to prod's existing image, so prod's MinIO is never recreated by this change.

The Google Cloud copy of each candidate (#381)

The org's release workflow will only promote a candidate that also exists in Google Artifact Registry (container-release.yml refuses a candidate without exactly one GAR destination). So every candidate is also copied, digest for digest, to:

us-central1-docker.pkg.dev/verjson-ci-640463/marketing-studio-candidates/marketing-studio-candidate-{api,web}

It lives in verjson-ci's Google Cloud project, next to verjson-ci's own container-canary repository, and is set up the same narrow way:

The trust condition names the contract SHA. When the generated files move to a new Verjson/.github pin, the publish fails at Google sign-in until the condition names the new SHA too. Update it in the same change, with gcloud iam workload-identity-pools providers update-oidc marketing-studio-main --workload-identity-pool=github-oci-candidates --location=global --project=verjson-ci-640463 --attribute-condition=... (swap the SHA in the current condition, read with describe).

Two kinds of "background work", and why both exist

They are not competing choices — they solve different problems, and using one for the other's job goes badly in a predictable way.

RabbitMQ because nearly every consumer here talks to a rate-limited third-party API. Per-queue prefetch, fair dispatch and a native dead-letter path are exactly the primitives that need — all of which a polling table has to reimplement, usually badly. See D-014.

trigger.dev for the agentic layer. An agent run is long, multi-step, and has to survive a deploy and a wait-for-human — which is precisely what a durable workflow engine is for and what a queue is not. See D-016.

Postgres keeps the rate limiter (a fixed-window counter, not a queue) and every read-model.

Queue topology

verjson.publish        → publish a post to one channel      (per-channel queues, prefetch 1)
verjson.metrics        → pull metrics for one account/day
verjson.tokens         → refresh OAuth tokens expiring soon
verjson.scrape         → fetch and parse one URL            (politeness delay per host)
verjson.whatsapp       → send one approved template
verjson.dlq            → anything that exhausted its retries

Each queue is bound to a direct exchange by routing key. Retries use a delayed retry queue rather than an in-process sleep, so a redeploy does not lose the backoff.

WhatsApp: Baileys

Baileys speaks the WhatsApp Web protocol from a persistent socket, so it runs as its own container with its own volume — a long-lived paired session cannot share a lifecycle with stateless API pods that get rescheduled. The volume holds the pairing; losing it means re-pairing by QR code.

The honest trade-off. Baileys is an unofficial, reverse-engineered client. Bulk marketing over it is against WhatsApp's terms, and the realistic downside is the number being banned — not a rate-limit, a ban. It is the pragmatic choice for an internal tool and for conversational replies; it is the wrong choice for a large cold broadcast.

The mitigation is structural, not procedural: the bridge sits behind the same ChannelProvider interface as every other channel, so the official WhatsApp Business Cloud API is a transport swap rather than a rewrite. If volume grows or a ban lands, changing one adapter moves the whole feature across. See D-015.

Operationally: opt-in is enforced before anything is queued, sends are paced, and template content still passes the approval gate.

The bridge's surface

services/whatsapp — a standalone ESM Node 22 service, outside the npm workspace so it can hold a stale Baileys pin without dragging the API's dependency tree along with it. Baileys is pinned to 6.7.24: npm's latest tag is a 7.0.0 release candidate, and legacy is the last stable line.

Every route but /health requires x-bridge-token, compared in constant time. /health is open because it is the container's HEALTHCHECK and the StatefulSet's probe, and it reveals only that the process is up.

Outbound, the bridge POSTs to WEBHOOK_URL (default http://api:4000/api/v1/whatsapp/events) with the same x-bridge-token. Payloads are { event, occurred_at, data } and snake_case, because that is what the API validates inbound; the bridge's own REST responses follow the table above instead.

Delivery is fire-and-forget. A webhook failure is logged and dropped: it must never take the socket down or fail a caller's send, because the API is the bridge's audience, not its dependency. Durability for anything that matters belongs on the API side of that hop.

A dropped socket reconnects with exponential backoff to 30 s. A close carrying loggedOut (the number was un-linked from the phone) clears the auth state instead of looping on dead credentials, so the bridge lands back at "show a QR".

Scrapers

Scraping is a queue consumer like any other (verjson.scrape), which means it inherits retries, dead-lettering and rate limiting for free rather than growing its own scheduler.

Rules that keep it defensible:

• Respect robots.txt and a per-host politeness delay. Enforced in the consumer, not left to each scraper.
• Identify honestly in the user agent, with a contact URL.
• Cache aggressively. A page fetched today is not fetched again tomorrow unless something asks for a refresh.
• Public pages only. Nothing behind a login, and nothing a robots.txt disallows.
• Personal data has a retention clock from the moment it lands — see the Apollo note in FEATURES.md (F-092).

Agent-Reach

Agent-Reach is a capability layer that gives agents unified read access across ~14 platforms (X, Reddit, LinkedIn, YouTube, GitHub, RSS, podcasts, web search) by routing to native CLI tools with ordered fallbacks, rather than wrapping them in a new abstraction.

Where it fits here: the research half of the growth-intelligence work (F-091, F-092, F-094) and the agentic layer (F-071). It is installable as an MCP server, which means our agents can reach it through the same MCP path they already use — no bespoke integration, and no scraper of our own for platforms it already covers.

Where it does not fit: anything that writes. It is a read and search layer; publishing stays with our own channel adapters, behind the approval gate.

Tracked as F-098.

Nonprod (#379, #383)

A second droplet, ms-nonprod (144.126.249.116, Pulumi stack nonprod), runs the same Compose stack at http://144.126.249.116.nip.io. It is deployed only by dispatching deploy nonprod (.github/workflows/deploy-nonprod.yml), which builds nothing: it rolls an existing candidate's signed digests through prod's own deploy/roll-vm.sh. Its GitHub environment nonprod holds its own SSH key and freshly generated secrets; none of prod's are shared. The database started empty; SEO providers run in mock mode. With no CLOUDFLARED_TOKEN, the cloudflared container restarts in a loop, which the health gate ignores; it stops once nonprod has a tunnel hostname.

Environment

Every value is listed in .env.example. The rule stands: server-side values are read only in apps/api/src/core/config.ts, and anything prefixed NEXT_PUBLIC_ is public and must never hold a secret.