Skip to article
HB CodeDocs
Get the app

HandbookContribute

Server, database roles, and operation

The account and relay service, the PostgreSQL role invariant, the configuration a development server needs, and what to verify beyond HTTP health.

Checked against the source on 5 October 2026

On this page

The server has account and relay responsibilities

The Actix application initializes shared services at startup: database pools, account authentication, connection management, and configured provider and push integrations. The DeviceConnectionManager is shared across HTTP workers in the process. Its live sockets, connection generations, and pending RPC records are process state, and the relay session store behind resume tokens is in memory too, with a 24-hour TTL and a 5-minute sweep. Every RPC and every heartbeat frame renews a session record. Redis is required for shared sign-in state, rate limiting, and credit reservations for paid requests, and the process exits at startup without it. It does not make a live socket registry portable between server processes.

Both devices and their pending request must reach that connection manager, so production runs one relay-serving process per region with explicit connection draining. The device routing chapter explains why a round-robin replica would break routing.

PostgreSQL roles are a startup invariant

The server exits at startup unless its two database logins are separate and confined by row-level security
The server exits at startup unless its two database logins are separate and confined by row-level securityInside PostgreSQL the migration role owns the tables, and 27 of them must have row-level security enabled and forced. The server connects through two logins, the system login hbcode and the tenant login authenticated. At startup it checks that neither login is a superuser, bypasses row-level security, inherits roles, or owns tables, that neither is the other or a member of it or of the table owner, and that the tenant login reads no rows from 16 tenant tables without a user ID. Any failure stops the process before it listens. Each server process opens its own system pool of 8 and tenant pool of 16 connections, shared by its HTTP workers.
PostgreSQL
Table ownerplantocode_migrator · SUPERUSER
27 tablesrow-level security enabled and forced
System loginhbcode
Tenant loginauthenticated
member of the owner
same login, or member of the other
reads 0 rows from 16 tables without a user ID
both: NOSUPERUSER · NOBYPASSRLS · NOINHERIT · own no tables
Each server process
system pool 8
tenant pool 16
shared by its HTTP workers
Any crossed line, a table without FORCE, or a visible row: create_dual_pools fails and the process exits before it listens.
Production also opens ACCOUNT_CONTROL_DATABASE_URL, with its own least-privilege check, and METRICS_DATABASE_URL.
  • Runtime loginsDATABASE_SYSTEM_URL · DATABASE_USER_URL

    assert_runtime_boundary checks the principals these URLs connect as, not the URLs. Before connecting, create_dual_pools also rejects two identical URLs.

  • Tenant contextapp.current_user_id

    Set with set_config for one transaction, so a pooled connection forgets it when the transaction ends. Row-level security policies written for each login decide what it may touch.

  • PoolsDB_POOL_SYSTEM_MAX · DB_POOL_USER_MAX

    Opened once per server process and shared by all its HTTP workers. Each extra process, including the second color during a blue-green cutover, opens its own.

create_dual_pools opens both pools and runs assert_runtime_boundary on the principals they actually connected as, so a URL that names the wrong role fails the same checks.

DefaultSystem poolTenant pool
Maximum connections816
Minimum connections24
application_namehbcode-systemhbcode-user
Statement timeout30 seconds8 seconds
Idle-in-transaction timeout60 seconds10 seconds
Lock timeout5 seconds2 seconds

Both pools use a 2-second acquire timeout, a 60-second idle timeout, and a maximum connection lifetime of 1,800 seconds. Every connection sets the time zone to UTC. A failed connect is tried up to five times, and the wait grows by one second with each attempt. Each server process opens its own pools, shared by its HTTP workers. More processes, including the second color of a blue-green cutover, multiply the connections and need a database capacity check first.

Configure a local server

  1. Copy server/.env.example to server/.env and configure a local database, Redis URL, JWT and refresh-token encryption settings, and Auth0 values.
  2. Create separate migration and runtime principals against a local development database.
  3. Apply server/scripts/run-migrations.sh with the required migration environment. server/README.md describes role provisioning and the preservation policy.
Read and run from the repository root
cp server/.env.example server/.env
# Configure the local services and database roles first.
pnpm server:check
pnpm dev:server

The migration runner consumes infrastructure/ansible/playbooks/plantocode/migration-execution-plan.yml, the same ordered plan deployment uses. The migration role must be a superuser that is none of postgres, hbcode, or authenticated and is absent from DATABASE_RUNTIME_ROLES. Files marked execution once are sha256-locked in plantocode_internal.migration_execution_ledger, and the runner stops when a runtime role can reach that ledger. A checksum mismatch, a once file without a single transaction, or an unplanned .sql file stops the run. PRESERVE_EXISTING_DATABASE_IDENTITIES=true is accepted only for the expand phase. Do not replace the runner with sqlx migrate run.

PurposeSettings and when they are required
Core servicesDATABASE_SYSTEM_URL, DATABASE_USER_URL, REDIS_URL. Core database and shared sign-in services.
Account loginAUTH0_DOMAIN, AUTH0_API_AUDIENCE, AUTH0_ALLOWED_CLIENT_IDS. The configured account login trust boundary.
TokensJWT_SECRET, REFRESH_TOKEN_ENCRYPTION_KEY. Issuing app credentials and storing refresh material.
ProvidersProvider API keys. Only the corresponding configured provider routes.
IntegrationsStripe, APNs, FCM, mail and other integration settings. Their respective account or notification features.
Extra databasesMETRICS_DATABASE_URL, ACCOUNT_CONTROL_DATABASE_URL. Optional in development. Production refuses to start without both, and each must target its own PostgreSQL database.
Service healthSERVICE_HEALTH_REGION, SERVICE_HEALTH_RELEASE. Required when METRICS_DATABASE_URL is set. The region is eu or us. The release is the SHA-256 of the deployed binary as 64 hex characters.
PreviewsWORKSPACE_PREVIEW_HMAC_SECRET. Optional in development, required in production. At least 32 characters.
DeploymentPLANTOCODE_DEPLOYMENT_TOKEN. The protected /health/deployment endpoint and its action /health/deployment/drain-device-links. Without the token both answer 503. A wrong token gets 401.

Verify behavior beyond HTTP health

GET /health checks basic server health. Protected /health/deployment supports deployment inspection. Then verify an account login, two device registrations, one read-only desktop RPC, and one controlled outbox operation. A healthy HTTP listener alone does not test database role separation, a native callback, provider configuration, or relay routing.

A useful relay trace follows the request ID, source client, target desktop, method, and connection generation. The forwarding code extracts W3C trace context and injects the child context into the next hop. For ambiguous delivery, inspect the desktop’s remote_rpc_idempotency claim and its chat_operation_ledger rows before inferring whether work ran.

A release finishes on the US canary before it starts in the EU, so the two regions run different releases for a while
A release finishes on the US canary before it starts in the EU, so the two regions run different releases for a whileTwo host lanes on a time axis that is not to scale. The lane interserver-us runs five steps in order: the database backup, api-expand, api-binary-deploy, the soak, and api-migrate. After that it serves the new release. During all of this the lane hetzner-primary serves the previous release. An operator reviews the canary result. Only then does hetzner-primary run the same five steps in the same order.
interserver-usUS · canary
hetzner-primaryEU
the regions run different releases
backup
expand
binary
soak
migrate
serves the new release
serves the previous release
canary result reviewed
the same five steps in the same order
not to scale
No generic rollback exists. A return after api-migrate needs a maintenance window, the matching backup from the first step, and the archived release that fits it.
  • Backup/usr/local/bin/backup_postgresql.sh

    Runs first on each host. Its verified output is the only way back after a forward migration.

  • Expandsite-app.yml --tags api-expand

    Applies the backward-compatible part of the migration plan. The binary that still runs keeps working on the expanded schema.

  • Binary and soaksite-app.yml --tags api-binary-deploy

    Activates the new binary with the blue-green services. The operator checks health and logs and waits for the soak to end.

  • Migratesite-app.yml --tags api-migrate

    Commits both release colors and applies the contract part of the plan. There is no generic rollback action after this step.

  • One host per run--limit interserver-us · --limit hetzner-primary

    Each action that changes production targets exactly one inventory host. The EU sequence starts only after the canary result is reviewed.

Production has two hosts in the Ansible inventory: interserver-us in the US and hetzner-primary in the EU. Each runs the server as two services, plantocode-blue on port 8080 and plantocode-green on port 8081. Every action that changes production runs through site-app.yml with one explicit --limit host. infrastructure/ansible/operations.md is the runbook. Its inventory and secrets are operator configuration.

Step on one hostAction and result
Backupbackup_postgresql.sh in /usr/local/bin. A verified database backup that the operator keeps.
Expand--tags api-expand. The backward-compatible phase of the migration plan is applied and attested.
Binary--tags api-binary-deploy. The new binary is active. The operator checks health and logs and completes the soak.
Migrate--tags api-migrate. Both release colors are committed and the contract phase is applied.

The runbook forbids three shortcuts. Never copy a release SQL file to a server and run psql by hand. Never run an application or database change without one explicit --limit. FLUSHALL on Redis is not a routine recovery action and needs explicit operator approval.

Why one relay process per region

Live sockets and pending requests are process state, so a drain comes before a process stops.