nick-doc/10 - Services/deployment.md

title, tags

title	tags
Deployment	services deployment infrastructure docker

Deployment

The deployment/ sub-project contains Docker Compose definitions, reverse-proxy configs, Gatus monitoring, and migration bundles for running the Amanat escrow platform. It covers three distinct stacks: a legacy compose (reference only), the dev-amn active dev stack (dev.amn.gg), and the escrow-multi white-label stack (multi.amn.gg).

---

1. Overview

File	Status	Host	Notes
deployment/docker-compose.yml	Legacy / reference	Any	nginx + traefik_public network; images from git.manko.yoga registry. Do not deploy from this.
deployment/dev-amn/docker-compose.yml	Active	89.58.32.32	shared-web + infra-caddy ingress; images from git.tbs.amn.gg/escrow
deployment/escrow-multi/docker-compose.yml	Active multi-shop	89.58.32.32	Isolated stack for multi.amn.gg; images tagged :multi; fresh Postgres/Redis; Drizzle migrations

The dev-amn stack is the authoritative dev deployment. The escrow-multi stack is the only valid target for feature/white-label-shops branch work.

[!warning] Branch / stack isolation Work on feature/white-label-shops must NEVER touch escrow-dev — no restart, redeploy, or env change. Work on main must NEVER touch escrow-multi. Each stack must have its own TELEGRAM_BOT_TOKEN (different bots). See deploy_architecture_two_stacks.

---

2. Services

2.1 dev-amn stack (active, dev.amn.gg)

Service	Image	Internal Port	Role
backend	git.tbs.amn.gg/escrow/backend:dev	5001	Express 5 API + Socket.IO + admin seed
frontend	git.tbs.amn.gg/escrow/frontend:dev	8083	Next.js SSR app
refscanner	git.tbs.amn.gg/escrow/scanner:dev	8080	In-house AMN payment scanner (SQLite)
postgres	postgres:18-alpine	5432	Primary datastore (auth + all 8 Postgres domain stores)
redis	redis:8-alpine	6379	Session cache, rate-limit counters, pub/sub
mongodb	mongo:8.0-noble	27017	Legacy datastore — dev boot parity only; retire once remaining reads migrated

2.2 escrow-multi stack (multi.amn.gg)

Service	Image	Internal Port	Role
migrate	node:22-alpine	n/a	One-shot Drizzle migration runner
backend	git.tbs.amn.gg/escrow/backend:multi	5001	Express API, tenant services, storefront API, tenant bot webhook
frontend	git.tbs.amn.gg/escrow/frontend:multi	8083	Next.js for multi.amn.gg, tenant subdomains, dashboards
postgres	postgres:18-alpine	5432	Isolated multi-stack database (escrow_multi)
redis	redis:8-alpine	6379	Isolated multi-stack cache/session/pub-sub

2.3 Legacy compose services (deployment/docker-compose.yml)

These are documented for reference only. Do not deploy from this file.

Service	Image	Host Port	Role
nginx	nginx:alpine	80 (via Traefik)	Reverse proxy in front of backend and frontend
nickDev-marketplace	git.manko.yoga/manawenuz/escrow-backend:dev	—	Backend (legacy registry)
mongodb	mongo:8.0-noble	—	Mongo datastore
postgres	postgres:18-alpine	—	Postgres datastore
redis	redis:8-alpine	—	Cache/sessions
nickDev-frontend	git.manko.yoga/manawenuz/escrow-frontend:dev	8083	Frontend (legacy registry)
gatus	twinproduction/gatus:latest	8084→8080	Uptime monitoring + Telegram alerting

---

3. Architecture Diagram

dev-amn (active)

Internet (HTTPS 443 / HTTP 80)
        │
        ▼
┌───────────────────────────────┐
│   Cloudflare CDN / Proxy      │
│   amn.gg  /  dev.amn.gg       │
└─────────────┬─────────────────┘
              │ (origin request)
              ▼
┌─────────────────────────────────────────────────────┐
│  Host: 89.58.32.32                                  │
│                                                     │
│  ┌──────────────────────────────────────────────┐   │
│  │  infra-caddy  (Arcane project "infra")       │   │
│  │  ports 80:80, 443:443 bound to host          │   │
│  │  Caddyfile: /opt/arcane/data/projects/       │   │
│  │             infra/Caddyfile                  │   │
│  └───────┬─────────────────────────┬────────────┘   │
│          │ /api/* /socket.io/*     │ /*             │
│          │ /uploads/*              │                │
│          ▼                         ▼                │
│  ┌───────────────┐        ┌────────────────────┐    │
│  │   backend     │        │     frontend       │    │
│  │   :5001       │        │     :8083          │    │
│  │   shared-web  │        │     shared-web     │    │
│  └──┬──┬────┬────┘        └────────────────────┘    │
│     │  │    │                                       │
│     │  │    └────────────────────┐                  │
│     │  │                        ▼                   │
│     │  │             ┌──────────────────────┐       │
│     │  │             │     refscanner       │       │
│     │  │             │     :8080            │       │
│     │  │             │  (default only)      │       │
│     │  │             └──────────────────────┘       │
│     │  │                                            │
│     ▼  ▼                                            │
│  ┌──────────┐  ┌──────────┐  ┌────────────────┐    │
│  │ postgres │  │  redis   │  │   mongodb      │    │
│  │  :5432   │  │  :6379   │  │   :27017       │    │
│  │ (default │  │ (default │  │ (default only, │    │
│  │  only)   │  │  only)   │  │   legacy)      │    │
│  └──────────┘  └──────────┘  └────────────────┘    │
│                                                     │
└─────────────────────────────────────────────────────┘

Networks:
  shared-web  (external) ─ backend + frontend (reachable by infra-caddy)
  default     (bridge)   ─ all containers on the stack

Legacy compose (reference only)

Internet (HTTPS 443)
        │
        ▼
┌─────────────────────────┐
│   Traefik (external)    │
│   escrowdev.ch.manko.   │
│   yoga → nginx:80       │
│   gatus.ch.manko.yoga → │
│   gatus:8080            │
└────────────┬────────────┘
             │
             ▼
┌────────────────────────────────────────┐
│  nginx (traefik_public + default)      │
│  nickDev-nginx                         │
│  conf: /var/data/escrowDev/nginx/      │
└────────┬──────────────────┬────────────┘
         │ /api /socket.io  │ /*
         ▼                  ▼
┌────────────────┐  ┌────────────────────┐
│ nickDev-       │  │  nickDev-frontend  │
│ marketplace    │  │  :8083             │
│ backend        │  │  (watchtower)      │
│ (watchtower)   │  └────────────────────┘
└──┬──┬──────────┘
   │  │
   ▼  ▼
┌──────────┐ ┌──────────┐ ┌────────────┐
│ mongodb  │ │ postgres │ │  redis     │
│ :27017   │ │ :5432    │ │  :6379     │
└──────────┘ └──────────┘ └────────────┘

┌────────────────────────────────────────┐
│  gatus :8084→:8080  (traefik_public)   │
└────────────────────────────────────────┘

---

4. Networks

Network	Type	Present in	Services Attached	Purpose
default (bridge)	Internal auto	dev-amn, legacy	All services	Container-to-container communication
shared-web	External (pre-existing)	dev-amn, escrow-multi	backend, frontend	Allows infra-caddy to proxy by container name
traefik_public	External (pre-existing)	Legacy compose only	nginx, gatus	Old Traefik-based ingress on git.manko.yoga host

Key rules:

• postgres, redis, mongodb are on default only — never externally reachable.
• refscanner is on default only; backend reaches it via alias refscanner:8080.
• Any new public-facing service must join shared-web AND get a Caddyfile vhost block.
• shared-web must exist on the host before docker compose up — it is created by the Arcane infra project.

---

5. Volumes and Bind Mounts

dev-amn stack

All data volumes use relative bind mounts under ./data/ (resolved to /opt/arcane/data/projects/escrow-dev/data/ on the server):

Service	Host Path	Container Path	Notes
backend	./data/uploads	/app/uploads	User-uploaded files (served via /uploads/*)
refscanner	./data/scanner	/data	SQLite DB at /data/scanner.db
postgres	./data/postgres	/var/lib/postgresql/data	PGDATA=/var/lib/postgresql/data/pgdata (subdir workaround)
redis	./data/redis	/data	RDB persistence dump
mongodb	./data/mongo	/data/db	Legacy; can be removed once Mongo retired

The Gatus config (deployment/gatus/config.yaml) is bind-mounted read-only into the gatus container at /config/config.yaml. It lives in the repo, not in ./data/.

Postgres volume note: postgres:18 uses a version-scoped data directory layout and refuses to init into a volume root that already contains files from a different layout. PGDATA is set to a subdirectory (/var/lib/postgresql/data/pgdata) inside the mount to avoid init conflicts.

Legacy compose bind mounts (/var/data/escrowDev/)

Service	Host Path	Container Path
nginx	/var/data/escrowDev/nginx/nginx.conf	/etc/nginx/nginx.conf (ro)
nginx	/var/data/escrowDev/nginx/logs	/var/log/nginx
nginx / backend	/var/data/escrowDev/uploads	/uploads / /app/uploads
mongodb	/var/data/escrowDev/mongodb_data	/data/db
mongodb	/var/data/escrowDev/mongo-init	/docker-entrypoint-initdb.d
postgres	/var/data/escrowDev/postgres_data	/var/lib/postgresql
redis	/var/data/escrowDev/redis_data	/data

---

6. Reverse Proxy (infra-caddy) Integration

Ingress for 89.58.32.32 is handled exclusively by infra-caddy — the Caddy container in the Arcane project infra. It owns host ports 80 and 443. No service should bind those ports directly.

Caddyfile block for dev.amn.gg

Live location on server: /opt/arcane/data/projects/infra/Caddyfile Reference copy in repo: deployment/dev-amn/Caddyfile

{
    email manwe@manko.yoga
    auto_https disable_redirects
}

dev.amn.gg {
    encode zstd gzip

    @backend path /api/* /socket.io/* /uploads/*
    reverse_proxy @backend backend:5001

    reverse_proxy frontend:8083
}

• auto_https disable_redirects — Cloudflare proxy sits in front; Caddy must not force HTTP→HTTPS redirects at origin.
• Routes by path prefix: /api/*, /socket.io/*, /uploads/* → backend:5001; everything else → frontend:8083.
• Container names resolve via the shared-web network (both backend and frontend join it).

Adding a new public service

1. Add the service to deployment/dev-amn/docker-compose.yml with networks: shared-web: {}.
2. Edit /opt/arcane/data/projects/infra/Caddyfile on the server — add a new vhost block or path matcher.
3. Reload Caddy without restarting:

   ssh -i ~/CascadeProjects/wzp root@89.58.32.32 \
     "docker exec infra-caddy caddy reload --config /etc/caddy/Caddyfile"
   

4. Verify: curl -I https://dev.amn.gg/<new-path>

See also: Shared Infra (89.58.32.32)

Legacy compose: Traefik labels

In the legacy compose, nginx and gatus expose themselves to Traefik via Docker labels:

# nginx
traefik.http.routers.escrowDev.rule=Host(`escrowdev.ch.manko.yoga`)
traefik.http.routers.escrowDev.entrypoints=https
traefik.http.services.escrowDev.loadbalancer.server.port=80

# gatus
traefik.http.routers.gatus.rule=Host(`gatus.ch.manko.yoga`)
traefik.http.routers.gatus.entrypoints=https
traefik.http.services.gatus.loadbalancer.server.port=8080

---

7. Gatus Monitoring

Gatus runs as a sidecar in the default network. Config is at deployment/gatus/config.yaml (bind-mounted read-only). Alerts are delivered via Telegram to GATUS_TELEGRAM_CHAT_ID.

In the legacy compose, Gatus is exposed on host port 8084 (mapped from container :8080) and publicly accessible via Traefik at gatus.ch.manko.yoga.

Alert policy

Setting	Value
Default failure threshold	3 consecutive failures
Default success threshold	2 consecutive successes
Send on resolved	Yes
Alert channel	Telegram (GATUS_TELEGRAM_BOT_TOKEN / GATUS_TELEGRAM_CHAT_ID)

Prod endpoints use failure-threshold: 2 (faster alerting).

Monitored endpoints

Name	Group	URL	Interval	Key Conditions
backend-dev-version	backend-dev	https://dev.amn.gg/api/version	60s	HTTP 200, body.version not empty
backend-dev-health	backend-dev	https://dev.amn.gg/api/health	30s	HTTP 200, all 8 PG store modes = postgres, redis ok, RN chain+token registry loaded
backend-prod-version	backend-prod	https://amn.gg/api/version	60s	HTTP 200, body.version not empty
backend-prod-health	backend-prod	https://amn.gg/api/health	30s	HTTP 200, db/postgres/redis/RN registries ok
frontend-dev	frontend	https://dev.amn.gg/	60s	HTTP 200, response time < 3000ms
frontend-prod	frontend	https://amn.gg/	60s	HTTP 200, response time < 3000ms
rn-api-reachable	external	https://api.request.network/v2/health	5m	HTTP 200/401/404 (checks reachability only)
chainalysis-public-api	external	https://public.chainalysis.com/api/v1/address/0x000…	5m	HTTP 200 or 404
bsc-rpc-publicnode	external	https://bsc-rpc.publicnode.com (POST)	2m	HTTP 200, result == "0x38" (BSC mainnet chain ID)

The backend-dev-health endpoint validates all 8 domain stores running on Postgres: auth, config, address, category, levelConfig, shopSettings, review, notification. A failure here means a store mode regression or a broken PG_URL.

Gatus dashboard: :8084 on the host locally (not publicly proxied by default — access via SSH tunnel, or add a Caddyfile block if public exposure is needed).

---

8. Environment Variables

All vars are injected via .env at the stack root. The server file is chmod 600 and never committed. The deployment/.env in the repo serves as the live dev reference / template.

8.1 Runtime / Node

Variable	Description	Default
NODE_ENV	Runtime environment	production
PORT	Express listen port	5001
TRUST_PROXY	Express trust-proxy (required behind Caddy/nginx)	true
DEBUG	Debug namespaces	(empty)
LOG_LEVEL	Winston log level	info

8.2 Database — Postgres

Variable	Description	Default in compose
PG_URL	Postgres DSN	postgres://amanat:amanat_local@postgres:5432/amanat_dev
POSTGRES_USER	Postgres superuser	amanat
POSTGRES_PASSWORD	Postgres superuser password	—
POSTGRES_DB	Postgres database name	amanat_dev
AUTO_SEED_ON_START	Run seed on boot	true

8.3 Database — Mongo (legacy)

Variable	Description	Default
MONGODB_URI	Mongo connection string	mongodb://admin:pass@mongodb:27017/marketplace?authSource=admin
MONGO_INITDB_ROOT_USERNAME	Mongo root user	admin
MONGO_INITDB_ROOT_PASSWORD	Mongo root password	changeme_local
MONGO_INITDB_DATABASE	Mongo init DB	marketplace
DB_NAME	Mongo database name used by app	amn-db

8.4 Store modes (dual-write seam)

All default to postgres in the dev-amn compose. Changing any to mongo re-routes that domain's reads/writes to MongoDB.

Variable	Domain	Default
AUTH_STORE	Auth / user accounts	postgres
CONFIG_STORE	App config	postgres
ADDRESS_STORE	User addresses	postgres
CATEGORY_STORE	Marketplace categories	postgres
LEVEL_CONFIG_STORE	Gamification level config	postgres
SHOP_SETTINGS_STORE	Per-shop settings	postgres
REVIEW_STORE	Product / seller reviews	postgres
NOTIFICATION_STORE	User notifications	postgres

See mongo_retirement_status and mongo-to-pg-migration-guide.

8.5 Auth / Sessions

Variable	Description
JWT_SECRET	JWT signing secret
JWT_EXPIRES_IN	Access token TTL (e.g. 7d)
REFRESH_TOKEN_EXPIRES_IN	Refresh token TTL (e.g. 30d)

8.6 Redis

Variable	Description
REDIS_URI	Redis connection string (includes password)
REDIS_PASSWORD	Redis auth password (standalone form)

8.7 URLs / CORS

Variable	Description
BASE_URL	Canonical origin (https://dev.amn.gg)
API_URL	API base URL
FRONTEND_URL	Frontend origin
BACKEND_URL	Backend origin
CORS_ORIGIN	Allowed CORS origin

8.8 File uploads

Variable	Description	Default
UPLOAD_PATH	Upload directory inside container	/app/uploads
MAX_FILE_SIZE	Max upload bytes	52428800 (50 MB)

8.9 Rate limiting

Variable	Description	Default
RATE_LIMIT_WINDOW_MS	Window for rate limiter	900000 (15 min)
RATE_LIMIT_MAX_REQUESTS	Max requests per window	100

GET /api/payment/:id must bypass paymentLimiter (30 req/15 min) — see backend_rate_limits.

8.10 SMTP

Variable	Description
SMTP_HOST	SMTP server hostname
SMTP_PORT	SMTP port
SMTP_SECURE	TLS (true/false)
SMTP_USER	SMTP username
SMTP_PASS	SMTP password
SMTP_FROM	From address

8.11 WebAuthn (Passkeys)

Variable	Description
WEBAUTHN_RP_ID	Relying party ID (domain)
WEBAUTHN_RP_NAME	Relying party display name
WEBAUTHN_RP_ORIGIN	Relying party origin URL

8.12 Admin seed

Variable	Description
ADMIN_EMAIL	Bootstrap admin email
ADMIN_PASSWORD	Bootstrap admin password
ADMIN_FIRST_NAME	Admin first name
ADMIN_LAST_NAME	Admin last name

8.13 Google OAuth

Variable	Description
GOOGLE_CLIENT_ID	Google OAuth client ID
GOOGLE_CLIENT_SECRET	Google OAuth client secret

8.14 OpenAI

Variable	Description
OPENAI_API_KEY	OpenAI API key
OPENAI_DEFAULT_MODEL	Default model (e.g. gpt-4)
OPENAI_MAX_TOKENS	Max tokens per request
OPENAI_TEMPERATURE	Sampling temperature

8.15 Sentry

Variable	Description
SENTRY_DSN	Sentry ingest DSN

8.16 Wallets / Blockchain

Variable	Description
ESCROW_WALLET_ADDRESS	Platform escrow wallet
BSC_USDT_CONTRACT	BSC USDT token contract address
ADMIN_PAYOUT_WALLET_ADDRESS	Admin payout destination
RECEIVER_WALLET_ADDRESS	Default receiver wallet

8.17 DePay

Variable	Description
DEPAY_INTEGRATION_ID	DePay integration UUID
DEPAY_WEBHOOK_SECRET	Webhook verification secret
DEPAY_NETWORKS	Enabled chains (e.g. bsc)
DEPAY_ALLOWED_TOKENS	Allowed payment tokens
DEPAY_PUBLIC_KEY	DePay public key (PEM)

8.18 SHKeeper

Variable	Description
SHKEEPER_API_KEY	SHKeeper API key
SHKEEPER_BASE_URL	SHKeeper service base URL
SHKEEPER_API_URL	Payment request endpoint
SHKEEPER_ENVIRONMENT	production or sandbox
SHKEEPER_WALLET_ID	Destination wallet
SHKEEPER_NETWORKS	Enabled chains
SHKEEPER_ALLOWED_TOKENS	Allowed tokens
SHKEEPER_FORCE_REAL	Bypass test mode
SHKEEPER_TOKEN	Token type (e.g. USDT)
SHKEEPER_CALLBACK_SECRET	Callback verification secret
SHKEEPER_WEBHOOK_SECRET	Webhook verification secret

8.19 Request Network

Variable	Description	Default
REQUEST_NETWORK_ENABLED	Enable RN provider	—
REQUEST_NETWORK_WEBHOOK_SECRET	Webhook signature secret	—
REQUEST_NETWORK_WEBHOOK_CALLBACK_URL	Public callback URL	—
REQUEST_NETWORK_PAYMENT_CURRENCY	Currency (e.g. USDC)	—
REQUEST_NETWORK_NETWORK	Chain (e.g. bsc)	—
REQUEST_NETWORK_MERCHANT_REFERENCE	Merchant address reference	—
REQUEST_NETWORK_API_BASE_URL	RN API root	—
REQUEST_NETWORK_API_KEY	RN API key	—
REQUEST_NETWORK_ORIGIN	Origin header sent to RN	—
REQUEST_NETWORK_ALLOW_TEST_WEBHOOKS	Allow test events	false

RN webhook discriminator is payload.event (not eventType) — see rn_webhook_event_field. RN proxy addresses differ per chain (not canonical CREATE2 addresses) — see rn_proxy_addresses_per_chain.

8.20 Transaction safety

Variable	Description	Default
TRANSACTION_SAFETY_ENABLED	Enable on-chain verification	true
TRANSACTION_SAFETY_REQUIRE_TX_HASH	Require tx hash	true
TRANSACTION_SAFETY_REQUIRE_TRANSFER_MATCH	Require transfer match	true
TRANSACTION_SAFETY_MIN_CONFIRMATIONS	Min block confirmations	12
TRANSACTION_SAFETY_AML_PROVIDER	AML provider (none, chainalysis)	none

8.21 Payment routing

Variable	Description
PAYMENT_PROVIDER	Active provider
PAYMENT_ENABLED_PROVIDERS	Comma-separated enabled providers
PAYMENT_PROVIDER_MODE	live or test
PAYMENT_ROLLBACK_PROVIDER	Fallback provider

8.22 Telegram

Variable	Description
TELEGRAM_FEATURE_ENABLED	Enable Telegram integration
TELEGRAM_MINIAPP_ENABLED	Enable Mini App
TELEGRAM_WEBHOOK_ENABLED	Enable webhook receiver
TELEGRAM_BOT_TOKEN	Main bot token (@amnescrow_Bot for dev)
TELEGRAM_WEBHOOK_SECRET_TOKEN	Webhook secret for validation
TELEGRAM_INITDATA_MAX_AGE_SEC	Max age for initData
TELEGRAM_INITDATA_REPLAY_WINDOW_MS	Replay protection window
TELEGRAM_WEBHOOK_REPLAY_WINDOW_MS	Webhook replay protection window
TELEGRAM_SESSION_TTL_SEC	Session TTL
TG_NOTIFY_BOT_TOKEN	Ops/monitoring bot token (amnGG_MonitorBot)
TG_NOTIFY_CHATS	Comma-separated chat IDs for ops notifications

Each stack (dev, multi) must have a different TELEGRAM_BOT_TOKEN — sharing a bot token kills one stack's webhook when the other registers. See escrow_multi_woodpecker_deploy and stack isolation warning above.

8.23 Pangolin / Newt (optional VPN mesh)

Variable	Description
PANGOLIN_ENDPOINT	Pangolin tunnel endpoint
NEWT_ID	Newt node ID
NEWT_SECRET	Newt node secret

8.24 Frontend (NEXT_PUBLIC_*)

Variable	Description
NEXT_PUBLIC_API_URL	Backend API URL (browser-visible)
NEXT_PUBLIC_SOCKET_URL	Socket.IO server URL
NEXT_PUBLIC_WALLETCONNECT_PROJECT_ID	WalletConnect project ID
NEXT_PUBLIC_ALCHEMY_API_KEY_MAINNET	Alchemy mainnet key
NEXT_PUBLIC_ALCHEMY_API_KEY_SEPOLIA	Alchemy Sepolia key
NEXT_PUBLIC_ALCHEMY_API_KEY_POLYGON	Alchemy Polygon key
NEXT_PUBLIC_ESCROW_WALLET_ADDRESS	Escrow wallet (shown in UI)
NEXT_PUBLIC_APP_NAME	App display name
NEXT_PUBLIC_APP_VERSION	App version string
NEXT_PUBLIC_MAPBOX_API_KEY	Mapbox key (address autocomplete)
NEXT_PUBLIC_PASSKEY_RP_NAME	WebAuthn RP name
NEXT_PUBLIC_PASSKEY_RP_ID	WebAuthn RP ID
NEXT_PUBLIC_PASSKEY_ORIGIN	WebAuthn origin
NEXT_PUBLIC_BACKEND_URL	Backend origin (direct calls)
NEXT_PUBLIC_DEPAY_INTEGRATION_ID	DePay integration ID
NEXT_PUBLIC_IS_DEVELOPMENT	Development flag
NEXT_PUBLIC_ENABLE_DEBUG	Enable client debug logging
NEXT_PUBLIC_APP_URL	Canonical app URL
NEXT_PUBLIC_TELEGRAM_BOT_ID	Telegram bot numeric ID
BUILD_STATIC_EXPORT	Enable next export mode (false for SSR)

NEXT_PUBLIC_* vars are baked into the frontend bundle at build time. Never put secrets in them. Frontend env changes require a fresh image build and redeploy.

8.25 Gatus

Variable	Description
GATUS_TELEGRAM_BOT_TOKEN	Telegram bot for alert delivery
GATUS_TELEGRAM_CHAT_ID	Target chat ID for alerts

---

9. Deploy Workflow

9.1 Normal image update (CI-driven — dev-amn)

Woodpecker CI builds backend and frontend images, pushes to git.tbs.amn.gg/escrow/, then triggers an Arcane GitOps sync which pulls the new image and recreates the container.

git push origin dev
  └─► Woodpecker build pipeline
        └─► docker push git.tbs.amn.gg/escrow/backend:dev
        └─► docker push git.tbs.amn.gg/escrow/frontend:dev
        └─► arcane-cli gitops sync cf6c9eab…  (or watchtower polls)
              └─► escrow-backend restarted with new image
              └─► escrow-frontend restarted with new image

Always bump package.json version before pushing. See version_bump_before_ci.

9.2 escrow-multi deploys (white-label stack)

Always use Woodpecker. Never use manual rsync/docker-build/ssh for escrow-multi.

# 1. Make changes, bump version in package.json
# 2. Commit
git commit -m "fix: description (vX.Y.Z)"
# 3. Push to Forgejo (remote is "forgejo", not "origin")
git push forgejo feature/white-label-shops
# 4. Monitor Woodpecker pipeline
source ~/CascadeProjects/escrow/.env
WOODPECKER_SERVER=$WOODPECKER_SERVER WOODPECKER_TOKEN=$WOODPECKER_TOKEN \
  woodpecker-cli pipeline ls escrow/backend

Frontend is a separate Woodpecker project (escrow/frontend). Both push targets trigger their respective pipelines.

9.3 Manual hotfix deploy (backend only — no registry cycle)

For urgent fixes without a full CI cycle, build locally on the server:

# 1. Copy changed files to build tree
scp -i ~/CascadeProjects/wzp src/services/auth/authRoutes.ts \
    root@89.58.32.32:/tmp/escrow-backend-build/src/services/auth/

# 2. Rebuild image on server (~3 min, ARM64)
ssh -i ~/CascadeProjects/wzp root@89.58.32.32 \
  "cd /tmp/escrow-backend-build && docker build -f Dockerfile.prod \
   -t escrow-backend-local:dev ."

# 3. Restart the backend container
ssh -i ~/CascadeProjects/wzp root@89.58.32.32 \
  "cd /opt/arcane/data/projects/escrow-dev && docker compose up -d backend"

The docker-compose.override.yml at /opt/arcane/data/projects/escrow-dev/docker-compose.override.yml sets pull_policy: never for escrow-backend-local:dev so watchtower never clobbers it.

9.4 Bringing the stack up/down

# via Arcane CLI (preferred)
arcane-cli project start devEscrow
arcane-cli project stop devEscrow

# via SSH + docker compose (direct)
ssh -i ~/CascadeProjects/wzp root@89.58.32.32 \
  "cd /opt/arcane/data/projects/escrow-dev && docker compose up -d"

ssh -i ~/CascadeProjects/wzp root@89.58.32.32 \
  "cd /opt/arcane/data/projects/escrow-dev && docker compose down"

9.5 Reloading Caddy after Caddyfile edits

ssh -i ~/CascadeProjects/wzp root@89.58.32.32 \
  "docker exec infra-caddy caddy reload --config /etc/caddy/Caddyfile"

No container restart needed.

9.6 Updating env vars

1. Edit .env on the server: /opt/arcane/data/projects/escrow-dev/.env
2. Restart affected container:

   ssh -i ~/CascadeProjects/wzp root@89.58.32.32 \
     "cd /opt/arcane/data/projects/escrow-dev && docker compose up -d backend"
   

3. Frontend NEXT_PUBLIC_* vars are baked at build time — they require a fresh image build and full redeploy via CI.

9.7 Verifying a deploy

# Check running containers
arcane-cli project status devEscrow

# Check backend version
curl https://dev.amn.gg/api/version

# Check health (all stores + RN registries)
curl https://dev.amn.gg/api/health | jq .

# Tail backend logs
ssh -i ~/CascadeProjects/wzp root@89.58.32.32 \
  "docker logs -f escrow-backend --tail 100"

CI ✓ green does NOT guarantee the new image was pushed. Always verify curl /api/version returns the expected version. See woodpecker_silent_build_fail.

---

10. Dev vs Prod Differences

Aspect	dev-amn (dev.amn.gg)	Prod (amn.gg)
Compose file	deployment/dev-amn/docker-compose.yml	Separate prod stack (not in this repo)
Image registry	git.tbs.amn.gg/escrow	Same registry, prod tags
Image tag	:dev	:latest or versioned
MongoDB	Present (dev parity — retired in prod)	Not present
ENABLE_TESTNET_CHAINS	true (compose override)	Not set / false
NODE_ENV	production	production
NEXT_PUBLIC_IS_DEVELOPMENT	false	false
PAYMENT_PROVIDER_MODE	live	live
REQUEST_NETWORK_ALLOW_TEST_WEBHOOKS	May be true for RN testing	false
TRANSACTION_SAFETY_MIN_CONFIRMATIONS	12 (default)	12 (default)
Gatus monitoring	Monitors both dev + prod endpoints	Shared Gatus instance
TLS	Cloudflare proxy → Caddy (disable_redirects)	Same
Version bump	Required before CI push	Required
Watchtower labels	Present in legacy compose	Prod stack may differ

---

11. Secret Management

The .env file on the server is the single source of runtime secrets. It is never committed.

• Server location: /opt/arcane/data/projects/escrow-dev/.env
• Permissions: chmod 600, owned by root
• Repo template: deployment/.env — contains live dev values, treated as low-sensitivity dev config; rotate all values before using in production

Rules

1. Never commit .env or any file containing real tokens, passwords, or private keys.
2. Never pass secrets as Dockerfile ARG/ENV at build time — they appear in image layers. All secrets are runtime-injected via env_file.
3. NEXT_PUBLIC_* vars are baked into the frontend bundle. Never place secrets in them.
4. Wallet addresses are public on-chain but still kept out of the repo for operational hygiene.
5. For new deployments: copy deployment/.env to the server, fill in real values, chmod 600.
6. Gatus vars (GATUS_TELEGRAM_BOT_TOKEN, GATUS_TELEGRAM_CHAT_ID) go into the same .env.
7. Telegram bot tokens are high-value — rotate immediately if accidentally pushed.

Sensitive variable groups

Group	Variables	Risk if leaked
JWT	JWT_SECRET	Full session forgery
DB credentials	POSTGRES_PASSWORD, REDIS_PASSWORD, MONGO_INITDB_ROOT_PASSWORD	Database access
Payment webhook secrets	REQUEST_NETWORK_WEBHOOK_SECRET, DEPAY_WEBHOOK_SECRET, SHKEEPER_CALLBACK_SECRET, SHKEEPER_WEBHOOK_SECRET	Fake payment injection
Bot tokens	TELEGRAM_BOT_TOKEN, TG_NOTIFY_BOT_TOKEN	Bot takeover / webhook hijack
OAuth secrets	GOOGLE_CLIENT_SECRET	OAuth impersonation
API keys	OPENAI_API_KEY, REQUEST_NETWORK_API_KEY, SHKEEPER_API_KEY	Billing / data access
Sentry DSN	SENTRY_DSN	Error data exfiltration