nick-doc/10 - Services/deployment.md

---
title: Deployment
tags: [services, deployment, infrastructure, docker]
---

# Deployment

The `deployment/` sub-project contains Docker Compose definitions, reverse-proxy configs, Gatus monitoring, and migration bundles for running the Amanat escrow platform. It covers three distinct stacks: a **legacy compose** (reference only), the **dev-amn active dev stack** (`dev.amn.gg`), and the **escrow-multi white-label stack** (`multi.amn.gg`).

---

## 1. Overview

| File | Status | Host | Notes |
|---|---|---|---|
| `deployment/docker-compose.yml` | **Legacy / reference** | Any | nginx + traefik_public network; images from `git.manko.yoga` registry. Do not deploy from this. |
| `deployment/dev-amn/docker-compose.yml` | **Active** | `89.58.32.32` | `shared-web` + infra-caddy ingress; images from `git.tbs.amn.gg/escrow` |
| `deployment/escrow-multi/docker-compose.yml` | **Active multi-shop** | `89.58.32.32` | Isolated stack for `multi.amn.gg`; images tagged `:multi`; fresh Postgres/Redis; Drizzle migrations |

The `dev-amn` stack is the authoritative dev deployment. The `escrow-multi` stack is the only valid target for `feature/white-label-shops` branch work.

> [!warning] Branch / stack isolation
> Work on `feature/white-label-shops` must NEVER touch `escrow-dev` — no restart, redeploy, or env change. Work on `main` must NEVER touch `escrow-multi`. Each stack must have its own `TELEGRAM_BOT_TOKEN` (different bots). See [[deploy_architecture_two_stacks]].

---

## 2. Services

### 2.1 dev-amn stack (active, `dev.amn.gg`)

| Service | Image | Internal Port | Role |
|---|---|---|---|
| `backend` | `git.tbs.amn.gg/escrow/backend:dev` | 5001 | Express 5 API + Socket.IO + admin seed |
| `frontend` | `git.tbs.amn.gg/escrow/frontend:dev` | 8083 | Next.js SSR app |
| `refscanner` | `git.tbs.amn.gg/escrow/scanner:dev` | 8080 | In-house AMN payment scanner (SQLite) |
| `postgres` | `postgres:18-alpine` | 5432 | Primary datastore (auth + all 8 Postgres domain stores) |
| `redis` | `redis:8-alpine` | 6379 | Session cache, rate-limit counters, pub/sub |
| `mongodb` | `mongo:8.0-noble` | 27017 | Legacy datastore — dev boot parity only; retire once remaining reads migrated |

### 2.2 escrow-multi stack (`multi.amn.gg`)

| Service | Image | Internal Port | Role |
|---|---|---|---|
| `migrate` | `node:22-alpine` | n/a | One-shot Drizzle migration runner |
| `backend` | `git.tbs.amn.gg/escrow/backend:multi` | 5001 | Express API, tenant services, storefront API, tenant bot webhook |
| `frontend` | `git.tbs.amn.gg/escrow/frontend:multi` | 8083 | Next.js for `multi.amn.gg`, tenant subdomains, dashboards |
| `postgres` | `postgres:18-alpine` | 5432 | Isolated multi-stack database (`escrow_multi`) |
| `redis` | `redis:8-alpine` | 6379 | Isolated multi-stack cache/session/pub-sub |

### 2.3 Legacy compose services (`deployment/docker-compose.yml`)

> These are documented for reference only. Do not deploy from this file.

| Service | Image | Host Port | Role |
|---|---|---|---|
| `nginx` | `nginx:alpine` | 80 (via Traefik) | Reverse proxy in front of backend and frontend |
| `nickDev-marketplace` | `git.manko.yoga/manawenuz/escrow-backend:dev` | — | Backend (legacy registry) |
| `mongodb` | `mongo:8.0-noble` | — | Mongo datastore |
| `postgres` | `postgres:18-alpine` | — | Postgres datastore |
| `redis` | `redis:8-alpine` | — | Cache/sessions |
| `nickDev-frontend` | `git.manko.yoga/manawenuz/escrow-frontend:dev` | 8083 | Frontend (legacy registry) |
| `gatus` | `twinproduction/gatus:latest` | 8084→8080 | Uptime monitoring + Telegram alerting |

---

## 3. Architecture Diagram

### dev-amn (active)

```
Internet (HTTPS 443 / HTTP 80)
        │
        ▼
┌───────────────────────────────┐
│   Cloudflare CDN / Proxy      │
│   amn.gg  /  dev.amn.gg       │
└─────────────┬─────────────────┘
              │ (origin request)
              ▼
┌─────────────────────────────────────────────────────┐
│  Host: 89.58.32.32                                  │
│                                                     │
│  ┌──────────────────────────────────────────────┐   │
│  │  infra-caddy  (Arcane project "infra")       │   │
│  │  ports 80:80, 443:443 bound to host          │   │
│  │  Caddyfile: /opt/arcane/data/projects/       │   │
│  │             infra/Caddyfile                  │   │
│  └───────┬─────────────────────────┬────────────┘   │
│          │ /api/* /socket.io/*     │ /*             │
│          │ /uploads/*              │                │
│          ▼                         ▼                │
│  ┌───────────────┐        ┌────────────────────┐    │
│  │   backend     │        │     frontend       │    │
│  │   :5001       │        │     :8083          │    │
│  │   shared-web  │        │     shared-web     │    │
│  └──┬──┬────┬────┘        └────────────────────┘    │
│     │  │    │                                       │
│     │  │    └────────────────────┐                  │
│     │  │                        ▼                   │
│     │  │             ┌──────────────────────┐       │
│     │  │             │     refscanner       │       │
│     │  │             │     :8080            │       │
│     │  │             │  (default only)      │       │
│     │  │             └──────────────────────┘       │
│     │  │                                            │
│     ▼  ▼                                            │
│  ┌──────────┐  ┌──────────┐  ┌────────────────┐    │
│  │ postgres │  │  redis   │  │   mongodb      │    │
│  │  :5432   │  │  :6379   │  │   :27017       │    │
│  │ (default │  │ (default │  │ (default only, │    │
│  │  only)   │  │  only)   │  │   legacy)      │    │
│  └──────────┘  └──────────┘  └────────────────┘    │
│                                                     │
└─────────────────────────────────────────────────────┘

Networks:
  shared-web  (external) ─ backend + frontend (reachable by infra-caddy)
  default     (bridge)   ─ all containers on the stack
```

### Legacy compose (reference only)

```
Internet (HTTPS 443)
        │
        ▼
┌─────────────────────────┐
│   Traefik (external)    │
│   escrowdev.ch.manko.   │
│   yoga → nginx:80       │
│   gatus.ch.manko.yoga → │
│   gatus:8080            │
└────────────┬────────────┘
             │
             ▼
┌────────────────────────────────────────┐
│  nginx (traefik_public + default)      │
│  nickDev-nginx                         │
│  conf: /var/data/escrowDev/nginx/      │
└────────┬──────────────────┬────────────┘
         │ /api /socket.io  │ /*
         ▼                  ▼
┌────────────────┐  ┌────────────────────┐
│ nickDev-       │  │  nickDev-frontend  │
│ marketplace    │  │  :8083             │
│ backend        │  │  (watchtower)      │
│ (watchtower)   │  └────────────────────┘
└──┬──┬──────────┘
   │  │
   ▼  ▼
┌──────────┐ ┌──────────┐ ┌────────────┐
│ mongodb  │ │ postgres │ │  redis     │
│ :27017   │ │ :5432    │ │  :6379     │
└──────────┘ └──────────┘ └────────────┘

┌────────────────────────────────────────┐
│  gatus :8084→:8080  (traefik_public)   │
└────────────────────────────────────────┘
```

---

## 4. Networks

| Network | Type | Present in | Services Attached | Purpose |
|---|---|---|---|---|
| `default` (bridge) | Internal auto | dev-amn, legacy | All services | Container-to-container communication |
| `shared-web` | External (pre-existing) | dev-amn, escrow-multi | `backend`, `frontend` | Allows infra-caddy to proxy by container name |
| `traefik_public` | External (pre-existing) | Legacy compose only | `nginx`, `gatus` | Old Traefik-based ingress on `git.manko.yoga` host |

**Key rules:**
- `postgres`, `redis`, `mongodb` are on `default` only — never externally reachable.
- `refscanner` is on `default` only; backend reaches it via alias `refscanner:8080`.
- Any new public-facing service must join `shared-web` AND get a Caddyfile vhost block.
- `shared-web` must exist on the host before `docker compose up` — it is created by the Arcane `infra` project.

---

## 5. Volumes and Bind Mounts

### dev-amn stack

All data volumes use relative bind mounts under `./data/` (resolved to `/opt/arcane/data/projects/escrow-dev/data/` on the server):

| Service | Host Path | Container Path | Notes |
|---|---|---|---|
| `backend` | `./data/uploads` | `/app/uploads` | User-uploaded files (served via `/uploads/*`) |
| `refscanner` | `./data/scanner` | `/data` | SQLite DB at `/data/scanner.db` |
| `postgres` | `./data/postgres` | `/var/lib/postgresql/data` | `PGDATA=/var/lib/postgresql/data/pgdata` (subdir workaround) |
| `redis` | `./data/redis` | `/data` | RDB persistence dump |
| `mongodb` | `./data/mongo` | `/data/db` | Legacy; can be removed once Mongo retired |

The Gatus config (`deployment/gatus/config.yaml`) is bind-mounted read-only into the gatus container at `/config/config.yaml`. It lives in the repo, not in `./data/`.

> **Postgres volume note:** `postgres:18` uses a version-scoped data directory layout and refuses to init into a volume root that already contains files from a different layout. `PGDATA` is set to a subdirectory (`/var/lib/postgresql/data/pgdata`) inside the mount to avoid init conflicts.

### Legacy compose bind mounts (`/var/data/escrowDev/`)

| Service | Host Path | Container Path |
|---|---|---|
| `nginx` | `/var/data/escrowDev/nginx/nginx.conf` | `/etc/nginx/nginx.conf` (ro) |
| `nginx` | `/var/data/escrowDev/nginx/logs` | `/var/log/nginx` |
| `nginx` / `backend` | `/var/data/escrowDev/uploads` | `/uploads` / `/app/uploads` |
| `mongodb` | `/var/data/escrowDev/mongodb_data` | `/data/db` |
| `mongodb` | `/var/data/escrowDev/mongo-init` | `/docker-entrypoint-initdb.d` |
| `postgres` | `/var/data/escrowDev/postgres_data` | `/var/lib/postgresql` |
| `redis` | `/var/data/escrowDev/redis_data` | `/data` |

---

## 6. Reverse Proxy (infra-caddy) Integration

Ingress for `89.58.32.32` is handled exclusively by **infra-caddy** — the Caddy container in the Arcane project `infra`. It owns host ports 80 and 443. No service should bind those ports directly.

### Caddyfile block for dev.amn.gg

Live location on server: `/opt/arcane/data/projects/infra/Caddyfile`
Reference copy in repo: `deployment/dev-amn/Caddyfile`

```caddy
{
    email manwe@manko.yoga
    auto_https disable_redirects
}

dev.amn.gg {
    encode zstd gzip

    @backend path /api/* /socket.io/* /uploads/*
    reverse_proxy @backend backend:5001

    reverse_proxy frontend:8083
}
```

- `auto_https disable_redirects` — Cloudflare proxy sits in front; Caddy must not force HTTP→HTTPS redirects at origin.
- Routes by path prefix: `/api/*`, `/socket.io/*`, `/uploads/*` → `backend:5001`; everything else → `frontend:8083`.
- Container names resolve via the `shared-web` network (both `backend` and `frontend` join it).

### Adding a new public service

1. Add the service to `deployment/dev-amn/docker-compose.yml` with `networks: shared-web: {}`.
2. Edit `/opt/arcane/data/projects/infra/Caddyfile` on the server — add a new vhost block or path matcher.
3. Reload Caddy without restarting:
   ```bash
   ssh -i ~/CascadeProjects/wzp root@89.58.32.32 \
     "docker exec infra-caddy caddy reload --config /etc/caddy/Caddyfile"
   ```
4. Verify: `curl -I https://dev.amn.gg/<new-path>`

See also: [[Shared Infra (89.58.32.32)]]

### Legacy compose: Traefik labels

In the legacy compose, nginx and gatus expose themselves to Traefik via Docker labels:

```yaml
# nginx
traefik.http.routers.escrowDev.rule=Host(`escrowdev.ch.manko.yoga`)
traefik.http.routers.escrowDev.entrypoints=https
traefik.http.services.escrowDev.loadbalancer.server.port=80

# gatus
traefik.http.routers.gatus.rule=Host(`gatus.ch.manko.yoga`)
traefik.http.routers.gatus.entrypoints=https
traefik.http.services.gatus.loadbalancer.server.port=8080
```

---

## 7. Gatus Monitoring

Gatus runs as a sidecar in the `default` network. Config is at `deployment/gatus/config.yaml` (bind-mounted read-only). Alerts are delivered via Telegram to `GATUS_TELEGRAM_CHAT_ID`.

In the legacy compose, Gatus is exposed on host port `8084` (mapped from container `:8080`) and publicly accessible via Traefik at `gatus.ch.manko.yoga`.

### Alert policy

| Setting | Value |
|---|---|
| Default failure threshold | 3 consecutive failures |
| Default success threshold | 2 consecutive successes |
| Send on resolved | Yes |
| Alert channel | Telegram (`GATUS_TELEGRAM_BOT_TOKEN` / `GATUS_TELEGRAM_CHAT_ID`) |

Prod endpoints use `failure-threshold: 2` (faster alerting).

### Monitored endpoints

| Name | Group | URL | Interval | Key Conditions |
|---|---|---|---|---|
| `backend-dev-version` | backend-dev | `https://dev.amn.gg/api/version` | 60s | HTTP 200, `body.version` not empty |
| `backend-dev-health` | backend-dev | `https://dev.amn.gg/api/health` | 30s | HTTP 200, all 8 PG store modes = postgres, redis ok, RN chain+token registry loaded |
| `backend-prod-version` | backend-prod | `https://amn.gg/api/version` | 60s | HTTP 200, `body.version` not empty |
| `backend-prod-health` | backend-prod | `https://amn.gg/api/health` | 30s | HTTP 200, db/postgres/redis/RN registries ok |
| `frontend-dev` | frontend | `https://dev.amn.gg/` | 60s | HTTP 200, response time < 3000ms |
| `frontend-prod` | frontend | `https://amn.gg/` | 60s | HTTP 200, response time < 3000ms |
| `rn-api-reachable` | external | `https://api.request.network/v2/health` | 5m | HTTP 200/401/404 (checks reachability only) |
| `chainalysis-public-api` | external | `https://public.chainalysis.com/api/v1/address/0x000…` | 5m | HTTP 200 or 404 |
| `bsc-rpc-publicnode` | external | `https://bsc-rpc.publicnode.com` (POST) | 2m | HTTP 200, `result == "0x38"` (BSC mainnet chain ID) |

The `backend-dev-health` endpoint validates **all 8 domain stores running on Postgres**: `auth`, `config`, `address`, `category`, `levelConfig`, `shopSettings`, `review`, `notification`. A failure here means a store mode regression or a broken `PG_URL`.

Gatus dashboard: `:8084` on the host locally (not publicly proxied by default — access via SSH tunnel, or add a Caddyfile block if public exposure is needed).

---

## 8. Environment Variables

All vars are injected via `.env` at the stack root. The server file is `chmod 600` and never committed. The `deployment/.env` in the repo serves as the live dev reference / template.

### 8.1 Runtime / Node

| Variable | Description | Default |
|---|---|---|
| `NODE_ENV` | Runtime environment | `production` |
| `PORT` | Express listen port | `5001` |
| `TRUST_PROXY` | Express trust-proxy (required behind Caddy/nginx) | `true` |
| `DEBUG` | Debug namespaces | _(empty)_ |
| `LOG_LEVEL` | Winston log level | `info` |

### 8.2 Database — Postgres

| Variable | Description | Default in compose |
|---|---|---|
| `PG_URL` | Postgres DSN | `postgres://amanat:amanat_local@postgres:5432/amanat_dev` |
| `POSTGRES_USER` | Postgres superuser | `amanat` |
| `POSTGRES_PASSWORD` | Postgres superuser password | — |
| `POSTGRES_DB` | Postgres database name | `amanat_dev` |
| `AUTO_SEED_ON_START` | Run seed on boot | `true` |

### 8.3 Database — Mongo (legacy)

| Variable | Description | Default |
|---|---|---|
| `MONGODB_URI` | Mongo connection string | `mongodb://admin:pass@mongodb:27017/marketplace?authSource=admin` |
| `MONGO_INITDB_ROOT_USERNAME` | Mongo root user | `admin` |
| `MONGO_INITDB_ROOT_PASSWORD` | Mongo root password | `changeme_local` |
| `MONGO_INITDB_DATABASE` | Mongo init DB | `marketplace` |
| `DB_NAME` | Mongo database name used by app | `amn-db` |

### 8.4 Store modes (dual-write seam)

All default to `postgres` in the dev-amn compose. Changing any to `mongo` re-routes that domain's reads/writes to MongoDB.

| Variable | Domain | Default |
|---|---|---|
| `AUTH_STORE` | Auth / user accounts | `postgres` |
| `CONFIG_STORE` | App config | `postgres` |
| `ADDRESS_STORE` | User addresses | `postgres` |
| `CATEGORY_STORE` | Marketplace categories | `postgres` |
| `LEVEL_CONFIG_STORE` | Gamification level config | `postgres` |
| `SHOP_SETTINGS_STORE` | Per-shop settings | `postgres` |
| `REVIEW_STORE` | Product / seller reviews | `postgres` |
| `NOTIFICATION_STORE` | User notifications | `postgres` |

See [[mongo_retirement_status]] and [[mongo-to-pg-migration-guide]].

### 8.5 Auth / Sessions

| Variable | Description |
|---|---|
| `JWT_SECRET` | JWT signing secret |
| `JWT_EXPIRES_IN` | Access token TTL (e.g. `7d`) |
| `REFRESH_TOKEN_EXPIRES_IN` | Refresh token TTL (e.g. `30d`) |

### 8.6 Redis

| Variable | Description |
|---|---|
| `REDIS_URI` | Redis connection string (includes password) |
| `REDIS_PASSWORD` | Redis auth password (standalone form) |

### 8.7 URLs / CORS

| Variable | Description |
|---|---|
| `BASE_URL` | Canonical origin (`https://dev.amn.gg`) |
| `API_URL` | API base URL |
| `FRONTEND_URL` | Frontend origin |
| `BACKEND_URL` | Backend origin |
| `CORS_ORIGIN` | Allowed CORS origin |

### 8.8 File uploads

| Variable | Description | Default |
|---|---|---|
| `UPLOAD_PATH` | Upload directory inside container | `/app/uploads` |
| `MAX_FILE_SIZE` | Max upload bytes | `52428800` (50 MB) |

### 8.9 Rate limiting

| Variable | Description | Default |
|---|---|---|
| `RATE_LIMIT_WINDOW_MS` | Window for rate limiter | `900000` (15 min) |
| `RATE_LIMIT_MAX_REQUESTS` | Max requests per window | `100` |

> GET `/api/payment/:id` must bypass `paymentLimiter` (30 req/15 min) — see [[backend_rate_limits]].

### 8.10 SMTP

| Variable | Description |
|---|---|
| `SMTP_HOST` | SMTP server hostname |
| `SMTP_PORT` | SMTP port |
| `SMTP_SECURE` | TLS (`true`/`false`) |
| `SMTP_USER` | SMTP username |
| `SMTP_PASS` | SMTP password |
| `SMTP_FROM` | From address |

### 8.11 WebAuthn (Passkeys)

| Variable | Description |
|---|---|
| `WEBAUTHN_RP_ID` | Relying party ID (domain) |
| `WEBAUTHN_RP_NAME` | Relying party display name |
| `WEBAUTHN_RP_ORIGIN` | Relying party origin URL |

### 8.12 Admin seed

| Variable | Description |
|---|---|
| `ADMIN_EMAIL` | Bootstrap admin email |
| `ADMIN_PASSWORD` | Bootstrap admin password |
| `ADMIN_FIRST_NAME` | Admin first name |
| `ADMIN_LAST_NAME` | Admin last name |

### 8.13 Google OAuth

| Variable | Description |
|---|---|
| `GOOGLE_CLIENT_ID` | Google OAuth client ID |
| `GOOGLE_CLIENT_SECRET` | Google OAuth client secret |

### 8.14 OpenAI

| Variable | Description |
|---|---|
| `OPENAI_API_KEY` | OpenAI API key |
| `OPENAI_DEFAULT_MODEL` | Default model (e.g. `gpt-4`) |
| `OPENAI_MAX_TOKENS` | Max tokens per request |
| `OPENAI_TEMPERATURE` | Sampling temperature |

### 8.15 Sentry

| Variable | Description |
|---|---|
| `SENTRY_DSN` | Sentry ingest DSN |

### 8.16 Wallets / Blockchain

| Variable | Description |
|---|---|
| `ESCROW_WALLET_ADDRESS` | Platform escrow wallet |
| `BSC_USDT_CONTRACT` | BSC USDT token contract address |
| `ADMIN_PAYOUT_WALLET_ADDRESS` | Admin payout destination |
| `RECEIVER_WALLET_ADDRESS` | Default receiver wallet |

### 8.17 DePay

| Variable | Description |
|---|---|
| `DEPAY_INTEGRATION_ID` | DePay integration UUID |
| `DEPAY_WEBHOOK_SECRET` | Webhook verification secret |
| `DEPAY_NETWORKS` | Enabled chains (e.g. `bsc`) |
| `DEPAY_ALLOWED_TOKENS` | Allowed payment tokens |
| `DEPAY_PUBLIC_KEY` | DePay public key (PEM) |

### 8.18 SHKeeper

| Variable | Description |
|---|---|
| `SHKEEPER_API_KEY` | SHKeeper API key |
| `SHKEEPER_BASE_URL` | SHKeeper service base URL |
| `SHKEEPER_API_URL` | Payment request endpoint |
| `SHKEEPER_ENVIRONMENT` | `production` or `sandbox` |
| `SHKEEPER_WALLET_ID` | Destination wallet |
| `SHKEEPER_NETWORKS` | Enabled chains |
| `SHKEEPER_ALLOWED_TOKENS` | Allowed tokens |
| `SHKEEPER_FORCE_REAL` | Bypass test mode |
| `SHKEEPER_TOKEN` | Token type (e.g. `USDT`) |
| `SHKEEPER_CALLBACK_SECRET` | Callback verification secret |
| `SHKEEPER_WEBHOOK_SECRET` | Webhook verification secret |

### 8.19 Request Network

| Variable | Description | Default |
|---|---|---|
| `REQUEST_NETWORK_ENABLED` | Enable RN provider | — |
| `REQUEST_NETWORK_WEBHOOK_SECRET` | Webhook signature secret | — |
| `REQUEST_NETWORK_WEBHOOK_CALLBACK_URL` | Public callback URL | — |
| `REQUEST_NETWORK_PAYMENT_CURRENCY` | Currency (e.g. `USDC`) | — |
| `REQUEST_NETWORK_NETWORK` | Chain (e.g. `bsc`) | — |
| `REQUEST_NETWORK_MERCHANT_REFERENCE` | Merchant address reference | — |
| `REQUEST_NETWORK_API_BASE_URL` | RN API root | — |
| `REQUEST_NETWORK_API_KEY` | RN API key | — |
| `REQUEST_NETWORK_ORIGIN` | Origin header sent to RN | — |
| `REQUEST_NETWORK_ALLOW_TEST_WEBHOOKS` | Allow test events | `false` |

> RN webhook discriminator is `payload.event` (not `eventType`) — see [[rn_webhook_event_field]].
> RN proxy addresses differ per chain (not canonical CREATE2 addresses) — see [[rn_proxy_addresses_per_chain]].

### 8.20 Transaction safety

| Variable | Description | Default |
|---|---|---|
| `TRANSACTION_SAFETY_ENABLED` | Enable on-chain verification | `true` |
| `TRANSACTION_SAFETY_REQUIRE_TX_HASH` | Require tx hash | `true` |
| `TRANSACTION_SAFETY_REQUIRE_TRANSFER_MATCH` | Require transfer match | `true` |
| `TRANSACTION_SAFETY_MIN_CONFIRMATIONS` | Min block confirmations | `12` |
| `TRANSACTION_SAFETY_AML_PROVIDER` | AML provider (`none`, `chainalysis`) | `none` |

### 8.21 Payment routing

| Variable | Description |
|---|---|
| `PAYMENT_PROVIDER` | Active provider |
| `PAYMENT_ENABLED_PROVIDERS` | Comma-separated enabled providers |
| `PAYMENT_PROVIDER_MODE` | `live` or `test` |
| `PAYMENT_ROLLBACK_PROVIDER` | Fallback provider |

### 8.22 Telegram

| Variable | Description |
|---|---|
| `TELEGRAM_FEATURE_ENABLED` | Enable Telegram integration |
| `TELEGRAM_MINIAPP_ENABLED` | Enable Mini App |
| `TELEGRAM_WEBHOOK_ENABLED` | Enable webhook receiver |
| `TELEGRAM_BOT_TOKEN` | Main bot token (`@amnescrow_Bot` for dev) |
| `TELEGRAM_WEBHOOK_SECRET_TOKEN` | Webhook secret for validation |
| `TELEGRAM_INITDATA_MAX_AGE_SEC` | Max age for initData |
| `TELEGRAM_INITDATA_REPLAY_WINDOW_MS` | Replay protection window |
| `TELEGRAM_WEBHOOK_REPLAY_WINDOW_MS` | Webhook replay protection window |
| `TELEGRAM_SESSION_TTL_SEC` | Session TTL |
| `TG_NOTIFY_BOT_TOKEN` | Ops/monitoring bot token (`amnGG_MonitorBot`) |
| `TG_NOTIFY_CHATS` | Comma-separated chat IDs for ops notifications |

> Each stack (dev, multi) must have a **different `TELEGRAM_BOT_TOKEN`** — sharing a bot token kills one stack's webhook when the other registers. See [[escrow_multi_woodpecker_deploy]] and stack isolation warning above.

### 8.23 Pangolin / Newt (optional VPN mesh)

| Variable | Description |
|---|---|
| `PANGOLIN_ENDPOINT` | Pangolin tunnel endpoint |
| `NEWT_ID` | Newt node ID |
| `NEWT_SECRET` | Newt node secret |

### 8.24 Frontend (NEXT_PUBLIC_*)

| Variable | Description |
|---|---|
| `NEXT_PUBLIC_API_URL` | Backend API URL (browser-visible) |
| `NEXT_PUBLIC_SOCKET_URL` | Socket.IO server URL |
| `NEXT_PUBLIC_WALLETCONNECT_PROJECT_ID` | WalletConnect project ID |
| `NEXT_PUBLIC_ALCHEMY_API_KEY_MAINNET` | Alchemy mainnet key |
| `NEXT_PUBLIC_ALCHEMY_API_KEY_SEPOLIA` | Alchemy Sepolia key |
| `NEXT_PUBLIC_ALCHEMY_API_KEY_POLYGON` | Alchemy Polygon key |
| `NEXT_PUBLIC_ESCROW_WALLET_ADDRESS` | Escrow wallet (shown in UI) |
| `NEXT_PUBLIC_APP_NAME` | App display name |
| `NEXT_PUBLIC_APP_VERSION` | App version string |
| `NEXT_PUBLIC_MAPBOX_API_KEY` | Mapbox key (address autocomplete) |
| `NEXT_PUBLIC_PASSKEY_RP_NAME` | WebAuthn RP name |
| `NEXT_PUBLIC_PASSKEY_RP_ID` | WebAuthn RP ID |
| `NEXT_PUBLIC_PASSKEY_ORIGIN` | WebAuthn origin |
| `NEXT_PUBLIC_BACKEND_URL` | Backend origin (direct calls) |
| `NEXT_PUBLIC_DEPAY_INTEGRATION_ID` | DePay integration ID |
| `NEXT_PUBLIC_IS_DEVELOPMENT` | Development flag |
| `NEXT_PUBLIC_ENABLE_DEBUG` | Enable client debug logging |
| `NEXT_PUBLIC_APP_URL` | Canonical app URL |
| `NEXT_PUBLIC_TELEGRAM_BOT_ID` | Telegram bot numeric ID |
| `BUILD_STATIC_EXPORT` | Enable `next export` mode (`false` for SSR) |

> `NEXT_PUBLIC_*` vars are baked into the frontend bundle at build time. Never put secrets in them. Frontend env changes require a fresh image build and redeploy.

### 8.25 Gatus

| Variable | Description |
|---|---|
| `GATUS_TELEGRAM_BOT_TOKEN` | Telegram bot for alert delivery |
| `GATUS_TELEGRAM_CHAT_ID` | Target chat ID for alerts |

---

## 9. Deploy Workflow

### 9.1 Normal image update (CI-driven — dev-amn)

Woodpecker CI builds backend and frontend images, pushes to `git.tbs.amn.gg/escrow/`, then triggers an Arcane GitOps sync which pulls the new image and recreates the container.

```
git push origin dev
  └─► Woodpecker build pipeline
        └─► docker push git.tbs.amn.gg/escrow/backend:dev
        └─► docker push git.tbs.amn.gg/escrow/frontend:dev
        └─► arcane-cli gitops sync cf6c9eab…  (or watchtower polls)
              └─► escrow-backend restarted with new image
              └─► escrow-frontend restarted with new image
```

> Always bump `package.json` version before pushing. See [[version_bump_before_ci]].

### 9.2 escrow-multi deploys (white-label stack)

**Always use Woodpecker. Never use manual rsync/docker-build/ssh for escrow-multi.**

```bash
# 1. Make changes, bump version in package.json
# 2. Commit
git commit -m "fix: description (vX.Y.Z)"
# 3. Push to Forgejo (remote is "forgejo", not "origin")
git push forgejo feature/white-label-shops
# 4. Monitor Woodpecker pipeline
source ~/CascadeProjects/escrow/.env
WOODPECKER_SERVER=$WOODPECKER_SERVER WOODPECKER_TOKEN=$WOODPECKER_TOKEN \
  woodpecker-cli pipeline ls escrow/backend
```

Frontend is a separate Woodpecker project (`escrow/frontend`). Both push targets trigger their respective pipelines.

### 9.3 Manual hotfix deploy (backend only — no registry cycle)

For urgent fixes without a full CI cycle, build locally on the server:

```bash
# 1. Copy changed files to build tree
scp -i ~/CascadeProjects/wzp src/services/auth/authRoutes.ts \
    root@89.58.32.32:/tmp/escrow-backend-build/src/services/auth/

# 2. Rebuild image on server (~3 min, ARM64)
ssh -i ~/CascadeProjects/wzp root@89.58.32.32 \
  "cd /tmp/escrow-backend-build && docker build -f Dockerfile.prod \
   -t escrow-backend-local:dev ."

# 3. Restart the backend container
ssh -i ~/CascadeProjects/wzp root@89.58.32.32 \
  "cd /opt/arcane/data/projects/escrow-dev && docker compose up -d backend"
```

The `docker-compose.override.yml` at `/opt/arcane/data/projects/escrow-dev/docker-compose.override.yml` sets `pull_policy: never` for `escrow-backend-local:dev` so watchtower never clobbers it.

### 9.4 Bringing the stack up/down

```bash
# via Arcane CLI (preferred)
arcane-cli project start devEscrow
arcane-cli project stop devEscrow

# via SSH + docker compose (direct)
ssh -i ~/CascadeProjects/wzp root@89.58.32.32 \
  "cd /opt/arcane/data/projects/escrow-dev && docker compose up -d"

ssh -i ~/CascadeProjects/wzp root@89.58.32.32 \
  "cd /opt/arcane/data/projects/escrow-dev && docker compose down"
```

### 9.5 Reloading Caddy after Caddyfile edits

```bash
ssh -i ~/CascadeProjects/wzp root@89.58.32.32 \
  "docker exec infra-caddy caddy reload --config /etc/caddy/Caddyfile"
```

No container restart needed.

### 9.6 Updating env vars

1. Edit `.env` on the server: `/opt/arcane/data/projects/escrow-dev/.env`
2. Restart affected container:
   ```bash
   ssh -i ~/CascadeProjects/wzp root@89.58.32.32 \
     "cd /opt/arcane/data/projects/escrow-dev && docker compose up -d backend"
   ```
3. Frontend `NEXT_PUBLIC_*` vars are baked at build time — they require a fresh image build and full redeploy via CI.

### 9.7 Verifying a deploy

```bash
# Check running containers
arcane-cli project status devEscrow

# Check backend version
curl https://dev.amn.gg/api/version

# Check health (all stores + RN registries)
curl https://dev.amn.gg/api/health | jq .

# Tail backend logs
ssh -i ~/CascadeProjects/wzp root@89.58.32.32 \
  "docker logs -f escrow-backend --tail 100"
```

> CI ✓ green does NOT guarantee the new image was pushed. Always verify `curl /api/version` returns the expected version. See [[woodpecker_silent_build_fail]].

---

## 10. Dev vs Prod Differences

| Aspect | dev-amn (`dev.amn.gg`) | Prod (`amn.gg`) |
|---|---|---|
| Compose file | `deployment/dev-amn/docker-compose.yml` | Separate prod stack (not in this repo) |
| Image registry | `git.tbs.amn.gg/escrow` | Same registry, prod tags |
| Image tag | `:dev` | `:latest` or versioned |
| MongoDB | Present (dev parity — retired in prod) | Not present |
| `ENABLE_TESTNET_CHAINS` | `true` (compose override) | Not set / `false` |
| `NODE_ENV` | `production` | `production` |
| `NEXT_PUBLIC_IS_DEVELOPMENT` | `false` | `false` |
| `PAYMENT_PROVIDER_MODE` | `live` | `live` |
| `REQUEST_NETWORK_ALLOW_TEST_WEBHOOKS` | May be `true` for RN testing | `false` |
| `TRANSACTION_SAFETY_MIN_CONFIRMATIONS` | `12` (default) | `12` (default) |
| Gatus monitoring | Monitors both dev + prod endpoints | Shared Gatus instance |
| TLS | Cloudflare proxy → Caddy (`disable_redirects`) | Same |
| Version bump | Required before CI push | Required |
| Watchtower labels | Present in legacy compose | Prod stack may differ |

---

## 11. Secret Management

The `.env` file on the server is the single source of runtime secrets. It is never committed.

- **Server location:** `/opt/arcane/data/projects/escrow-dev/.env`
- **Permissions:** `chmod 600`, owned by root
- **Repo template:** `deployment/.env` — contains live dev values, treated as low-sensitivity dev config; rotate all values before using in production

### Rules

1. Never commit `.env` or any file containing real tokens, passwords, or private keys.
2. Never pass secrets as Dockerfile `ARG`/`ENV` at build time — they appear in image layers. All secrets are runtime-injected via `env_file`.
3. `NEXT_PUBLIC_*` vars are baked into the frontend bundle. Never place secrets in them.
4. Wallet addresses are public on-chain but still kept out of the repo for operational hygiene.
5. For new deployments: copy `deployment/.env` to the server, fill in real values, `chmod 600`.
6. Gatus vars (`GATUS_TELEGRAM_BOT_TOKEN`, `GATUS_TELEGRAM_CHAT_ID`) go into the same `.env`.
7. Telegram bot tokens are high-value — rotate immediately if accidentally pushed.

### Sensitive variable groups

| Group | Variables | Risk if leaked |
|---|---|---|
| JWT | `JWT_SECRET` | Full session forgery |
| DB credentials | `POSTGRES_PASSWORD`, `REDIS_PASSWORD`, `MONGO_INITDB_ROOT_PASSWORD` | Database access |
| Payment webhook secrets | `REQUEST_NETWORK_WEBHOOK_SECRET`, `DEPAY_WEBHOOK_SECRET`, `SHKEEPER_CALLBACK_SECRET`, `SHKEEPER_WEBHOOK_SECRET` | Fake payment injection |
| Bot tokens | `TELEGRAM_BOT_TOKEN`, `TG_NOTIFY_BOT_TOKEN` | Bot takeover / webhook hijack |
| OAuth secrets | `GOOGLE_CLIENT_SECRET` | OAuth impersonation |
| API keys | `OPENAI_API_KEY`, `REQUEST_NETWORK_API_KEY`, `SHKEEPER_API_KEY` | Billing / data access |
| Sentry DSN | `SENTRY_DSN` | Error data exfiltration |