docs: add sub-project service docs + sync vault 2026-06-08
Add 10 - Services/ docs for all sub-projects: backend, frontend, scanner, deployment (new), update amanat-assist. Update Scanner Architecture, Telegram Mini App flow, and Activity Log. Add payment safety edge cases.
This commit is contained in:
Siavash Sameni
53
10 - Services/README.md
Normal file
53
10 - Services/README.md
Normal file
@@ -0,0 +1,53 @@
|
||||
# 10 - Services
|
||||
|
||||
This section documents each deployable service (sub-project) in the Amanat/Escrow platform. Each article covers the service's purpose, configuration, build process, and operational notes.
|
||||
|
||||
See also: [[01 - Architecture]] · [[08 - Operations]] · [[03 - API Reference]]
|
||||
|
||||
---
|
||||
|
||||
## Service Inventory
|
||||
|
||||
| Service | Language / Framework | Status | URL | Doc |
|
||||
|---|---|---|---|---|
|
||||
| Backend | Node.js / TypeScript (Express) | Live | `api.dev.amn.gg` | [[backend]] |
|
||||
| Frontend | Next.js / React / TypeScript | Live | `dev.amn.gg` | [[frontend]] |
|
||||
| Scanner | Go | Live | internal | [[scanner]] |
|
||||
| Amanat Assist | Node.js / TypeScript + LLM proxy | Live | `assist.dev.amn.gg` | [[amanat-assist]] |
|
||||
| Deployment | Docker Compose + Caddy + Watchtower | Live | — | [[deployment]] |
|
||||
|
||||
---
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
```
|
||||
Browser / Telegram Mini App
|
||||
│
|
||||
▼
|
||||
infra-caddy (reverse proxy, TLS)
|
||||
├── dev.amn.gg → [[frontend]] (Next.js SSR)
|
||||
├── api.dev.amn.gg → [[backend]] (Express REST + WebSocket)
|
||||
└── assist.dev.amn.gg → [[amanat-assist]] (LLM proxy / Telegram bot)
|
||||
|
||||
[[backend]]
|
||||
├── MongoDB / PostgreSQL (dual-write seam, PG cutover in progress)
|
||||
├── Redis (sessions, rate-limit, pub-sub)
|
||||
└── emits payment events
|
||||
│
|
||||
▼
|
||||
[[scanner]] (Go — watches EVM chains for on-chain payments)
|
||||
│ webhook callback
|
||||
└──────────────────▶ [[backend]] /api/payment/callback
|
||||
```
|
||||
|
||||
- All containers share the `shared-web` Docker network managed by [[deployment]].
|
||||
- [[amanat-assist]] is a separate Telegram Mini App; it calls [[backend]] APIs on behalf of users.
|
||||
- [[scanner]] is stateless; it probes RPC endpoints and forwards confirmations to the backend.
|
||||
|
||||
---
|
||||
|
||||
## Related Sections
|
||||
|
||||
- [[01 - Architecture]] — system-wide design decisions, data model, and sequence diagrams
|
||||
- [[03 - API Reference]] — REST endpoints, WebSocket events, auth headers
|
||||
- [[08 - Operations]] — deployment runbooks, monitoring, secrets management
|
||||
@@ -246,12 +246,17 @@ trigger: push/manual to main
|
||||
agent: linux/arm64 (same host as assist.amn.gg)
|
||||
|
||||
steps:
|
||||
1. build-frontend: npm ci + npm run build (Vite)
|
||||
2. deploy:
|
||||
1. build-frontend (node:22-alpine):
|
||||
- npm ci + npm run build (Vite)
|
||||
- Bakes VITE_ env vars into the static bundle at build time
|
||||
2. deploy (docker:27-cli, docker socket volume-mounted — no registry push):
|
||||
- Copy dist/ to /opt/amanat-assist/dist/ (nginx bind-mount)
|
||||
- Rebuild amanat-llm-proxy Docker image in-place
|
||||
- docker compose up -d --no-deps llm-proxy
|
||||
3. notify: Telegram CI notification
|
||||
- Sync docker-compose.yml to /opt/amanat-assist/
|
||||
- Rebuild amanat-llm-proxy Docker image in-place (locally, never pushed)
|
||||
- docker compose up -d (recreates llm-proxy container)
|
||||
3. notify (node:22-alpine):
|
||||
- Runs scripts/ci/tg-notify.cjs on success or failure
|
||||
- Uses TG_TOKEN + TG_USERS secrets
|
||||
```
|
||||
|
||||
Nginx picks up new static files from the bind-mount without restart.
|
||||
@@ -267,7 +272,7 @@ The proxy container is recreated with the new image.
|
||||
| `MISTRAL_API_KEY` | llm-proxy runtime | Mistral API key (server-side only) |
|
||||
| `KIMI_API_KEY` | llm-proxy runtime | Optional Kimi API key |
|
||||
| `DEEPSEEK_API_KEY` | llm-proxy runtime | Optional DeepSeek API key (auto-fallback) |
|
||||
| `OPENCODE_PROXY_URL` | llm-proxy runtime | OpenCode local proxy URL |
|
||||
| `OPENCODE_PROXY_URL` | llm-proxy runtime | OpenCode local proxy URL (default `http://127.0.0.1:3456`) |
|
||||
| `ALLOWED_ORIGINS` | llm-proxy runtime | CORS whitelist (comma-separated) |
|
||||
| `PORT` | llm-proxy runtime | Port (default 3001) |
|
||||
|
||||
@@ -294,3 +299,5 @@ See `src/sections/assist/` in the frontend repo for the implementation.
|
||||
- **Session storage is local only** — history lives in `localStorage`, not synced to backend
|
||||
- **Vision model not streaming** — responses may feel slow for image analysis
|
||||
- **categoryId from vision disabled** — vision returns category names, not ObjectIds; name→ID matching is left to the LLM in the follow-up turn
|
||||
- **llm-proxy is zero-dependency** — `llm-proxy/index.mjs` uses only Node.js built-ins (`http`, native `fetch`); no npm packages. Logs rotate at 10 MB.
|
||||
- **No registry push** — CI builds the llm-proxy image directly on the host via a docker socket volume mount; `docker pull` will always fail (intentional — image is local-only)
|
||||
|
||||
466
10 - Services/backend.md
Normal file
466
10 - Services/backend.md
Normal file
@@ -0,0 +1,466 @@
|
||||
# Backend Service — amn-backend
|
||||
|
||||
## 1. Overview
|
||||
|
||||
**amn-backend** is the Express 5 / TypeScript API server powering the Amanat escrow marketplace (`dev.amn.gg`). It handles all buyer-seller escrow workflow logic, crypto payment processing across multiple chains and providers, real-time socket events, authentication, admin tooling, and the in-progress Mongo→PostgreSQL migration.
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Current version | **2.10.5** |
|
||||
| Status | Active — production at `dev.amn.gg` |
|
||||
| Repo | `git@git.tbs.amn.gg:escrow/backend.git` |
|
||||
| Runtime port | 8083 (production Docker), 8080 (dev Docker), 5001 (dev default) |
|
||||
| Database | PostgreSQL (Drizzle ORM) — sole persistence layer as of v2.9.12 |
|
||||
| Node version | 22 (`.nvmrc`) |
|
||||
|
||||
PostgreSQL is the sole active database. MongoDB references remain in some env-var config for the dual-write seam during migration, but no Mongo-backed stores remain active in normal operation (all 11 repository domains use Drizzle repos).
|
||||
|
||||
---
|
||||
|
||||
## 2. Tech Stack
|
||||
|
||||
| Layer | Technology |
|
||||
|---|---|
|
||||
| Framework | Express 5 (TypeScript) |
|
||||
| Runtime | Node.js 22 |
|
||||
| Primary DB | PostgreSQL via Drizzle ORM (`drizzle-orm ^0.45.2`, `pg ^8.21.0`) |
|
||||
| Migrations | Drizzle Kit (`drizzle-kit ^0.31.1`) — 19 landed SQL migrations |
|
||||
| Session / Cache | Redis (`ioredis`) with Socket.IO pub-sub adapter |
|
||||
| Realtime | Socket.IO with Redis adapter (seller/buyer rooms) |
|
||||
| Auth | JWT (`jsonwebtoken`), Google OAuth, WebAuthn passkeys (`@simplewebauthn/server`), Telegram Mini App initData |
|
||||
| Crypto payments | Request Network, amn.scanner (in-house), DePay, SHKeeper |
|
||||
| Rate limiting | In-memory (express-rate-limit) — Redis adapter planned |
|
||||
| AI integration | OpenAI (listing descriptions, moderation) |
|
||||
| Email | Nodemailer via Resend SMTP |
|
||||
| Telegram | Bot webhook + Mini App session + identity linking |
|
||||
| Security | Helmet, CORS, Cloudflare Turnstile CAPTCHA, HMAC webhook verification |
|
||||
| Containerization | Docker (Dockerfile.prod, Dockerfile.dev) |
|
||||
| CI/CD | Woodpecker CI (4 pipelines) |
|
||||
|
||||
---
|
||||
|
||||
## 3. Directory Structure
|
||||
|
||||
```
|
||||
backend/src/
|
||||
├── app.ts # Express bootstrap, middleware chain, route registration, graceful shutdown
|
||||
├── cluster.ts # Node.js cluster mode entry point (multi-core)
|
||||
├── controllers/ # HTTP request handlers — thin layer, delegate to services
|
||||
├── db/ # Drizzle/Postgres layer
|
||||
│ ├── schema/ # Per-table Drizzle schema files + index.ts barrel
|
||||
│ ├── migrations/ # 19 numbered SQL migration files (0000–0018)
|
||||
│ └── repositories/ # Drizzle repos, factory.ts, backfill scripts, verify utilities
|
||||
├── infrastructure/
|
||||
│ └── socket/ # Socket.IO server init, room helpers, emit wrappers
|
||||
├── models/ # Removed — replaced by Drizzle schemas in db/schema/
|
||||
├── routes/ # Express Router definitions (mounted in app.ts)
|
||||
│ ├── amnScannerWebhookRoutes.ts
|
||||
│ ├── blogRoutes.ts
|
||||
│ ├── disputeRoutes.ts
|
||||
│ └── pointsRoutes.ts
|
||||
├── scripts/ # CLI utilities (seed:users, seed:categories, backfill, etc.)
|
||||
├── seeds/ # Seed data fixtures (Postgres-capable, store-aware, idempotent)
|
||||
├── services/ # Feature domain services (self-contained per domain)
|
||||
│ ├── address/ # Address management
|
||||
│ ├── admin/ # Admin-only operations, AML config, break-glass, data cleanup
|
||||
│ ├── ai/ # OpenAI integration (descriptions, moderation)
|
||||
│ ├── auth/ # JWT, OAuth, Passkey, Telegram, password reset
|
||||
│ ├── blockchain/ # Web3 read/verify helpers
|
||||
│ ├── blog/ # Posts, categories, comments
|
||||
│ ├── chat/ # Conversations, messages, attachments
|
||||
│ ├── config/ # Runtime config service
|
||||
│ ├── delivery/ # Delivery tracking
|
||||
│ ├── dispute/ # Dispute lifecycle, evidence, mediator assignment
|
||||
│ ├── email/ # Nodemailer transport + templates
|
||||
│ ├── file/ # Multer uploads, MIME validation
|
||||
│ ├── health/ # Health check endpoint logic
|
||||
│ ├── marketplace/ # PurchaseRequest, SellerOffer, Template, Shop
|
||||
│ ├── notification/ # Templates, delivery, mark-as-read
|
||||
│ ├── payment/ # Payment orchestration + provider adapters + ledger
|
||||
│ │ ├── adapters/ # Provider-neutral adapter interface + registry
|
||||
│ │ ├── amnScanner/ # amn.scanner in-house pay-in detection
|
||||
│ │ ├── ledger/ # Internal funds ledger (available / held / releasable)
|
||||
│ │ ├── migration/ # Legacy data backfill utilities
|
||||
│ │ ├── observability/ # Logging and incident controls
|
||||
│ │ ├── orchestration/ # High-level payment flow coordination
|
||||
│ │ ├── priceOracle/ # Chainlink + off-chain FX oracle, depeg protection
|
||||
│ │ ├── reconciliation/ # Webhook + status reconciliation per provider
|
||||
│ │ ├── request-network/# Request Network routes and webhook signature
|
||||
│ │ ├── requestNetwork/ # Request Network service logic
|
||||
│ │ ├── safety/ # Transaction Safety Provider + confirmation thresholds
|
||||
│ │ ├── tokens/ # On-chain token registry / decimals lookup
|
||||
│ │ └── wallets/ # Derived destination wallets + sweep orchestration
|
||||
│ ├── points/ # Loyalty points, levels, redemption
|
||||
│ ├── redis/ # Redis client, cache helpers, pub-sub
|
||||
│ ├── telegram/ # Bot webhook, Mini App session, identity linking, notifications
|
||||
│ ├── trezor/ # Trezor hardware-wallet signing for admin approvals
|
||||
│ └── user/ # Profile, preferences, addresses
|
||||
├── shared/
|
||||
│ ├── config/index.ts # Centralised typed env-var loader
|
||||
│ ├── middleware/ # authMiddleware, errorHandler, roleGuard, validators
|
||||
│ ├── types/ # Cross-cutting TypeScript types
|
||||
│ └── utils/response-handler.ts # Standard success/error response envelope
|
||||
└── utils/ # Pure utilities (logger, currencyUtils, etc.)
|
||||
```
|
||||
|
||||
Each service folder is self-contained: `<feature>Service.ts`, `<feature>Controller.ts`, `<feature>Routes.ts`, `<feature>Validation.ts`. This design allows future extraction to microservices with minimal coupling.
|
||||
|
||||
---
|
||||
|
||||
## 4. Key Services / Modules
|
||||
|
||||
| Module | Description |
|
||||
|---|---|
|
||||
| `services/auth/` | JWT issuance/refresh, Google OAuth, WebAuthn passkeys, Telegram initData verification, password reset |
|
||||
| `services/marketplace/` | Core escrow domain: PurchaseRequest, SellerOffer, Template, Shop lifecycle |
|
||||
| `services/payment/` | Payment orchestration, provider adapters, internal ledger, reconciliation |
|
||||
| `services/payment/ledger/` | Double-spend guard: tracks available / held / releasable balances per payment |
|
||||
| `services/payment/wallets/` | Derived destination address derivation (xpub) + sweep orchestration |
|
||||
| `services/payment/priceOracle/` | Chainlink + off-chain FX oracle for multi-currency pricing + stablecoin depeg protection |
|
||||
| `services/payment/safety/` | Transaction Safety Provider: confirmation thresholds, tx hash and transfer match enforcement |
|
||||
| `services/payment/amnScanner/` | In-house blockchain scanner webhook adapter (replaces Request Network for pay-in detection) |
|
||||
| `services/payment/requestNetwork/` | Request Network pay-in routes, webhook signature verification, invoice creation |
|
||||
| `infrastructure/socket/` | Socket.IO server init, buyer/seller room management, emit helpers |
|
||||
| `services/redis/` | Redis client wrapper, pub-sub channel helpers, session cache |
|
||||
| `services/chat/` | Conversations and message threading between buyer and seller |
|
||||
| `services/dispute/` | Dispute lifecycle: open, evidence, mediator, resolution |
|
||||
| `services/admin/` | Admin RBAC operations: AML config, break-glass, dispute management, data cleanup |
|
||||
| `services/telegram/` | Bot webhook handler, Mini App session auth, Telegram identity linking, push notifications |
|
||||
| `services/trezor/` | Trezor hardware-wallet approval gate for high-value admin actions (break-glass overrideable) |
|
||||
| `services/notification/` | In-app notification templates, delivery, mark-as-read |
|
||||
| `services/ai/` | OpenAI integration: AI-assisted listing descriptions and content moderation |
|
||||
| `services/email/` | Nodemailer transport via Resend SMTP, HTML email templates |
|
||||
| `services/points/` | Loyalty points engine, tier levels, redemption |
|
||||
| `services/blog/` | Blog posts, categories, comments |
|
||||
| `services/file/` | Multer-based file upload handler, MIME validation, upload path management |
|
||||
| `services/blockchain/` | Low-level Web3 read helpers: balance checks, tx confirmation polling |
|
||||
| `db/repositories/` | Drizzle ORM repository layer for all 11 domain entities |
|
||||
| `seeds/` | Idempotent Postgres seed fixtures for users, categories, shops, configs |
|
||||
| `scripts/` | CLI backfill, migration verify, seeding, and maintenance scripts |
|
||||
|
||||
---
|
||||
|
||||
## 5. API Surface Summary
|
||||
|
||||
All routes are mounted under `/api/*`. See [[03 - API Reference/API Overview]] for the full endpoint reference.
|
||||
|
||||
Key route groups:
|
||||
|
||||
| Prefix | Domain |
|
||||
|---|---|
|
||||
| `/api/auth/*` | Registration, login, OAuth, passkeys, Telegram auth |
|
||||
| `/api/payment/*` | Payment CRUD, status polling, provider webhooks |
|
||||
| `/api/payment/request-network/*` | Request Network webhook + invoice endpoints |
|
||||
| `/api/amn-scanner/*` | amn.scanner webhook receiver |
|
||||
| `/api/marketplace/*` | Purchase requests, seller offers, templates |
|
||||
| `/api/chat/*` | Conversations, messages, attachments |
|
||||
| `/api/dispute/*` | Dispute lifecycle |
|
||||
| `/api/admin/*` | Admin operations (role-gated) |
|
||||
| `/api/notification/*` | In-app notifications |
|
||||
| `/api/blog/*` | Blog posts and comments |
|
||||
| `/api/points/*` | Loyalty points |
|
||||
| `/api/user/*` | User profile, preferences |
|
||||
| `/health` | Docker healthcheck + active store mode listing |
|
||||
|
||||
**Rate limits (active):**
|
||||
|
||||
| Scope | Limit |
|
||||
|---|---|
|
||||
| Auth endpoints | 10 req / 15 min |
|
||||
| Payment endpoints | 30 req / 15 min |
|
||||
| AI endpoints | 20 req / 15 min |
|
||||
| Global | 100 req / 15 min |
|
||||
| `GET /api/payment/:id` | Exempt (polling route) |
|
||||
| RN + Telegram webhooks | Exempt from global limiter |
|
||||
|
||||
---
|
||||
|
||||
## 6. Database
|
||||
|
||||
### PostgreSQL (primary — active)
|
||||
|
||||
- **ORM:** Drizzle ORM (`drizzle-orm ^0.45.2`)
|
||||
- **Driver:** `pg ^8.21.0`
|
||||
- **Migrations:** 19 SQL files under `src/db/migrations/` (0000–0018), managed by `drizzle-kit`
|
||||
- **Schemas:** per-table files in `src/db/schema/`, exported via `index.ts` barrel
|
||||
- **Repositories:** `src/db/repositories/` — one Drizzle repo per domain; `factory.ts` provides DI
|
||||
- **Connection:** `PG_URL` env var (`postgres://user:pass@host:5432/db`)
|
||||
- **Migrations run:** `npx drizzle-kit migrate` (or via `drizzle.config.ts`)
|
||||
|
||||
### MongoDB (legacy — migration in progress)
|
||||
|
||||
MongoDB and Mongoose were removed at the code level as of v2.9.12. The `MONGO_CONNECT_MODE` env var and `*_STORE` vars remain for the dual-write seam but all active domain stores use Drizzle exclusively. Remaining migration work:
|
||||
|
||||
- Backfill execution for remaining legacy records
|
||||
- Per-domain read cutover verification
|
||||
- Chat domain normalization (current blocker)
|
||||
- Full runtime coupling severance
|
||||
|
||||
See [[PRD - Mongo Retirement (Full Nuke).md]] and [[MIGRATION_TODO.md]] for status.
|
||||
|
||||
---
|
||||
|
||||
## 7. Auth Model
|
||||
|
||||
| Method | Mechanism |
|
||||
|---|---|
|
||||
| Password | bcrypt hashed, JWT access + refresh token pair |
|
||||
| Google OAuth | OAuth 2.0 code flow via `google-auth-library` |
|
||||
| WebAuthn / Passkeys | `@simplewebauthn/server` — RP ID: `dev.amn.gg` |
|
||||
| Telegram Mini App | initData HMAC verification (bot token), replay window: 120 s, TTL: 24 h |
|
||||
| Telegram Bot | Webhook secret token header verification |
|
||||
| Sessions | Stateless JWT; refresh token stored in Redis |
|
||||
| CAPTCHA | Cloudflare Turnstile, triggered after 3 failed login attempts from same IP |
|
||||
|
||||
**RBAC roles:** `admin`, `buyer`, `seller`, `resolver`, `guard`
|
||||
|
||||
`roleGuard(role)` middleware is applied per-route after `authMiddleware`. The admin role unlocks break-glass, AML config, dispute management, and data cleanup endpoints.
|
||||
|
||||
**Trezor safekeeping:** when `TREZOR_SAFEKEEPING_REQUIRED=true`, high-value admin actions (release, refund, payout) require a Trezor-signed approval message. Break-glass overrides this for 1 hour and fires a Telegram alarm.
|
||||
|
||||
---
|
||||
|
||||
## 8. Realtime (Socket.IO)
|
||||
|
||||
- **Adapter:** Redis pub-sub (`@socket.io/redis-adapter`) — scales across multiple backend instances
|
||||
- **Init:** `infrastructure/socket/socketService.ts` — attaches to the HTTP server after Express bootstraps
|
||||
- **Room model:**
|
||||
- `buyer:<userId>` — buyer-facing events (payment status, offer updates, cart)
|
||||
- `seller:<userId>` — seller-facing events (new requests, offer accepted)
|
||||
- Admin rooms for dispute/notification broadcasts
|
||||
- **Auth:** Socket handshake verified with JWT before room join
|
||||
- **Known issue:** Global payment broadcasts previously wiped all users' carts (fixed in frontend v2.8.4 with a provider gate). Backend room-scoping is an open follow-up item.
|
||||
|
||||
Key emitted events (non-exhaustive):
|
||||
|
||||
| Event | Direction | Description |
|
||||
|---|---|---|
|
||||
| `payment:status` | Server → client | Payment state change (pending → confirmed → released) |
|
||||
| `offer:new` | Server → seller | New purchase request from buyer |
|
||||
| `offer:accepted` | Server → buyer | Seller accepted the offer |
|
||||
| `notification:new` | Server → client | In-app notification delivery |
|
||||
| `dispute:update` | Server → both | Dispute state change |
|
||||
| `chat:message` | Server → both | New chat message in conversation |
|
||||
|
||||
---
|
||||
|
||||
## 9. Payment Providers
|
||||
|
||||
The payment layer uses a provider-neutral adapter interface (`services/payment/adapters/`). All providers register in the adapter registry. The ledger (`services/payment/ledger/`) enforces double-spend prevention across all providers.
|
||||
|
||||
| Provider | Type | Chains | Status |
|
||||
|---|---|---|---|
|
||||
| **amn.scanner** | In-house blockchain scanner | ETH, BSC, Base, TON | Active — default for new payments when `AMN_SCANNER_DEFAULT=true` |
|
||||
| **Request Network** | Decentralized payment protocol | BSC (USDC/USDT) + ETH | Active — legacy in-flight payments; webhook-driven |
|
||||
| **DePay** | Widget-based crypto payments | Multi-chain | Available via adapter |
|
||||
| **SHKeeper** | Self-hosted crypto gateway | Bitcoin + EVM | Available via adapter |
|
||||
|
||||
### Payment flow
|
||||
|
||||
1. Buyer creates intent (`POST /api/payment`) → provider adapter creates invoice / watch address
|
||||
2. Provider webhook arrives → HMAC-verified → reconciliation service updates ledger
|
||||
3. Escrow holds funds → seller fulfills → admin/resolver releases or refunds
|
||||
4. Ledger enforces: held → releasable → released (no double-spend)
|
||||
|
||||
### amn.scanner specifics
|
||||
|
||||
- Webhook endpoint: `POST /api/amn-scanner/webhook`
|
||||
- HMAC verification via `AMN_SCANNER_WEBHOOK_SECRET`
|
||||
- Discriminator field: `payload.event` (not `eventType`) — always check this field
|
||||
- Provider scoped by `provider: "amn.scanner"` in payment records
|
||||
- Read token decimals on-chain, not from registry
|
||||
|
||||
### Request Network specifics
|
||||
|
||||
- Webhook endpoint: `POST /api/payment/request-network/webhook`
|
||||
- Webhook secret: `REQUEST_NETWORK_WEBHOOK_SECRET`
|
||||
- Network: BSC mainnet, currency: USDC
|
||||
- Canonical proxy addresses differ per chain (ETH: `0x370DE2…`, Base: `0x189219…`) — probe before trusting
|
||||
|
||||
### Safety layer
|
||||
|
||||
- `TRANSACTION_SAFETY_MIN_CONFIRMATIONS=12` (default)
|
||||
- Requires tx hash match and on-chain transfer match before releasing funds
|
||||
- AML screening: `none` (default), `ofac` (OFAC SDN list, local, free), or `chainalysis`
|
||||
|
||||
### Price oracle / depeg protection
|
||||
|
||||
- Providers: Chainlink + off-chain FX (`OFFCHAIN_FX_URL`)
|
||||
- Chains: ETH (RPC via `CHAINLINK_RPC_1`), BSC (via `CHAINLINK_RPC_56`)
|
||||
- Depeg hard cap: `DEPEG_HARD_CAP_BPS` (default 500 bps = 5%)
|
||||
- Oracle max staleness: `ORACLE_MAX_STALENESS_S=120`
|
||||
- Currently disabled (`ORACLE_QUOTING_ENABLED=false`) — enable after FX feeds are configured
|
||||
|
||||
---
|
||||
|
||||
## 10. CI/CD (Woodpecker)
|
||||
|
||||
Four pipelines in `backend/.woodpecker/`:
|
||||
|
||||
### `production.yml` — primary deploy pipeline
|
||||
|
||||
Trigger: `push` to `main`/`master` · Platform: `linux/arm64`
|
||||
|
||||
| Step | Description |
|
||||
|---|---|
|
||||
| `get-version` | Reads `package.json` version, writes `dev-<version>` to `.tags` |
|
||||
| `typecheck` | `npm ci` + `npm run typecheck` — gates image build on clean TypeScript (cached npm on host) |
|
||||
| `build-and-deploy` | `docker build -t git.tbs.amn.gg/escrow/backend:dev` locally on the agent, then `docker compose up -d --no-deps --pull never backend` — no registry push, image stays local |
|
||||
| `notify` | Posts plain-text result to Telegram via `scripts/ci/tg-notify.cjs` (no parse_mode) |
|
||||
|
||||
> No registry push on production pipeline — agent is co-located with the stack; pushing large images over Tailscale times out.
|
||||
|
||||
### `development.yml` — parked
|
||||
|
||||
Trigger: `event: cron` (no cron configured — effectively disabled). Targets legacy `git.manko.yoga` registry and retired Arcane deploy. Use `manual.yml` for manual playground builds.
|
||||
|
||||
### `manual.yml` — manual build playground
|
||||
|
||||
Trigger: manual. Builds and pushes to `git.tbs.amn.gg/escrow/backend`. Used for testing the pipeline independently.
|
||||
|
||||
### `cleanup.yml` — image cleanup
|
||||
|
||||
Trigger: scheduled/manual. Removes old image tags from the registry.
|
||||
|
||||
**Important CI notes:**
|
||||
- Always bump `package.json` version before pushing a CI-triggering commit — otherwise the build tag doesn't change and the deployed image may be stale.
|
||||
- CI green does not guarantee the image was pushed — verify `git.tbs.amn.gg` has the `dev-<version>` tag before trusting the deploy.
|
||||
- Woodpecker eats `${VAR}` in commands — use `$VAR` or `$$VAR`; prefer plugins over raw curl for notifications.
|
||||
|
||||
---
|
||||
|
||||
## 11. Local Development Quick-Start
|
||||
|
||||
```bash
|
||||
# Clone and install
|
||||
git clone git@git.tbs.amn.gg:escrow/backend.git
|
||||
cd backend
|
||||
npm install
|
||||
|
||||
# Copy environment file
|
||||
cp .env.example .env.local
|
||||
# Edit .env.local — set PG_URL, REDIS_URI, JWT_SECRET at minimum
|
||||
|
||||
# Start dependencies (Postgres + Redis)
|
||||
docker compose -f docker-compose.local.yml up -d
|
||||
|
||||
# Run DB migrations
|
||||
npx drizzle-kit migrate
|
||||
|
||||
# Start dev server (hot-reload)
|
||||
npm run dev
|
||||
# → listens on http://localhost:5001
|
||||
|
||||
# OR run in dev Docker
|
||||
docker compose -f docker-compose.dev.yml up
|
||||
# → listens on http://localhost:8080
|
||||
|
||||
# Seed database
|
||||
npm run seed:users
|
||||
npm run seed:categories
|
||||
```
|
||||
|
||||
**Typecheck (required before push):**
|
||||
```bash
|
||||
npm run typecheck
|
||||
```
|
||||
A pre-push git hook blocks the push on tsc errors. If a parallel agent's mid-refactor tree has errors, use explicit `git add <path>` — never `git add -A`.
|
||||
|
||||
**Run tests:**
|
||||
```bash
|
||||
npm test
|
||||
```
|
||||
Test files live in `__tests__/`.
|
||||
|
||||
---
|
||||
|
||||
## 12. Environment Variables
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `NODE_ENV` | `production` / `development` / `test` |
|
||||
| `PORT` | HTTP listen port (default 5001) |
|
||||
| `TRUST_PROXY_HOPS` | Number of reverse-proxy hops in front of app |
|
||||
| `FRONTEND_URL` | Allowed CORS origin for frontend |
|
||||
| `BACKEND_URL` | Public backend base URL |
|
||||
| `PG_URL` | PostgreSQL connection string |
|
||||
| `POSTGRES_USER` | Postgres username (Docker init) |
|
||||
| `POSTGRES_PASSWORD` | Postgres password (Docker init) |
|
||||
| `POSTGRES_DB` | Postgres database name (Docker init) |
|
||||
| `MONGO_CONNECT_MODE` | `always` / `never` / `optional` — Mongo connection behavior (legacy) |
|
||||
| `REDIS_URI` | Redis connection URI |
|
||||
| `JWT_SECRET` | HS256 signing secret for access tokens |
|
||||
| `JWT_EXPIRES_IN` | Access token TTL (e.g. `7d`) |
|
||||
| `REFRESH_TOKEN_EXPIRES_IN` | Refresh token TTL (e.g. `30d`) |
|
||||
| `ADMIN_EMAIL` | Bootstrap admin account email |
|
||||
| `ADMIN_PASSWORD` | Bootstrap admin account password |
|
||||
| `SEED_USERS` | `true` to auto-seed users on dev boot |
|
||||
| `SEED_PASSWORD_ADMIN` | Admin seed account password |
|
||||
| `SEED_PASSWORD_SUPPORT` | Support seed account password |
|
||||
| `SEED_PASSWORD_BUYER` | Buyer seed account password |
|
||||
| `SEED_PASSWORD_SELLER` | Seller seed account password |
|
||||
| `GOOGLE_CLIENT_ID` | Google OAuth client ID |
|
||||
| `GOOGLE_CLIENT_SECRET` | Google OAuth client secret |
|
||||
| `WEBAUTHN_RP_ID` | WebAuthn relying party ID (e.g. `dev.amn.gg`) |
|
||||
| `WEBAUTHN_RP_NAME` | WebAuthn relying party display name |
|
||||
| `WEBAUTHN_RP_ORIGIN` | WebAuthn allowed origin |
|
||||
| `SMTP_HOST` | SMTP server host |
|
||||
| `SMTP_PORT` | SMTP server port |
|
||||
| `SMTP_SECURE` | `true` for TLS |
|
||||
| `SMTP_USER` | SMTP username |
|
||||
| `SMTP_PASS` | SMTP password |
|
||||
| `SMTP_FROM` | From address for outgoing email |
|
||||
| `RESEND_WEBHOOK_SECRET` | Resend inbound webhook signing secret (`whsec_…`) |
|
||||
| `TURNSTILE_SECRET_KEY` | Cloudflare Turnstile server-side secret (empty = CAPTCHA disabled) |
|
||||
| `RATE_LIMIT_WINDOW_MS` | Rate limit window in milliseconds |
|
||||
| `RATE_LIMIT_MAX_REQUESTS` | Max requests per window (global) |
|
||||
| `MAX_FILE_SIZE` | Upload max file size in bytes |
|
||||
| `UPLOAD_PATH` | Server-side upload directory |
|
||||
| `PAYMENT_PROVIDER_MODE` | `live` / `test` |
|
||||
| `PAYMENT_LEDGER_ENFORCEMENT` | `true` to enforce double-spend ledger guard |
|
||||
| `ESCROW_WALLET_ADDRESS` | Platform escrow wallet address |
|
||||
| `RECEIVER_WALLET_ADDRESS` | Platform receiver wallet address |
|
||||
| `REQUEST_NETWORK_ENABLED` | Enable Request Network provider |
|
||||
| `REQUEST_NETWORK_API_KEY` | Request Network API key |
|
||||
| `REQUEST_NETWORK_NETWORK` | Target chain (`bsc`, `eth`, etc.) |
|
||||
| `REQUEST_NETWORK_WEBHOOK_SECRET` | HMAC secret for RN webhook verification |
|
||||
| `AMN_SCANNER_URL` | amn.scanner service base URL |
|
||||
| `AMN_SCANNER_WEBHOOK_SECRET` | HMAC secret for scanner webhook verification |
|
||||
| `AMN_SCANNER_DEFAULT` | `true` to make amn.scanner the default provider |
|
||||
| `ORACLE_QUOTING_ENABLED` | Enable on-chain oracle pricing + depeg protection |
|
||||
| `PRICE_ORACLE_PROVIDERS` | Comma-separated oracle providers (`chainlink,offchain_fx`) |
|
||||
| `ORACLE_MAX_STALENESS_S` | Max oracle data age in seconds |
|
||||
| `DEPEG_HARD_CAP_BPS` | Stablecoin depeg hard cap in basis points |
|
||||
| `OFFCHAIN_FX_URL` | Off-chain FX rate source URL (required for IRR/TRY) |
|
||||
| `CHAINLINK_RPC_1` | Private RPC override for Chainlink on ETH mainnet |
|
||||
| `CHAINLINK_RPC_56` | Private RPC override for Chainlink on BSC |
|
||||
| `DERIVED_DESTINATION_XPUB` | xPub for derived payment address derivation |
|
||||
| `DERIVED_DESTINATION_SWEEP_SIGNER` | Sweep signing mode: `build-only` / `hot-key` / `kms` / `trezor` |
|
||||
| `DERIVED_DESTINATION_SWEEP_INTERVAL_MS` | Sweep cron interval in ms (0 = disabled) |
|
||||
| `SWEEP_MASTER_PRIVKEY` | Master sweep wallet private key (gas funder) |
|
||||
| `TREZOR_SAFEKEEPING_REQUIRED` | `true` to require Trezor approval for admin actions |
|
||||
| `TRANSACTION_SAFETY_ENABLED` | Enable transaction safety layer |
|
||||
| `TRANSACTION_SAFETY_MIN_CONFIRMATIONS` | Minimum on-chain confirmations before release |
|
||||
| `TRANSACTION_SAFETY_AML_PROVIDER` | AML provider: `none` / `ofac` / `chainalysis` |
|
||||
| `CHAINALYSIS_API_KEY` | Chainalysis API key (when AML provider = chainalysis) |
|
||||
| `TELEGRAM_BOT_TOKEN` | Telegram bot token |
|
||||
| `TELEGRAM_WEBHOOK_SECRET_TOKEN` | Telegram webhook secret token header value |
|
||||
| `TELEGRAM_INITDATA_MAX_AGE_SEC` | Max age for Telegram initData (default 86400 s) |
|
||||
| `TG_NOTIFY_CHATS` | Comma-separated Telegram chat IDs for CI/admin notifications |
|
||||
|
||||
---
|
||||
|
||||
## 13. Known Issues / Open Items
|
||||
|
||||
| Issue | Status | Notes |
|
||||
|---|---|---|
|
||||
| Mongo→PG migration incomplete | In progress | Chat normalization is the current blocker; read cutover and backfill exec pending |
|
||||
| Backend room-scoping for socket events | Open | Frontend provider gate is in place (v2.8.4); backend should scope payment events to `seller:<id>` rooms to prevent cross-user leakage |
|
||||
| Rate limit counters are in-memory | Open | Not shared across instances; Redis adapter planned for distributed deployments |
|
||||
| Oracle quoting disabled | Open | `ORACLE_QUOTING_ENABLED=false`; requires FX feed configuration before enabling |
|
||||
| amn.scanner multi-seller + multi-chain gap | Open | Current scanner watches one chain; multi-seller and multi-chain support not yet verified |
|
||||
| Woodpecker development.yml parked | Known | Targets legacy registry; needs repointing to `git.tbs.amn.gg` and new Arcane deploy before re-enabling |
|
||||
| Trezor safekeeping off by default | By design | `TREZOR_SAFEKEEPING_REQUIRED=false`; must be enabled explicitly in production once admin xpub is registered |
|
||||
| Request Network canonical proxy addresses | Known | RN's CREATE2 canonical-address claim is false for ETH and Base — probe actual address before trusting |
|
||||
| JSON assets not copied to dist/ | Fixed (requires postbuild) | `tsc` does not copy `.json` files; explicit `postbuild` copy step required for any `fs.readFileSync` on JSON assets |
|
||||
| Parallel agent push conflicts | Operational | mojtaba agent pushes to same branches; always `git fetch --rebase` before pushing; expect version-bump conflicts |
|
||||
620
10 - Services/deployment.md
Normal file
620
10 - Services/deployment.md
Normal file
@@ -0,0 +1,620 @@
|
||||
---
|
||||
title: Deployment
|
||||
tags: [services, deployment, infrastructure, docker]
|
||||
---
|
||||
|
||||
# Deployment
|
||||
|
||||
The `deployment/` sub-project contains all Docker Compose definitions, Caddyfile configurations, Gatus monitoring config, and environment templates for running the Amanat escrow platform. Two compose files exist side-by-side reflecting a legacy setup and the current live stack.
|
||||
|
||||
---
|
||||
|
||||
## 1. Overview
|
||||
|
||||
| File | Status | Host | Notes |
|
||||
|---|---|---|---|
|
||||
| `deployment/docker-compose.yml` | Legacy | Any | nginx + traefik_public network, images from `git.manko.yoga` registry |
|
||||
| `deployment/dev-amn/docker-compose.yml` | **Active** | `89.58.32.32` | shared-web + infra-caddy ingress, images from `git.tbs.amn.gg/escrow` |
|
||||
|
||||
The `dev-amn` stack is the authoritative deployment. It runs under Arcane project **devEscrow** (`77c10db2…`) on the ARM64 host at `89.58.32.32`. All operational decisions, env var edits, and container restarts target this stack.
|
||||
|
||||
The legacy compose (`deployment/docker-compose.yml`) is kept for historical reference. It uses an nginx sidecar, Traefik labels, and images from the old `git.manko.yoga` registry. Do not deploy from it.
|
||||
|
||||
---
|
||||
|
||||
## 2. Services
|
||||
|
||||
| Service | Image | Internal Port | Role |
|
||||
|---|---|---|---|
|
||||
| `backend` | `git.tbs.amn.gg/escrow/backend:dev` | 5001 | Express 5 API + Socket.IO + admin seed |
|
||||
| `frontend` | `git.tbs.amn.gg/escrow/frontend:dev` | 8083 | Next.js SSR app |
|
||||
| `refscanner` | `git.tbs.amn.gg/escrow/scanner:dev` | 8080 | In-house AMN payment scanner (SQLite) |
|
||||
| `postgres` | `postgres:18-alpine` | 5432 | Primary datastore (auth, marketplace, PG stores) |
|
||||
| `redis` | `redis:8-alpine` | 6379 | Session cache, rate-limit counters, job queues |
|
||||
| `mongodb` | `mongo:8.0-noble` | 27017 | Legacy datastore — dev boot parity only, retired in prod |
|
||||
| `gatus` | `twinproduction/gatus:latest` | 8080 (mapped 8084) | Uptime monitoring + Telegram alerting |
|
||||
|
||||
> **Note on refscanner:** The in-house scanner (`provider: "amn.scanner"`) persists state in a SQLite file at `/data/scanner.db` inside the container. It does not expose a port on `shared-web`; the backend calls it via the `default` bridge by container alias `refscanner`.
|
||||
|
||||
> **Note on mongodb:** The Mongo container is retained for dev stack parity because `MONGODB_URI` is still present in the env. It will be removed once the backend's remaining Mongo reads are migrated to Postgres. See [[mongo-to-pg-migration-guide]] and [[mongo_retirement_status]].
|
||||
|
||||
---
|
||||
|
||||
## 3. Architecture Diagram
|
||||
|
||||
```
|
||||
Internet (HTTPS 443 / HTTP 80)
|
||||
│
|
||||
▼
|
||||
┌───────────────────────────┐
|
||||
│ Cloudflare CDN / Proxy │
|
||||
│ amn.gg / dev.amn.gg │
|
||||
└─────────────┬─────────────┘
|
||||
│
|
||||
▼ (origin)
|
||||
┌─────────────────────────────────────────────────┐
|
||||
│ Host: 89.58.32.32 │
|
||||
│ │
|
||||
│ ┌────────────────────────────────────────────┐ │
|
||||
│ │ infra-caddy (Arcane project "infra") │ │
|
||||
│ │ ports 80:80, 443:443 on host │ │
|
||||
│ │ reads Caddyfile at │ │
|
||||
│ │ /opt/arcane/data/projects/infra/Caddyfile │ │
|
||||
│ └───┬───────────────────────────┬────────────┘ │
|
||||
│ │ /api/* /socket.io/* │ /* │
|
||||
│ │ /uploads/* │ │
|
||||
│ ▼ ▼ │
|
||||
│ ┌────────────┐ ┌────────────────┐ │
|
||||
│ │ backend │ │ frontend │ │
|
||||
│ │ :5001 │ │ :8083 │ │
|
||||
│ │ shared-web │ │ shared-web │ │
|
||||
│ └──┬──┬──┬───┘ └────────────────┘ │
|
||||
│ │ │ │ │
|
||||
│ │ │ └──────────────────────┐ │
|
||||
│ │ │ ▼ │
|
||||
│ │ │ ┌────────────────────┐ │
|
||||
│ │ │ │ refscanner │ │
|
||||
│ │ │ │ :8080 (default │ │
|
||||
│ │ │ │ bridge only) │ │
|
||||
│ │ │ └────────────────────┘ │
|
||||
│ │ │ │
|
||||
│ ▼ ▼ │
|
||||
│ ┌──────────┐ ┌──────────┐ ┌───────────────┐ │
|
||||
│ │ postgres │ │ redis │ │ mongodb │ │
|
||||
│ │ :5432 │ │ :6379 │ │ :27017 │ │
|
||||
│ │ (default │ │ (default │ │ (default only,│ │
|
||||
│ │ only) │ │ only) │ │ legacy) │ │
|
||||
│ └──────────┘ └──────────┘ └───────────────┘ │
|
||||
│ │
|
||||
│ ┌────────────────────────────────────────────┐ │
|
||||
│ │ gatus :8084 (mapped from :8080) │ │
|
||||
│ │ monitors dev.amn.gg + amn.gg + external │ │
|
||||
│ └────────────────────────────────────────────┘ │
|
||||
└─────────────────────────────────────────────────┘
|
||||
|
||||
Networks:
|
||||
shared-web ─── external, attached: backend, frontend
|
||||
default ─── internal bridge: all services
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 4. Networks
|
||||
|
||||
| Network | Type | Services Attached | Purpose |
|
||||
|---|---|---|---|
|
||||
| `default` (bridge) | Internal | All services | Container-to-container communication |
|
||||
| `shared-web` | External (pre-existing) | `backend`, `frontend` | Allows infra-caddy to proxy by container name |
|
||||
| `traefik_public` | External (legacy only) | nginx, gatus (legacy compose) | Old Traefik-based ingress on `git.manko.yoga` host |
|
||||
|
||||
**Key rules:**
|
||||
- `postgres`, `redis`, `mongodb` are on `default` only — no external exposure.
|
||||
- `refscanner` is on `default` only; backend reaches it via alias `refscanner:8080`.
|
||||
- Any new public-facing service must join `shared-web` AND get a Caddyfile block. See [[Shared Infra (89.58.32.32)]] and section 6 below.
|
||||
- `shared-web` must exist on the host before `docker compose up`. It is created by the `infra` project.
|
||||
|
||||
---
|
||||
|
||||
## 5. Volumes and Bind Mounts
|
||||
|
||||
All data volumes in the `dev-amn` stack use relative bind mounts under `./data/` (resolved to `/opt/arcane/data/projects/escrow-dev/data/` on the server):
|
||||
|
||||
| Service | Host Path | Container Path | Notes |
|
||||
|---|---|---|---|
|
||||
| `backend` | `./data/uploads` | `/app/uploads` | User-uploaded files |
|
||||
| `refscanner` | `./data/scanner` | `/data` | SQLite DB at `/data/scanner.db` |
|
||||
| `postgres` | `./data/postgres` | `/var/lib/postgresql/data` | `PGDATA` subdir workaround: actual data at `./data/postgres/pgdata` |
|
||||
| `redis` | `./data/redis` | `/data` | Persistence dump |
|
||||
| `mongodb` | `./data/mongo` | `/data/db` | Legacy; can be deleted once Mongo retired |
|
||||
| `gatus` | `./gatus/config.yaml` | `/config/config.yaml` (ro) | Monitoring config — part of repo |
|
||||
|
||||
**Postgres volume note:** `postgres:18` introduced a version-scoped data directory layout and refuses to init directly into a volume root that already contains files from a different layout. The compose file sets `PGDATA=/var/lib/postgresql/data/pgdata` to place actual data in a subdirectory of the mount, avoiding init conflicts.
|
||||
|
||||
**Legacy compose** (`deployment/docker-compose.yml`) uses absolute host paths under `/var/data/escrowDev/` and does not share volumes with the dev-amn stack.
|
||||
|
||||
---
|
||||
|
||||
## 6. Reverse Proxy (infra-caddy) Integration
|
||||
|
||||
Ingress for `89.58.32.32` is handled exclusively by **infra-caddy** — the Caddy container in the Arcane project `infra`. It owns host ports 80 and 443. No service should bind those ports directly.
|
||||
|
||||
### Current Caddyfile block (dev.amn.gg)
|
||||
|
||||
Located at `/opt/arcane/data/projects/infra/Caddyfile` on the server (and mirrored in `deployment/dev-amn/Caddyfile` for reference):
|
||||
|
||||
```
|
||||
{
|
||||
email manwe@manko.yoga
|
||||
auto_https disable_redirects
|
||||
}
|
||||
|
||||
dev.amn.gg {
|
||||
encode zstd gzip
|
||||
|
||||
@backend path /api/* /socket.io/* /uploads/*
|
||||
reverse_proxy @backend backend:5001
|
||||
|
||||
reverse_proxy frontend:8083
|
||||
}
|
||||
```
|
||||
|
||||
- `auto_https disable_redirects` — Cloudflare proxy sits in front; Caddy should not force HTTP→HTTPS redirects at origin.
|
||||
- Routes by path prefix: `/api/*`, `/socket.io/*`, `/uploads/*` go to `backend:5001`; everything else to `frontend:8083`.
|
||||
- Container names are resolved via the `shared-web` network.
|
||||
|
||||
### Adding a new public service
|
||||
|
||||
1. Add the service to `deployment/dev-amn/docker-compose.yml` with `networks: shared-web: {}`.
|
||||
2. Edit `/opt/arcane/data/projects/infra/Caddyfile` on the server — add a new vhost block or path matcher.
|
||||
3. Reload Caddy (no restart needed):
|
||||
```bash
|
||||
docker exec infra-caddy caddy reload --config /etc/caddy/Caddyfile
|
||||
```
|
||||
4. Verify via `curl -I https://dev.amn.gg/<new-path>`.
|
||||
|
||||
---
|
||||
|
||||
## 7. Gatus Monitoring
|
||||
|
||||
Gatus runs as a sidecar in the `default` network. Config is at `deployment/gatus/config.yaml` (bind-mounted read-only). Alerts are delivered via Telegram.
|
||||
|
||||
### Alert policy
|
||||
|
||||
| Setting | Value |
|
||||
|---|---|
|
||||
| Default failure threshold | 3 consecutive failures |
|
||||
| Default success threshold | 2 consecutive successes |
|
||||
| Send on resolved | Yes |
|
||||
| Alert channel | Telegram (`GATUS_TELEGRAM_BOT_TOKEN` / `GATUS_TELEGRAM_CHAT_ID`) |
|
||||
|
||||
### Monitored endpoints
|
||||
|
||||
| Name | Group | URL | Interval | Key Conditions |
|
||||
|---|---|---|---|---|
|
||||
| `backend-dev-version` | backend-dev | `https://dev.amn.gg/api/version` | 60s | HTTP 200, body.version not empty |
|
||||
| `backend-dev-health` | backend-dev | `https://dev.amn.gg/api/health` | 30s | HTTP 200, all PG store modes = postgres, redis ok, RN chain+token registry loaded |
|
||||
| `backend-prod-version` | backend-prod | `https://amn.gg/api/version` | 60s | HTTP 200, body.version not empty (failure-threshold 2) |
|
||||
| `backend-prod-health` | backend-prod | `https://amn.gg/api/health` | 30s | HTTP 200, db/postgres/redis/RN registries ok (failure-threshold 2) |
|
||||
| `frontend-dev` | frontend | `https://dev.amn.gg/` | 60s | HTTP 200, response < 3000ms |
|
||||
| `frontend-prod` | frontend | `https://amn.gg/` | 60s | HTTP 200, response < 3000ms (failure-threshold 2) |
|
||||
| `rn-api-reachable` | external | `https://api.request.network/v2/health` | 5m | HTTP 200/401/404 (accepts auth errors — just checks reachability) |
|
||||
| `chainalysis-public-api` | external | `https://public.chainalysis.com/api/v1/address/0x000…` | 5m | HTTP 200 or 404 |
|
||||
| `bsc-rpc-publicnode` | external | `https://bsc-rpc.publicnode.com` (POST) | 2m | HTTP 200, `result == "0x38"` (BSC mainnet chain ID) |
|
||||
|
||||
The `backend-dev-health` check validates that **all 8 domain stores are running on Postgres** (`auth`, `config`, `address`, `category`, `levelConfig`, `shopSettings`, `review`, `notification`). A failure here means a store mode regression or a broken `PG_URL`.
|
||||
|
||||
Gatus dashboard is accessible at `:8084` on the host (not publicly proxied by default — access via SSH tunnel or add a Caddyfile block if needed).
|
||||
|
||||
---
|
||||
|
||||
## 8. Environment Variables
|
||||
|
||||
All vars are passed to containers via `.env` at the stack root (`deployment/dev-amn/.env` on the server, `deployment/.env` in the repo as the live dev reference). The file is `chmod 600` and never committed.
|
||||
|
||||
### Backend
|
||||
|
||||
| Variable | Description | Example / Default |
|
||||
|---|---|---|
|
||||
| `NODE_ENV` | Runtime environment | `production` |
|
||||
| `PORT` | Express listen port | `5001` |
|
||||
| `TRUST_PROXY` | Express trust-proxy (required behind Caddy) | `true` |
|
||||
| `DEBUG` | Debug namespaces | _(empty)_ |
|
||||
| `LOG_LEVEL` | Winston log level | `info` |
|
||||
|
||||
#### Database
|
||||
|
||||
| Variable | Description | Example |
|
||||
|---|---|---|
|
||||
| `MONGODB_URI` | Mongo connection string | `mongodb://admin:pass@mongodb:27017/marketplace?authSource=admin` |
|
||||
| `MONGO_INITDB_ROOT_USERNAME` | Mongo root user | `admin` |
|
||||
| `MONGO_INITDB_ROOT_PASSWORD` | Mongo root password | — |
|
||||
| `MONGO_INITDB_DATABASE` | Mongo init database | `marketplace` |
|
||||
| `DB_NAME` | Mongo database name used by app | `amn-db` |
|
||||
| `PG_URL` | Postgres DSN | `postgres://amanat:pass@amanat-postgres:5432/amanat_dev` |
|
||||
| `POSTGRES_USER` | Postgres superuser | `amanat` |
|
||||
| `POSTGRES_PASSWORD` | Postgres superuser password | — |
|
||||
| `POSTGRES_DB` | Postgres database name | `amanat_dev` |
|
||||
| `AUTO_SEED_ON_START` | Run seed on boot | `true` |
|
||||
|
||||
#### Store modes (dual-write seam)
|
||||
|
||||
| Variable | Description | Default |
|
||||
|---|---|---|
|
||||
| `AUTH_STORE` | Auth domain store backend | `postgres` |
|
||||
| `CONFIG_STORE` | Config domain | `postgres` |
|
||||
| `ADDRESS_STORE` | Address domain | `postgres` |
|
||||
| `CATEGORY_STORE` | Category domain | `postgres` |
|
||||
| `LEVEL_CONFIG_STORE` | Level config domain | `postgres` |
|
||||
| `SHOP_SETTINGS_STORE` | Shop settings domain | `postgres` |
|
||||
| `REVIEW_STORE` | Review domain | `postgres` |
|
||||
| `NOTIFICATION_STORE` | Notification domain | `postgres` |
|
||||
|
||||
#### Auth / Sessions
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `JWT_SECRET` | JWT signing secret |
|
||||
| `JWT_EXPIRES_IN` | Access token TTL (e.g. `7d`) |
|
||||
| `REFRESH_TOKEN_EXPIRES_IN` | Refresh token TTL (e.g. `30d`) |
|
||||
|
||||
#### Redis
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `REDIS_URI` | Redis connection string (includes password) |
|
||||
| `REDIS_PASSWORD` | Redis auth password (standalone, if not in URI) |
|
||||
|
||||
#### URLs / CORS
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `BASE_URL` | Canonical origin (`https://dev.amn.gg`) |
|
||||
| `API_URL` | API base URL |
|
||||
| `FRONTEND_URL` | Frontend origin |
|
||||
| `BACKEND_URL` | Backend origin |
|
||||
| `CORS_ORIGIN` | Allowed CORS origin |
|
||||
|
||||
#### File uploads
|
||||
|
||||
| Variable | Description | Default |
|
||||
|---|---|---|
|
||||
| `UPLOAD_PATH` | Upload directory inside container | `/app/uploads` |
|
||||
| `MAX_FILE_SIZE` | Max upload bytes | `52428800` (50 MB) |
|
||||
|
||||
#### Rate limiting
|
||||
|
||||
| Variable | Description | Default |
|
||||
|---|---|---|
|
||||
| `RATE_LIMIT_WINDOW_MS` | Window for rate limiter | `900000` (15 min) |
|
||||
| `RATE_LIMIT_MAX_REQUESTS` | Max requests per window | `100` |
|
||||
|
||||
> GET `/api/payment/:id` must bypass `paymentLimiter` — see [[backend_rate_limits]].
|
||||
|
||||
#### SMTP
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `SMTP_HOST` | SMTP server hostname |
|
||||
| `SMTP_PORT` | SMTP port |
|
||||
| `SMTP_SECURE` | TLS (`true`/`false`) |
|
||||
| `SMTP_USER` | SMTP username |
|
||||
| `SMTP_PASS` | SMTP password |
|
||||
| `SMTP_FROM` | From address |
|
||||
|
||||
#### WebAuthn (Passkeys)
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `WEBAUTHN_RP_ID` | Relying party ID (domain) |
|
||||
| `WEBAUTHN_RP_NAME` | Relying party display name |
|
||||
| `WEBAUTHN_RP_ORIGIN` | Relying party origin URL |
|
||||
|
||||
#### Admin seed
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `ADMIN_EMAIL` | Bootstrap admin email |
|
||||
| `ADMIN_PASSWORD` | Bootstrap admin password |
|
||||
| `ADMIN_FIRST_NAME` | Admin first name |
|
||||
| `ADMIN_LAST_NAME` | Admin last name |
|
||||
|
||||
#### Google OAuth
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `GOOGLE_CLIENT_ID` | Google OAuth client ID |
|
||||
| `GOOGLE_CLIENT_SECRET` | Google OAuth client secret |
|
||||
|
||||
#### OpenAI
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `OPENAI_API_KEY` | OpenAI API key |
|
||||
| `OPENAI_DEFAULT_MODEL` | Default model (e.g. `gpt-4`) |
|
||||
| `OPENAI_MAX_TOKENS` | Max tokens per request |
|
||||
| `OPENAI_TEMPERATURE` | Sampling temperature |
|
||||
|
||||
#### Sentry
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `SENTRY_DSN` | Sentry ingest DSN |
|
||||
|
||||
#### Wallets / Blockchain
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `ESCROW_WALLET_ADDRESS` | Platform escrow wallet |
|
||||
| `BSC_USDT_CONTRACT` | BSC USDT token contract address |
|
||||
| `ADMIN_PAYOUT_WALLET_ADDRESS` | Admin payout destination |
|
||||
| `RECEIVER_WALLET_ADDRESS` | Default receiver wallet |
|
||||
|
||||
#### DePay
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `DEPAY_INTEGRATION_ID` | DePay integration UUID |
|
||||
| `DEPAY_WEBHOOK_SECRET` | Webhook verification secret |
|
||||
| `DEPAY_NETWORKS` | Enabled chains (e.g. `bsc`) |
|
||||
| `DEPAY_ALLOWED_TOKENS` | Allowed payment tokens |
|
||||
| `DEPAY_PUBLIC_KEY` | DePay public key (PEM) |
|
||||
|
||||
#### SHKeeper
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `SHKEEPER_API_KEY` | SHKeeper API key |
|
||||
| `SHKEEPER_BASE_URL` | SHKeeper service base URL |
|
||||
| `SHKEEPER_API_URL` | Payment request endpoint |
|
||||
| `SHKEEPER_ENVIRONMENT` | `production` or `sandbox` |
|
||||
| `SHKEEPER_WALLET_ID` | Destination wallet |
|
||||
| `SHKEEPER_NETWORKS` | Enabled chains |
|
||||
| `SHKEEPER_ALLOWED_TOKENS` | Allowed tokens |
|
||||
| `SHKEEPER_FORCE_REAL` | Bypass test mode |
|
||||
| `SHKEEPER_TOKEN` | Token type (e.g. `USDT`) |
|
||||
| `SHKEEPER_CALLBACK_SECRET` | Callback verification secret |
|
||||
| `SHKEEPER_WEBHOOK_SECRET` | Webhook verification secret |
|
||||
|
||||
#### Request Network
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `REQUEST_NETWORK_ENABLED` | Enable RN provider |
|
||||
| `REQUEST_NETWORK_WEBHOOK_SECRET` | Webhook signature secret |
|
||||
| `REQUEST_NETWORK_WEBHOOK_CALLBACK_URL` | Public callback URL |
|
||||
| `REQUEST_NETWORK_PAYMENT_CURRENCY` | Currency (e.g. `USDC`) |
|
||||
| `REQUEST_NETWORK_NETWORK` | Chain (e.g. `bsc`) |
|
||||
| `REQUEST_NETWORK_MERCHANT_REFERENCE` | Merchant address reference |
|
||||
| `REQUEST_NETWORK_API_BASE_URL` | RN API root |
|
||||
| `REQUEST_NETWORK_API_KEY` | RN API key |
|
||||
| `REQUEST_NETWORK_ORIGIN` | Origin header sent to RN |
|
||||
| `REQUEST_NETWORK_ALLOW_TEST_WEBHOOKS` | Allow test events (default `false`) |
|
||||
|
||||
> RN webhook discriminator is `payload.event` (not `eventType`) — see [[rn_webhook_event_field]].
|
||||
|
||||
#### Transaction safety
|
||||
|
||||
| Variable | Description | Default |
|
||||
|---|---|---|
|
||||
| `TRANSACTION_SAFETY_ENABLED` | Enable on-chain verification | `true` |
|
||||
| `TRANSACTION_SAFETY_REQUIRE_TX_HASH` | Require tx hash | `true` |
|
||||
| `TRANSACTION_SAFETY_REQUIRE_TRANSFER_MATCH` | Require transfer match | `true` |
|
||||
| `TRANSACTION_SAFETY_MIN_CONFIRMATIONS` | Min block confirmations | `12` |
|
||||
| `TRANSACTION_SAFETY_AML_PROVIDER` | AML provider (`none`, `chainalysis`) | `none` |
|
||||
|
||||
#### Payment routing
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `PAYMENT_PROVIDER` | Active provider |
|
||||
| `PAYMENT_ENABLED_PROVIDERS` | Comma-separated enabled providers |
|
||||
| `PAYMENT_PROVIDER_MODE` | `live` or `test` |
|
||||
| `PAYMENT_ROLLBACK_PROVIDER` | Fallback provider |
|
||||
|
||||
#### Telegram
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `TELEGRAM_FEATURE_ENABLED` | Enable Telegram integration |
|
||||
| `TELEGRAM_MINIAPP_ENABLED` | Enable Mini App |
|
||||
| `TELEGRAM_WEBHOOK_ENABLED` | Enable webhook receiver |
|
||||
| `TELEGRAM_BOT_TOKEN` | Main bot token |
|
||||
| `TELEGRAM_WEBHOOK_SECRET_TOKEN` | Webhook secret for validation |
|
||||
| `TELEGRAM_INITDATA_MAX_AGE_SEC` | Max age for initData |
|
||||
| `TELEGRAM_INITDATA_REPLAY_WINDOW_MS` | Replay protection window |
|
||||
| `TELEGRAM_WEBHOOK_REPLAY_WINDOW_MS` | Webhook replay protection window |
|
||||
| `TELEGRAM_SESSION_TTL_SEC` | Session TTL |
|
||||
| `TG_NOTIFY_BOT_TOKEN` | Ops/monitoring bot token (amnGG_MonitorBot) |
|
||||
| `TG_NOTIFY_CHATS` | Comma-separated chat IDs for ops notifications |
|
||||
|
||||
#### Pangolin / Newt (VPN mesh — optional)
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `PANGOLIN_ENDPOINT` | Pangolin tunnel endpoint |
|
||||
| `NEWT_ID` | Newt node ID |
|
||||
| `NEWT_SECRET` | Newt node secret |
|
||||
|
||||
#### Testnet chains
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `ENABLE_TESTNET_CHAINS` | Expose testnet chain configs | Set to `true` in dev-amn compose override |
|
||||
|
||||
### Frontend (NEXT_PUBLIC_*)
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `NEXT_PUBLIC_API_URL` | Backend API URL (browser-visible) |
|
||||
| `NEXT_PUBLIC_SOCKET_URL` | Socket.IO server URL |
|
||||
| `NEXT_PUBLIC_WALLETCONNECT_PROJECT_ID` | WalletConnect project ID |
|
||||
| `NEXT_PUBLIC_ALCHEMY_API_KEY_MAINNET` | Alchemy mainnet key |
|
||||
| `NEXT_PUBLIC_ALCHEMY_API_KEY_SEPOLIA` | Alchemy Sepolia key |
|
||||
| `NEXT_PUBLIC_ALCHEMY_API_KEY_POLYGON` | Alchemy Polygon key |
|
||||
| `NEXT_PUBLIC_ESCROW_WALLET_ADDRESS` | Escrow wallet (shown in UI) |
|
||||
| `NEXT_PUBLIC_APP_NAME` | App display name |
|
||||
| `NEXT_PUBLIC_APP_VERSION` | App version string |
|
||||
| `NEXT_PUBLIC_MAPBOX_API_KEY` | Mapbox key (address autocomplete) |
|
||||
| `NEXT_PUBLIC_PASSKEY_RP_NAME` | WebAuthn RP name |
|
||||
| `NEXT_PUBLIC_PASSKEY_RP_ID` | WebAuthn RP ID |
|
||||
| `NEXT_PUBLIC_PASSKEY_ORIGIN` | WebAuthn origin |
|
||||
| `NEXT_PUBLIC_BACKEND_URL` | Backend origin (used for direct calls) |
|
||||
| `NEXT_PUBLIC_DEPAY_INTEGRATION_ID` | DePay integration ID |
|
||||
| `NEXT_PUBLIC_IS_DEVELOPMENT` | Development flag |
|
||||
| `NEXT_PUBLIC_ENABLE_DEBUG` | Enable client debug logging |
|
||||
| `NEXT_PUBLIC_APP_URL` | Canonical app URL |
|
||||
| `NEXT_PUBLIC_TELEGRAM_BOT_ID` | Telegram bot numeric ID |
|
||||
| `BUILD_STATIC_EXPORT` | Enable `next export` mode (`false` for SSR) |
|
||||
|
||||
### Gatus
|
||||
|
||||
| Variable | Description |
|
||||
|---|---|
|
||||
| `GATUS_TELEGRAM_BOT_TOKEN` | Telegram bot for alert delivery |
|
||||
| `GATUS_TELEGRAM_CHAT_ID` | Target chat ID for alerts |
|
||||
|
||||
---
|
||||
|
||||
## 9. Deploy Workflow
|
||||
|
||||
### 9.1 Normal image update (CI-driven)
|
||||
|
||||
Woodpecker CI builds `backend` and `frontend` images, pushes tags to `git.tbs.amn.gg/escrow/` on merge to `dev`, then triggers an Arcane GitOps sync which pulls the new image and recreates the container.
|
||||
|
||||
```
|
||||
git push origin dev
|
||||
└─► Woodpecker build pipeline
|
||||
└─► docker push git.tbs.amn.gg/escrow/backend:dev
|
||||
└─► docker push git.tbs.amn.gg/escrow/frontend:dev
|
||||
└─► arcane-cli gitops sync cf6c9eab… (or watchtower polls)
|
||||
└─► escrow-backend container restarted with new image
|
||||
└─► escrow-frontend container restarted with new image
|
||||
```
|
||||
|
||||
> Always bump the version in `package.json` + lock before pushing, otherwise the CI build may not register as a new deploy. See [[version_bump_before_ci]].
|
||||
|
||||
### 9.2 Manual deploy (backend hotfix — no registry)
|
||||
|
||||
For urgent backend fixes without a full CI cycle, use the local-build pattern (the dev stack has `pull_policy: always` but the override `docker-compose.override.yml` sets `pull_policy: never` for the `escrow-backend-local:dev` image path):
|
||||
|
||||
```bash
|
||||
# 1. Copy changed files to build tree on server
|
||||
scp -i ~/CascadeProjects/wzp src/services/auth/authRoutes.ts \
|
||||
root@89.58.32.32:/tmp/escrow-backend-build/src/services/auth/
|
||||
|
||||
# 2. Rebuild image on server (~3 min, ARM64)
|
||||
ssh -i ~/CascadeProjects/wzp root@89.58.32.32 \
|
||||
"cd /tmp/escrow-backend-build && docker build -f Dockerfile.prod -t escrow-backend-local:dev ."
|
||||
|
||||
# 3. Restart the backend container
|
||||
ssh -i ~/CascadeProjects/wzp root@89.58.32.32 \
|
||||
"cd /opt/arcane/data/projects/escrow-dev && docker compose up -d backend"
|
||||
```
|
||||
|
||||
### 9.3 Bringing the stack up/down
|
||||
|
||||
```bash
|
||||
# via Arcane CLI (preferred)
|
||||
arcane-cli project start devEscrow
|
||||
arcane-cli project stop devEscrow
|
||||
|
||||
# via SSH + docker compose (direct)
|
||||
ssh -i ~/CascadeProjects/wzp root@89.58.32.32 \
|
||||
"cd /opt/arcane/data/projects/escrow-dev && docker compose up -d"
|
||||
|
||||
ssh -i ~/CascadeProjects/wzp root@89.58.32.32 \
|
||||
"cd /opt/arcane/data/projects/escrow-dev && docker compose down"
|
||||
```
|
||||
|
||||
### 9.4 Reloading Caddy after Caddyfile edits
|
||||
|
||||
Edit `/opt/arcane/data/projects/infra/Caddyfile` on the server, then:
|
||||
|
||||
```bash
|
||||
ssh -i ~/CascadeProjects/wzp root@89.58.32.32 \
|
||||
"docker exec infra-caddy caddy reload --config /etc/caddy/Caddyfile"
|
||||
```
|
||||
|
||||
No container restart needed.
|
||||
|
||||
### 9.5 Updating env vars
|
||||
|
||||
1. Edit `.env` on the server: `/opt/arcane/data/projects/escrow-dev/.env`
|
||||
2. Restart affected service:
|
||||
```bash
|
||||
ssh -i ~/CascadeProjects/wzp root@89.58.32.32 \
|
||||
"cd /opt/arcane/data/projects/escrow-dev && docker compose up -d backend"
|
||||
```
|
||||
Frontend env vars baked at build time (via `NEXT_PUBLIC_*`) require a fresh image rebuild.
|
||||
|
||||
### 9.6 Verifying a deploy
|
||||
|
||||
```bash
|
||||
# Check running containers
|
||||
arcane-cli project status devEscrow
|
||||
|
||||
# Check backend version
|
||||
curl https://dev.amn.gg/api/version
|
||||
|
||||
# Check health (all stores + registries)
|
||||
curl https://dev.amn.gg/api/health | jq .
|
||||
|
||||
# Tail backend logs
|
||||
ssh -i ~/CascadeProjects/wzp root@89.58.32.32 \
|
||||
"docker logs -f escrow-backend --tail 100"
|
||||
```
|
||||
|
||||
> CI ✓ green does NOT guarantee the new image was pushed to the registry. Always verify `curl /api/version` returns the expected version. See [[woodpecker_silent_build_fail]].
|
||||
|
||||
---
|
||||
|
||||
## 10. Dev vs Prod Differences
|
||||
|
||||
| Aspect | dev-amn (dev.amn.gg) | Prod (amn.gg) |
|
||||
|---|---|---|
|
||||
| Compose file | `deployment/dev-amn/docker-compose.yml` | Separate prod stack (not in this repo) |
|
||||
| Image registry | `git.tbs.amn.gg/escrow` | Same registry, prod tags |
|
||||
| Image tag | `:dev` | `:latest` or versioned |
|
||||
| MongoDB | Present (dev parity) | Retired |
|
||||
| `ENABLE_TESTNET_CHAINS` | `true` (compose override) | Not set / `false` |
|
||||
| `NODE_ENV` | `production` (same) | `production` |
|
||||
| `NEXT_PUBLIC_IS_DEVELOPMENT` | `false` | `false` |
|
||||
| `PAYMENT_PROVIDER_MODE` | `live` | `live` |
|
||||
| `REQUEST_NETWORK_ALLOW_TEST_WEBHOOKS` | can be `true` for RN testing | `false` |
|
||||
| Watchtower labels | Present in legacy compose | Prod stack may differ |
|
||||
| Gatus monitoring | Monitors both dev + prod endpoints | N/A (shared gatus instance) |
|
||||
| TLS | Cloudflare proxy → Caddy (disable_redirects) | Same |
|
||||
| Version bump requirement | Required before CI push | Required |
|
||||
|
||||
---
|
||||
|
||||
## 11. Secret Management
|
||||
|
||||
**The `.env` file on the server is the single source of runtime secrets. It is never committed.**
|
||||
|
||||
- Location on server: `/opt/arcane/data/projects/escrow-dev/.env`
|
||||
- Permissions: `chmod 600` owned by root
|
||||
- Reference template: `deployment/.env` (in repo — contains live dev values, treated as low-sensitivity dev config; rotate before prod use)
|
||||
- `.gitleaks.toml` in `deployment/` configures secret scanning exclusions for the repo
|
||||
|
||||
### Rules
|
||||
|
||||
1. Never commit `.env` or any file containing real tokens, passwords, or private keys.
|
||||
2. Never pass secrets as Dockerfile `ARG`/`ENV` at build time — they appear in image layers. All secrets are runtime-injected via `env_file`.
|
||||
3. `NEXT_PUBLIC_*` vars are baked into the frontend bundle at build time. Do not place secrets in any `NEXT_PUBLIC_` variable.
|
||||
4. Wallet addresses (e.g. `ESCROW_WALLET_ADDRESS`) are public on-chain but still kept out of the repo for operational hygiene.
|
||||
5. For new deployments: copy `deployment/.env` to the server, fill in real values, then `chmod 600`.
|
||||
6. Gatus bot token and chat ID go into the same `.env` — they are read by the gatus container via `environment:` directives.
|
||||
7. Telegram bot tokens are high-value secrets — rotate immediately if accidentally pushed.
|
||||
|
||||
### Sensitive variable groups
|
||||
|
||||
| Group | Variables | Risk if leaked |
|
||||
|---|---|---|
|
||||
| JWT | `JWT_SECRET` | Full session forgery |
|
||||
| DB credentials | `POSTGRES_PASSWORD`, `REDIS_PASSWORD`, `MONGO_INITDB_ROOT_PASSWORD` | Database access |
|
||||
| Payment webhook secrets | `REQUEST_NETWORK_WEBHOOK_SECRET`, `DEPAY_WEBHOOK_SECRET`, `SHKEEPER_CALLBACK_SECRET`, `SHKEEPER_WEBHOOK_SECRET` | Fake payment injection |
|
||||
| Bot tokens | `TELEGRAM_BOT_TOKEN`, `TG_NOTIFY_BOT_TOKEN` | Bot takeover |
|
||||
| OAuth secrets | `GOOGLE_CLIENT_SECRET` | OAuth impersonation |
|
||||
| API keys | `OPENAI_API_KEY`, `REQUEST_NETWORK_API_KEY`, `SHKEEPER_API_KEY` | Billing / data access |
|
||||
| Sentry DSN | `SENTRY_DSN` | Error data exfiltration |
|
||||
452
10 - Services/frontend.md
Normal file
452
10 - Services/frontend.md
Normal file
@@ -0,0 +1,452 @@
|
||||
---
|
||||
title: Frontend Service — amn-frontend
|
||||
tags: [service, frontend, nextjs, react, web3, telegram]
|
||||
created: 2026-06-08
|
||||
updated: 2026-06-08
|
||||
---
|
||||
|
||||
# Frontend Service — amn-frontend
|
||||
|
||||
## 1. Overview
|
||||
|
||||
`amn-frontend` is the primary user-facing application for the Amanat (AMN) escrow marketplace. It serves buyers, sellers, and admins through a unified Next.js 16 App Router application with a Persian-first (RTL) UI.
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Package name | `amn-frontend` |
|
||||
| Version | **2.10.5** |
|
||||
| Status | Active — deployed on `dev.amn.gg` |
|
||||
| Framework | Next.js 16 (App Router + Turbopack), React 19, TypeScript strict |
|
||||
| Dev port | `8083` (both local and Docker) |
|
||||
| Package manager | `yarn@1.22.22` |
|
||||
| Node requirement | `>=20` (host runs v26.0.0) |
|
||||
| Repo | `git@git.manko.yoga:222/nick/frontend.git` |
|
||||
|
||||
The app covers the full escrow lifecycle: request creation, multi-seller offer collection, negotiation, on-chain payment (BSC/ETH/Base), delivery confirmation, dispute handling, loyalty points, and a Telegram Mini App shell for mobile-native access.
|
||||
|
||||
---
|
||||
|
||||
## 2. Tech Stack
|
||||
|
||||
### Core
|
||||
|
||||
| Layer | Library / Version | Notes |
|
||||
|---|---|---|
|
||||
| Framework | `next@^16.1.1` | App Router, Turbopack dev, standalone output |
|
||||
| UI runtime | `react@^19.1.0` + `react-dom@^19.1.0` | |
|
||||
| Language | TypeScript `^6.0.3` strict | `noEmit` check required before push |
|
||||
| Component library | `@mui/material@^9.0.1` | MUI v9 + `@mui/lab`, `@mui/x-data-grid`, `@mui/x-date-pickers`, `@mui/x-tree-view` |
|
||||
| Styling | `@emotion/react` + `@emotion/styled` + `stylis-plugin-rtl` | RTL support via stylis |
|
||||
| Animation | `framer-motion@^12.13.0` | |
|
||||
| Icon system | `@iconify/react@^6.0.0` | |
|
||||
|
||||
### Data Fetching & State
|
||||
|
||||
| Layer | Library | Notes |
|
||||
|---|---|---|
|
||||
| Server-state cache | `@tanstack/react-query@^5.83.0` | Primary async state manager |
|
||||
| Lightweight fetch | `swr@^2.3.3` | Used in some hooks alongside RQ |
|
||||
| HTTP client | `axios@^1.11.0` | Centralized instance with interceptors in `src/lib/axios.ts` |
|
||||
| Forms | `react-hook-form@^7.77.0` + `zod@^4.0.10` + `@hookform/resolvers@^5.0.1` | |
|
||||
| Real-time | `socket.io-client@^4.8.1` | `src/contexts/` socket context; used for request/offer/chat events |
|
||||
|
||||
### Web3
|
||||
|
||||
| Layer | Library | Notes |
|
||||
|---|---|---|
|
||||
| Wallet connection | `wagmi@^2.19.5` | Primary Web3 state manager |
|
||||
| EVM low-level | `viem@^2.31.7` | ABI encoding, RPC calls |
|
||||
| Compat layer | `ethers@^6.15.0` | Legacy compatibility |
|
||||
| Chain indexing | `alchemy-sdk@^3.6.1` | Mainnet / Sepolia / Polygon queries |
|
||||
| TON wallet | `@tonconnect/ui-react@^2.4.4` + `@ton/core@^0.63.1` | TON Connect in Telegram Mini App |
|
||||
| Hardware wallet | `@trezor/connect-web@^9.7.3` | Trezor signing flow |
|
||||
|
||||
### Internationalization & Localisation
|
||||
|
||||
| Layer | Library | Notes |
|
||||
|---|---|---|
|
||||
| i18n engine | `i18next@^26.3.0` + `react-i18next@^17.0.8` | |
|
||||
| Language detection | `i18next-browser-languagedetector@^8.1.0` | |
|
||||
| Lazy loading | `i18next-resources-to-backend@^1.2.1` | |
|
||||
| Persian date | `date-fns-jalali@^4.1.0-0` | Jalali calendar date formatting |
|
||||
| RTL styling | `stylis-plugin-rtl@^2.1.1` | Emotion cache flips properties for RTL |
|
||||
|
||||
### Observability & Testing
|
||||
|
||||
| Layer | Library | Notes |
|
||||
|---|---|---|
|
||||
| Error tracking | `@sentry/nextjs@^10.22.0` | Configured in `src/instrumentation.ts` |
|
||||
| Unit tests | `jest@^30.4.2` + `@testing-library/react@^16.3.0` | |
|
||||
| E2E tests | `@playwright/test@^1.56.1` | `e2e/` directory; performance spec included |
|
||||
| Notifications | `notistack@^3.0.2` + `sonner@^2.0.3` | Toast system |
|
||||
|
||||
### Editor & Rich Content
|
||||
|
||||
| Layer | Library | Notes |
|
||||
|---|---|---|
|
||||
| Rich text | `@tiptap/react@^3.23.6` + extensions | Code blocks, links, images, alignment, underline |
|
||||
| Markdown render | `react-markdown@10.1.0` + rehype plugins | With GFM, syntax highlight, sanitization |
|
||||
| Maps | `mapbox-gl@^3.12.0` + `react-map-gl@^8.0.4` | Address / delivery location picker |
|
||||
| Charts | `react-apexcharts@^2.1.0` | Dashboard KPI charts |
|
||||
| Carousels | `embla-carousel-react@8.6.0` | Auto-scroll and autoplay plugins |
|
||||
|
||||
---
|
||||
|
||||
## 3. App Router Page Structure
|
||||
|
||||
The Next.js App Router root is `src/app/`. Pages are thin wrappers that import a View component from `src/sections/<feature>/view/`. No business logic lives in `page.tsx` files.
|
||||
|
||||
### Top-level routes
|
||||
|
||||
| Route segment | Type | Purpose |
|
||||
|---|---|---|
|
||||
| `/` | Public | Landing / marketing page |
|
||||
| `/api/health` | API route | Health check endpoint |
|
||||
| `/api/llm` | API route | LLM proxy (amanat-assist integration) |
|
||||
| `/auth/jwt/*` | Auth | Sign-in, sign-up, verify email, reset password, update password |
|
||||
| `/checkout/` | Protected | Checkout flow entry (redirects to payment) |
|
||||
| `/dashboard/` | Protected | Main authenticated shell (see sub-routes below) |
|
||||
| `/design-preview/` | Dev | Component / theme preview (non-production) |
|
||||
| `/error/` | Public | Global error page |
|
||||
| `/payment/` | Protected | Payment status / callback landing |
|
||||
| `/post/[slug]` | Public | Blog / post reader |
|
||||
| `/shop/[seller]/[id]` | Public | Public seller shop and product view |
|
||||
| `/telegram/` | Mini App | Telegram Mini App shell (dedicated layout, see §7) |
|
||||
| `not-found.tsx` | Public | 404 page |
|
||||
|
||||
### Dashboard sub-routes (`/dashboard/*`)
|
||||
|
||||
All dashboard routes are wrapped in `AuthGuard` + `EmailVerificationGuard`.
|
||||
|
||||
| Sub-route | Purpose |
|
||||
|---|---|
|
||||
| `account/` | Profile, avatar, address book, notification prefs, passkey, wallet linking |
|
||||
| `admin/` | Admin control panel |
|
||||
| `assist/` | AI assistant chat (amanat-assist integration) |
|
||||
| `chat/` | Real-time escrow negotiation chat |
|
||||
| `disputes/` | Dispute hub — raise, view, respond |
|
||||
| `payment/` | Payment history and detail view |
|
||||
| `points/` | Loyalty hub — transaction log, referral tracking, level tiers |
|
||||
| `post/` | Admin blog editor (Tiptap) |
|
||||
| `request/` | Buyer purchase request management (create, track, accept offer) |
|
||||
| `request-template/` | Seller request templates management |
|
||||
| `seller/` | Seller profile and analytics |
|
||||
| `shop-settings/` | Seller shop configuration (name, policies, payment rails) |
|
||||
| `shops/` | Browse shops / checkout within dashboard scope |
|
||||
| `user/` | Admin user management |
|
||||
|
||||
---
|
||||
|
||||
## 4. Key Sections & Features
|
||||
|
||||
### Marketplace
|
||||
|
||||
- `src/sections/shop-settings/` — seller configures shop, accepted payment chains/tokens, delivery policy.
|
||||
- `src/sections/request/` — core escrow lifecycle feature. Status flow:
|
||||
```
|
||||
pending_payment → pending → active → received_offers → in_negotiation
|
||||
→ payment → processing → delivery → delivered → confirming → seller_paid → completed
|
||||
(or cancelled at most stages)
|
||||
```
|
||||
- Shared status/urgency color and label maps live in `src/sections/request/constants.ts`. Do not redefine per-view; use `getStatusColor / getStatusLabel / getUrgencyColor / getUrgencyLabel`.
|
||||
- Role-based views (buyer / seller / admin) dispatched from `role-based-<feature>-view.tsx` components.
|
||||
|
||||
### Escrow Flow
|
||||
|
||||
The escrow flow spans multiple sections:
|
||||
|
||||
1. **Buyer** creates a purchase request (`/dashboard/request/new`) — wizard in `src/sections/request/components/steps/`.
|
||||
2. **Sellers** receive notifications via Socket.io and submit offers (`received_offers` state).
|
||||
3. **Negotiation** phase: real-time chat (`/dashboard/chat/`) with offer counter-proposals.
|
||||
4. **Payment**: buyer pays on-chain (BSC primary, ETH/Base/Polygon/Arbitrum supported). Funds held in escrow wallet (`NEXT_PUBLIC_ESCROW_WALLET_ADDRESS`). USDT is the primary escrow currency; BSC USDT uses 18 decimals (non-standard — handled in `src/utils/currencyUtils.ts`).
|
||||
5. **Delivery & confirmation**: seller marks delivered, buyer confirms → `confirming → seller_paid → completed`.
|
||||
6. **Disputes**: either party can raise at `src/sections/dispute/`.
|
||||
|
||||
### Dashboard & Admin
|
||||
|
||||
- Overview tiles with ApexCharts KPI cards.
|
||||
- Admin panel: user management, shop review, dispute arbitration, blog post management.
|
||||
- Points / loyalty system: transaction ledger, referral tracking, tier levels at `src/sections/points/`.
|
||||
- AI assist panel: embedded `amanat-assist` chat at `/dashboard/assist/`.
|
||||
|
||||
### Telegram Mini App
|
||||
|
||||
See §7 for full detail.
|
||||
|
||||
---
|
||||
|
||||
## 5. State Management
|
||||
|
||||
The app uses a layered approach — no single global store:
|
||||
|
||||
| Layer | Tool | Scope |
|
||||
|---|---|---|
|
||||
| Server state & cache | `@tanstack/react-query` | All API calls — fetching, mutations, invalidation |
|
||||
| Supplementary fetch | `swr` | Some lightweight hooks |
|
||||
| Local component state | `React.useState` / `useReducer` | Component-local UI state |
|
||||
| Cross-tree shared state | React Context | Socket connection (`src/contexts/`), Auth (`src/auth/context/`), Web3, Settings drawer, Localization |
|
||||
| Form state | `react-hook-form` | All form instances, with `zod` schemas as resolvers |
|
||||
| Settings (theme/locale) | Context + `localStorage` | Theme mode, layout direction, color preset, font — managed by `src/settings/` |
|
||||
|
||||
There is no Zustand or Redux in the dependency tree. Global state is passed via Context providers stacked in `src/app/layout.tsx`.
|
||||
|
||||
Key contexts:
|
||||
|
||||
- `SocketContext` (`src/contexts/`) — wraps `socket.io-client`, exposes live event subscriptions.
|
||||
- `AuthContext` (`src/auth/context/`) — JWT session, user object, sign-in/out actions.
|
||||
- `Web3Context` / wagmi `WagmiProvider` (`src/web3/context/`) — wallet connection, chain switching.
|
||||
- `SettingsContext` (`src/settings/`) — UI preferences (RTL, color scheme, font).
|
||||
- `LocalizationProvider` (`src/locales/`) — i18next + MUI date picker locale.
|
||||
|
||||
---
|
||||
|
||||
## 6. Internationalization
|
||||
|
||||
The app is RTL-first with Persian (Farsi) as the primary production language.
|
||||
|
||||
| Aspect | Implementation |
|
||||
|---|---|
|
||||
| Engine | `i18next` + `react-i18next` |
|
||||
| Supported languages | `fa` (Persian), `ar` (Arabic), `en` (English), `fr` (French), `cn` (Chinese), `vi` (Vietnamese) |
|
||||
| Translation files | `src/locales/langs/<lang>/*.json` — split by feature namespace |
|
||||
| RTL flip | `stylis-plugin-rtl` applied to the Emotion cache — physical CSS properties (margin-left, padding-right, etc.) are automatically mirrored |
|
||||
| LTR islands | Inline `dir="ltr"` on elements containing URLs, wallet addresses, token amounts, or other inherently LTR content |
|
||||
| Persian calendar | `date-fns-jalali` for Jalali date formatting; MUI date pickers use the Jalali locale adapter |
|
||||
| Direction state | Controlled via `SettingsContext` — users can toggle in the settings drawer |
|
||||
| Config | `src/locales/locales-config.ts` + `src/locales/i18n-provider.tsx` |
|
||||
|
||||
Language detection priority: URL `?lng=` param → browser `Accept-Language` → localStorage fallback → `fa`.
|
||||
|
||||
---
|
||||
|
||||
## 7. Telegram Mini App Integration
|
||||
|
||||
The Telegram Mini App (TMA) is a first-class feature with a dedicated route segment, layout, and 40+ purpose-built components.
|
||||
|
||||
### Loading & Auth Flow
|
||||
|
||||
- The TMA loads via Telegram's `webApp.openWebApp()` into the `/telegram/` route.
|
||||
- The root layout at `src/app/telegram/layout.tsx` provides a minimal provider stack:
|
||||
- `TonConnectUIProvider` (TON wallet)
|
||||
- No standard app shell (no top nav, no side drawer) — uses native Telegram chrome instead.
|
||||
- User identity: `window.Telegram.WebApp.initData` is parsed by `src/utils/telegram-webapp.ts` (a custom wrapper around the `window.Telegram` global — **no `@twa-dev` or `@telegram-apps` SDK package** is used).
|
||||
- Auth is linked to the existing JWT session: on first open the app prompts the user to connect their AMN account (onboarding sheet). Subsequent opens re-use the stored token.
|
||||
|
||||
### Key File Locations
|
||||
|
||||
| Path | Purpose |
|
||||
|---|---|
|
||||
| `src/app/telegram/layout.tsx` | TMA root layout — minimal providers |
|
||||
| `src/app/telegram/page.tsx` | TMA entry point |
|
||||
| `src/utils/telegram-webapp.ts` | Custom `window.Telegram.WebApp` wrapper / SDK util |
|
||||
| `src/sections/telegram/` | All TMA feature code |
|
||||
| `src/sections/telegram/view/` | ~18 view components (one per TMA screen) |
|
||||
| `src/sections/telegram/components/` | ~28 TMA-specific UI primitives |
|
||||
| `src/sections/telegram/hooks/` | TMA-scoped hooks including `use-telegram-live-context` |
|
||||
| `src/sections/telegram/telegram-shell-css.ts` | Native Telegram shell CSS variables integration |
|
||||
|
||||
### TMA Views
|
||||
|
||||
| View file | Screen |
|
||||
|---|---|
|
||||
| `telegram-mini-app-view.tsx` | Main shell / router (23 KB — primary orchestrator) |
|
||||
| `telegram-home-view.tsx` | Home tab |
|
||||
| `telegram-shop-view.tsx` | Shop list |
|
||||
| `telegram-seller-shop-view.tsx` | Individual seller shop products |
|
||||
| `telegram-cart-view.tsx` | Cart |
|
||||
| `telegram-checkout-view.tsx` | Checkout |
|
||||
| `telegram-payment-view.tsx` | Payment status |
|
||||
| `telegram-requests-view.tsx` | Buyer requests list |
|
||||
| `telegram-request-detail-view.tsx` | Request detail + offer management (31 KB) |
|
||||
| `telegram-new-request-view.tsx` | New request wizard |
|
||||
| `telegram-template-detail-view.tsx` | Seller template detail |
|
||||
| `telegram-chat-view.tsx` | In-app chat thread list |
|
||||
| `telegram-chat-thread-view.tsx` | Single chat thread |
|
||||
| `telegram-archived-chats-view.tsx` | Archived chats |
|
||||
| `telegram-account-view.tsx` | Account settings (18 KB) |
|
||||
| `telegram-addresses-view.tsx` | Address book (15 KB) |
|
||||
| `telegram-points-view.tsx` | Loyalty points |
|
||||
| `telegram-notifications-view.tsx` | Notification centre |
|
||||
| `telegram-settings-view.tsx` | App settings (14 KB) |
|
||||
|
||||
### TMA-specific Components
|
||||
|
||||
Key primitives: `telegram-chat-row`, `telegram-request-stepper`, `telegram-onboarding-sheet`, `telegram-tab-bar`, `telegram-header`, `telegram-quick-actions`, `telegram-cart-fab`, `telegram-support-fab`, `telegram-welcome-banner`, `telegram-unlinked-state`, `telegram-unsupported-state`.
|
||||
|
||||
### TON Wallet in TMA
|
||||
|
||||
TON Connect (`@tonconnect/ui-react`, `@ton/core`) is active only inside the TMA layout. BSC payments via wagmi/viem are also available in-TMA but TON is the preferred rail for Telegram users.
|
||||
|
||||
---
|
||||
|
||||
## 8. Web3 Integration
|
||||
|
||||
All Web3 code lives under `src/web3/`.
|
||||
|
||||
### Architecture
|
||||
|
||||
```
|
||||
src/web3/
|
||||
├── config.ts # WEB3_CONFIG — chains, WalletConnect project ID
|
||||
├── index.ts # Public barrel
|
||||
├── types.ts # Shared Web3 types
|
||||
├── utils.ts # Misc helpers
|
||||
├── payment-rails.ts # Chain+token routing logic
|
||||
├── decentralizedPayment.ts # Core payment execution (16 KB)
|
||||
├── web3Service.ts # Service layer (9 KB)
|
||||
├── paymentBackendService.ts # Backend sync after on-chain tx (12 KB)
|
||||
├── tonconnect-provider.tsx # TonConnect provider wrapper
|
||||
├── context/ # Web3Context provider
|
||||
├── contracts/ # ABI definitions
|
||||
├── hooks/
|
||||
│ ├── use-web3-wagmi.ts # wagmi-based wallet + tx hooks (5.5 KB)
|
||||
│ ├── use-alchemy.ts # Alchemy SDK hooks — balance, tx history (3.8 KB)
|
||||
│ ├── use-chainlink.ts # Chainlink price feed hooks (2.6 KB)
|
||||
│ └── use-web3-context.ts # Context consumer hook
|
||||
├── services/ # Additional service modules
|
||||
├── trezor/ # Trezor Connect integration
|
||||
└── utils/ # Chain-specific utilities
|
||||
```
|
||||
|
||||
### Supported Chains
|
||||
|
||||
Declared in `WEB3_CONFIG.supportedChains`: **BSC** (default, lowest fees), Base, Polygon, Arbitrum, Ethereum.
|
||||
|
||||
Primary escrow payments run on BSC. BSC USDT is 18 decimals (non-standard; handled in `src/utils/currencyUtils.ts` — do not hardcode decimals).
|
||||
|
||||
### Wallet Support
|
||||
|
||||
| Wallet | Integration |
|
||||
|---|---|
|
||||
| MetaMask / injected | wagmi `injected()` connector |
|
||||
| WalletConnect | wagmi WalletConnect connector (`NEXT_PUBLIC_WALLETCONNECT_PROJECT_ID`) |
|
||||
| Trezor | `@trezor/connect-web` in `src/web3/trezor/` |
|
||||
| TON wallet | `@tonconnect/ui-react` (TMA only) |
|
||||
|
||||
### Oracle / Price Feeds
|
||||
|
||||
- Chainlink price feeds via `use-chainlink.ts` — used for USDT/USD peg monitoring.
|
||||
- Alchemy SDK (`alchemy-sdk`) for on-chain data queries (balances, tx receipts).
|
||||
- Depeg protection feature in development — see `nick-doc/` oracle depeg protection design doc.
|
||||
|
||||
---
|
||||
|
||||
## 9. CI/CD
|
||||
|
||||
### Pipeline
|
||||
|
||||
CI is managed by Woodpecker at `frontend/.woodpecker/production.yml`.
|
||||
|
||||
**Trigger:** push to `main` or `master` branch.
|
||||
|
||||
**Agent:** `platform: linux/arm64` (netcup agent on `89.58.32.32`).
|
||||
|
||||
**Steps:**
|
||||
|
||||
| Step | Image | Action |
|
||||
|---|---|---|
|
||||
| `get-version` | `node:22-alpine` | Reads `package.json` version → writes `dev-<version>` to `.tags` |
|
||||
| `build-and-deploy` | `docker:27-cli` | `docker build -t git.tbs.amn.gg/escrow/frontend:dev .` then `docker compose up -d --no-deps --pull never frontend` against `/opt/escrow-dev/docker-compose.yml` |
|
||||
| `notify` | `node:22-alpine` | Posts Telegram notification (success or failure) via `scripts/ci/tg-notify.cjs` using `TG_TOKEN` + `TG_USERS` secrets |
|
||||
|
||||
**Important CI notes:**
|
||||
|
||||
- The image is built locally on the host — it does **not** pull from a registry. `docker-compose.override.yml` sets `pull_policy: never`.
|
||||
- Turbopack is dev-only. The production build uses standard `next build` (webpack).
|
||||
- `next build` runs a strict TypeScript type-check. The build fails on type errors.
|
||||
- **Always bump `package.json` version before pushing** to `main`/`master`. Docker tags use `dev-<version>`. Reusing the same version overwrites the previous image tag and breaks rollback.
|
||||
- A CI green check does not guarantee the image was pushed to the registry. Verify the registry tag manually if deployment seems stale.
|
||||
|
||||
### Docker Build
|
||||
|
||||
- Output mode: `standalone` (set in `next.config.js`).
|
||||
- Start command: `PORT=8083 node .next/standalone/server.js`.
|
||||
- Post-build: `cp -r .next/static .next/standalone/.next/ && cp -r public .next/standalone/` (required for static assets with standalone output).
|
||||
- Build cache: `--mount=type=cache` for apk, yarn, and `.next/cache` — incremental Next.js rebuilds on unchanged packages.
|
||||
|
||||
---
|
||||
|
||||
## 10. Local Development Quick-Start
|
||||
|
||||
```bash
|
||||
# Prerequisites: Node >=20, yarn 1.22.22
|
||||
|
||||
cd frontend/
|
||||
|
||||
# Install dependencies
|
||||
yarn install
|
||||
|
||||
# Copy env file and fill in values
|
||||
cp .env.local.example .env.local
|
||||
# Edit .env.local — see §11 for required vars
|
||||
|
||||
# Start dev server (Turbopack, port 8083)
|
||||
yarn dev
|
||||
|
||||
# Alternative: webpack (slower, more compatible)
|
||||
yarn dev:webpack
|
||||
|
||||
# Type check (must pass before push)
|
||||
npx tsc --noEmit --ignoreDeprecations 6.0
|
||||
|
||||
# Lint
|
||||
yarn lint
|
||||
yarn lint:fix
|
||||
|
||||
# Unit tests
|
||||
yarn test
|
||||
|
||||
# E2E tests (requires running app)
|
||||
yarn playwright:install
|
||||
yarn test:e2e
|
||||
|
||||
# Production build (validates types + builds)
|
||||
yarn build
|
||||
|
||||
# Run standalone production build
|
||||
yarn start
|
||||
```
|
||||
|
||||
**Note:** The dev server binds to `http://localhost:8083`. When proxied via infra-caddy on the dev server, it maps to `https://dev.amn.gg`.
|
||||
|
||||
---
|
||||
|
||||
## 11. Environment Variables
|
||||
|
||||
All public vars are prefixed `NEXT_PUBLIC_` and baked into the client bundle at build time.
|
||||
|
||||
| Variable | Required | Description |
|
||||
|---|---|---|
|
||||
| `NEXT_PUBLIC_APP_NAME` | Yes | Application display name (e.g. `Amanat`) |
|
||||
| `NEXT_PUBLIC_APP_VERSION` | Yes | App version string — should match `package.json` version |
|
||||
| `NEXT_PUBLIC_BACKEND_URL` | Yes | Base URL for backend API (e.g. `https://api.dev.amn.gg`) |
|
||||
| `NEXT_PUBLIC_API_URL` | Yes | API endpoint root (often same as `BACKEND_URL` + `/api`) |
|
||||
| `NEXT_PUBLIC_SOCKET_URL` | No | Socket.IO server URL — falls back to `NEXT_PUBLIC_BACKEND_URL` |
|
||||
| `NEXT_PUBLIC_ESCROW_WALLET_ADDRESS` | Yes | On-chain escrow holding wallet address |
|
||||
| `NEXT_PUBLIC_WALLETCONNECT_PROJECT_ID` | Yes | WalletConnect Cloud project ID |
|
||||
| `NEXT_PUBLIC_ALCHEMY_API_KEY_MAINNET` | Yes | Alchemy API key for Ethereum mainnet |
|
||||
| `NEXT_PUBLIC_ALCHEMY_API_KEY_SEPOLIA` | Yes | Alchemy API key for Sepolia testnet |
|
||||
| `NEXT_PUBLIC_ALCHEMY_API_KEY_POLYGON` | Yes | Alchemy API key for Polygon |
|
||||
| `NEXT_PUBLIC_MAPBOX_API_KEY` | No | Mapbox GL token for map components |
|
||||
| `NEXT_PUBLIC_ASSETS_DIR` | No | Custom assets base URL — defaults to empty (local `/public`) |
|
||||
| `BUILD_STATIC_EXPORT` | No | Set `true` to enable static export mode |
|
||||
| `NODE_ENV` | Auto | Set by Next.js (`development` / `production`) |
|
||||
|
||||
Actual values for the dev deployment are stored in `~/.agentSecrets/escrow/CLAUDE.md` (not in the repo).
|
||||
|
||||
---
|
||||
|
||||
## 12. Known Issues / Open Items
|
||||
|
||||
| # | Issue | Status |
|
||||
|---|---|---|
|
||||
| 1 | **Socket room scoping** — global payment socket broadcasts previously wiped every user's cart. A provider gate was added in frontend `v2.8.4` to filter by provider. Backend-side room scoping is still an open follow-up. | Open |
|
||||
| 2 | **Backend rate limiter on GET /payment/:id** — `paymentLimiter` (30 req/15 min) applies to the payment status poll endpoint. Results in 429 during rapid callback polling, leaving payments stuck in "processing". Fix is on backend side but frontend polling interval could be increased as a mitigation. | Open (backend fix pending) |
|
||||
| 3 | **Offer rejection UI** — "all sellers stuck at step 4" was a UI-only bug; backend rejects and notifies correctly. Telegram seller step must use `mojtaba`'s `StepContext` (introduced in `fe v2.9.13`). | Resolved in v2.9.13 |
|
||||
| 4 | **Cart wipe regression risk** — any new global socket event handler must be scoped by `provider:` to avoid touching RN or other payment records. | Ongoing convention |
|
||||
| 5 | **Performance is network-bound** — Mongo API profiling shows 300–800 ms response times due to WAN RTT (~235 ms). Server-side processing is 3–12 ms. Frontend-side CDN / edge caching is the recommended fix; DB migration will not help. | Open |
|
||||
| 6 | **Oracle depeg protection** — server-side oracle quoting for multi-currency pricing + stablecoin depeg protection is designed and approved. Build starts on a new dev branch. | In progress |
|
||||
| 7 | **Multi-chain for amn.scanner** — the in-house scanner pay-in path is not yet multi-chain; verify scanner watches mainnet addresses before enabling multi-chain selection in the UI for scanner provider. | Open |
|
||||
| 8 | **Parallel agent pushes** — a second agent (moojttaba) pushes to the same branches. Always `git fetch --rebase` before pushing. Version-bump conflicts are expected. | Ongoing |
|
||||
| 9 | **Tiptap / rich text in TMA** — the Tiptap editor is desktop-optimised; its usability on mobile Telegram is untested at scale. | Not verified |
|
||||
| 10 | **`design-preview/` route** — present in the app router but should be excluded or protected in production builds. | Low priority |
|
||||
513
10 - Services/scanner.md
Normal file
513
10 - Services/scanner.md
Normal file
@@ -0,0 +1,513 @@
|
||||
---
|
||||
title: AMN Pay Scanner
|
||||
tags: [service, scanner, payment, go, blockchain]
|
||||
version: 0.1.10
|
||||
created: 2026-06-08
|
||||
---
|
||||
|
||||
# AMN Pay Scanner
|
||||
|
||||
> [!info]
|
||||
> Version: **0.1.10** — Go 1.25 — Status: **production (BSC + Ethereum + BSC Testnet)**, other chains staged
|
||||
> Repo: `scanner/` within the escrow monorepo.
|
||||
> Cross-ref: [[Scanner Architecture]] | [[Scanner API]]
|
||||
|
||||
---
|
||||
|
||||
## 1. Overview
|
||||
|
||||
AMN Pay Scanner is a standalone Go microservice that watches on-chain payment events and notifies the backend via signed webhook when a payment is confirmed.
|
||||
|
||||
### What it replaces
|
||||
|
||||
Previously, the AMN escrow platform relied on **Request Network** as the payment infrastructure layer. Request Network introduced:
|
||||
|
||||
- An external smart-contract dependency (`ERC20FeeProxy`) on RN's deployment schedule
|
||||
- A closed fee-proxy address registry that differs per chain and is not reliably canonical (see memory note on RN proxy addresses)
|
||||
- A separate webhook/event pipeline managed by RN's infrastructure
|
||||
- A hard coupling between the backend and RN's SDK
|
||||
|
||||
AMN Pay Scanner removes all of these by:
|
||||
|
||||
1. Deploying the same `ERC20FeeProxy` contract under our own control
|
||||
2. Polling RPC endpoints directly (no RN nodes)
|
||||
3. Deriving payment references in-house using the same keccak256 formula as the proxy contract
|
||||
4. Delivering webhooks signed with a backend-controlled HMAC secret
|
||||
|
||||
The scanner also supports **direct-address payment rails** (Tron, TON, and manual EVM flows) where no proxy contract is involved at all.
|
||||
|
||||
---
|
||||
|
||||
## 2. How It Works
|
||||
|
||||
### Step-by-step flow
|
||||
|
||||
```
|
||||
Backend Scanner Chain
|
||||
│ │ │
|
||||
│ POST /intents │ │
|
||||
│ {chainId, token, amount, │ │
|
||||
│ destination, callbackUrl}│ │
|
||||
├──────────────────────────► │ │
|
||||
│ │ persist intent (SQLite) │
|
||||
│ {intentId, │ derive paymentReference │
|
||||
│ paymentReference, │ compute topicRef (EVM) │
|
||||
│ checkoutBlock} │ │
|
||||
◄──────────────────────────── │ │
|
||||
│ │ │
|
||||
│ (frontend builds tx using │ │
|
||||
│ proxyAddress + │ │
|
||||
│ paymentReference) │ │
|
||||
│ │ poll eth_getLogs │
|
||||
│ ├────────────────────────────►│
|
||||
│ │ logs [] │
|
||||
│ ◄────────────────────────────┤
|
||||
│ │ match Topics[1] → topicRef │
|
||||
│ │ validate token+amount+dest │
|
||||
│ │ status → confirming │
|
||||
│ │ │
|
||||
│ │ (wait confirmationThreshold│
|
||||
│ │ blocks / finality signal) │
|
||||
│ │ │
|
||||
│ POST callbackUrl │ │
|
||||
│ {intentId, txHash, │ │
|
||||
│ status:"confirmed", ...} │ │
|
||||
◄──────────────────────────── │ │
|
||||
│ 200 OK │ │
|
||||
├──────────────────────────► │ │
|
||||
│ │ status → confirmed │
|
||||
│ │ record webhookDeliveredAt │
|
||||
```
|
||||
|
||||
### Intent status lifecycle
|
||||
|
||||
```
|
||||
pending ──(tx seen)──► confirming ──(depth reached)──► confirmed ──(webhook ok)──► [done]
|
||||
│ │ │
|
||||
│ │ (deep reorg / TTL) │ (all retries fail)
|
||||
└────────────────────────┴──────────────► expired webhook_failed
|
||||
```
|
||||
|
||||
- **Tron / TON** skip `confirming` — their API only returns finalized events, so the status jumps directly to `confirmed`.
|
||||
- **Startup reconciliation**: on startup, any `confirmed` intent with `webhook_delivered_at IS NULL` created in the last 7 days has its webhook re-delivered. This covers crashes between finalization and delivery.
|
||||
- **`webhook_failed`** intents are retried on `WEBHOOK_RETRY_HOURS` schedule (default 6 h) and immediately via `POST /admin/webhooks/retry`.
|
||||
|
||||
---
|
||||
|
||||
## 3. Supported Chains
|
||||
|
||||
> Chains marked **verified: false** in `supported-chains.json` do NOT start a worker goroutine at runtime. Override with `SCANNER_ENABLED_CHAINS` env var to force-enable specific chain IDs without a code change.
|
||||
|
||||
| Chain | Chain ID | Type | Proxy Address | Confirmation Depth | Verified |
|
||||
|---|---|---|---|---|---|
|
||||
| BNB Smart Chain | 56 | EVM | `0x0DfbEe143b42B41eFC5A6F87bFD1fFC78c2f0aC9` | 200 blocks | **yes** |
|
||||
| Ethereum Mainnet | 1 | EVM | `0x370DE27fdb7D1Ff1e1BaA7D11c5820a324Cf623C` | 50 blocks | **yes** |
|
||||
| BSC Testnet | 97 | EVM | `0x0DfbEe143b42B41eFC5A6F87bFD1fFC78c2f0aC9` | 5 blocks | **yes** (testnet) |
|
||||
| Arbitrum One | 42161 | EVM | `0x0DfbEe143b42B41eFC5A6F87bFD1fFC78c2f0aC9` | 2400 blocks | no |
|
||||
| Polygon | 137 | EVM | `0x0DfbEe143b42B41eFC5A6F87bFD1fFC78c2f0aC9` | 300 blocks | no |
|
||||
| Base | 8453 | EVM | `0x1892196E80C4c17ea5100Da765Ab48c1fE2Fb814` | 300 blocks | no |
|
||||
| Tron Mainnet | 728126428 | Tron | TRC20 USDT contract (`TR7NHqjeKQxGTCi8q8ZY4pL8otSzgjLj6t`) | TronGrid confirmed (~200 reported) | no |
|
||||
| TON Mainnet | 1100 | TON | USDT Jetton master (`EQCxE6mUtQJKFnGfaROTKOt1lZbDiiX1kCixRv7Nw2Id_sDs`) | TonCenter finalized (~120 reported) | no |
|
||||
|
||||
> [!warning] Chain notes
|
||||
> - **Ethereum**: uses the older v0.1.0 proxy ABI at `0x370DE27…`. A v0.2.0-next proxy is also deployed at `0x0DfbEe…` on ETH but the scanner checkout uses the v0.1.0 ABI — do not swap addresses silently.
|
||||
> - **Base**: proxy address is non-canonical (differs from the CREATE2 expected address per RN smart-contracts artifact v0.2.0). See memory note on RN proxy addresses.
|
||||
> - **Tron**: no fee-proxy contract exists. Matching is by unique destination address, not payment reference.
|
||||
> - **TON**: lag is reported in **seconds** (not blocks); per-intent polling is O(pending intents) API calls per cycle — known scaling concern.
|
||||
|
||||
---
|
||||
|
||||
## 4. Architecture Diagram
|
||||
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────────────┐
|
||||
│ scanner binary │
|
||||
│ │
|
||||
│ ┌─────────────┐ ┌──────────────────────────────────────────┐ │
|
||||
│ │ HTTP API │ │ Worker Pool │ │
|
||||
│ │ (api.go) │ │ │ │
|
||||
│ │ │ │ ┌──────────────┐ eth_getLogs / eth_ │ │
|
||||
│ │ POST /intents│ │ │ ChainWorker │─► blockNumber (JSON-RPC│ │
|
||||
│ │ GET /intents│ │ │ (EVM×N) │ per chain) │ │
|
||||
│ │ /balances │ │ └──────────────┘ │ │
|
||||
│ │ /balance- │ │ ┌──────────────┐ TronGrid REST API │ │
|
||||
│ │ watches │ │ │ TronChain- │─► /v1/contracts/events │ │
|
||||
│ │ /scanner/ │ │ │ Worker │ │ │
|
||||
│ │ status │ │ └──────────────┘ │ │
|
||||
│ │ /admin/ │ │ ┌──────────────┐ TonCenter v3 REST │ │
|
||||
│ │ webhooks/ │ │ │ TonChain- │─► /jetton/transfers │ │
|
||||
│ │ retry │ │ │ Worker │ │ │
|
||||
│ └──────┬──────┘ │ └──────────────┘ │ │
|
||||
│ │ └─────────────┬────────────────────────────┘ │
|
||||
│ │ │ match / confirm │
|
||||
│ ▼ ▼ │
|
||||
│ ┌──────────────────────────────────────────────────────────┐ │
|
||||
│ │ SQLite (WAL) │ │
|
||||
│ │ intents · checkpoints · balance_watches │ │
|
||||
│ └──────────────────────────────────────────────────────────┘ │
|
||||
│ │ │ │
|
||||
│ ▼ ▼ │
|
||||
│ ┌────────────────┐ ┌─────────────────────────────────────┐ │
|
||||
│ │BalanceWatch- │ │ webhook.go │ │
|
||||
│ │Scheduler │ │ HMAC-SHA256 sign → POST callbackUrl│ │
|
||||
│ │(balance.go) │ │ retry: 5s→30s→2m→10m→1h→failed │ │
|
||||
│ └────────────────┘ └─────────────────────────────────────┘ │
|
||||
│ │
|
||||
│ Background loops (main.go): │
|
||||
│ • intent TTL expiry (INTENT_TTL_HOURS) │
|
||||
│ • webhook retry loop (WEBHOOK_RETRY_HOURS) │
|
||||
│ • startup reconciliation (confirmed, no delivery) │
|
||||
└──────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. API Routes
|
||||
|
||||
All endpoints except `/health` require `Authorization: Bearer <SCANNER_API_KEY>`. Request bodies are capped at 64 KB.
|
||||
|
||||
| Method | Path | Auth | Purpose |
|
||||
|---|---|---|---|
|
||||
| `GET` | `/health` | none | Liveness probe — returns `{"status":"ok"}` |
|
||||
| `GET` | `/scanner/status` | Bearer | Chain lag, last scanned block, pending intent counts per chain |
|
||||
| `POST` | `/intents` | Bearer | Register a payment intent; returns `intentId`, `paymentReference`, `checkoutBlock` |
|
||||
| `GET` | `/intents/{id}` | Bearer | Fetch full intent record including current status and tx details |
|
||||
| `DELETE` | `/intents/{id}` | Bearer | Cancel a pending intent (sets status to `expired`) |
|
||||
| `POST` | `/balances/check` | Bearer | Read current ERC-20 balance for an address+token on a given chain |
|
||||
| `POST` | `/balance-watches` | Bearer | Start a balance-change watch on an EVM address/token |
|
||||
| `GET` | `/balance-watches/{id}` | Bearer | Fetch balance-watch status and current balance |
|
||||
| `DELETE` | `/balance-watches/{id}` | Bearer | Stop a balance watch |
|
||||
| `POST` | `/admin/webhooks/retry` | Bearer | Force immediate retry of all `webhook_failed` intents |
|
||||
|
||||
Full request/response schemas: [[Scanner API]]
|
||||
|
||||
---
|
||||
|
||||
## 6. Payment Reference Derivation (EVM)
|
||||
|
||||
The ERC20FeeProxy contract indexes payments by a `bytes8` reference. The scanner derives it deterministically so the worker's scan loop only needs a single indexed DB lookup per log.
|
||||
|
||||
```
|
||||
# Step 1: build raw reference
|
||||
input = lower(intentId) + lower(salt) + lower(destination)
|
||||
paymentReference = last8Bytes(keccak256(input)) ← bytes8, 16 hex chars
|
||||
|
||||
# Step 2: build EVM log index key
|
||||
topicRef = keccak256(paymentReferenceBytes) ← bytes32, 64 hex chars
|
||||
↑ this is Topics[1] in the emitted log
|
||||
```
|
||||
|
||||
- `salt` is a 32-byte random hex string generated at intent creation time.
|
||||
- `destination` is the EVM address of the AMN treasury / seller wallet, lowercased.
|
||||
- Both `paymentReference` and `topicRef` are stored in the `intents` table at creation time. The scan loop performs `SELECT * FROM intents WHERE topic_ref = ? AND status = 'pending'` — O(1) with index, regardless of how many pending intents exist.
|
||||
|
||||
**Event signature** (used as `Topics[0]` filter):
|
||||
```
|
||||
TransferWithReferenceAndFee → keccak256 = 0xbc8b749850bdc0c5e4a49d50f87ec40dca2acdb70a84e7c6823ccdc7b3b3d4b3
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 7. EVM / Tron / TON Matching Logic
|
||||
|
||||
### EVM
|
||||
|
||||
1. Worker fetches `eth_getLogs` for the proxy contract address + `TransferWithReferenceAndFee` event topic in 2000-block chunks.
|
||||
2. For each log, extract `Topics[1]` (= `topicRef`).
|
||||
3. Query DB: `WHERE topic_ref = ? AND status = 'pending'`.
|
||||
4. On match: decode `log.Data` to extract `tokenAddress`, `amount`, `destination`, `feeAmount`. Validate all four against the intent record.
|
||||
5. Update status to `confirming`, record `txHash`, `blockNumber`, `logIndex`.
|
||||
6. On next poll: check `head - blockNumber + 1 >= confirmationsRequired`. When met, finalize and deliver webhook.
|
||||
|
||||
**Reorg protection**: the checkpoint is rewound by `3 × confirmationThreshold` blocks (clamped to 20–500) on each poll. Any log from a recently reorganized block will be re-fetched and re-matched.
|
||||
|
||||
### Tron
|
||||
|
||||
- No proxy contract — each intent receives a unique HD-derived destination address.
|
||||
- Worker polls TronGrid `/v1/contracts/{usdtContract}/events?event_name=Transfer` filtered to the intent's destination address.
|
||||
- Match criterion: `to == destination AND amount >= intent.Amount`.
|
||||
- TronGrid returns only already-confirmed transactions. No multi-block wait — status jumps directly to `confirmed`.
|
||||
- Addresses from TronGrid arrive as `41xxxx` (21-byte hex). The worker normalizes these to `0x`-prefixed 20-byte EVM style for storage and comparison.
|
||||
- Checkpoint stored as a millisecond Unix timestamp in `last_scanned_block`.
|
||||
- Pagination follows `meta.links.next` until nil.
|
||||
|
||||
### TON
|
||||
|
||||
- Also uses per-intent unique destination addresses (no proxy contract).
|
||||
- Worker calls TonCenter v3 `/jetton/transfers?account={destination}&jetton_master={usdtJetton}&start_utime={checkpoint}` for each pending TON intent.
|
||||
- Match criterion: `destination == intent.Destination AND amount >= intent.Amount`.
|
||||
- TonCenter returns only finalized transactions — status jumps directly to `confirmed`.
|
||||
- TON addresses are base64url (`EQ…`/`UQ…`), case-sensitive. Never lowercased.
|
||||
- Checkpoint stored as Unix seconds.
|
||||
- Lag is reported in seconds, not blocks.
|
||||
- **Scaling note**: each pending TON intent triggers one HTTP round-trip per scan cycle. High concurrency will exhaust TonCenter rate limits.
|
||||
|
||||
---
|
||||
|
||||
## 8. Webhook Payload
|
||||
|
||||
### Payment confirmed (intent webhook)
|
||||
|
||||
Posted to `callbackUrl` on intent confirmation:
|
||||
|
||||
```json
|
||||
{
|
||||
"intentId": "018f1a2b-3c4d-7e8f-9a0b-c1d2e3f4a5b6",
|
||||
"paymentReference": "0xa1b2c3d4e5f60718",
|
||||
"txHash": "0x4a3b2c1d...",
|
||||
"blockNumber": 39000010,
|
||||
"confirmations": 200,
|
||||
"amount": "10000000000000000000",
|
||||
"token": "0x55d398326f99059fF775485246999027B3197955",
|
||||
"chainId": 56,
|
||||
"status": "confirmed"
|
||||
}
|
||||
```
|
||||
|
||||
Header: `X-AMN-Signature: hex(HMAC-SHA256(rawBody, callbackSecret))`
|
||||
|
||||
The `confirmations` value is **capped** at the chain's acceptance threshold once confirmed. The scanner does not continue incrementing after the payment is safe to credit.
|
||||
|
||||
**Retry schedule on delivery failure**: `5 s → 30 s → 2 min → 10 min → 1 h → webhook_failed`
|
||||
|
||||
After exhausting retries the intent is set to `webhook_failed`. Manual recovery: `POST /admin/webhooks/retry` or wait for the `WEBHOOK_RETRY_HOURS` background sweep.
|
||||
|
||||
### Balance changed (balance-watch webhook)
|
||||
|
||||
Posted to the watch's `callbackUrl` when balance delta is detected:
|
||||
|
||||
```json
|
||||
{
|
||||
"eventType": "balance_changed",
|
||||
"watchId": "payment-123-c56-USDT",
|
||||
"chainId": 56,
|
||||
"chainType": "evm",
|
||||
"address": "0xabc...",
|
||||
"tokenAddress": "0x55d398326f99059fF775485246999027B3197955",
|
||||
"tokenSymbol": "USDT",
|
||||
"decimals": 18,
|
||||
"previousBalance": "0",
|
||||
"currentBalance": "10000000000000000000",
|
||||
"delta": "10000000000000000000",
|
||||
"changeCount": 1,
|
||||
"checkedAt": "2026-06-08T12:00:00Z",
|
||||
"status": "balance_changed"
|
||||
}
|
||||
```
|
||||
|
||||
Additional headers: `X-AMN-Delivery-ID: <watchId>`, `X-AMN-Event-Type: balance_changed`
|
||||
|
||||
The scanner only advances `current_balance` after a successful (2xx) delivery, so a down backend will retry on the next scheduled check.
|
||||
|
||||
---
|
||||
|
||||
## 9. SQLite DB Schema
|
||||
|
||||
Database at `DB_PATH` (default `./scanner.db`; Docker: `/data/scanner.db`). WAL mode enabled, busy timeout 5 000 ms.
|
||||
|
||||
### `intents`
|
||||
|
||||
| Column | Type | Notes |
|
||||
|---|---|---|
|
||||
| `intent_id` | TEXT PK | caller-supplied UUID |
|
||||
| `chain_id` | INTEGER | numeric chain ID |
|
||||
| `chain_type` | TEXT | `evm` / `tron` / `ton` |
|
||||
| `token_address` | TEXT | EVM/Tron: lowercase `0x` hex; TON: base64url |
|
||||
| `destination` | TEXT | receiving address |
|
||||
| `amount` | TEXT | base-10 wei / token smallest unit |
|
||||
| `payment_reference` | TEXT | 8-byte hex — EVM only |
|
||||
| `topic_ref` | TEXT | keccak256 of paymentReference — scan index for EVM |
|
||||
| `status` | TEXT | `pending` / `confirming` / `confirmed` / `expired` / `webhook_failed` |
|
||||
| `callback_url` | TEXT | backend webhook endpoint |
|
||||
| `callback_secret` | TEXT | HMAC key — excluded from all JSON responses |
|
||||
| `confirmations_required` | INTEGER | floored at the chain acceptance threshold |
|
||||
| `tx_hash` | TEXT NULL | set once the transaction is seen on-chain |
|
||||
| `log_index` | INTEGER NULL | log position within tx (EVM only) |
|
||||
| `block_number` | INTEGER NULL | block when seen (EVM); ms timestamp (Tron); unix s (TON) |
|
||||
| `confirmations` | INTEGER | depth while confirming; capped at threshold after confirmation |
|
||||
| `salt` | TEXT | 32-byte random hex used in reference derivation |
|
||||
| `webhook_delivered_at` | TEXT NULL | RFC3339 timestamp of first successful delivery |
|
||||
| `created_at` / `updated_at` | DATETIME | UTC |
|
||||
|
||||
Unique index on `(tx_hash, log_index)` prevents double-confirmation.
|
||||
|
||||
### `checkpoints`
|
||||
|
||||
| Column | Notes |
|
||||
|---|---|
|
||||
| `chain_id` PK | |
|
||||
| `last_scanned_block` | block number (EVM), ms timestamp (Tron), unix seconds (TON) |
|
||||
|
||||
### `balance_watches`
|
||||
|
||||
| Column | Type | Notes |
|
||||
|---|---|---|
|
||||
| `watch_id` | TEXT PK | caller-supplied idempotency key |
|
||||
| `chain_id` / `chain_type` | INTEGER / TEXT | currently EVM only |
|
||||
| `token_address` / `token_symbol` | TEXT | ERC-20 contract + optional registry symbol |
|
||||
| `decimals` | INTEGER | registry decimals for display |
|
||||
| `address` | TEXT | watched holder address |
|
||||
| `baseline_balance` | TEXT | base-unit balance at watch creation |
|
||||
| `current_balance` | TEXT | last successfully delivered balance |
|
||||
| `status` | TEXT | `watching` / `stopped` / `expired` |
|
||||
| `callback_url` / `callback_secret` | TEXT | signed webhook destination |
|
||||
| `last_checked_at` / `next_check_at` | DATETIME | scheduler state |
|
||||
| `change_count` / `last_notified_at` | INTEGER / DATETIME | notification audit |
|
||||
| `expires_at` | DATETIME | hard stop after 7 days |
|
||||
| `created_at` / `updated_at` | DATETIME | UTC |
|
||||
|
||||
Indexes: `(status, next_check_at)` for due-scan queries; `(chain_id, status)` for status reporting.
|
||||
|
||||
---
|
||||
|
||||
## 10. Configuration
|
||||
|
||||
All configuration via environment variables. Copy `.env.example` and populate before first run.
|
||||
|
||||
| Variable | Default | Required | Notes |
|
||||
|---|---|---|---|
|
||||
| `PORT` | `8080` | no | HTTP listen port |
|
||||
| `DB_PATH` | `./scanner.db` | no | SQLite file path. Docker: mount `/data` and set `/data/scanner.db` |
|
||||
| `CHAINS_JSON_PATH` | `./supported-chains.json` | no | Chain registry file |
|
||||
| `TOKENS_JSON_PATH` | `./tokens.json` | no | Token registry for symbol/decimals metadata |
|
||||
| `SCANNER_API_KEY` | _(none)_ | **yes (prod)** | Shared secret for Bearer auth. Generate: `openssl rand -hex 32` |
|
||||
| `POLL_INTERVAL_SEC` | `15` | no | Chain polling interval in seconds |
|
||||
| `INTENT_TTL_HOURS` | `24` | no | Expire pending intents after N hours. `0` = disabled |
|
||||
| `WEBHOOK_RETRY_HOURS` | `6` | no | Background re-delivery interval for `webhook_failed` intents. `0` = disabled |
|
||||
| `BALANCE_WATCH_TICK_SEC` | `60` | no | How often the scheduler checks for due balance watches |
|
||||
| `BALANCE_WATCH_BATCH_SIZE` | `50` | no | Max due watches processed per tick |
|
||||
| `RPC_BSC` | chain config | no | Override BSC JSON-RPC URL |
|
||||
| `RPC_ARB` | chain config | no | Override Arbitrum JSON-RPC URL |
|
||||
| `RPC_ETH` | chain config | no | Override Ethereum JSON-RPC URL |
|
||||
| `RPC_POLYGON` | chain config | no | Override Polygon JSON-RPC URL |
|
||||
| `RPC_BASE` | chain config | no | Override Base JSON-RPC URL |
|
||||
| `TRONGRID_API_KEY` | _(none)_ | recommended | Free tier is very low; required for any real Tron traffic |
|
||||
| `TONCENTER_API_KEY` | _(none)_ | recommended | Rate-limited without key |
|
||||
| `SCANNER_ENABLED_CHAINS` | JSON `verified` flags | no | Comma-separated chain IDs to enable, overriding `verified`. E.g. `56,1` |
|
||||
| `SCANNER_CALLBACK_ALLOWED_HOSTS` | _(none)_ | prod recommended | Comma-separated hosts for SSRF guard on `callbackUrl` targets |
|
||||
|
||||
---
|
||||
|
||||
## 11. Docker Deployment
|
||||
|
||||
```bash
|
||||
# Build
|
||||
docker build -t amn-scanner .
|
||||
|
||||
# Run
|
||||
docker run -d \
|
||||
--name amn-scanner \
|
||||
--network shared-web \
|
||||
-p 8080:8080 \
|
||||
-v /opt/arcane/data/projects/escrow-dev/scanner-data:/data \
|
||||
--env-file .env \
|
||||
-e DB_PATH=/data/scanner.db \
|
||||
amn-scanner
|
||||
```
|
||||
|
||||
**On the dev server** (`89.58.32.32`): the scanner is part of the `escrow-dev` Arcane project. Images are built locally from source at `/tmp/escrow-backend-build/` — the dev stack does **not** pull from any registry.
|
||||
|
||||
```bash
|
||||
# Copy changed scanner source files
|
||||
scp -i ~/CascadeProjects/wzp scanner/*.go root@89.58.32.32:/tmp/escrow-backend-build/scanner/
|
||||
|
||||
# Rebuild + restart (on server)
|
||||
ssh -i ~/CascadeProjects/wzp root@89.58.32.32 \
|
||||
"cd /tmp/escrow-backend-build/scanner && docker build -t amn-scanner-local:dev . && \
|
||||
cd /opt/arcane/data/projects/escrow-dev && docker compose up -d scanner"
|
||||
```
|
||||
|
||||
Health check URL (via infra-caddy): check project Caddyfile for the current vhost. Direct internal: `http://amn-scanner:8080/health`.
|
||||
|
||||
---
|
||||
|
||||
## 12. Integration with the Backend
|
||||
|
||||
### Registering a payment intent
|
||||
|
||||
```typescript
|
||||
// backend: src/services/amnScanner/...
|
||||
const resp = await fetch(`${SCANNER_URL}/intents`, {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Authorization': `Bearer ${SCANNER_API_KEY}`,
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({
|
||||
intentId: payment._id.toString(), // MongoDB ObjectId string
|
||||
chainId: 56,
|
||||
tokenAddress: '0x55d398326f99059fF775485246999027B3197955', // USDT BSC
|
||||
destination: sellerWalletAddress,
|
||||
amount: amountInWei, // base-10 string
|
||||
callbackUrl: `${BACKEND_URL}/api/payment/amn-scanner/webhook`,
|
||||
callbackSecret: process.env.SCANNER_CALLBACK_SECRET,
|
||||
}),
|
||||
});
|
||||
const { intentId, paymentReference, checkoutBlock } = await resp.json();
|
||||
// store intentId + checkoutBlock in the payment record
|
||||
// pass checkoutBlock to the frontend for transaction construction
|
||||
```
|
||||
|
||||
The `checkoutBlock` contains everything the frontend needs to call the `ERC20FeeProxy.transferWithReferenceAndFee()` function:
|
||||
|
||||
```json
|
||||
{
|
||||
"destination": "0x...",
|
||||
"tokenAddress": "0x55d...",
|
||||
"tokenSymbol": "USDT",
|
||||
"decimals": 18,
|
||||
"chainId": 56,
|
||||
"proxyAddress": "0x0DfbEe143b42B41eFC5A6F87bFD1fFC78c2f0aC9",
|
||||
"paymentReference": "0xa1b2c3d4e5f60718",
|
||||
"feeAmount": "0",
|
||||
"feeAddress": "0x0000000000000000000000000000000000000000",
|
||||
"amountWei": "10000000000000000000"
|
||||
}
|
||||
```
|
||||
|
||||
### Receiving the webhook callback
|
||||
|
||||
```typescript
|
||||
// POST /api/payment/amn-scanner/webhook
|
||||
app.post('/api/payment/amn-scanner/webhook', async (req, res) => {
|
||||
const signature = req.headers['x-amn-signature'];
|
||||
const expected = hmacSha256Hex(req.rawBody, process.env.SCANNER_CALLBACK_SECRET);
|
||||
if (!timingSafeEqual(signature, expected)) return res.status(401).end();
|
||||
|
||||
const { intentId, status, txHash, amount, chainId } = req.body;
|
||||
if (status !== 'confirmed') return res.status(200).end(); // ignore non-confirmed
|
||||
|
||||
await paymentService.markConfirmed({ intentId, txHash, amount, chainId });
|
||||
res.status(200).end();
|
||||
});
|
||||
```
|
||||
|
||||
> [!warning]
|
||||
> The backend must always scope payment lookups by `provider: "amn.scanner"`. Sweeping all pending payments to mark them confirmed/failed will destroy records from other providers (RN, manual). See memory note on payment cleanup + provider filter.
|
||||
|
||||
### Backend env vars required
|
||||
|
||||
```
|
||||
SCANNER_URL=http://amn-scanner:8080
|
||||
SCANNER_API_KEY=<same value as scanner SCANNER_API_KEY>
|
||||
SCANNER_CALLBACK_SECRET=<same value as scanner intent callbackSecret>
|
||||
```
|
||||
|
||||
See memory note: [[amn_scanner_payin_wiring]] for full wiring details and token-decimal notes.
|
||||
|
||||
---
|
||||
|
||||
## 13. Known Limitations / Open Items
|
||||
|
||||
| # | Area | Description |
|
||||
|---|---|---|
|
||||
| 1 | **TON scaling** | O(pending intents) TonCenter API calls per scan cycle. Becomes a bottleneck above ~50 concurrent TON intents. Future fix: batch by address or use a streaming API. |
|
||||
| 2 | **Tron/TON balance watches** | `POST /balances/check` and `POST /balance-watches` currently only support EVM ERC-20 reads. Tron TRC-20 and TON Jetton balance reads are future scope. |
|
||||
| 3 | **Non-EVM chains unverified** | Arbitrum, Polygon, Base, Tron, TON are `"verified": false` — workers do not start by default. Needs production testing + `SCANNER_ENABLED_CHAINS` opt-in before enabling for real traffic. |
|
||||
| 4 | **Base proxy address** | `0x1892196E80C4c17ea5100Da765Ab48c1fE2Fb814` is non-canonical (differs from RN's CREATE2 expected address for that chain). Verify before enabling Base in production. |
|
||||
| 5 | **Ethereum proxy version** | Ethereum uses the v0.1.0 proxy at `0x370DE27…`. A v0.2.0 proxy is also deployed on ETH. The two have different ABI signatures — do not silently swap addresses. |
|
||||
| 6 | **Single SQLite instance** | The embedded SQLite database is not horizontally scalable. Running multiple scanner replicas would require coordination or migration to a shared DB. Acceptable for current load. |
|
||||
| 7 | **No native-token support** | Only ERC-20/TRC-20/Jetton transfers are scanned. Native token (BNB, ETH, TRX, TON coin) payments are not supported. |
|
||||
| 8 | **Multi-seller / multi-chain** | AMN Scanner pay-in supports single-seller flow only. Multi-seller cart payments and cross-chain routing are not implemented. |
|
||||
| 9 | **Webhook signature algorithm** | HMAC-SHA256 with a pre-shared secret. There is no key rotation mechanism — changing `callbackSecret` requires intent re-registration. |
|
||||
| 10 | **No EVM testnet for ETH/Polygon/Base** | Only BSC Testnet (97) has a testnet entry in `supported-chains.json`. Developers testing on Ethereum Sepolia or Polygon Amoy need to add chain entries manually. |
|
||||
Reference in New Issue
Block a user