docs: sync vault with codebase state (2026-06-12)

- Update backend, frontend, scanner, deployment, amanat-assist service docs
- Update System Overview, Scanner Architecture, Telegram Mini App flow
- Update 10 - Services/README.md
- Add Tenant data model, Tenant API reference, Tenant Storefront Flow
- Add Multi-Shop Branch Project Scan (2026-06-10)
- Add tenant.md service doc
- Append activity log entry
- Reflects archived/search/stats route fix and new E2E test suite

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Siavash Sameni
2026-06-12 11:42:18 +04:00
parent 18073afb52
commit e52ffce48a
18 changed files with 2619 additions and 1102 deletions

View File

@@ -3,12 +3,13 @@ title: AMN Pay Scanner
tags: [service, scanner, payment, go, blockchain]
version: 0.1.10
created: 2026-06-08
updated: 2026-06-12
---
# AMN Pay Scanner
> [!info]
> Version: **0.1.10** — Go 1.25 — Status: **production (BSC + Ethereum + BSC Testnet)**, other chains staged
> Version: **0.1.10** — Go 1.25 — Status: **production (BSC + Ethereum + BSC Testnet)**, other chains staged behind `SCANNER_ENABLED_CHAINS`
> Repo: `scanner/` within the escrow monorepo.
> Cross-ref: [[Scanner Architecture]] | [[Scanner API]]
@@ -16,25 +17,37 @@ created: 2026-06-08
## 1. Overview
AMN Pay Scanner is a standalone Go microservice that watches on-chain payment events and notifies the backend via signed webhook when a payment is confirmed.
AMN Pay Scanner is a standalone Go microservice that watches on-chain payment events across EVM chains, Tron, and TON, and notifies the backend via signed webhook when a payment is confirmed.
### What it replaces
Previously, the AMN escrow platform relied on **Request Network** as the payment infrastructure layer. Request Network introduced:
The platform previously relied on **Request Network** as its payment infrastructure layer. That dependency introduced:
- An external smart-contract dependency (`ERC20FeeProxy`) on RN's deployment schedule
- A closed fee-proxy address registry that differs per chain and is not reliably canonical (see memory note on RN proxy addresses)
- A separate webhook/event pipeline managed by RN's infrastructure
- A hard coupling between the backend and RN's SDK
- An external smart-contract registry whose canonical proxy addresses differ per chain and cannot be trusted without on-chain verification (see memory note on RN proxy addresses)
- A closed RN event/webhook pipeline that the backend had no control over
- A hard SDK coupling between the backend and RN's versioned contracts
- Inability to support Tron or TON (not in RN's network)
AMN Pay Scanner removes all of these by:
AMN Pay Scanner replaces this entirely by:
1. Deploying the same `ERC20FeeProxy` contract under our own control
2. Polling RPC endpoints directly (no RN nodes)
3. Deriving payment references in-house using the same keccak256 formula as the proxy contract
4. Delivering webhooks signed with a backend-controlled HMAC secret
1. Deploying an in-house `ERC20FeeProxy` contract on each EVM chain under our own control
2. Polling RPC endpoints directly no RN nodes, no RN SDK
3. Deriving payment references in-house using the same keccak256 formula the proxy contract expects
4. Delivering signed webhooks using a backend-controlled HMAC secret
5. Supporting **direct-address rails** (Tron, TON, manual EVM) where no proxy contract is needed
The scanner also supports **direct-address payment rails** (Tron, TON, and manual EVM flows) where no proxy contract is involved at all.
### Current status
| Chain | Status |
|---|---|
| BNB Smart Chain (56) | Production |
| Ethereum Mainnet (1) | Production |
| BSC Testnet (97) | Production (testnet) |
| Arbitrum One (42161) | Staged — `verified: false` |
| Polygon (137) | Staged — `verified: false` |
| Base (8453) | Staged — `verified: false` |
| Tron Mainnet (728126428) | Staged — `verified: false` |
| TON Mainnet (1100) | Staged — `verified: false` |
---
@@ -50,8 +63,10 @@ Backend Scanner Chain
│ destination, callbackUrl}│ │
├──────────────────────────► │ │
│ │ persist intent (SQLite) │
{intentId, │ derive paymentReference │
paymentReference, │ compute topicRef (EVM) │
│ derive paymentReference │
│ compute topicRef (EVM) │
│ {intentId, │ │
│ paymentReference, │ │
│ checkoutBlock} │ │
◄──────────────────────────── │ │
│ │ │
@@ -70,12 +85,12 @@ Backend Scanner Chain
│ │ blocks / finality signal) │
│ │ │
│ POST callbackUrl │ │
│ X-AMN-Signature: ... │ │
│ {intentId, txHash, │ │
│ status:"confirmed", ...} │ │
◄──────────────────────────── │ │
│ 200 OK │ │
├──────────────────────────► │ │
│ │ status → confirmed │
│ │ record webhookDeliveredAt │
```
@@ -88,78 +103,83 @@ pending ──(tx seen)──► confirming ──(depth reached)──► confi
└────────────────────────┴──────────────► expired webhook_failed
```
- **Tron / TON** skip `confirming` — their API only returns finalized events, so the status jumps directly to `confirmed`.
- **Startup reconciliation**: on startup, any `confirmed` intent with `webhook_delivered_at IS NULL` created in the last 7 days has its webhook re-delivered. This covers crashes between finalization and delivery.
- **`webhook_failed`** intents are retried on `WEBHOOK_RETRY_HOURS` schedule (default 6 h) and immediately via `POST /admin/webhooks/retry`.
- **Tron / TON** skip `confirming` — their chain APIs only surface already-finalized transactions. Status jumps directly to `confirmed`.
- **Startup reconciliation**: on startup, any `confirmed` intent with `webhook_delivered_at IS NULL` created within the last 7 days has its webhook re-delivered. This recovers from crashes between `finalizeIntent` and `deliverWebhook`.
- **`webhook_failed`** intents are retried every `WEBHOOK_RETRY_HOURS` (default 6 h) and immediately on `POST /admin/webhooks/retry`.
---
## 3. Supported Chains
> Chains marked **verified: false** in `supported-chains.json` do NOT start a worker goroutine at runtime. Override with `SCANNER_ENABLED_CHAINS` env var to force-enable specific chain IDs without a code change.
> [!note]
> Chains marked `verified: false` in `supported-chains.json` do **not** start a worker goroutine at runtime. Force-enable specific chain IDs without a rebuild by setting `SCANNER_ENABLED_CHAINS=56,1,42161`.
| Chain | Chain ID | Type | Proxy Address | Confirmation Depth | Verified |
| Chain | Chain ID | Type | Proxy Address | Confirmation Depth | Active by Default |
|---|---|---|---|---|---|
| BNB Smart Chain | 56 | EVM | `0x0DfbEe143b42B41eFC5A6F87bFD1fFC78c2f0aC9` | 200 blocks | **yes** |
| Ethereum Mainnet | 1 | EVM | `0x370DE27fdb7D1Ff1e1BaA7D11c5820a324Cf623C` | 50 blocks | **yes** |
| BNB Smart Chain | 56 | EVM | `0x0DfbEe143b42B41eFC5A6F87bFD1fFC78c2f0aC9` | 200 blocks (~10 min) | **yes** |
| Ethereum Mainnet | 1 | EVM | `0x370DE27fdb7D1Ff1e1BaA7D11c5820a324Cf623C` | 50 blocks (~10 min) | **yes** |
| BSC Testnet | 97 | EVM | `0x0DfbEe143b42B41eFC5A6F87bFD1fFC78c2f0aC9` | 5 blocks | **yes** (testnet) |
| Arbitrum One | 42161 | EVM | `0x0DfbEe143b42B41eFC5A6F87bFD1fFC78c2f0aC9` | 2400 blocks | no |
| Arbitrum One | 42161 | EVM | `0x0DfbEe143b42B41eFC5A6F87bFD1fFC78c2f0aC9` | 2400 blocks (~54 min) | no |
| Polygon | 137 | EVM | `0x0DfbEe143b42B41eFC5A6F87bFD1fFC78c2f0aC9` | 300 blocks | no |
| Base | 8453 | EVM | `0x1892196E80C4c17ea5100Da765Ab48c1fE2Fb814` | 300 blocks | no |
| Tron Mainnet | 728126428 | Tron | TRC20 USDT contract (`TR7NHqjeKQxGTCi8q8ZY4pL8otSzgjLj6t`) | TronGrid confirmed (~200 reported) | no |
| TON Mainnet | 1100 | TON | USDT Jetton master (`EQCxE6mUtQJKFnGfaROTKOt1lZbDiiX1kCixRv7Nw2Id_sDs`) | TonCenter finalized (~120 reported) | no |
| Tron Mainnet | 728126428 | Tron | TRC20 USDT contract `TR7NHqjeKQxGTCi8q8ZY4pL8otSzgjLj6t` | TronGrid confirmed (~200 reported) | no |
| TON Mainnet | 1100 | TON | USDT Jetton master `EQCxE6mUtQJKFnGfaROTKOt1lZbDiiX1kCixRv7Nw2Id_sDs` | TonCenter finalized (~120 reported) | no |
> [!warning] Chain notes
> - **Ethereum**: uses the older v0.1.0 proxy ABI at `0x370DE27…`. A v0.2.0-next proxy is also deployed at `0x0DfbEe…` on ETH but the scanner checkout uses the v0.1.0 ABI — do not swap addresses silently.
> - **Base**: proxy address is non-canonical (differs from the CREATE2 expected address per RN smart-contracts artifact v0.2.0). See memory note on RN proxy addresses.
> - **Tron**: no fee-proxy contract exists. Matching is by unique destination address, not payment reference.
> - **TON**: lag is reported in **seconds** (not blocks); per-intent polling is O(pending intents) API calls per cycle — known scaling concern.
> [!warning] Chain-specific notes
> - **Ethereum**: uses the v0.1.0 proxy ABI at `0x370DE27…`. A v0.2.0-next proxy is also deployed at `0x0DfbEe…` on ETH but checkout uses the v0.1.0 ABI — do not swap addresses.
> - **Arbitrum**: 2400-block threshold covers the optimistic rollup challenge window (~54 min at ~1.3 s/block).
> - **Base**: proxy address `0x1892196…` is non-canonical — it differs from the RN CREATE2 expected address for this chain. Verify on-chain before enabling in production.
> - **Tron**: no fee-proxy contract exists on Tron. Matching is by unique HD-derived destination address, not payment reference.
> - **TON**: lag is reported in **seconds**, not blocks. Per-intent polling is O(pending intents) TonCenter calls per cycle.
---
## 4. Architecture Diagram
```
┌──────────────────────────────────────────────────────────────────┐
│ scanner binary │
│ │
│ ┌─────────────┐ ┌──────────────────────────────────────────┐ │
│ │ HTTP API │ │ Worker Pool │ │
│ │ (api.go) │ │ │ │
│ │ │ │ ┌──────────────┐ eth_getLogs / eth_ │ │
│ │ POST /intents │ │ ChainWorker ─► blockNumber (JSON-RPC│
│ │ GET /intents│ │ │ (EVM×N) per chain) │ │
│ │ /balances │ │ └──────────────┘ │ │
│ │ /balance- │ │ ┌──────────────┐ TronGrid REST API │ │
│ │ watches │ │ │ TronChain- ─► /v1/contracts/events │ │
│ │ /scanner/ │ │ │ Worker │ │ │
│ │ status │ │ └──────────────┘ │ │
│ │ /admin/ │ │ ┌──────────────┐ TonCenter v3 REST │ │
│ │ webhooks/ │ │ │ TonChain- ─► /jetton/transfers │ │
│ │ retry │ │ │ Worker │
└──────┬──────┘ │ └──────────────┘ │ │
└─────────────┬────────────────────────────┘
│ match / confirm
│ ┌──────────────────────────────────────────────────────────┐
SQLite (WAL)
│ │ intents · checkpoints · balance_watches
└──────────────────────────────────────────────────────────┘
│ ┌────────────────┐ ┌─────────────────────────────────────┐ │
│ │BalanceWatch- │ │ webhook.go │ │
│ │Scheduler │ │ HMAC-SHA256 sign → POST callbackUrl│ │
│ │(balance.go) │ │ retry: 5s→30s→2m→10m→1h→failed │ │
└────────────────┘ └─────────────────────────────────────┘
Background loops (main.go):
• intent TTL expiry (INTENT_TTL_HOURS)
│ • webhook retry loop (WEBHOOK_RETRY_HOURS)
│ • startup reconciliation (confirmed, no delivery)
└──────────────────────────────────────────────────────────────────┘
┌──────────────────────────────────────────────────────────────────────
scanner binary
│ ┌──────────────────┐ ┌────────────────────────────────────────┐
│ │ HTTP API │ │ Worker Pool │
│ │ (api.go) │ │ │
│ │ │ │ ┌────────────────┐ eth_getLogs / │
│ │ POST /intents │ │ ChainWorker ─► eth_blockNumber
│ │ GET /intents │ │ (EVM × N) │ (JSON-RPC)
│ │ DELETE /intents │ │ └────────────────┘ │
│ │ POST /balances │ │ ┌────────────────┐ TronGrid REST │
│ │ /check │ │ │ TronChain- ─► /v1/contracts/
│ │ POST /balance- │ │ │ Worker {addr}/events
│ │ watches │ │ └────────────────┘ │
│ │ GET /balance- │ │ ┌────────────────┐ TonCenter v3 │
│ │ watches/id │ │ │ TonChain- ─► /jetton/
│ │ DEL /balance- │ │ │ Worker transfers │
│ watches/id │ │ └────────────────┘ │
│ GET /scanner/ └─────────────┬────────────────────────
status │ match / confirm │
│ POST /admin/ ▼ │
│ webhooks/retry │ ┌────────────────────────────────────────
└────────┬──────────┘ SQLite (WAL mode) │ │
│ intents · checkpoints · balance_watches
│ └───────────────────────────────────────┘ │
│ │
▼ │
│ ┌────────────────┐ ┌────────────────────────────────────────┐ │
│ │ BalanceWatch- │ │ webhook.go │ │
│ │ Scheduler │ │ HMAC-SHA256 sign → POST callbackUrl │ │
│ │ (balance_ │ │ retry: 5s → 30s → 2m → 10m → 1h │ │
│ watch.go) │ │ → webhook_failed│
└─────────────────┘ └────────────────────────────────────────┘
Background loops (main.go):
│ • intent TTL expiry (INTENT_TTL_HOURS)
│ • webhook retry loop (WEBHOOK_RETRY_HOURS)
│ • startup reconciliation (confirmed intents, no delivery) │
└──────────────────────────────────────────────────────────────────────┘
```
One worker goroutine is spawned per active chain. All three chain types implement a common `Worker` interface (`start()`, `stop()`, `getHead()`). Workers poll on `POLL_INTERVAL_SEC` (default 15 s).
---
## 5. API Routes
@@ -169,14 +189,14 @@ All endpoints except `/health` require `Authorization: Bearer <SCANNER_API_KEY>`
| Method | Path | Auth | Purpose |
|---|---|---|---|
| `GET` | `/health` | none | Liveness probe — returns `{"status":"ok"}` |
| `GET` | `/scanner/status` | Bearer | Chain lag, last scanned block, pending intent counts per chain |
| `GET` | `/scanner/status` | Bearer | Chain lag, last scanned block, pending intent count, active balance-watch count per chain |
| `POST` | `/intents` | Bearer | Register a payment intent; returns `intentId`, `paymentReference`, `checkoutBlock` |
| `GET` | `/intents/{id}` | Bearer | Fetch full intent record including current status and tx details |
| `DELETE` | `/intents/{id}` | Bearer | Cancel a pending intent (sets status to `expired`) |
| `POST` | `/balances/check` | Bearer | Read current ERC-20 balance for an address+token on a given chain |
| `POST` | `/balance-watches` | Bearer | Start a balance-change watch on an EVM address/token |
| `GET` | `/balance-watches/{id}` | Bearer | Fetch balance-watch status and current balance |
| `DELETE` | `/balance-watches/{id}` | Bearer | Stop a balance watch |
| `POST` | `/balances/check` | Bearer | Read current ERC-20 balance for an address+token on a given EVM chain |
| `POST` | `/balance-watches` | Bearer | Start an async balance-change watch on an EVM address/token pair |
| `GET` | `/balance-watches/{id}` | Bearer | Fetch balance-watch status, current balance, and check schedule |
| `DELETE` | `/balance-watches/{id}` | Bearer | Stop a balance watch (also: `POST /balance-watches/{id}/stop`) |
| `POST` | `/admin/webhooks/retry` | Bearer | Force immediate retry of all `webhook_failed` intents |
Full request/response schemas: [[Scanner API]]
@@ -185,25 +205,27 @@ Full request/response schemas: [[Scanner API]]
## 6. Payment Reference Derivation (EVM)
The ERC20FeeProxy contract indexes payments by a `bytes8` reference. The scanner derives it deterministically so the worker's scan loop only needs a single indexed DB lookup per log.
The `ERC20FeeProxy` contract indexes payments by a `bytes8` reference. The scanner derives it deterministically so the scan loop needs only one indexed DB lookup per log.
```
# Step 1: build raw reference
# Step 1 — derive the bytes8 payment reference
input = lower(intentId) + lower(salt) + lower(destination)
paymentReference = last8Bytes(keccak256(input)) ← bytes8, 16 hex chars
paymentReference = last8Bytes(keccak256(input)) ← bytes8, stored as 16 hex chars
# Step 2: build EVM log index key
# Step 2 — derive the EVM log topic index key
topicRef = keccak256(paymentReferenceBytes) ← bytes32, 64 hex chars
↑ this is Topics[1] in the emitted log
↑ this is Topics[1] in every TransferWithReferenceAndFee log
```
- `salt` is a 32-byte random hex string generated at intent creation time.
- `destination` is the EVM address of the AMN treasury / seller wallet, lowercased.
- Both `paymentReference` and `topicRef` are stored in the `intents` table at creation time. The scan loop performs `SELECT * FROM intents WHERE topic_ref = ? AND status = 'pending'` — O(1) with index, regardless of how many pending intents exist.
- `salt` is a 32-byte random hex string generated at intent creation time to prevent reference collisions.
- `destination` is the EVM treasury/seller wallet address, always lowercased before hashing.
- Both `paymentReference` and `topicRef` are written to the `intents` table at creation time.
- The scan inner loop executes `SELECT * FROM intents WHERE topic_ref = ? AND status = 'pending'` — O(1) with index regardless of how many pending intents exist.
**Event signature** (used as `Topics[0]` filter):
**Event signature** used as `Topics[0]` filter:
```
TransferWithReferenceAndFee → keccak256 = 0xbc8b749850bdc0c5e4a49d50f87ec40dca2acdb70a84e7c6823ccdc7b3b3d4b3
TransferWithReferenceAndFee
keccak256 = 0xbc8b749850bdc0c5e4a49d50f87ec40dca2acdb70a84e7c6823ccdc7b3b3d4b3
```
---
@@ -212,34 +234,33 @@ TransferWithReferenceAndFee → keccak256 = 0xbc8b749850bdc0c5e4a49d50f87ec40dca
### EVM
1. Worker fetches `eth_getLogs` for the proxy contract address + `TransferWithReferenceAndFee` event topic in 2000-block chunks.
2. For each log, extract `Topics[1]` (= `topicRef`).
1. Worker calls `eth_getLogs` for the proxy contract address + `TransferWithReferenceAndFee` event topic in 2 000-block chunks.
2. For each log, extract `Topics[1]` (the `topicRef`).
3. Query DB: `WHERE topic_ref = ? AND status = 'pending'`.
4. On match: decode `log.Data` to extract `tokenAddress`, `amount`, `destination`, `feeAmount`. Validate all four against the intent record.
5. Update status to `confirming`, record `txHash`, `blockNumber`, `logIndex`.
6. On next poll: check `head - blockNumber + 1 >= confirmationsRequired`. When met, finalize and deliver webhook.
5. Update status to `confirming`; record `txHash`, `blockNumber`, `logIndex`.
6. On subsequent polls: if `chainHead - blockNumber + 1 >= confirmationsRequired`, finalize and deliver webhook.
**Reorg protection**: the checkpoint is rewound by `3 × confirmationThreshold` blocks (clamped to 20500) on each poll. Any log from a recently reorganized block will be re-fetched and re-matched.
**Reorg protection**: the EVM checkpoint is rewound by `3 × confirmationThreshold` blocks (clamped 20500) on every tick. Any log from a reorganized block will be re-fetched and re-matched. The unique index on `(tx_hash, log_index)` prevents double-confirmation if the same log is matched on two consecutive ticks.
### Tron
- No proxy contract — each intent receives a unique HD-derived destination address.
- Worker polls TronGrid `/v1/contracts/{usdtContract}/events?event_name=Transfer` filtered to the intent's destination address.
- No proxy contract on Tron. Each intent receives a unique HD-derived destination address.
- Worker polls TronGrid `/v1/contracts/{usdtTrc20}/events?event_name=Transfer` filtered to the intent's destination address.
- Match criterion: `to == destination AND amount >= intent.Amount`.
- TronGrid returns only already-confirmed transactions. No multi-block wait — status jumps directly to `confirmed`.
- Addresses from TronGrid arrive as `41xxxx` (21-byte hex). The worker normalizes these to `0x`-prefixed 20-byte EVM style for storage and comparison.
- Checkpoint stored as a millisecond Unix timestamp in `last_scanned_block`.
- TronGrid only surfaces already-confirmed transactions — status jumps directly to `confirmed` with no `confirming` intermediate state.
- Addresses from TronGrid arrive in `41xxxx` (21-byte hex) format. The worker normalizes them to `0x`-prefixed 20-byte EVM format for storage and comparison.
- Checkpoint is stored as a millisecond Unix timestamp in `last_scanned_block`.
- Pagination follows `meta.links.next` until nil.
### TON
- Also uses per-intent unique destination addresses (no proxy contract).
- Worker calls TonCenter v3 `/jetton/transfers?account={destination}&jetton_master={usdtJetton}&start_utime={checkpoint}` for each pending TON intent.
- Worker calls TonCenter v3 `/jetton/transfers?account={destination}&jetton_master={usdtJetton}&start_utime={checkpoint}` for each pending TON intent individually.
- Match criterion: `destination == intent.Destination AND amount >= intent.Amount`.
- TonCenter returns only finalized transactions — status jumps directly to `confirmed`.
- TON addresses are base64url (`EQ…`/`UQ…`), case-sensitive. Never lowercased.
- Checkpoint stored as Unix seconds.
- Lag is reported in seconds, not blocks.
- TON addresses are base64url (`EQ…`/`UQ…`), case-sensitive. They must never be lowercased.
- Checkpoint stored as Unix seconds. Lag reported in seconds, not blocks.
- **Scaling note**: each pending TON intent triggers one HTTP round-trip per scan cycle. High concurrency will exhaust TonCenter rate limits.
---
@@ -248,7 +269,7 @@ TransferWithReferenceAndFee → keccak256 = 0xbc8b749850bdc0c5e4a49d50f87ec40dca
### Payment confirmed (intent webhook)
Posted to `callbackUrl` on intent confirmation:
Posted to `callbackUrl` when an intent reaches `confirmed` status:
```json
{
@@ -264,17 +285,18 @@ Posted to `callbackUrl` on intent confirmation:
}
```
Header: `X-AMN-Signature: hex(HMAC-SHA256(rawBody, callbackSecret))`
Headers:
- `X-AMN-Signature: hex(HMAC-SHA256(rawBody, callbackSecret))`
The `confirmations` value is **capped** at the chain's acceptance threshold once confirmed. The scanner does not continue incrementing after the payment is safe to credit.
The `confirmations` value is **capped** at the chain acceptance threshold once confirmed. The scanner does not keep incrementing after the payment is safe to credit.
**Retry schedule on delivery failure**: `5 s → 30 s → 2 min → 10 min → 1 h → webhook_failed`
After exhausting retries the intent is set to `webhook_failed`. Manual recovery: `POST /admin/webhooks/retry` or wait for the `WEBHOOK_RETRY_HOURS` background sweep.
After exhausting retries the intent status becomes `webhook_failed`. Recovery: `POST /admin/webhooks/retry` or wait for the `WEBHOOK_RETRY_HOURS` background sweep (default 6 h).
### Balance changed (balance-watch webhook)
Posted to the watch's `callbackUrl` when balance delta is detected:
Posted to the watch's `callbackUrl` when a balance delta is detected:
```json
{
@@ -295,65 +317,77 @@ Posted to the watch's `callbackUrl` when balance delta is detected:
}
```
Additional headers: `X-AMN-Delivery-ID: <watchId>`, `X-AMN-Event-Type: balance_changed`
Additional headers:
- `X-AMN-Delivery-ID: <watchId>`
- `X-AMN-Event-Type: balance_changed`
- `X-AMN-Signature: hex(HMAC-SHA256(rawBody, callbackSecret))`
The scanner only advances `current_balance` after a successful (2xx) delivery, so a down backend will retry on the next scheduled check.
The scanner only advances `current_balance` in the DB after a successful (2xx) delivery. A down backend will get the same notification on the next scheduled check.
**Watch polling cadence** (age-decayed):
- First 24 hours: every 5 minutes
- 2448 hours: every 10 minutes
- 4896 hours: every 20 minutes
- 96+ hours: every 40 minutes
- Hard expiry: 7 days after creation
---
## 9. SQLite DB Schema
Database at `DB_PATH` (default `./scanner.db`; Docker: `/data/scanner.db`). WAL mode enabled, busy timeout 5 000 ms.
Database at `DB_PATH` (default `./scanner.db`; Docker: `/data/scanner.db`). WAL mode with 5 000 ms busy timeout. Connection pool capped at 1 to serialize writes.
### `intents`
| Column | Type | Notes |
|---|---|---|
| `intent_id` | TEXT PK | caller-supplied UUID |
| `chain_id` | INTEGER | numeric chain ID |
| `intent_id` | TEXT PK | Caller-supplied UUID |
| `chain_id` | INTEGER | Numeric chain ID |
| `chain_type` | TEXT | `evm` / `tron` / `ton` |
| `token_address` | TEXT | EVM/Tron: lowercase `0x` hex; TON: base64url |
| `destination` | TEXT | receiving address |
| `amount` | TEXT | base-10 wei / token smallest unit |
| `payment_reference` | TEXT | 8-byte hex — EVM only |
| `topic_ref` | TEXT | keccak256 of paymentReference scan index for EVM |
| `destination` | TEXT | Receiving address |
| `amount` | TEXT | Base-10 smallest unit (wei / TRC-20 units / nanoton) |
| `payment_reference` | TEXT | 8-byte hex — EVM proxy rail only |
| `topic_ref` | TEXT | `keccak256(paymentReferenceBytes)` — EVM scan index |
| `status` | TEXT | `pending` / `confirming` / `confirmed` / `expired` / `webhook_failed` |
| `callback_url` | TEXT | backend webhook endpoint |
| `callback_secret` | TEXT | HMAC key — excluded from all JSON responses |
| `confirmations_required` | INTEGER | floored at the chain acceptance threshold |
| `tx_hash` | TEXT NULL | set once the transaction is seen on-chain |
| `log_index` | INTEGER NULL | log position within tx (EVM only) |
| `block_number` | INTEGER NULL | block when seen (EVM); ms timestamp (Tron); unix s (TON) |
| `confirmations` | INTEGER | depth while confirming; capped at threshold after confirmation |
| `callback_url` | TEXT | Backend webhook endpoint |
| `callback_secret` | TEXT | HMAC key — excluded from all JSON responses (`json:"-"`) |
| `confirmations_required` | INTEGER | Set to chain acceptance floor at intent creation |
| `tx_hash` | TEXT NULL | Set once the transaction is seen on-chain |
| `log_index` | INTEGER NULL | Log position within tx (EVM only) |
| `block_number` | INTEGER NULL | Block number (EVM); ms timestamp (Tron); unix seconds (TON) |
| `confirmations` | INTEGER | Depth while confirming; capped at threshold after confirmation |
| `salt` | TEXT | 32-byte random hex used in reference derivation |
| `webhook_delivered_at` | TEXT NULL | RFC3339 timestamp of first successful delivery |
| `created_at` / `updated_at` | DATETIME | UTC |
Unique index on `(tx_hash, log_index)` prevents double-confirmation.
Indexes: `(status)`, `(chain_id, status)`, `(payment_reference)`, `(topic_ref)`.
Unique index: `(tx_hash, log_index) WHERE tx_hash IS NOT NULL` — prevents double-confirmation.
### `checkpoints`
| Column | Notes |
|---|---|
| `chain_id` PK | |
| `last_scanned_block` | block number (EVM), ms timestamp (Tron), unix seconds (TON) |
| `chain_id` PK | Numeric chain ID |
| `last_scanned_block` | Block number (EVM), ms timestamp (Tron), unix seconds (TON) |
| `updated_at` | UTC |
### `balance_watches`
| Column | Type | Notes |
|---|---|---|
| `watch_id` | TEXT PK | caller-supplied idempotency key |
| `chain_id` / `chain_type` | INTEGER / TEXT | currently EVM only |
| `watch_id` | TEXT PK | Caller-supplied idempotency key |
| `chain_id` / `chain_type` | INTEGER / TEXT | Currently EVM only |
| `token_address` / `token_symbol` | TEXT | ERC-20 contract + optional registry symbol |
| `decimals` | INTEGER | registry decimals for display |
| `address` | TEXT | watched holder address |
| `baseline_balance` | TEXT | base-unit balance at watch creation |
| `current_balance` | TEXT | last successfully delivered balance |
| `decimals` | INTEGER | Token decimals for display |
| `address` | TEXT | Watched holder address |
| `baseline_balance` | TEXT | Base-unit balance at watch creation |
| `current_balance` | TEXT | Last successfully delivered balance |
| `status` | TEXT | `watching` / `stopped` / `expired` |
| `callback_url` / `callback_secret` | TEXT | signed webhook destination |
| `last_checked_at` / `next_check_at` | DATETIME | scheduler state |
| `change_count` / `last_notified_at` | INTEGER / DATETIME | notification audit |
| `expires_at` | DATETIME | hard stop after 7 days |
| `callback_url` / `callback_secret` | TEXT | Signed webhook destination + HMAC key |
| `last_checked_at` / `next_check_at` | DATETIME | Scheduler state |
| `change_count` / `last_notified_at` | INTEGER / DATETIME | Notification audit |
| `expires_at` | DATETIME | Hard stop 7 days after creation |
| `created_at` / `updated_at` | DATETIME | UTC |
Indexes: `(status, next_check_at)` for due-scan queries; `(chain_id, status)` for status reporting.
@@ -362,15 +396,15 @@ Indexes: `(status, next_check_at)` for due-scan queries; `(chain_id, status)` fo
## 10. Configuration
All configuration via environment variables. Copy `.env.example` and populate before first run.
All configuration via environment variables. Copy `.env.example` before first run.
| Variable | Default | Required | Notes |
|---|---|---|---|
| `PORT` | `8080` | no | HTTP listen port |
| `DB_PATH` | `./scanner.db` | no | SQLite file path. Docker: mount `/data` and set `/data/scanner.db` |
| `CHAINS_JSON_PATH` | `./supported-chains.json` | no | Chain registry file |
| `DB_PATH` | `./scanner.db` | no | SQLite file path. Docker: mount `/data`, set `/data/scanner.db` |
| `CHAINS_JSON_PATH` | `./supported-chains.json` | no | Chain registry JSON file |
| `TOKENS_JSON_PATH` | `./tokens.json` | no | Token registry for symbol/decimals metadata |
| `SCANNER_API_KEY` | _(none)_ | **yes (prod)** | Shared secret for Bearer auth. Generate: `openssl rand -hex 32` |
| `SCANNER_API_KEY` | _(none)_ | **yes (prod)** | Shared secret for Bearer auth. Generate: `openssl rand -hex 32`. Unset = all requests allowed (dev only) |
| `POLL_INTERVAL_SEC` | `15` | no | Chain polling interval in seconds |
| `INTENT_TTL_HOURS` | `24` | no | Expire pending intents after N hours. `0` = disabled |
| `WEBHOOK_RETRY_HOURS` | `6` | no | Background re-delivery interval for `webhook_failed` intents. `0` = disabled |
@@ -381,10 +415,10 @@ All configuration via environment variables. Copy `.env.example` and populate be
| `RPC_ETH` | chain config | no | Override Ethereum JSON-RPC URL |
| `RPC_POLYGON` | chain config | no | Override Polygon JSON-RPC URL |
| `RPC_BASE` | chain config | no | Override Base JSON-RPC URL |
| `TRONGRID_API_KEY` | _(none)_ | recommended | Free tier is very low; required for any real Tron traffic |
| `TRONGRID_API_KEY` | _(none)_ | strongly recommended | Free tier is severely rate-limited; required for real Tron traffic |
| `TONCENTER_API_KEY` | _(none)_ | recommended | Rate-limited without key |
| `SCANNER_ENABLED_CHAINS` | JSON `verified` flags | no | Comma-separated chain IDs to enable, overriding `verified`. E.g. `56,1` |
| `SCANNER_CALLBACK_ALLOWED_HOSTS` | _(none)_ | prod recommended | Comma-separated hosts for SSRF guard on `callbackUrl` targets |
| `SCANNER_ENABLED_CHAINS` | JSON `verified` flags | no | Comma-separated chain IDs to activate, overriding `verified` field. E.g. `56,1` |
| `SCANNER_CALLBACK_ALLOWED_HOSTS` | _(none)_ | prod recommended | Comma-separated hosts/IPs allowed as `callbackUrl` targets (SSRF guard) |
---
@@ -394,7 +428,7 @@ All configuration via environment variables. Copy `.env.example` and populate be
# Build
docker build -t amn-scanner .
# Run
# Run (standalone)
docker run -d \
--name amn-scanner \
--network shared-web \
@@ -405,28 +439,41 @@ docker run -d \
amn-scanner
```
**On the dev server** (`89.58.32.32`): the scanner is part of the `escrow-dev` Arcane project. Images are built locally from source at `/tmp/escrow-backend-build/` — the dev stack does **not** pull from any registry.
### Dev server (89.58.32.32)
The scanner is part of the `escrow-dev` Arcane project. The dev stack builds images locally — it does **not** pull from any registry.
```bash
# Copy changed scanner source files
# 1. Copy changed scanner source files
scp -i ~/CascadeProjects/wzp scanner/*.go root@89.58.32.32:/tmp/escrow-backend-build/scanner/
# Rebuild + restart (on server)
# 2. Rebuild image on server (~23 min)
ssh -i ~/CascadeProjects/wzp root@89.58.32.32 \
"cd /tmp/escrow-backend-build/scanner && docker build -t amn-scanner-local:dev . && \
cd /opt/arcane/data/projects/escrow-dev && docker compose up -d scanner"
"cd /tmp/escrow-backend-build/scanner && \
docker build -t amn-scanner-local:dev . && \
cd /opt/arcane/data/projects/escrow-dev && \
docker compose up -d scanner"
```
Health check URL (via infra-caddy): check project Caddyfile for the current vhost. Direct internal: `http://amn-scanner:8080/health`.
Health check: `curl http://amn-scanner:8080/health` (internal) or via the Caddyfile vhost.
### Health probe
```
GET /health
→ {"status":"ok"}
```
---
## 12. Integration with the Backend
The backend wires the scanner through the `amn.scanner` provider. See memory note [[amn_scanner_payin_wiring]] for full service/dispatch registration and the 6 required env vars.
### Registering a payment intent
```typescript
// backend: src/services/amnScanner/...
// src/services/amnScanner/...
const resp = await fetch(`${SCANNER_URL}/intents`, {
method: 'POST',
headers: {
@@ -434,26 +481,26 @@ const resp = await fetch(`${SCANNER_URL}/intents`, {
'Content-Type': 'application/json',
},
body: JSON.stringify({
intentId: payment._id.toString(), // MongoDB ObjectId string
intentId: payment._id.toString(), // MongoDB ObjectId string
chainId: 56,
tokenAddress: '0x55d398326f99059fF775485246999027B3197955', // USDT BSC
destination: sellerWalletAddress,
amount: amountInWei, // base-10 string
amount: amountInWei, // base-10 string, smallest unit
callbackUrl: `${BACKEND_URL}/api/payment/amn-scanner/webhook`,
callbackSecret: process.env.SCANNER_CALLBACK_SECRET,
}),
});
const { intentId, paymentReference, checkoutBlock } = await resp.json();
// store intentId + checkoutBlock in the payment record
// pass checkoutBlock to the frontend for transaction construction
// Store intentId in the payment record
// Pass checkoutBlock to the frontend for transaction construction
```
The `checkoutBlock` contains everything the frontend needs to call the `ERC20FeeProxy.transferWithReferenceAndFee()` function:
The `checkoutBlock` response contains everything the frontend needs to call `ERC20FeeProxy.transferWithReferenceAndFee()`:
```json
{
"destination": "0x...",
"tokenAddress": "0x55d...",
"tokenAddress": "0x55d398326f99059fF775485246999027B3197955",
"tokenSymbol": "USDT",
"decimals": 18,
"chainId": 56,
@@ -465,49 +512,77 @@ The `checkoutBlock` contains everything the frontend needs to call the `ERC20Fee
}
```
> [!note] Token decimals
> Read token decimals on-chain, not from an internal registry. The scanner's `checkoutBlock.decimals` comes from `tokens.json`, which may lag registry updates.
### Receiving the webhook callback
```typescript
// POST /api/payment/amn-scanner/webhook
app.post('/api/payment/amn-scanner/webhook', async (req, res) => {
const signature = req.headers['x-amn-signature'];
const expected = hmacSha256Hex(req.rawBody, process.env.SCANNER_CALLBACK_SECRET);
const expected = hmacSha256Hex(req.rawBody, process.env.SCANNER_CALLBACK_SECRET);
if (!timingSafeEqual(signature, expected)) return res.status(401).end();
const { intentId, status, txHash, amount, chainId } = req.body;
if (status !== 'confirmed') return res.status(200).end(); // ignore non-confirmed
if (status !== 'confirmed') return res.status(200).end(); // ignore non-terminal
await paymentService.markConfirmed({ intentId, txHash, amount, chainId });
res.status(200).end();
});
```
> [!warning]
> The backend must always scope payment lookups by `provider: "amn.scanner"`. Sweeping all pending payments to mark them confirmed/failed will destroy records from other providers (RN, manual). See memory note on payment cleanup + provider filter.
> [!warning] Always scope by provider
> The backend must always scope payment lookups to `provider: "amn.scanner"`. Sweeping all pending payments will destroy records from other providers (RN, manual). See memory note on payment cleanup + provider filter.
### Backend env vars required
### Using direct balance checks (non-proxy flows)
```typescript
// Synchronous balance read (manual payment flow)
const { balance } = await scannerClient.post('/balances/check', {
chainId: 56,
address: sellerWalletAddress,
token: 'USDT',
});
// Store baseline, then re-check when buyer clicks "I paid"
// Async balance watch
await scannerClient.post('/balance-watches', {
watchId: `payment-${paymentId}-c56-USDT`,
chainId: 56,
address: sellerWalletAddress,
token: 'USDT',
callbackUrl: `${BACKEND_URL}/api/payment/amn-scanner/webhook`,
callbackSecret: process.env.SCANNER_CALLBACK_SECRET,
baselineBalance: '0',
});
// Stop watch after payment resolved
await scannerClient.delete(`/balance-watches/payment-${paymentId}-c56-USDT`);
```
### Backend environment variables
```
SCANNER_URL=http://amn-scanner:8080
SCANNER_API_KEY=<same value as scanner SCANNER_API_KEY>
SCANNER_CALLBACK_SECRET=<same value as scanner intent callbackSecret>
SCANNER_CALLBACK_SECRET=<shared HMAC key, same value used in callbackSecret field>
```
See memory note: [[amn_scanner_payin_wiring]] for full wiring details and token-decimal notes.
---
## 13. Known Limitations / Open Items
| # | Area | Description |
|---|---|---|
| 1 | **TON scaling** | O(pending intents) TonCenter API calls per scan cycle. Becomes a bottleneck above ~50 concurrent TON intents. Future fix: batch by address or use a streaming API. |
| 2 | **Tron/TON balance watches** | `POST /balances/check` and `POST /balance-watches` currently only support EVM ERC-20 reads. Tron TRC-20 and TON Jetton balance reads are future scope. |
| 3 | **Non-EVM chains unverified** | Arbitrum, Polygon, Base, Tron, TON are `"verified": false` — workers do not start by default. Needs production testing + `SCANNER_ENABLED_CHAINS` opt-in before enabling for real traffic. |
| 4 | **Base proxy address** | `0x1892196E80C4c17ea5100Da765Ab48c1fE2Fb814` is non-canonical (differs from RN's CREATE2 expected address for that chain). Verify before enabling Base in production. |
| 5 | **Ethereum proxy version** | Ethereum uses the v0.1.0 proxy at `0x370DE27…`. A v0.2.0 proxy is also deployed on ETH. The two have different ABI signatures — do not silently swap addresses. |
| 6 | **Single SQLite instance** | The embedded SQLite database is not horizontally scalable. Running multiple scanner replicas would require coordination or migration to a shared DB. Acceptable for current load. |
| 7 | **No native-token support** | Only ERC-20/TRC-20/Jetton transfers are scanned. Native token (BNB, ETH, TRX, TON coin) payments are not supported. |
| 8 | **Multi-seller / multi-chain** | AMN Scanner pay-in supports single-seller flow only. Multi-seller cart payments and cross-chain routing are not implemented. |
| 9 | **Webhook signature algorithm** | HMAC-SHA256 with a pre-shared secret. There is no key rotation mechanism — changing `callbackSecret` requires intent re-registration. |
| 10 | **No EVM testnet for ETH/Polygon/Base** | Only BSC Testnet (97) has a testnet entry in `supported-chains.json`. Developers testing on Ethereum Sepolia or Polygon Amoy need to add chain entries manually. |
| 1 | **TON scaling** | O(pending intents) TonCenter API calls per scan cycle. Becomes a bottleneck above ~50 concurrent TON intents. Future fix: batch by address or use a streaming/webhook API. |
| 2 | **Tron/TON balance watches** | `POST /balances/check` and `POST /balance-watches` only support EVM ERC-20 (`eth_call` balanceOf). Tron TRC-20 and TON Jetton balance reads are future scope. |
| 3 | **Non-EVM chains unverified** | Arbitrum, Polygon, Base, Tron, TON are `"verified": false` — workers do not start by default. Needs production testing and `SCANNER_ENABLED_CHAINS` opt-in before enabling for real traffic. |
| 4 | **Base proxy address non-canonical** | `0x1892196E80C4c17ea5100Da765Ab48c1fE2Fb814` differs from the RN CREATE2 expected address for Base. Must be verified on-chain before enabling Base in production. |
| 5 | **Ethereum proxy version** | Chain 1 uses the v0.1.0 proxy at `0x370DE27…`. A v0.2.0 proxy is also deployed on ETH. The two have different ABI signatures — do not silently swap addresses. |
| 6 | **Single SQLite instance** | The embedded SQLite database is not horizontally scalable. Running multiple scanner replicas would require coordination or migration to a shared DB (Postgres). Acceptable for current load. |
| 7 | **No native-token support** | Only ERC-20, TRC-20, and Jetton (TON) transfers are scanned. Native token payments (BNB, ETH, TRX, TON coin) are not supported. |
| 8 | **Single-seller only** | AMN Scanner pay-in supports single-seller flow. Multi-seller cart payments and cross-chain routing are not implemented. |
| 9 | **No webhook key rotation** | HMAC-SHA256 with a pre-shared `callbackSecret`. There is no key rotation mechanism — changing the secret requires re-registering intents. |
| 10 | **No EVM testnet for ETH/Polygon/Base** | Only BSC Testnet (97) has a testnet entry in `supported-chains.json`. Testing on Ethereum Sepolia or Polygon Amoy requires manually adding chain entries. |
| 11 | **Arbitrum threshold latency** | The 2400-block Arbitrum threshold (~54 min) is deliberately conservative for the optimistic rollup challenge window. This makes Arbitrum slow for real-time escrow use. |