251 lines
12 KiB
Markdown
251 lines
12 KiB
Markdown
---
|
||
title: Scanner Architecture
|
||
tags: [architecture, scanner, payment]
|
||
created: 2026-05-30
|
||
---
|
||
|
||
# Scanner Architecture
|
||
|
||
AMN Pay Scanner is a standalone Go microservice that watches on-chain payment events and notifies the backend via webhook when a payment is confirmed. It replaces the Request Network integration with an in-house polling scanner that supports EVM chains, Tron, and TON.
|
||
|
||
> [!info]
|
||
> Repo: `scanner/` within the escrow monorepo. Binary: `scanner`. Written in Go 1.25. SQLite (WAL mode) for state. No external dependencies beyond the chain APIs. Scanner `0.1.8` also supports direct-address EVM ERC-20 balance reads and balance watches for non-smart-contract payment rails.
|
||
|
||
---
|
||
|
||
## 1. Responsibilities
|
||
|
||
- Accept payment **intents** from the backend (POST /intents)
|
||
- Watch the relevant chain for matching on-chain transfers
|
||
- Track confirmation depth (EVM) or rely on finality from the chain API (Tron, TON)
|
||
- Deliver a signed webhook to the backend callback URL when confirmed
|
||
- Retry failed webhook deliveries
|
||
- Expire stale pending intents on a configurable TTL
|
||
- Read an EVM ERC-20 balance on demand for a backend-supplied address/token
|
||
- Watch an EVM address/token pair for balance changes, decay polling cadence over time, and stop after backend cancellation or 7-day expiry
|
||
|
||
---
|
||
|
||
## 2. Component map
|
||
|
||
```
|
||
┌─────────────────────────────────────────────────────────┐
|
||
│ scanner binary │
|
||
│ │
|
||
│ main.go │
|
||
│ ├── loadConfig() config.go │
|
||
│ ├── initDB() intent.go (SQLite schema) │
|
||
│ ├── startup reconcile intent.go │
|
||
│ ├── newServer() api.go │
|
||
│ │ └── startWorkers() api.go │
|
||
│ │ ├── ChainWorker chain.go (EVM) │
|
||
│ │ ├── TronChainWorker tron_chain.go (Tron) │
|
||
│ │ └── TonChainWorker ton_chain.go (TON) │
|
||
│ ├── HTTP routes api.go / main.go │
|
||
│ ├── intent TTL expiry main.go + intent.go │
|
||
│ ├── webhook retry loop main.go + webhook.go │
|
||
│ └── BalanceWatchScheduler balance_watch.go │
|
||
│ │
|
||
│ reference.go — payment reference / topic hash math │
|
||
│ webhook.go — delivery, HMAC signing, retry │
|
||
│ balance.go — EVM ERC-20 balanceOf(address) reads │
|
||
│ balance_watch.go — balance_watches state + webhooks │
|
||
└─────────────────────────────────────────────────────────┘
|
||
```
|
||
|
||
---
|
||
|
||
## 3. Chain worker model
|
||
|
||
All three chain types implement the `Worker` interface:
|
||
|
||
```go
|
||
type Worker interface {
|
||
start()
|
||
stop()
|
||
getHead(ctx context.Context) (int64, error)
|
||
}
|
||
```
|
||
|
||
One worker goroutine is spawned per chain marked `"verified": true` in `supported-chains.json`. Workers are selected by `chainType`:
|
||
|
||
| chainType | Worker struct | API used |
|
||
|---|---|---|
|
||
| `evm` (default) | `ChainWorker` | JSON-RPC 2.0 (`eth_getLogs`, `eth_blockNumber`) |
|
||
| `tron` | `TronChainWorker` | TronGrid REST (`/v1/contracts/{contract}/events`) |
|
||
| `ton` | `TonChainWorker` | TonCenter v3 REST (`/jetton/transfers`) |
|
||
|
||
Workers poll on `POLL_INTERVAL_SEC` (default 15 s). On first run, each worker starts scanning from the current chain head minus a small buffer (10 blocks for EVM, 24 h for Tron/TON).
|
||
|
||
---
|
||
|
||
## 4. EVM scanning detail
|
||
|
||
```
|
||
for each tick:
|
||
head = eth_blockNumber
|
||
from = max(checkpoint − ReorgBuffer(), 0)
|
||
chunks = split [from..head] into 2000-block ranges
|
||
for each chunk:
|
||
logs = eth_getLogs(proxyAddress, EventTopic, from, to)
|
||
for each log:
|
||
topicRef = Topics[1] (keccak256 of paymentReference — pre-indexed)
|
||
intent = DB lookup by topicRef WHERE status='pending'
|
||
validate(log.Data, intent) ← token + destination + amount check
|
||
confirmIntentPending() ← status → 'confirming'
|
||
saveCheckpoint(to)
|
||
checkConfirmations():
|
||
for each confirming intent:
|
||
confs = head - blockNumber + 1
|
||
if confs >= required: finalizeIntent(capped at required) + deliverWebhook()
|
||
```
|
||
|
||
**Reorg protection**: `ReorgBuffer()` re-scans `3 × confirmationThreshold` blocks before the checkpoint (clamped 20–500). This catches any log that appeared in a block that was later reorganised off the canonical chain.
|
||
|
||
**Event signature**: `TransferWithReferenceAndFee` keccak256 = `0xbc8b749850bdc0c5e4a49d50f87ec40dca2acdb70a84e7c6823ccdc7b3b3d4b3`
|
||
|
||
---
|
||
|
||
## 5. Tron scanning detail
|
||
|
||
TronGrid does not expose a fee-proxy contract. Each intent is assigned a unique HD-derived destination address. The scanner watches TRC20 `Transfer` events on the USDT contract and matches by `to` address.
|
||
|
||
- Checkpoint: block timestamp in milliseconds (`last_scanned_block` column)
|
||
- TronGrid addresses arrive as `41xxxx` hex (21 bytes); normalized to `0x` (20 bytes EVM style)
|
||
- Tron transactions reported by TronGrid are already confirmed; status goes directly to `confirmed` (no multi-block wait)
|
||
- Pagination follows `meta.links.next` until empty
|
||
|
||
---
|
||
|
||
## 6. TON scanning detail
|
||
|
||
TON uses TonCenter v3. Per-intent polling: for each pending TON intent, a separate HTTP call fetches incoming Jetton transfers to that destination since the checkpoint.
|
||
|
||
- Checkpoint: Unix timestamp in seconds
|
||
- TON addresses are base64url (`EQ…`/`UQ…`) — case-sensitive, never lowercased
|
||
- `proxyAddress` = USDT Jetton master address (`EQCxE6mUtQJKFnGfaROTKOt1lZbDiiX1kCixRv7Nw2Id_sDs`)
|
||
- TonCenter returns only finalized transactions; status goes directly to `confirmed`
|
||
- Lag is reported in seconds, not blocks
|
||
- Known scaling limitation: O(pending intents) API calls per scan cycle
|
||
|
||
---
|
||
|
||
## 7. Intent lifecycle
|
||
|
||
```
|
||
pending ──(tx seen)──► confirming ──(enough blocks)──► confirmed ──(webhook ok)──► [done]
|
||
│ │ │
|
||
│ │ (deep reorg / TTL) │ (all retries fail)
|
||
└───────────────────────┴──────────► expired webhook_failed
|
||
```
|
||
|
||
- **Tron / TON** skip `confirming` and jump directly to `confirmed`.
|
||
- `webhook_failed` intents are retried every `WEBHOOK_RETRY_HOURS` (default 6 h) and on `POST /admin/webhooks/retry`.
|
||
- **Startup reconciliation**: on startup, `confirmed` intents with `webhook_delivered_at IS NULL` and created in the last 7 days have their webhook re-delivered. This recovers from a crash between `finalizeIntent` and `deliverWebhook`.
|
||
|
||
---
|
||
|
||
## 8. Direct balance watch model
|
||
|
||
Direct-address payments are a separate rail from scanner intents. They do not require `ERC20FeeProxy`, `paymentReference`, or a smart-contract event. The backend gives the scanner a public address, token, callback URL, and HMAC secret.
|
||
|
||
Two backend usage modes are supported:
|
||
|
||
- **Synchronous check mode**: backend calls `POST /balances/check` when an address is allocated to the buyer and again when the buyer clicks "I paid". The backend compares the current base-unit balance to its stored baseline and target amount.
|
||
- **Watch mode**: backend calls `POST /balance-watches`; the scanner stores a row in `balance_watches`, reads the initial balance, and checks the same address/token periodically. If the balance changes, scanner sends a signed `balance_changed` webhook. Backend stops the watch with `DELETE /balance-watches/{watchId}` once the payment is accepted, cancelled, or otherwise resolved.
|
||
|
||
Cadence is age-based:
|
||
|
||
| Watch age | Next check interval |
|
||
|---|---|
|
||
| 0–24 h | 5 min |
|
||
| 24–48 h | 10 min |
|
||
| 48–72 h | 20 min |
|
||
| 72 h–7 d | 40 min |
|
||
| 7 d+ | status becomes `expired` |
|
||
|
||
The scanner only advances `current_balance` after a changed-balance webhook is delivered successfully. If the backend is down or returns non-2xx, the same change is retried on the next scheduled due check.
|
||
|
||
Current implementation scope: EVM ERC-20 `balanceOf(address)` via JSON-RPC `eth_call`. Tron/TON balance reads are future scope.
|
||
|
||
---
|
||
|
||
## 9. Payment reference math (EVM)
|
||
|
||
```
|
||
paymentReference = last8Bytes(keccak256(lower(intentId + salt + destination)))
|
||
topicRef (index) = keccak256(paymentReferenceBytes)
|
||
```
|
||
|
||
The ERC20FeeProxy indexes `paymentReference` so `Topics[1]` in the log is `topicRef`, not the raw reference. The DB stores `topic_ref` pre-computed per intent so the scan loop is a single indexed SQL lookup instead of O(n) hashing.
|
||
|
||
---
|
||
|
||
## 10. Database schema (SQLite WAL)
|
||
|
||
Three main tables:
|
||
|
||
**`intents`** — one row per payment intent
|
||
|
||
| Column | Type | Notes |
|
||
|---|---|---|
|
||
| `intent_id` | TEXT PK | caller-supplied UUID |
|
||
| `chain_id` | INTEGER | numeric chain ID |
|
||
| `chain_type` | TEXT | `evm` / `tron` / `ton` |
|
||
| `token_address` | TEXT | EVM/Tron: lowercase 0x hex; TON: base64url |
|
||
| `destination` | TEXT | receiving address |
|
||
| `amount` | TEXT | base-10 wei / token smallest unit |
|
||
| `payment_reference` | TEXT | 8-byte hex (EVM only) |
|
||
| `topic_ref` | TEXT | keccak256 of paymentReference (EVM index) |
|
||
| `status` | TEXT | `pending` / `confirming` / `confirmed` / `expired` / `webhook_failed` |
|
||
| `callback_url` | TEXT | backend webhook endpoint |
|
||
| `callback_secret` | TEXT | HMAC key (not returned in GET) |
|
||
| `confirmations_required` | INTEGER | from chain config or caller override, floored at the chain acceptance threshold |
|
||
| `tx_hash` | TEXT NULL | transaction hash once seen |
|
||
| `log_index` | INTEGER NULL | log position within tx (EVM) |
|
||
| `block_number` | INTEGER NULL | block / timestamp when seen |
|
||
| `confirmations` | INTEGER | current depth while confirming; capped at the accepted threshold after confirmation |
|
||
| `salt` | TEXT | 32-byte random hex for reference derivation |
|
||
| `webhook_delivered_at` | TEXT NULL | RFC3339 timestamp of successful delivery |
|
||
| `created_at` / `updated_at` | DATETIME | |
|
||
|
||
Unique index on `(tx_hash, log_index)` prevents duplicate intent confirmation.
|
||
|
||
**`checkpoints`** — one row per chain, tracks scan progress
|
||
|
||
| Column | Notes |
|
||
|---|---|
|
||
| `chain_id` | PK |
|
||
| `last_scanned_block` | block number (EVM), ms timestamp (Tron), unix seconds (TON) |
|
||
|
||
**`balance_watches`** — one row per direct-address balance watch
|
||
|
||
| Column | Type | Notes |
|
||
|---|---|---|
|
||
| `watch_id` | TEXT PK | caller-supplied or scanner-generated idempotency key |
|
||
| `chain_id` / `chain_type` | INTEGER / TEXT | currently EVM only for direct balance reads |
|
||
| `token_address` / `token_symbol` | TEXT | ERC-20 contract and optional registry symbol |
|
||
| `decimals` | INTEGER | registry decimals for display/metadata |
|
||
| `address` | TEXT | watched holder address |
|
||
| `baseline_balance` | TEXT | base-unit balance at backend baseline |
|
||
| `current_balance` | TEXT | last delivered/current scanner balance |
|
||
| `status` | TEXT | `watching` / `stopped` / `expired` |
|
||
| `callback_url` / `callback_secret` | TEXT | signed webhook destination |
|
||
| `last_checked_at` / `next_check_at` | DATETIME | scheduler state |
|
||
| `change_count` / `last_notified_at` | INTEGER / DATETIME | notification audit fields |
|
||
| `expires_at` | DATETIME | hard stop after 7 days |
|
||
| `created_at` / `updated_at` | DATETIME | UTC timestamps |
|
||
|
||
Indexes: `(status, next_check_at)` for due scans and `(chain_id, status)` for status reporting.
|
||
|
||
---
|
||
|
||
## 11. Security model
|
||
|
||
- All non-health endpoints require `Authorization: Bearer <SCANNER_API_KEY>` (constant-time compare).
|
||
- If `SCANNER_API_KEY` is unset the server logs a warning and allows all requests — intended for local dev only.
|
||
- Webhooks are signed with HMAC-SHA256: `X-AMN-Signature: hex(hmac(body, callbackSecret))`.
|
||
- The `callbackSecret` is stored in the DB but excluded from all JSON responses (`json:"-"` tag).
|
||
- Request bodies are limited to 64 KB.
|
||
- Balance-watch callbacks use the same HMAC header plus `X-AMN-Delivery-ID=<watchId>` and `X-AMN-Event-Type=balance_changed`.
|