nick/nick-doc

Fork 0

Files

Siavash Sameni 9dcdb420fc docs: sync from backend 22ae0bd — scanner balance watches

2026-06-03 21:23:50 +04:00

12 KiB

Raw Blame History

title, tags, created

title

Scanner Architecture

AMN Pay Scanner is a standalone Go microservice that watches on-chain payment events and notifies the backend via webhook when a payment is confirmed. It replaces the Request Network integration with an in-house polling scanner that supports EVM chains, Tron, and TON.

[!info] Repo: scanner/ within the escrow monorepo. Binary: scanner. Written in Go 1.25. SQLite (WAL mode) for state. No external dependencies beyond the chain APIs. Scanner 0.1.8 also supports direct-address EVM ERC-20 balance reads and balance watches for non-smart-contract payment rails.

1. Responsibilities

Accept payment intents from the backend (POST /intents)
Watch the relevant chain for matching on-chain transfers
Track confirmation depth (EVM) or rely on finality from the chain API (Tron, TON)
Deliver a signed webhook to the backend callback URL when confirmed
Retry failed webhook deliveries
Expire stale pending intents on a configurable TTL
Read an EVM ERC-20 balance on demand for a backend-supplied address/token
Watch an EVM address/token pair for balance changes, decay polling cadence over time, and stop after backend cancellation or 7-day expiry

2. Component map

┌─────────────────────────────────────────────────────────┐
│                       scanner binary                    │
│                                                         │
│  main.go                                                │
│  ├── loadConfig()          config.go                   │
│  ├── initDB()              intent.go (SQLite schema)   │
│  ├── startup reconcile     intent.go                   │
│  ├── newServer()           api.go                      │
│  │   └── startWorkers()   api.go                       │
│  │       ├── ChainWorker   chain.go        (EVM)       │
│  │       ├── TronChainWorker tron_chain.go (Tron)      │
│  │       └── TonChainWorker  ton_chain.go  (TON)       │
│  ├── HTTP routes           api.go / main.go            │
│  ├── intent TTL expiry     main.go + intent.go         │
│  ├── webhook retry loop    main.go + webhook.go        │
│  └── BalanceWatchScheduler balance_watch.go            │
│                                                         │
│  reference.go  — payment reference / topic hash math   │
│  webhook.go    — delivery, HMAC signing, retry         │
│  balance.go    — EVM ERC-20 balanceOf(address) reads   │
│  balance_watch.go — balance_watches state + webhooks   │
└─────────────────────────────────────────────────────────┘

3. Chain worker model

All three chain types implement the Worker interface:

type Worker interface {
    start()
    stop()
    getHead(ctx context.Context) (int64, error)
}

One worker goroutine is spawned per chain marked "verified": true in supported-chains.json. Workers are selected by chainType:

chainType	Worker struct	API used
`evm` (default)	`ChainWorker`	JSON-RPC 2.0 (`eth_getLogs`, `eth_blockNumber`)
`tron`	`TronChainWorker`	TronGrid REST (`/v1/contracts/{contract}/events`)
`ton`	`TonChainWorker`	TonCenter v3 REST (`/jetton/transfers`)

Workers poll on POLL_INTERVAL_SEC (default 15 s). On first run, each worker starts scanning from the current chain head minus a small buffer (10 blocks for EVM, 24 h for Tron/TON).

4. EVM scanning detail

for each tick:
  head   = eth_blockNumber
  from   = max(checkpoint − ReorgBuffer(), 0)
  chunks = split [from..head] into 2000-block ranges
  for each chunk:
    logs = eth_getLogs(proxyAddress, EventTopic, from, to)
    for each log:
      topicRef = Topics[1]  (keccak256 of paymentReference — pre-indexed)
      intent   = DB lookup by topicRef WHERE status='pending'
      validate(log.Data, intent)  ← token + destination + amount check
      confirmIntentPending()      ← status → 'confirming'
    saveCheckpoint(to)
  checkConfirmations():
    for each confirming intent:
      confs = head - blockNumber + 1
      if confs >= required: finalizeIntent(capped at required) + deliverWebhook()

Reorg protection: ReorgBuffer() re-scans 3 × confirmationThreshold blocks before the checkpoint (clamped 20–500). This catches any log that appeared in a block that was later reorganised off the canonical chain.

Event signature: TransferWithReferenceAndFee keccak256 = 0xbc8b749850bdc0c5e4a49d50f87ec40dca2acdb70a84e7c6823ccdc7b3b3d4b3

5. Tron scanning detail

TronGrid does not expose a fee-proxy contract. Each intent is assigned a unique HD-derived destination address. The scanner watches TRC20 Transfer events on the USDT contract and matches by to address.

Checkpoint: block timestamp in milliseconds (last_scanned_block column)
TronGrid addresses arrive as 41xxxx hex (21 bytes); normalized to 0x (20 bytes EVM style)
Tron transactions reported by TronGrid are already confirmed; status goes directly to confirmed (no multi-block wait)
Pagination follows meta.links.next until empty

6. TON scanning detail

TON uses TonCenter v3. Per-intent polling: for each pending TON intent, a separate HTTP call fetches incoming Jetton transfers to that destination since the checkpoint.

Checkpoint: Unix timestamp in seconds
TON addresses are base64url (EQ…/UQ…) — case-sensitive, never lowercased
proxyAddress = USDT Jetton master address (EQCxE6mUtQJKFnGfaROTKOt1lZbDiiX1kCixRv7Nw2Id_sDs)
TonCenter returns only finalized transactions; status goes directly to confirmed
Lag is reported in seconds, not blocks
Known scaling limitation: O(pending intents) API calls per scan cycle

7. Intent lifecycle

pending ──(tx seen)──► confirming ──(enough blocks)──► confirmed ──(webhook ok)──► [done]
   │                       │                               │
   │                       │ (deep reorg / TTL)            │ (all retries fail)
   └───────────────────────┴──────────► expired      webhook_failed

Tron / TON skip confirming and jump directly to confirmed.
webhook_failed intents are retried every WEBHOOK_RETRY_HOURS (default 6 h) and on POST /admin/webhooks/retry.
Startup reconciliation: on startup, confirmed intents with webhook_delivered_at IS NULL and created in the last 7 days have their webhook re-delivered. This recovers from a crash between finalizeIntent and deliverWebhook.

8. Direct balance watch model

Direct-address payments are a separate rail from scanner intents. They do not require ERC20FeeProxy, paymentReference, or a smart-contract event. The backend gives the scanner a public address, token, callback URL, and HMAC secret.

Two backend usage modes are supported:

Synchronous check mode: backend calls POST /balances/check when an address is allocated to the buyer and again when the buyer clicks "I paid". The backend compares the current base-unit balance to its stored baseline and target amount.
Watch mode: backend calls POST /balance-watches; the scanner stores a row in balance_watches, reads the initial balance, and checks the same address/token periodically. If the balance changes, scanner sends a signed balance_changed webhook. Backend stops the watch with DELETE /balance-watches/{watchId} once the payment is accepted, cancelled, or otherwise resolved.

Cadence is age-based:

Watch age	Next check interval
0–24 h	5 min
24–48 h	10 min
48–72 h	20 min
72 h–7 d	40 min
7 d+	status becomes `expired`

The scanner only advances current_balance after a changed-balance webhook is delivered successfully. If the backend is down or returns non-2xx, the same change is retried on the next scheduled due check.

Current implementation scope: EVM ERC-20 balanceOf(address) via JSON-RPC eth_call. Tron/TON balance reads are future scope.

9. Payment reference math (EVM)

paymentReference  = last8Bytes(keccak256(lower(intentId + salt + destination)))
topicRef (index)  = keccak256(paymentReferenceBytes)

The ERC20FeeProxy indexes paymentReference so Topics[1] in the log is topicRef, not the raw reference. The DB stores topic_ref pre-computed per intent so the scan loop is a single indexed SQL lookup instead of O(n) hashing.

10. Database schema (SQLite WAL)

Three main tables:

intents — one row per payment intent

Column	Type	Notes
`intent_id`	TEXT PK	caller-supplied UUID
`chain_id`	INTEGER	numeric chain ID
`chain_type`	TEXT	`evm` / `tron` / `ton`
`token_address`	TEXT	EVM/Tron: lowercase 0x hex; TON: base64url
`destination`	TEXT	receiving address
`amount`	TEXT	base-10 wei / token smallest unit
`payment_reference`	TEXT	8-byte hex (EVM only)
`topic_ref`	TEXT	keccak256 of paymentReference (EVM index)
`status`	TEXT	`pending` / `confirming` / `confirmed` / `expired` / `webhook_failed`
`callback_url`	TEXT	backend webhook endpoint
`callback_secret`	TEXT	HMAC key (not returned in GET)
`confirmations_required`	INTEGER	from chain config or caller override, floored at the chain acceptance threshold
`tx_hash`	TEXT NULL	transaction hash once seen
`log_index`	INTEGER NULL	log position within tx (EVM)
`block_number`	INTEGER NULL	block / timestamp when seen
`confirmations`	INTEGER	current depth while confirming; capped at the accepted threshold after confirmation
`salt`	TEXT	32-byte random hex for reference derivation
`webhook_delivered_at`	TEXT NULL	RFC3339 timestamp of successful delivery
`created_at` / `updated_at`	DATETIME

Unique index on (tx_hash, log_index) prevents duplicate intent confirmation.

checkpoints — one row per chain, tracks scan progress

Column	Notes
`chain_id`	PK
`last_scanned_block`	block number (EVM), ms timestamp (Tron), unix seconds (TON)

balance_watches — one row per direct-address balance watch

Column	Type	Notes
`watch_id`	TEXT PK	caller-supplied or scanner-generated idempotency key
`chain_id` / `chain_type`	INTEGER / TEXT	currently EVM only for direct balance reads
`token_address` / `token_symbol`	TEXT	ERC-20 contract and optional registry symbol
`decimals`	INTEGER	registry decimals for display/metadata
`address`	TEXT	watched holder address
`baseline_balance`	TEXT	base-unit balance at backend baseline
`current_balance`	TEXT	last delivered/current scanner balance
`status`	TEXT	`watching` / `stopped` / `expired`
`callback_url` / `callback_secret`	TEXT	signed webhook destination
`last_checked_at` / `next_check_at`	DATETIME	scheduler state
`change_count` / `last_notified_at`	INTEGER / DATETIME	notification audit fields
`expires_at`	DATETIME	hard stop after 7 days
`created_at` / `updated_at`	DATETIME	UTC timestamps

Indexes: (status, next_check_at) for due scans and (chain_id, status) for status reporting.

11. Security model

All non-health endpoints require Authorization: Bearer <SCANNER_API_KEY> (constant-time compare).
If SCANNER_API_KEY is unset the server logs a warning and allows all requests — intended for local dev only.
Webhooks are signed with HMAC-SHA256: X-AMN-Signature: hex(hmac(body, callbackSecret)).
The callbackSecret is stored in the DB but excluded from all JSON responses (json:"-" tag).
Request bodies are limited to 64 KB.
Balance-watch callbacks use the same HMAC header plus X-AMN-Delivery-ID=<watchId> and X-AMN-Event-Type=balance_changed.

12 KiB Raw Blame History Unescape Escape