nick-doc/01 - Architecture/Scanner Architecture.md

title, tags, created

title	tags	created
Scanner Architecture	architecture scanner payment	2026-05-30

Scanner Architecture

AMN Pay Scanner is a standalone Go microservice that watches on-chain payment events and notifies the backend via webhook when a payment is confirmed. It replaces the Request Network integration with an in-house polling scanner that supports EVM chains, Tron, and TON.

[!info] Repo: scanner/ within the escrow monorepo. Binary: scanner. Written in Go 1.25. SQLite (WAL mode) for state. No external dependencies beyond the chain APIs.

---

1. Responsibilities

• Accept payment intents from the backend (POST /intents)
• Watch the relevant chain for matching on-chain transfers
• Track confirmation depth (EVM) or rely on finality from the chain API (Tron, TON)
• Deliver a signed webhook to the backend callback URL when confirmed
• Retry failed webhook deliveries
• Expire stale pending intents on a configurable TTL

---

2. Component map

┌─────────────────────────────────────────────────────────┐
│                       scanner binary                    │
│                                                         │
│  main.go                                                │
│  ├── loadConfig()          config.go                   │
│  ├── initDB()              intent.go (SQLite schema)   │
│  ├── startup reconcile     intent.go                   │
│  ├── newServer()           api.go                      │
│  │   └── startWorkers()   api.go                       │
│  │       ├── ChainWorker   chain.go        (EVM)       │
│  │       ├── TronChainWorker tron_chain.go (Tron)      │
│  │       └── TonChainWorker  ton_chain.go  (TON)       │
│  ├── HTTP routes           api.go / main.go            │
│  ├── intent TTL expiry     main.go + intent.go         │
│  └── webhook retry loop    main.go + webhook.go        │
│                                                         │
│  reference.go  — payment reference / topic hash math   │
│  webhook.go    — delivery, HMAC signing, retry         │
└─────────────────────────────────────────────────────────┘

---

3. Chain worker model

All three chain types implement the Worker interface:

type Worker interface {
    start()
    stop()
    getHead(ctx context.Context) (int64, error)
}

One worker goroutine is spawned per chain marked "verified": true in supported-chains.json. Workers are selected by chainType:

chainType	Worker struct	API used
evm (default)	ChainWorker	JSON-RPC 2.0 (eth_getLogs, eth_blockNumber)
tron	TronChainWorker	TronGrid REST (/v1/contracts/{contract}/events)
ton	TonChainWorker	TonCenter v3 REST (/jetton/transfers)

Workers poll on POLL_INTERVAL_SEC (default 15 s). On first run, each worker starts scanning from the current chain head minus a small buffer (10 blocks for EVM, 24 h for Tron/TON).

---

4. EVM scanning detail

for each tick:
  head   = eth_blockNumber
  from   = max(checkpoint − ReorgBuffer(), 0)
  chunks = split [from..head] into 2000-block ranges
  for each chunk:
    logs = eth_getLogs(proxyAddress, EventTopic, from, to)
    for each log:
      topicRef = Topics[1]  (keccak256 of paymentReference — pre-indexed)
      intent   = DB lookup by topicRef WHERE status='pending'
      validate(log.Data, intent)  ← token + destination + amount check
      confirmIntentPending()      ← status → 'confirming'
    saveCheckpoint(to)
  checkConfirmations():
    for each confirming intent:
      confs = head - blockNumber + 1
      if confs >= required: finalizeIntent() + deliverWebhook()

Reorg protection: ReorgBuffer() re-scans 3 × confirmationThreshold blocks before the checkpoint (clamped 20–500). This catches any log that appeared in a block that was later reorganised off the canonical chain.

Event signature: TransferWithReferenceAndFee keccak256 = 0xbc8b749850bdc0c5e4a49d50f87ec40dca2acdb70a84e7c6823ccdc7b3b3d4b3

---

5. Tron scanning detail

TronGrid does not expose a fee-proxy contract. Each intent is assigned a unique HD-derived destination address. The scanner watches TRC20 Transfer events on the USDT contract and matches by to address.

• Checkpoint: block timestamp in milliseconds (last_scanned_block column)
• TronGrid addresses arrive as 41xxxx hex (21 bytes); normalized to 0x (20 bytes EVM style)
• Tron transactions reported by TronGrid are already confirmed; status goes directly to confirmed (no multi-block wait)
• Pagination follows meta.links.next until empty

---

6. TON scanning detail

TON uses TonCenter v3. Per-intent polling: for each pending TON intent, a separate HTTP call fetches incoming Jetton transfers to that destination since the checkpoint.

• Checkpoint: Unix timestamp in seconds
• TON addresses are base64url (EQ…/UQ…) — case-sensitive, never lowercased
• proxyAddress = USDT Jetton master address (EQCxE6mUtQJKFnGfaROTKOt1lZbDiiX1kCixRv7Nw2Id_sDs)
• TonCenter returns only finalized transactions; status goes directly to confirmed
• Lag is reported in seconds, not blocks
• Known scaling limitation: O(pending intents) API calls per scan cycle

---

7. Intent lifecycle

pending ──(tx seen)──► confirming ──(enough blocks)──► confirmed ──(webhook ok)──► [done]
   │                       │                               │
   │                       │ (deep reorg / TTL)            │ (all retries fail)
   └───────────────────────┴──────────► expired      webhook_failed

• Tron / TON skip confirming and jump directly to confirmed.
• webhook_failed intents are retried every WEBHOOK_RETRY_HOURS (default 6 h) and on POST /admin/webhooks/retry.
• Startup reconciliation: on startup, confirmed intents with webhook_delivered_at IS NULL and created in the last 7 days have their webhook re-delivered. This recovers from a crash between finalizeIntent and deliverWebhook.

---

8. Payment reference math (EVM)

paymentReference  = last8Bytes(keccak256(lower(intentId + salt + destination)))
topicRef (index)  = keccak256(paymentReferenceBytes)

The ERC20FeeProxy indexes paymentReference so Topics[1] in the log is topicRef, not the raw reference. The DB stores topic_ref pre-computed per intent so the scan loop is a single indexed SQL lookup instead of O(n) hashing.

---

9. Database schema (SQLite WAL)

Two tables:

intents — one row per payment intent

Column	Type	Notes
intent_id	TEXT PK	caller-supplied UUID
chain_id	INTEGER	numeric chain ID
chain_type	TEXT	evm / tron / ton
token_address	TEXT	EVM/Tron: lowercase 0x hex; TON: base64url
destination	TEXT	receiving address
amount	TEXT	base-10 wei / token smallest unit
payment_reference	TEXT	8-byte hex (EVM only)
topic_ref	TEXT	keccak256 of paymentReference (EVM index)
status	TEXT	pending / confirming / confirmed / expired / webhook_failed
callback_url	TEXT	backend webhook endpoint
callback_secret	TEXT	HMAC key (not returned in GET)
confirmations_required	INTEGER	from chain config or caller override
tx_hash	TEXT NULL	transaction hash once seen
log_index	INTEGER NULL	log position within tx (EVM)
block_number	INTEGER NULL	block / timestamp when seen
confirmations	INTEGER	current confirmation depth
salt	TEXT	32-byte random hex for reference derivation
webhook_delivered_at	TEXT NULL	RFC3339 timestamp of successful delivery
created_at / updated_at	DATETIME

Unique index on (tx_hash, log_index) prevents duplicate intent confirmation.

checkpoints — one row per chain, tracks scan progress

Column	Notes
chain_id	PK
last_scanned_block	block number (EVM), ms timestamp (Tron), unix seconds (TON)

---

10. Security model

• All non-health endpoints require Authorization: Bearer <SCANNER_API_KEY> (constant-time compare).
• If SCANNER_API_KEY is unset the server logs a warning and allows all requests — intended for local dev only.
• Webhooks are signed with HMAC-SHA256: X-AMN-Signature: hex(hmac(body, callbackSecret)).
• The callbackSecret is stored in the DB but excluded from all JSON responses (json:"-" tag).
• Request bodies are limited to 64 KB.