dceaf82934
Full-codebase-audit 2026-05-30 outputs: - Audit report: 09 - Audits/Full Codebase Audit - 2026-05-30.md - 81 issue files ISSUE-055..135 (decisions + 1 skipped no-brainer). - Scanner docs from scratch (was zero): architecture, data model, API ref, payment flow, operations runbook + repo README. - Doc-sync updates across API reference, data models, flows, design system. - Secret Rotation Runbook (08 - Operations) for the exposed credentials. - Reusable workflow guide (07 - Development) + .claude/workflows/full-codebase-audit.js. Issues remain status:open intentionally — the code fixes are uncommitted-then-committed working-tree changes per repo and aren't "resolved" until merged/deployed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
200 lines
8.7 KiB
Markdown
200 lines
8.7 KiB
Markdown
---
|
||
title: Scanner Architecture
|
||
tags: [architecture, scanner, payment]
|
||
created: 2026-05-30
|
||
---
|
||
|
||
# Scanner Architecture
|
||
|
||
AMN Pay Scanner is a standalone Go microservice that watches on-chain payment events and notifies the backend via webhook when a payment is confirmed. It replaces the Request Network integration with an in-house polling scanner that supports EVM chains, Tron, and TON.
|
||
|
||
> [!info]
|
||
> Repo: `scanner/` within the escrow monorepo. Binary: `scanner`. Written in Go 1.25. SQLite (WAL mode) for state. No external dependencies beyond the chain APIs.
|
||
|
||
---
|
||
|
||
## 1. Responsibilities
|
||
|
||
- Accept payment **intents** from the backend (POST /intents)
|
||
- Watch the relevant chain for matching on-chain transfers
|
||
- Track confirmation depth (EVM) or rely on finality from the chain API (Tron, TON)
|
||
- Deliver a signed webhook to the backend callback URL when confirmed
|
||
- Retry failed webhook deliveries
|
||
- Expire stale pending intents on a configurable TTL
|
||
|
||
---
|
||
|
||
## 2. Component map
|
||
|
||
```
|
||
┌─────────────────────────────────────────────────────────┐
|
||
│ scanner binary │
|
||
│ │
|
||
│ main.go │
|
||
│ ├── loadConfig() config.go │
|
||
│ ├── initDB() intent.go (SQLite schema) │
|
||
│ ├── startup reconcile intent.go │
|
||
│ ├── newServer() api.go │
|
||
│ │ └── startWorkers() api.go │
|
||
│ │ ├── ChainWorker chain.go (EVM) │
|
||
│ │ ├── TronChainWorker tron_chain.go (Tron) │
|
||
│ │ └── TonChainWorker ton_chain.go (TON) │
|
||
│ ├── HTTP routes api.go / main.go │
|
||
│ ├── intent TTL expiry main.go + intent.go │
|
||
│ └── webhook retry loop main.go + webhook.go │
|
||
│ │
|
||
│ reference.go — payment reference / topic hash math │
|
||
│ webhook.go — delivery, HMAC signing, retry │
|
||
└─────────────────────────────────────────────────────────┘
|
||
```
|
||
|
||
---
|
||
|
||
## 3. Chain worker model
|
||
|
||
All three chain types implement the `Worker` interface:
|
||
|
||
```go
|
||
type Worker interface {
|
||
start()
|
||
stop()
|
||
getHead(ctx context.Context) (int64, error)
|
||
}
|
||
```
|
||
|
||
One worker goroutine is spawned per chain marked `"verified": true` in `supported-chains.json`. Workers are selected by `chainType`:
|
||
|
||
| chainType | Worker struct | API used |
|
||
|---|---|---|
|
||
| `evm` (default) | `ChainWorker` | JSON-RPC 2.0 (`eth_getLogs`, `eth_blockNumber`) |
|
||
| `tron` | `TronChainWorker` | TronGrid REST (`/v1/contracts/{contract}/events`) |
|
||
| `ton` | `TonChainWorker` | TonCenter v3 REST (`/jetton/transfers`) |
|
||
|
||
Workers poll on `POLL_INTERVAL_SEC` (default 15 s). On first run, each worker starts scanning from the current chain head minus a small buffer (10 blocks for EVM, 24 h for Tron/TON).
|
||
|
||
---
|
||
|
||
## 4. EVM scanning detail
|
||
|
||
```
|
||
for each tick:
|
||
head = eth_blockNumber
|
||
from = max(checkpoint − ReorgBuffer(), 0)
|
||
chunks = split [from..head] into 2000-block ranges
|
||
for each chunk:
|
||
logs = eth_getLogs(proxyAddress, EventTopic, from, to)
|
||
for each log:
|
||
topicRef = Topics[1] (keccak256 of paymentReference — pre-indexed)
|
||
intent = DB lookup by topicRef WHERE status='pending'
|
||
validate(log.Data, intent) ← token + destination + amount check
|
||
confirmIntentPending() ← status → 'confirming'
|
||
saveCheckpoint(to)
|
||
checkConfirmations():
|
||
for each confirming intent:
|
||
confs = head - blockNumber + 1
|
||
if confs >= required: finalizeIntent() + deliverWebhook()
|
||
```
|
||
|
||
**Reorg protection**: `ReorgBuffer()` re-scans `3 × confirmationThreshold` blocks before the checkpoint (clamped 20–500). This catches any log that appeared in a block that was later reorganised off the canonical chain.
|
||
|
||
**Event signature**: `TransferWithReferenceAndFee` keccak256 = `0xbc8b749850bdc0c5e4a49d50f87ec40dca2acdb70a84e7c6823ccdc7b3b3d4b3`
|
||
|
||
---
|
||
|
||
## 5. Tron scanning detail
|
||
|
||
TronGrid does not expose a fee-proxy contract. Each intent is assigned a unique HD-derived destination address. The scanner watches TRC20 `Transfer` events on the USDT contract and matches by `to` address.
|
||
|
||
- Checkpoint: block timestamp in milliseconds (`last_scanned_block` column)
|
||
- TronGrid addresses arrive as `41xxxx` hex (21 bytes); normalized to `0x` (20 bytes EVM style)
|
||
- Tron transactions reported by TronGrid are already confirmed; status goes directly to `confirmed` (no multi-block wait)
|
||
- Pagination follows `meta.links.next` until empty
|
||
|
||
---
|
||
|
||
## 6. TON scanning detail
|
||
|
||
TON uses TonCenter v3. Per-intent polling: for each pending TON intent, a separate HTTP call fetches incoming Jetton transfers to that destination since the checkpoint.
|
||
|
||
- Checkpoint: Unix timestamp in seconds
|
||
- TON addresses are base64url (`EQ…`/`UQ…`) — case-sensitive, never lowercased
|
||
- `proxyAddress` = USDT Jetton master address (`EQCxE6mUtQJKFnGfaROTKOt1lZbDiiX1kCixRv7Nw2Id_sDs`)
|
||
- TonCenter returns only finalized transactions; status goes directly to `confirmed`
|
||
- Lag is reported in seconds, not blocks
|
||
- Known scaling limitation: O(pending intents) API calls per scan cycle
|
||
|
||
---
|
||
|
||
## 7. Intent lifecycle
|
||
|
||
```
|
||
pending ──(tx seen)──► confirming ──(enough blocks)──► confirmed ──(webhook ok)──► [done]
|
||
│ │ │
|
||
│ │ (deep reorg / TTL) │ (all retries fail)
|
||
└───────────────────────┴──────────► expired webhook_failed
|
||
```
|
||
|
||
- **Tron / TON** skip `confirming` and jump directly to `confirmed`.
|
||
- `webhook_failed` intents are retried every `WEBHOOK_RETRY_HOURS` (default 6 h) and on `POST /admin/webhooks/retry`.
|
||
- **Startup reconciliation**: on startup, `confirmed` intents with `webhook_delivered_at IS NULL` and created in the last 7 days have their webhook re-delivered. This recovers from a crash between `finalizeIntent` and `deliverWebhook`.
|
||
|
||
---
|
||
|
||
## 8. Payment reference math (EVM)
|
||
|
||
```
|
||
paymentReference = last8Bytes(keccak256(lower(intentId + salt + destination)))
|
||
topicRef (index) = keccak256(paymentReferenceBytes)
|
||
```
|
||
|
||
The ERC20FeeProxy indexes `paymentReference` so `Topics[1]` in the log is `topicRef`, not the raw reference. The DB stores `topic_ref` pre-computed per intent so the scan loop is a single indexed SQL lookup instead of O(n) hashing.
|
||
|
||
---
|
||
|
||
## 9. Database schema (SQLite WAL)
|
||
|
||
Two tables:
|
||
|
||
**`intents`** — one row per payment intent
|
||
|
||
| Column | Type | Notes |
|
||
|---|---|---|
|
||
| `intent_id` | TEXT PK | caller-supplied UUID |
|
||
| `chain_id` | INTEGER | numeric chain ID |
|
||
| `chain_type` | TEXT | `evm` / `tron` / `ton` |
|
||
| `token_address` | TEXT | EVM/Tron: lowercase 0x hex; TON: base64url |
|
||
| `destination` | TEXT | receiving address |
|
||
| `amount` | TEXT | base-10 wei / token smallest unit |
|
||
| `payment_reference` | TEXT | 8-byte hex (EVM only) |
|
||
| `topic_ref` | TEXT | keccak256 of paymentReference (EVM index) |
|
||
| `status` | TEXT | `pending` / `confirming` / `confirmed` / `expired` / `webhook_failed` |
|
||
| `callback_url` | TEXT | backend webhook endpoint |
|
||
| `callback_secret` | TEXT | HMAC key (not returned in GET) |
|
||
| `confirmations_required` | INTEGER | from chain config or caller override |
|
||
| `tx_hash` | TEXT NULL | transaction hash once seen |
|
||
| `log_index` | INTEGER NULL | log position within tx (EVM) |
|
||
| `block_number` | INTEGER NULL | block / timestamp when seen |
|
||
| `confirmations` | INTEGER | current confirmation depth |
|
||
| `salt` | TEXT | 32-byte random hex for reference derivation |
|
||
| `webhook_delivered_at` | TEXT NULL | RFC3339 timestamp of successful delivery |
|
||
| `created_at` / `updated_at` | DATETIME | |
|
||
|
||
Unique index on `(tx_hash, log_index)` prevents duplicate intent confirmation.
|
||
|
||
**`checkpoints`** — one row per chain, tracks scan progress
|
||
|
||
| Column | Notes |
|
||
|---|---|
|
||
| `chain_id` | PK |
|
||
| `last_scanned_block` | block number (EVM), ms timestamp (Tron), unix seconds (TON) |
|
||
|
||
---
|
||
|
||
## 10. Security model
|
||
|
||
- All non-health endpoints require `Authorization: Bearer <SCANNER_API_KEY>` (constant-time compare).
|
||
- If `SCANNER_API_KEY` is unset the server logs a warning and allows all requests — intended for local dev only.
|
||
- Webhooks are signed with HMAC-SHA256: `X-AMN-Signature: hex(hmac(body, callbackSecret))`.
|
||
- The `callbackSecret` is stored in the DB but excluded from all JSON responses (`json:"-"` tag).
|
||
- Request bodies are limited to 64 KB.
|