audit: 2026-05-30 full-codebase audit — report, issues, docs, runbooks
Full-codebase-audit 2026-05-30 outputs: - Audit report: 09 - Audits/Full Codebase Audit - 2026-05-30.md - 81 issue files ISSUE-055..135 (decisions + 1 skipped no-brainer). - Scanner docs from scratch (was zero): architecture, data model, API ref, payment flow, operations runbook + repo README. - Doc-sync updates across API reference, data models, flows, design system. - Secret Rotation Runbook (08 - Operations) for the exposed credentials. - Reusable workflow guide (07 - Development) + .claude/workflows/full-codebase-audit.js. Issues remain status:open intentionally — the code fixes are uncommitted-then-committed working-tree changes per repo and aren't "resolved" until merged/deployed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Siavash Sameni
199
01 - Architecture/Scanner Architecture.md
Normal file
199
01 - Architecture/Scanner Architecture.md
Normal file
@@ -0,0 +1,199 @@
|
||||
---
|
||||
title: Scanner Architecture
|
||||
tags: [architecture, scanner, payment]
|
||||
created: 2026-05-30
|
||||
---
|
||||
|
||||
# Scanner Architecture
|
||||
|
||||
AMN Pay Scanner is a standalone Go microservice that watches on-chain payment events and notifies the backend via webhook when a payment is confirmed. It replaces the Request Network integration with an in-house polling scanner that supports EVM chains, Tron, and TON.
|
||||
|
||||
> [!info]
|
||||
> Repo: `scanner/` within the escrow monorepo. Binary: `scanner`. Written in Go 1.25. SQLite (WAL mode) for state. No external dependencies beyond the chain APIs.
|
||||
|
||||
---
|
||||
|
||||
## 1. Responsibilities
|
||||
|
||||
- Accept payment **intents** from the backend (POST /intents)
|
||||
- Watch the relevant chain for matching on-chain transfers
|
||||
- Track confirmation depth (EVM) or rely on finality from the chain API (Tron, TON)
|
||||
- Deliver a signed webhook to the backend callback URL when confirmed
|
||||
- Retry failed webhook deliveries
|
||||
- Expire stale pending intents on a configurable TTL
|
||||
|
||||
---
|
||||
|
||||
## 2. Component map
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────┐
|
||||
│ scanner binary │
|
||||
│ │
|
||||
│ main.go │
|
||||
│ ├── loadConfig() config.go │
|
||||
│ ├── initDB() intent.go (SQLite schema) │
|
||||
│ ├── startup reconcile intent.go │
|
||||
│ ├── newServer() api.go │
|
||||
│ │ └── startWorkers() api.go │
|
||||
│ │ ├── ChainWorker chain.go (EVM) │
|
||||
│ │ ├── TronChainWorker tron_chain.go (Tron) │
|
||||
│ │ └── TonChainWorker ton_chain.go (TON) │
|
||||
│ ├── HTTP routes api.go / main.go │
|
||||
│ ├── intent TTL expiry main.go + intent.go │
|
||||
│ └── webhook retry loop main.go + webhook.go │
|
||||
│ │
|
||||
│ reference.go — payment reference / topic hash math │
|
||||
│ webhook.go — delivery, HMAC signing, retry │
|
||||
└─────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Chain worker model
|
||||
|
||||
All three chain types implement the `Worker` interface:
|
||||
|
||||
```go
|
||||
type Worker interface {
|
||||
start()
|
||||
stop()
|
||||
getHead(ctx context.Context) (int64, error)
|
||||
}
|
||||
```
|
||||
|
||||
One worker goroutine is spawned per chain marked `"verified": true` in `supported-chains.json`. Workers are selected by `chainType`:
|
||||
|
||||
| chainType | Worker struct | API used |
|
||||
|---|---|---|
|
||||
| `evm` (default) | `ChainWorker` | JSON-RPC 2.0 (`eth_getLogs`, `eth_blockNumber`) |
|
||||
| `tron` | `TronChainWorker` | TronGrid REST (`/v1/contracts/{contract}/events`) |
|
||||
| `ton` | `TonChainWorker` | TonCenter v3 REST (`/jetton/transfers`) |
|
||||
|
||||
Workers poll on `POLL_INTERVAL_SEC` (default 15 s). On first run, each worker starts scanning from the current chain head minus a small buffer (10 blocks for EVM, 24 h for Tron/TON).
|
||||
|
||||
---
|
||||
|
||||
## 4. EVM scanning detail
|
||||
|
||||
```
|
||||
for each tick:
|
||||
head = eth_blockNumber
|
||||
from = max(checkpoint − ReorgBuffer(), 0)
|
||||
chunks = split [from..head] into 2000-block ranges
|
||||
for each chunk:
|
||||
logs = eth_getLogs(proxyAddress, EventTopic, from, to)
|
||||
for each log:
|
||||
topicRef = Topics[1] (keccak256 of paymentReference — pre-indexed)
|
||||
intent = DB lookup by topicRef WHERE status='pending'
|
||||
validate(log.Data, intent) ← token + destination + amount check
|
||||
confirmIntentPending() ← status → 'confirming'
|
||||
saveCheckpoint(to)
|
||||
checkConfirmations():
|
||||
for each confirming intent:
|
||||
confs = head - blockNumber + 1
|
||||
if confs >= required: finalizeIntent() + deliverWebhook()
|
||||
```
|
||||
|
||||
**Reorg protection**: `ReorgBuffer()` re-scans `3 × confirmationThreshold` blocks before the checkpoint (clamped 20–500). This catches any log that appeared in a block that was later reorganised off the canonical chain.
|
||||
|
||||
**Event signature**: `TransferWithReferenceAndFee` keccak256 = `0xbc8b749850bdc0c5e4a49d50f87ec40dca2acdb70a84e7c6823ccdc7b3b3d4b3`
|
||||
|
||||
---
|
||||
|
||||
## 5. Tron scanning detail
|
||||
|
||||
TronGrid does not expose a fee-proxy contract. Each intent is assigned a unique HD-derived destination address. The scanner watches TRC20 `Transfer` events on the USDT contract and matches by `to` address.
|
||||
|
||||
- Checkpoint: block timestamp in milliseconds (`last_scanned_block` column)
|
||||
- TronGrid addresses arrive as `41xxxx` hex (21 bytes); normalized to `0x` (20 bytes EVM style)
|
||||
- Tron transactions reported by TronGrid are already confirmed; status goes directly to `confirmed` (no multi-block wait)
|
||||
- Pagination follows `meta.links.next` until empty
|
||||
|
||||
---
|
||||
|
||||
## 6. TON scanning detail
|
||||
|
||||
TON uses TonCenter v3. Per-intent polling: for each pending TON intent, a separate HTTP call fetches incoming Jetton transfers to that destination since the checkpoint.
|
||||
|
||||
- Checkpoint: Unix timestamp in seconds
|
||||
- TON addresses are base64url (`EQ…`/`UQ…`) — case-sensitive, never lowercased
|
||||
- `proxyAddress` = USDT Jetton master address (`EQCxE6mUtQJKFnGfaROTKOt1lZbDiiX1kCixRv7Nw2Id_sDs`)
|
||||
- TonCenter returns only finalized transactions; status goes directly to `confirmed`
|
||||
- Lag is reported in seconds, not blocks
|
||||
- Known scaling limitation: O(pending intents) API calls per scan cycle
|
||||
|
||||
---
|
||||
|
||||
## 7. Intent lifecycle
|
||||
|
||||
```
|
||||
pending ──(tx seen)──► confirming ──(enough blocks)──► confirmed ──(webhook ok)──► [done]
|
||||
│ │ │
|
||||
│ │ (deep reorg / TTL) │ (all retries fail)
|
||||
└───────────────────────┴──────────► expired webhook_failed
|
||||
```
|
||||
|
||||
- **Tron / TON** skip `confirming` and jump directly to `confirmed`.
|
||||
- `webhook_failed` intents are retried every `WEBHOOK_RETRY_HOURS` (default 6 h) and on `POST /admin/webhooks/retry`.
|
||||
- **Startup reconciliation**: on startup, `confirmed` intents with `webhook_delivered_at IS NULL` and created in the last 7 days have their webhook re-delivered. This recovers from a crash between `finalizeIntent` and `deliverWebhook`.
|
||||
|
||||
---
|
||||
|
||||
## 8. Payment reference math (EVM)
|
||||
|
||||
```
|
||||
paymentReference = last8Bytes(keccak256(lower(intentId + salt + destination)))
|
||||
topicRef (index) = keccak256(paymentReferenceBytes)
|
||||
```
|
||||
|
||||
The ERC20FeeProxy indexes `paymentReference` so `Topics[1]` in the log is `topicRef`, not the raw reference. The DB stores `topic_ref` pre-computed per intent so the scan loop is a single indexed SQL lookup instead of O(n) hashing.
|
||||
|
||||
---
|
||||
|
||||
## 9. Database schema (SQLite WAL)
|
||||
|
||||
Two tables:
|
||||
|
||||
**`intents`** — one row per payment intent
|
||||
|
||||
| Column | Type | Notes |
|
||||
|---|---|---|
|
||||
| `intent_id` | TEXT PK | caller-supplied UUID |
|
||||
| `chain_id` | INTEGER | numeric chain ID |
|
||||
| `chain_type` | TEXT | `evm` / `tron` / `ton` |
|
||||
| `token_address` | TEXT | EVM/Tron: lowercase 0x hex; TON: base64url |
|
||||
| `destination` | TEXT | receiving address |
|
||||
| `amount` | TEXT | base-10 wei / token smallest unit |
|
||||
| `payment_reference` | TEXT | 8-byte hex (EVM only) |
|
||||
| `topic_ref` | TEXT | keccak256 of paymentReference (EVM index) |
|
||||
| `status` | TEXT | `pending` / `confirming` / `confirmed` / `expired` / `webhook_failed` |
|
||||
| `callback_url` | TEXT | backend webhook endpoint |
|
||||
| `callback_secret` | TEXT | HMAC key (not returned in GET) |
|
||||
| `confirmations_required` | INTEGER | from chain config or caller override |
|
||||
| `tx_hash` | TEXT NULL | transaction hash once seen |
|
||||
| `log_index` | INTEGER NULL | log position within tx (EVM) |
|
||||
| `block_number` | INTEGER NULL | block / timestamp when seen |
|
||||
| `confirmations` | INTEGER | current confirmation depth |
|
||||
| `salt` | TEXT | 32-byte random hex for reference derivation |
|
||||
| `webhook_delivered_at` | TEXT NULL | RFC3339 timestamp of successful delivery |
|
||||
| `created_at` / `updated_at` | DATETIME | |
|
||||
|
||||
Unique index on `(tx_hash, log_index)` prevents duplicate intent confirmation.
|
||||
|
||||
**`checkpoints`** — one row per chain, tracks scan progress
|
||||
|
||||
| Column | Notes |
|
||||
|---|---|
|
||||
| `chain_id` | PK |
|
||||
| `last_scanned_block` | block number (EVM), ms timestamp (Tron), unix seconds (TON) |
|
||||
|
||||
---
|
||||
|
||||
## 10. Security model
|
||||
|
||||
- All non-health endpoints require `Authorization: Bearer <SCANNER_API_KEY>` (constant-time compare).
|
||||
- If `SCANNER_API_KEY` is unset the server logs a warning and allows all requests — intended for local dev only.
|
||||
- Webhooks are signed with HMAC-SHA256: `X-AMN-Signature: hex(hmac(body, callbackSecret))`.
|
||||
- The `callbackSecret` is stored in the DB but excluded from all JSON responses (`json:"-"` tag).
|
||||
- Request bodies are limited to 64 KB.
|
||||
Reference in New Issue
Block a user