nick-doc/10 - Services/scanner.md

title, tags, version, created

title	tags	version	created
AMN Pay Scanner	service scanner payment go blockchain	0.1.10	2026-06-08

AMN Pay Scanner

[!info] Version: 0.1.10 — Go 1.25 — Status: production (BSC + Ethereum + BSC Testnet), other chains staged Repo: scanner/ within the escrow monorepo. Cross-ref: Scanner Architecture | Scanner API

---

1. Overview

AMN Pay Scanner is a standalone Go microservice that watches on-chain payment events and notifies the backend via signed webhook when a payment is confirmed.

What it replaces

Previously, the AMN escrow platform relied on Request Network as the payment infrastructure layer. Request Network introduced:

• An external smart-contract dependency (ERC20FeeProxy) on RN's deployment schedule
• A closed fee-proxy address registry that differs per chain and is not reliably canonical (see memory note on RN proxy addresses)
• A separate webhook/event pipeline managed by RN's infrastructure
• A hard coupling between the backend and RN's SDK

AMN Pay Scanner removes all of these by:

1. Deploying the same ERC20FeeProxy contract under our own control
2. Polling RPC endpoints directly (no RN nodes)
3. Deriving payment references in-house using the same keccak256 formula as the proxy contract
4. Delivering webhooks signed with a backend-controlled HMAC secret

The scanner also supports direct-address payment rails (Tron, TON, and manual EVM flows) where no proxy contract is involved at all.

---

2. How It Works

Step-by-step flow

Backend                     Scanner                        Chain
  │                            │                             │
  │  POST /intents             │                             │
  │  {chainId, token, amount,  │                             │
  │   destination, callbackUrl}│                             │
  ├──────────────────────────► │                             │
  │                            │ persist intent (SQLite)     │
  │  {intentId,                │ derive paymentReference     │
  │   paymentReference,        │ compute topicRef (EVM)      │
  │   checkoutBlock}           │                             │
  ◄──────────────────────────── │                             │
  │                            │                             │
  │  (frontend builds tx using │                             │
  │   proxyAddress +           │                             │
  │   paymentReference)        │                             │
  │                            │    poll eth_getLogs         │
  │                            ├────────────────────────────►│
  │                            │    logs []                  │
  │                            ◄────────────────────────────┤
  │                            │ match Topics[1] → topicRef  │
  │                            │ validate token+amount+dest  │
  │                            │ status → confirming         │
  │                            │                             │
  │                            │  (wait confirmationThreshold│
  │                            │   blocks / finality signal) │
  │                            │                             │
  │  POST callbackUrl          │                             │
  │  {intentId, txHash,        │                             │
  │   status:"confirmed", ...} │                             │
  ◄──────────────────────────── │                             │
  │  200 OK                    │                             │
  ├──────────────────────────► │                             │
  │                            │ status → confirmed          │
  │                            │ record webhookDeliveredAt   │

Intent status lifecycle

pending ──(tx seen)──► confirming ──(depth reached)──► confirmed ──(webhook ok)──► [done]
   │                        │                               │
   │                        │ (deep reorg / TTL)            │ (all retries fail)
   └────────────────────────┴──────────────► expired   webhook_failed

• Tron / TON skip confirming — their API only returns finalized events, so the status jumps directly to confirmed.
• Startup reconciliation: on startup, any confirmed intent with webhook_delivered_at IS NULL created in the last 7 days has its webhook re-delivered. This covers crashes between finalization and delivery.
• webhook_failed intents are retried on WEBHOOK_RETRY_HOURS schedule (default 6 h) and immediately via POST /admin/webhooks/retry.

---

3. Supported Chains

Chains marked verified: false in supported-chains.json do NOT start a worker goroutine at runtime. Override with SCANNER_ENABLED_CHAINS env var to force-enable specific chain IDs without a code change.

Chain	Chain ID	Type	Proxy Address	Confirmation Depth	Verified
BNB Smart Chain	56	EVM	0x0DfbEe143b42B41eFC5A6F87bFD1fFC78c2f0aC9	200 blocks	yes
Ethereum Mainnet	1	EVM	0x370DE27fdb7D1Ff1e1BaA7D11c5820a324Cf623C	50 blocks	yes
BSC Testnet	97	EVM	0x0DfbEe143b42B41eFC5A6F87bFD1fFC78c2f0aC9	5 blocks	yes (testnet)
Arbitrum One	42161	EVM	0x0DfbEe143b42B41eFC5A6F87bFD1fFC78c2f0aC9	2400 blocks	no
Polygon	137	EVM	0x0DfbEe143b42B41eFC5A6F87bFD1fFC78c2f0aC9	300 blocks	no
Base	8453	EVM	0x1892196E80C4c17ea5100Da765Ab48c1fE2Fb814	300 blocks	no
Tron Mainnet	728126428	Tron	TRC20 USDT contract (TR7NHqjeKQxGTCi8q8ZY4pL8otSzgjLj6t)	TronGrid confirmed (~200 reported)	no
TON Mainnet	1100	TON	USDT Jetton master (EQCxE6mUtQJKFnGfaROTKOt1lZbDiiX1kCixRv7Nw2Id_sDs)	TonCenter finalized (~120 reported)	no

[!warning] Chain notes

• Ethereum: uses the older v0.1.0 proxy ABI at 0x370DE27…. A v0.2.0-next proxy is also deployed at 0x0DfbEe… on ETH but the scanner checkout uses the v0.1.0 ABI — do not swap addresses silently.
• Base: proxy address is non-canonical (differs from the CREATE2 expected address per RN smart-contracts artifact v0.2.0). See memory note on RN proxy addresses.
• Tron: no fee-proxy contract exists. Matching is by unique destination address, not payment reference.
• TON: lag is reported in seconds (not blocks); per-intent polling is O(pending intents) API calls per cycle — known scaling concern.

---

4. Architecture Diagram

┌──────────────────────────────────────────────────────────────────┐
│                       scanner binary                             │
│                                                                  │
│  ┌─────────────┐   ┌──────────────────────────────────────────┐ │
│  │  HTTP API   │   │              Worker Pool                  │ │
│  │  (api.go)   │   │                                          │ │
│  │             │   │  ┌──────────────┐  eth_getLogs / eth_    │ │
│  │ POST /intents│  │  │ ChainWorker  │─► blockNumber (JSON-RPC│ │
│  │ GET  /intents│  │  │ (EVM×N)      │  per chain)            │ │
│  │ /balances   │   │  └──────────────┘                        │ │
│  │ /balance-   │   │  ┌──────────────┐  TronGrid REST API     │ │
│  │  watches    │   │  │ TronChain-   │─► /v1/contracts/events │ │
│  │ /scanner/   │   │  │ Worker       │                         │ │
│  │  status     │   │  └──────────────┘                        │ │
│  │ /admin/     │   │  ┌──────────────┐  TonCenter v3 REST     │ │
│  │  webhooks/  │   │  │ TonChain-    │─► /jetton/transfers     │ │
│  │  retry      │   │  │ Worker       │                         │ │
│  └──────┬──────┘   │  └──────────────┘                        │ │
│         │          └─────────────┬────────────────────────────┘ │
│         │                        │ match / confirm               │
│         ▼                        ▼                               │
│  ┌──────────────────────────────────────────────────────────┐   │
│  │                      SQLite (WAL)                        │   │
│  │  intents · checkpoints · balance_watches                 │   │
│  └──────────────────────────────────────────────────────────┘   │
│         │                        │                               │
│         ▼                        ▼                               │
│  ┌────────────────┐   ┌─────────────────────────────────────┐   │
│  │BalanceWatch-   │   │  webhook.go                         │   │
│  │Scheduler       │   │  HMAC-SHA256 sign → POST callbackUrl│   │
│  │(balance.go)    │   │  retry: 5s→30s→2m→10m→1h→failed    │   │
│  └────────────────┘   └─────────────────────────────────────┘   │
│                                                                  │
│  Background loops (main.go):                                     │
│    • intent TTL expiry (INTENT_TTL_HOURS)                        │
│    • webhook retry loop (WEBHOOK_RETRY_HOURS)                    │
│    • startup reconciliation (confirmed, no delivery)             │
└──────────────────────────────────────────────────────────────────┘

---

5. API Routes

All endpoints except /health require Authorization: Bearer <SCANNER_API_KEY>. Request bodies are capped at 64 KB.

Method	Path	Auth	Purpose
GET	/health	none	Liveness probe — returns {"status":"ok"}
GET	/scanner/status	Bearer	Chain lag, last scanned block, pending intent counts per chain
POST	/intents	Bearer	Register a payment intent; returns intentId, paymentReference, checkoutBlock
GET	/intents/{id}	Bearer	Fetch full intent record including current status and tx details
DELETE	/intents/{id}	Bearer	Cancel a pending intent (sets status to expired)
POST	/balances/check	Bearer	Read current ERC-20 balance for an address+token on a given chain
POST	/balance-watches	Bearer	Start a balance-change watch on an EVM address/token
GET	/balance-watches/{id}	Bearer	Fetch balance-watch status and current balance
DELETE	/balance-watches/{id}	Bearer	Stop a balance watch
POST	/admin/webhooks/retry	Bearer	Force immediate retry of all webhook_failed intents

Full request/response schemas: Scanner API

---

6. Payment Reference Derivation (EVM)

The ERC20FeeProxy contract indexes payments by a bytes8 reference. The scanner derives it deterministically so the worker's scan loop only needs a single indexed DB lookup per log.

# Step 1: build raw reference
input            = lower(intentId) + lower(salt) + lower(destination)
paymentReference = last8Bytes(keccak256(input))     ← bytes8, 16 hex chars

# Step 2: build EVM log index key
topicRef         = keccak256(paymentReferenceBytes)  ← bytes32, 64 hex chars
                   ↑ this is Topics[1] in the emitted log

• salt is a 32-byte random hex string generated at intent creation time.
• destination is the EVM address of the AMN treasury / seller wallet, lowercased.
• Both paymentReference and topicRef are stored in the intents table at creation time. The scan loop performs SELECT * FROM intents WHERE topic_ref = ? AND status = 'pending' — O(1) with index, regardless of how many pending intents exist.

Event signature (used as Topics[0] filter):

TransferWithReferenceAndFee → keccak256 = 0xbc8b749850bdc0c5e4a49d50f87ec40dca2acdb70a84e7c6823ccdc7b3b3d4b3

---

7. EVM / Tron / TON Matching Logic

EVM

1. Worker fetches eth_getLogs for the proxy contract address + TransferWithReferenceAndFee event topic in 2000-block chunks.
2. For each log, extract Topics[1] (= topicRef).
3. Query DB: WHERE topic_ref = ? AND status = 'pending'.
4. On match: decode log.Data to extract tokenAddress, amount, destination, feeAmount. Validate all four against the intent record.
5. Update status to confirming, record txHash, blockNumber, logIndex.
6. On next poll: check head - blockNumber + 1 >= confirmationsRequired. When met, finalize and deliver webhook.

Reorg protection: the checkpoint is rewound by 3 × confirmationThreshold blocks (clamped to 20–500) on each poll. Any log from a recently reorganized block will be re-fetched and re-matched.

Tron

• No proxy contract — each intent receives a unique HD-derived destination address.
• Worker polls TronGrid /v1/contracts/{usdtContract}/events?event_name=Transfer filtered to the intent's destination address.
• Match criterion: to == destination AND amount >= intent.Amount.
• TronGrid returns only already-confirmed transactions. No multi-block wait — status jumps directly to confirmed.
• Addresses from TronGrid arrive as 41xxxx (21-byte hex). The worker normalizes these to 0x-prefixed 20-byte EVM style for storage and comparison.
• Checkpoint stored as a millisecond Unix timestamp in last_scanned_block.
• Pagination follows meta.links.next until nil.

TON

• Also uses per-intent unique destination addresses (no proxy contract).
• Worker calls TonCenter v3 /jetton/transfers?account={destination}&jetton_master={usdtJetton}&start_utime={checkpoint} for each pending TON intent.
• Match criterion: destination == intent.Destination AND amount >= intent.Amount.
• TonCenter returns only finalized transactions — status jumps directly to confirmed.
• TON addresses are base64url (EQ…/UQ…), case-sensitive. Never lowercased.
• Checkpoint stored as Unix seconds.
• Lag is reported in seconds, not blocks.
• Scaling note: each pending TON intent triggers one HTTP round-trip per scan cycle. High concurrency will exhaust TonCenter rate limits.

---

8. Webhook Payload

Payment confirmed (intent webhook)

Posted to callbackUrl on intent confirmation:

{
  "intentId": "018f1a2b-3c4d-7e8f-9a0b-c1d2e3f4a5b6",
  "paymentReference": "0xa1b2c3d4e5f60718",
  "txHash": "0x4a3b2c1d...",
  "blockNumber": 39000010,
  "confirmations": 200,
  "amount": "10000000000000000000",
  "token": "0x55d398326f99059fF775485246999027B3197955",
  "chainId": 56,
  "status": "confirmed"
}

Header: X-AMN-Signature: hex(HMAC-SHA256(rawBody, callbackSecret))

The confirmations value is capped at the chain's acceptance threshold once confirmed. The scanner does not continue incrementing after the payment is safe to credit.

Retry schedule on delivery failure: 5 s → 30 s → 2 min → 10 min → 1 h → webhook_failed

After exhausting retries the intent is set to webhook_failed. Manual recovery: POST /admin/webhooks/retry or wait for the WEBHOOK_RETRY_HOURS background sweep.

Balance changed (balance-watch webhook)

Posted to the watch's callbackUrl when balance delta is detected:

{
  "eventType": "balance_changed",
  "watchId": "payment-123-c56-USDT",
  "chainId": 56,
  "chainType": "evm",
  "address": "0xabc...",
  "tokenAddress": "0x55d398326f99059fF775485246999027B3197955",
  "tokenSymbol": "USDT",
  "decimals": 18,
  "previousBalance": "0",
  "currentBalance": "10000000000000000000",
  "delta": "10000000000000000000",
  "changeCount": 1,
  "checkedAt": "2026-06-08T12:00:00Z",
  "status": "balance_changed"
}

Additional headers: X-AMN-Delivery-ID: <watchId>, X-AMN-Event-Type: balance_changed

The scanner only advances current_balance after a successful (2xx) delivery, so a down backend will retry on the next scheduled check.

---

9. SQLite DB Schema

Database at DB_PATH (default ./scanner.db; Docker: /data/scanner.db). WAL mode enabled, busy timeout 5 000 ms.

intents

Column	Type	Notes
intent_id	TEXT PK	caller-supplied UUID
chain_id	INTEGER	numeric chain ID
chain_type	TEXT	evm / tron / ton
token_address	TEXT	EVM/Tron: lowercase 0x hex; TON: base64url
destination	TEXT	receiving address
amount	TEXT	base-10 wei / token smallest unit
payment_reference	TEXT	8-byte hex — EVM only
topic_ref	TEXT	keccak256 of paymentReference — scan index for EVM
status	TEXT	pending / confirming / confirmed / expired / webhook_failed
callback_url	TEXT	backend webhook endpoint
callback_secret	TEXT	HMAC key — excluded from all JSON responses
confirmations_required	INTEGER	floored at the chain acceptance threshold
tx_hash	TEXT NULL	set once the transaction is seen on-chain
log_index	INTEGER NULL	log position within tx (EVM only)
block_number	INTEGER NULL	block when seen (EVM); ms timestamp (Tron); unix s (TON)
confirmations	INTEGER	depth while confirming; capped at threshold after confirmation
salt	TEXT	32-byte random hex used in reference derivation
webhook_delivered_at	TEXT NULL	RFC3339 timestamp of first successful delivery
created_at / updated_at	DATETIME	UTC

Unique index on (tx_hash, log_index) prevents double-confirmation.

checkpoints

Column	Notes
chain_id PK
last_scanned_block	block number (EVM), ms timestamp (Tron), unix seconds (TON)

balance_watches

Column	Type	Notes
watch_id	TEXT PK	caller-supplied idempotency key
chain_id / chain_type	INTEGER / TEXT	currently EVM only
token_address / token_symbol	TEXT	ERC-20 contract + optional registry symbol
decimals	INTEGER	registry decimals for display
address	TEXT	watched holder address
baseline_balance	TEXT	base-unit balance at watch creation
current_balance	TEXT	last successfully delivered balance
status	TEXT	watching / stopped / expired
callback_url / callback_secret	TEXT	signed webhook destination
last_checked_at / next_check_at	DATETIME	scheduler state
change_count / last_notified_at	INTEGER / DATETIME	notification audit
expires_at	DATETIME	hard stop after 7 days
created_at / updated_at	DATETIME	UTC

Indexes: (status, next_check_at) for due-scan queries; (chain_id, status) for status reporting.

---

10. Configuration

All configuration via environment variables. Copy .env.example and populate before first run.

Variable	Default	Required	Notes
PORT	8080	no	HTTP listen port
DB_PATH	./scanner.db	no	SQLite file path. Docker: mount /data and set /data/scanner.db
CHAINS_JSON_PATH	./supported-chains.json	no	Chain registry file
TOKENS_JSON_PATH	./tokens.json	no	Token registry for symbol/decimals metadata
SCANNER_API_KEY	(none)	yes (prod)	Shared secret for Bearer auth. Generate: openssl rand -hex 32
POLL_INTERVAL_SEC	15	no	Chain polling interval in seconds
INTENT_TTL_HOURS	24	no	Expire pending intents after N hours. 0 = disabled
WEBHOOK_RETRY_HOURS	6	no	Background re-delivery interval for webhook_failed intents. 0 = disabled
BALANCE_WATCH_TICK_SEC	60	no	How often the scheduler checks for due balance watches
BALANCE_WATCH_BATCH_SIZE	50	no	Max due watches processed per tick
RPC_BSC	chain config	no	Override BSC JSON-RPC URL
RPC_ARB	chain config	no	Override Arbitrum JSON-RPC URL
RPC_ETH	chain config	no	Override Ethereum JSON-RPC URL
RPC_POLYGON	chain config	no	Override Polygon JSON-RPC URL
RPC_BASE	chain config	no	Override Base JSON-RPC URL
TRONGRID_API_KEY	(none)	recommended	Free tier is very low; required for any real Tron traffic
TONCENTER_API_KEY	(none)	recommended	Rate-limited without key
SCANNER_ENABLED_CHAINS	JSON verified flags	no	Comma-separated chain IDs to enable, overriding verified. E.g. 56,1
SCANNER_CALLBACK_ALLOWED_HOSTS	(none)	prod recommended	Comma-separated hosts for SSRF guard on callbackUrl targets

---

11. Docker Deployment

# Build
docker build -t amn-scanner .

# Run
docker run -d \
  --name amn-scanner \
  --network shared-web \
  -p 8080:8080 \
  -v /opt/arcane/data/projects/escrow-dev/scanner-data:/data \
  --env-file .env \
  -e DB_PATH=/data/scanner.db \
  amn-scanner

On the dev server (89.58.32.32): the scanner is part of the escrow-dev Arcane project. Images are built locally from source at /tmp/escrow-backend-build/ — the dev stack does not pull from any registry.

# Copy changed scanner source files
scp -i ~/CascadeProjects/wzp scanner/*.go root@89.58.32.32:/tmp/escrow-backend-build/scanner/

# Rebuild + restart (on server)
ssh -i ~/CascadeProjects/wzp root@89.58.32.32 \
  "cd /tmp/escrow-backend-build/scanner && docker build -t amn-scanner-local:dev . && \
   cd /opt/arcane/data/projects/escrow-dev && docker compose up -d scanner"

Health check URL (via infra-caddy): check project Caddyfile for the current vhost. Direct internal: http://amn-scanner:8080/health.

---

12. Integration with the Backend

Registering a payment intent

// backend: src/services/amnScanner/...
const resp = await fetch(`${SCANNER_URL}/intents`, {
  method: 'POST',
  headers: {
    'Authorization': `Bearer ${SCANNER_API_KEY}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    intentId: payment._id.toString(),   // MongoDB ObjectId string
    chainId: 56,
    tokenAddress: '0x55d398326f99059fF775485246999027B3197955',  // USDT BSC
    destination: sellerWalletAddress,
    amount: amountInWei,                // base-10 string
    callbackUrl: `${BACKEND_URL}/api/payment/amn-scanner/webhook`,
    callbackSecret: process.env.SCANNER_CALLBACK_SECRET,
  }),
});
const { intentId, paymentReference, checkoutBlock } = await resp.json();
// store intentId + checkoutBlock in the payment record
// pass checkoutBlock to the frontend for transaction construction

The checkoutBlock contains everything the frontend needs to call the ERC20FeeProxy.transferWithReferenceAndFee() function:

{
  "destination": "0x...",
  "tokenAddress": "0x55d...",
  "tokenSymbol": "USDT",
  "decimals": 18,
  "chainId": 56,
  "proxyAddress": "0x0DfbEe143b42B41eFC5A6F87bFD1fFC78c2f0aC9",
  "paymentReference": "0xa1b2c3d4e5f60718",
  "feeAmount": "0",
  "feeAddress": "0x0000000000000000000000000000000000000000",
  "amountWei": "10000000000000000000"
}

Receiving the webhook callback

// POST /api/payment/amn-scanner/webhook
app.post('/api/payment/amn-scanner/webhook', async (req, res) => {
  const signature = req.headers['x-amn-signature'];
  const expected = hmacSha256Hex(req.rawBody, process.env.SCANNER_CALLBACK_SECRET);
  if (!timingSafeEqual(signature, expected)) return res.status(401).end();

  const { intentId, status, txHash, amount, chainId } = req.body;
  if (status !== 'confirmed') return res.status(200).end(); // ignore non-confirmed

  await paymentService.markConfirmed({ intentId, txHash, amount, chainId });
  res.status(200).end();
});

Warning

The backend must always scope payment lookups by provider: "amn.scanner". Sweeping all pending payments to mark them confirmed/failed will destroy records from other providers (RN, manual). See memory note on payment cleanup + provider filter.

Backend env vars required

SCANNER_URL=http://amn-scanner:8080
SCANNER_API_KEY=<same value as scanner SCANNER_API_KEY>
SCANNER_CALLBACK_SECRET=<same value as scanner intent callbackSecret>

See memory note: amn_scanner_payin_wiring for full wiring details and token-decimal notes.

---

13. Known Limitations / Open Items

#	Area	Description
1	TON scaling	O(pending intents) TonCenter API calls per scan cycle. Becomes a bottleneck above ~50 concurrent TON intents. Future fix: batch by address or use a streaming API.
2	Tron/TON balance watches	POST /balances/check and POST /balance-watches currently only support EVM ERC-20 reads. Tron TRC-20 and TON Jetton balance reads are future scope.
3	Non-EVM chains unverified	Arbitrum, Polygon, Base, Tron, TON are "verified": false — workers do not start by default. Needs production testing + SCANNER_ENABLED_CHAINS opt-in before enabling for real traffic.
4	Base proxy address	0x1892196E80C4c17ea5100Da765Ab48c1fE2Fb814 is non-canonical (differs from RN's CREATE2 expected address for that chain). Verify before enabling Base in production.
5	Ethereum proxy version	Ethereum uses the v0.1.0 proxy at 0x370DE27…. A v0.2.0 proxy is also deployed on ETH. The two have different ABI signatures — do not silently swap addresses.
6	Single SQLite instance	The embedded SQLite database is not horizontally scalable. Running multiple scanner replicas would require coordination or migration to a shared DB. Acceptable for current load.
7	No native-token support	Only ERC-20/TRC-20/Jetton transfers are scanned. Native token (BNB, ETH, TRX, TON coin) payments are not supported.
8	Multi-seller / multi-chain	AMN Scanner pay-in supports single-seller flow only. Multi-seller cart payments and cross-chain routing are not implemented.
9	Webhook signature algorithm	HMAC-SHA256 with a pre-shared secret. There is no key rotation mechanism — changing callbackSecret requires intent re-registration.
10	No EVM testnet for ETH/Polygon/Base	Only BSC Testnet (97) has a testnet entry in supported-chains.json. Developers testing on Ethereum Sepolia or Polygon Amoy need to add chain entries manually.