nick-doc/PRD - Retire Request Network — In-House Payment Scanner.md

# PRD — Retire Request Network: AMN Pay Scanner (Standalone Microservice)

> Status: **Implemented — 2026-05-29**
> Task: #13
> Priority: High
> Effort estimate: ~4–6 days (Rust/Go scanner service + Node.js adapter swap)
> Depends on: Task #8 (done), Task #9 (confirmation thresholds), Task #11 (Trezor sweep — parallel, not blocking)
>
> | Component | Status |
> |---|---|
> | Go scanner service | ✅ Done — `scanner/*.go`, builds, tests pass |
> | Backend adapter (`amnPayAdapter.ts`) | ✅ Done — implements `PaymentProviderAdapter` |
> | Webhook receiver (`amnScannerWebhookRoutes.ts`) | ✅ Done — HMAC verify + PaymentCoordinator delegation |
> | Provider config (`providerConfig.ts`) | ✅ Done — `amn.scanner` enabled when `AMN_SCANNER_URL` set |
> | Admin scanner status proxy | ✅ Done — `GET /api/admin/scanner/status` |
> | Frontend lag column | ✅ Done — color-coded chips in networks list |
> | Docker compose (dev + prod) | ✅ Done — `amn-scanner` service with healthcheck |
> | Env vars (backend + scanner) | ✅ Done — `AMN_SCANNER_URL`, `AMN_SCANNER_WEBHOOK_SECRET`, `AMN_SCANNER_DEFAULT` |
> | Cross-language reference test | ✅ Done — Go `computePaymentReference` = TS `computeOnChainPaymentReference` |
> | Live end-to-end probe | ⏳ Pending — requires deployed scanner + real on-chain payment |

---

## Context

The platform currently uses Request Network (RN) as a payment infrastructure middleware. RN's API is called to:

1. Create a payment request (returns a `requestId` + `salt` used to derive an on-chain `paymentReference`)
2. Provide a hosted checkout page (already bypassed by our in-house checkout)
3. Deliver a webhook callback when payment is detected on-chain

The underlying payment mechanism — the `ERC20FeeProxy` smart contract — is **not proprietary to RN**. It is open-source, deployed on all five supported chains (BSC, Arbitrum, Ethereum, Polygon, Base), and already integrated into our in-house checkout. We call it directly today for payment execution.

RN's critical constraint: it validates the payment `destination` against the registered merchant wallet (`0x05E280...`). This makes it impossible to route payments to HD-derived per-(buyer, seller) addresses through RN's API.

This PRD describes replacing RN with **AMN Pay Scanner** — a standalone, lightweight microservice that exposes the same provider interface as RN but without the destination restriction and without any external dependency.

---

## Architecture Overview

```
┌─────────────────────────────────────────────────────────────────┐
│                        Node.js Backend                          │
│                                                                 │
│  POST /api/payment/intents                                      │
│    → amnPayAdapter.createIntent(...)                            │
│         ↓  HTTP POST /intents                                   │
│  ┌──────────────────────────────┐                               │
│  │      AMN Pay Scanner         │  ← replaces RN API            │
│  │   (Rust or Go binary)        │                               │
│  │                              │                               │
│  │  • generates salt + ref      │                               │
│  │  • eth_getLogs per chain     │                               │
│  │  • any destination address   │                               │
│  │  • webhooks backend on match │                               │
│  └──────────────────────────────┘                               │
│         ↑  POST /webhook (confirmed)                            │
│    → payment.status = 'confirmed'                               │
│    → emit socket event to buyer                                 │
└─────────────────────────────────────────────────────────────────┘
```

The Node.js backend treats AMN Pay Scanner as just another payment provider. The existing adapter pattern (`requestNetworkAdapter.ts`) is replaced with `amnPayAdapter.ts`. Everything upstream (checkout UI, payment model, status machine, socket events) is unchanged.

---

## What AMN Pay Scanner Is

A standalone HTTP service that:

1. **Accepts payment intents** — `POST /intents` with `{ chainId, tokenAddress, destination, amount, paymentReference?, callbackUrl }`. Returns `{ intentId, paymentReference, checkoutBlock }`.
2. **Generates paymentReference locally** if not provided — `last8bytes(keccak256(intentId + salt + destination))`, identical to RN's formula.
3. **Scans chains** — one async loop per chain, `eth_getLogs` on `ERC20FeeProxy` for `TransferWithReferenceAndFee` events matching pending intents.
4. **Delivers webhook** — `POST callbackUrl` with `{ intentId, paymentReference, txHash, blockNumber, amount, token, chainId, status: "confirmed" }` once confirmations ≥ threshold.
5. **Exposes status API** — `GET /intents/:intentId` returns current state.
6. **Exposes health/admin API** — `GET /health`, `GET /scanner/status` (per-chain: lastScannedBlock, chainHead, lag, pendingCount).

It holds its own state (SQLite or embedded key-value store — no MongoDB dependency). It is **stateless from the backend's perspective**: if it restarts, it replays from its checkpoint.

---

## Language Choice: Rust or Go

Both are appropriate. The choice affects implementation speed vs long-term performance.

### Go

- Faster to write for a networked service (goroutines, `net/http`, `encoding/json` all stdlib)
- Easier for a generalist agent to implement correctly
- Good RPC JSON libraries available
- Produces a small static binary (~5MB)
- **Recommended for first version**

### Rust

- Marginally faster at sustained high-throughput block scanning (thousands of chains, millions of events)
- Steeper implementation effort, especially async (`tokio`, `ethers-rs` or `alloy`)
- Better choice if the service becomes a platform product
- **Recommended if scaling beyond 5 chains or productizing**

**Decision for this PRD: Go for v1.** Rewrite to Rust if/when volume justifies it. The API surface is small; rewriting is low cost.

---

## Service API

### `POST /intents`

```json
Request:
{
  "intentId": "6847abc123...",        // caller-provided idempotency key (MongoDB Payment _id)
  "chainId": 56,
  "tokenAddress": "0x55d398...",      // USDT on BSC
  "destination": "0xDERIVED...",      // any address — no restriction
  "amount": "10000000000000000000",   // wei
  "callbackUrl": "https://api.amn.gg/api/payment/amn-scanner/webhook",
  "callbackSecret": "hmac_secret",    // used to sign webhook payload
  "confirmations": 12                 // optional override; defaults to chain threshold
}

Response:
{
  "intentId": "6847abc123...",
  "paymentReference": "0x1a2b3c4d5e6f7a8b",
  "checkoutBlock": {
    "destination": "0xDERIVED...",
    "tokenAddress": "0x55d398...",
    "tokenSymbol": "USDT",
    "decimals": 18,
    "chainId": 56,
    "proxyAddress": "0x0DfbEe...",
    "paymentReference": "0x1a2b3c4d5e6f7a8b",
    "feeAmount": "0",
    "feeAddress": "0x000000000000000000000000000000000000dEaD",
    "amountWei": "10000000000000000000"
  }
}
```

### `GET /intents/:intentId`

```json
{
  "intentId": "...",
  "status": "pending" | "confirmed" | "expired",
  "paymentReference": "0x...",
  "txHash": "0x..." | null,
  "blockNumber": 12345678 | null,
  "confirmations": 3,
  "requiredConfirmations": 12
}
```

### `POST /webhook` (outbound — scanner → backend)

```json
{
  "intentId": "6847abc123...",
  "paymentReference": "0x1a2b3c4d5e6f7a8b",
  "txHash": "0xabc...",
  "blockNumber": 39105482,
  "amount": "10000000000000000000",
  "token": "0x55d398...",
  "chainId": 56,
  "status": "confirmed"
}
// Header: X-AMN-Signature: hmac_sha256(body, callbackSecret)
```

### `GET /scanner/status`

```json
{
  "chains": [
    { "chainId": 56, "name": "BSC", "lastScannedBlock": 39105490, "chainHead": 39105492, "lag": 2, "pendingIntents": 1 },
    { "chainId": 42161, "name": "Arbitrum", "lastScannedBlock": 185234101, "chainHead": 185234104, "lag": 3, "pendingIntents": 0 }
  ]
}
```

---

## Node.js Backend Changes

### New adapter: `amnPayAdapter.ts`

Replaces `requestNetworkAdapter.ts`. Calls `AMN_SCANNER_URL` (env var) instead of the RN API. The adapter interface (`CreatePayInIntentInput` → `PayInIntentResult`) is identical — the rest of the backend is untouched.

### New webhook receiver: `POST /api/payment/amn-scanner/webhook`

Replaces the existing RN webhook handler. Verifies `X-AMN-Signature`, finds the Payment by `intentId`, advances status to `confirmed`, emits socket event. ~50 lines, same pattern as existing RN webhook handler.

### Adapter swap: `providerConfig.ts`

```typescript
// Before:
provider: 'request.network'

// After (new env var):
provider: process.env.AMN_SCANNER_URL ? 'amn.scanner' : 'request.network'
```

Allows parallel run: RN stays active for existing in-flight payments; new intents go to the scanner.

### Environment variables

| Variable | Where | Purpose |
|---|---|---|
| `AMN_SCANNER_URL` | Backend `.env` | Base URL of the scanner service (e.g. `http://amn-scanner:8080`) |
| `AMN_SCANNER_WEBHOOK_SECRET` | Backend `.env` | HMAC secret for incoming webhook verification |
| `AMN_SCANNER_CHAINS` | Scanner config | JSON array of chain configs (loaded from `supportedChains.json`) |
| `AMN_SCANNER_RPC_*` | Scanner config | Per-chain RPC URL overrides |

---

## Destination: master wallet vs derived addresses

With AMN Pay Scanner, **both options are available from day one** — there is no registered-wallet restriction.

**Option A — Master wallet (conservative, ship immediately):**
`destination = 0x05E280...`. Funds land in single wallet. No sweep needed. `derivedDestination` stays as metadata. Zero operational change.

**Option B — Derived HD addresses (full custody separation):**
`destination = derivedDestination.address` per (buyer, sellerOffer) pair. Funds land in unique addresses. Requires Trezor-signed sweep (Task #11). Full per-buyer custody separation.

Recommendation: ship with Option A; flip to Option B after Task #11 (Trezor) is merged — it is a one-line change in `amnPayAdapter.ts`.

---

## Benefits

- **No RN API dependency** — no API key, no RN rate limits, no RN downtime affecting checkouts
- **Any destination address** — removes the core RN constraint; derived HD addresses become real payment destinations
- **Faster checkout initiation** — `paymentReference` generated in the scanner in microseconds; no external HTTP round-trip on the critical path
- **Separation of concerns** — scanner is a dedicated process; a crash does not affect the Node.js backend's availability
- **Language-level performance** — Go/Rust binary uses far less memory than a Node.js process for sustained polling; more appropriate for an always-on IO-bound loop
- **Portable** — scanner can be deployed alongside the backend or on a separate host; scales independently
- **Reuses audited contracts** — same `ERC20FeeProxy` contracts; no new smart contract risk
- **Cost reduction** — removes RN subscription; RPC cost only (~$0–50/month at current volume)
- **Full observability** — every block, every log, every match is logged locally; no black-box

---

## Risks

- **Additional service to operate** — one more Docker container, one more process to monitor, one more thing that can be misconfigured. Mitigated by: `restart: always`, health endpoint, admin dashboard scanner-lag column.
- **Scanner downtime = delayed confirmation** — payments are not marked confirmed until the scanner catches up. Mitigated by: checkpoint resume (replays from `lastScannedBlock - 500` on startup), alerting on lag > 60s.
- **RPC reliability** — flaky RPC causes missed blocks. Mitigated by: two RPCs per chain with automatic fallback (already in `supportedChains.json`).
- **Block reorganisations** — shallow reorgs can temporarily confirm a payment that is later invalidated. Mitigated by: confirmation threshold (12 blocks on BSC ≈ 36s).
- **New codebase to maintain** — a Go/Rust service adds a new language to the stack. Mitigated by: the service is small (~500–800 lines), well-scoped, and unlikely to change frequently after initial ship.
- **Existing in-flight RN payments** — payments created before cutover have RN-generated references; only the RN webhook can detect them. Mitigated by: parallel-run window (both providers active); drain RN-pending payments before removing RN adapter.
- **HMAC secret rotation** — if `AMN_SCANNER_WEBHOOK_SECRET` leaks, an attacker can forge confirmations. Mitigated by: secret stored in deploy vault, not in source; webhook handler also checks `intentId` exists in DB before marking confirmed.

---

## Neutral Assessment

| Dimension | RN-hosted | AMN Pay Scanner |
|---|---|---|
| External dependency | RN API + RN webhook | RPC providers (two per chain, already present) |
| Destination restriction | Registered merchant wallet only | Any EVM address |
| Confirmation latency | RN webhook (seconds–minutes, opaque) | Scanner poll interval (15–30s, deterministic) |
| Cost | RN subscription + RPC | RPC only ($0–50/month) |
| Operational complexity | Low — RN runs it | Medium — one extra Docker container |
| Languages in stack | Node.js only | Node.js + Go (or Rust) |
| Custody flexibility | None | Full (any address, incl. derived wallets) |
| Auditability | RN-side logs only | Full local logs: every block, every match |
| Smart contract risk | RN's contracts (audited) | Same contracts — unchanged |
| Dev effort to ship | Zero | ~4–6 days |

The primary driver for choosing AMN Pay Scanner over the embedded-in-Node approach (previous version of this PRD) is **process isolation**: an always-on block-scanning loop is a better fit for a compiled binary than for the Node.js event loop, and a crash in the scanner cannot take down the API server.

---

## Acceptance Criteria

1. `POST /api/payment/intents` (or `/request-network/intents`) returns a checkout block within 300ms with no call to RN's API.
2. `paymentReference` is generated by the scanner and stored on the Payment record before the response is returned.
3. Scanner detects a `TransferWithReferenceAndFee` event on BSC within two poll cycles (≤30s) of the transaction being mined.
4. Matched payment is marked `confirmed` once `confirmations >= threshold`; socket event is emitted to the buyer.
5. Scanner resumes from `lastScannedBlock` after restart; no event is processed twice (idempotent on `txHash + logIndex`).
6. Webhook payload is rejected if `X-AMN-Signature` does not match.
7. Existing RN-originated payments continue to be detected via RN webhook during the parallel-run window.
8. `GET /api/admin/rn/scanner/status` returns per-chain lag; admin networks page shows lag column.
9. `REQUEST_NETWORK_API_KEY` can be removed from env without breaking startup.

---

## Files to Create / Modify

### Scanner service (new repo or `scanner/` directory in monorepo)

| File | Purpose |
|---|---|
| `scanner/main.go` | Entry point: load config, start chain workers, start HTTP server |
| `scanner/intent.go` | Intent store: create, lookup, update status (SQLite via `database/sql`) |
| `scanner/chain.go` | Per-chain scanner loop: `eth_getLogs`, match, confirm, webhook delivery |
| `scanner/reference.go` | `computePaymentReference(intentId, salt, destination)` — mirrors `paymentReference.ts` formula |
| `scanner/webhook.go` | Outbound webhook POST with HMAC signing and retry |
| `scanner/api.go` | HTTP handlers: `POST /intents`, `GET /intents/:id`, `GET /scanner/status`, `GET /health` |
| `scanner/config.go` | Load chains from `supported-chains.json` (copy of backend's file) |
| `scanner/Dockerfile` | Multi-stage build: `golang:1.22-alpine` → `alpine:3.19` |
| `scanner/supported-chains.json` | Copy (or symlink) of backend's `supportedChains.json` |

### Node.js backend

| File | Change |
|---|---|
| `backend/src/services/payment/adapters/amnPayAdapter.ts` | **CREATE** — HTTP client for AMN Pay Scanner; same interface as `requestNetworkAdapter.ts` |
| `backend/src/services/payment/requestNetwork/requestNetworkPayInService.ts` | Add branch: if provider is `amn.scanner`, call `amnPayAdapter` instead of `requestNetworkAdapter` |
| `backend/src/routes/amnScannerWebhookRoutes.ts` | **CREATE** — `POST /api/payment/amn-scanner/webhook`; verify HMAC, update payment status |
| `backend/src/app.ts` | Mount `amnScannerWebhookRoutes`; add `GET /api/admin/rn/scanner/status` proxy |
| `backend/src/services/payment/providerConfig.ts` | Register `amn.scanner` as a provider; read `AMN_SCANNER_URL` env var |
| `frontend/src/sections/admin/networks/networks-list-view.tsx` | Add scanner lag column per chain |

### Deployment

| File | Change |
|---|---|
| `docker-compose.dev.yml` | Add `amn-scanner` service: build from `./scanner`, port 8080, env from `.env` |
| `docker-compose.production.yml` | Same |
| `.env.example` (backend) | Add `AMN_SCANNER_URL`, `AMN_SCANNER_WEBHOOK_SECRET` |
| `scanner/.env.example` | `PORT`, `CHAINS_JSON_PATH`, `RPC_BSC`, `RPC_ARB`, `RPC_ETH`, `RPC_POLYGON`, `RPC_BASE`, `DB_PATH` |

---

## Implementation Notes for Agent (Kimi)

### paymentReference formula (Go)

```go
// Mirrors backend/src/services/payment/requestNetwork/paymentReference.ts
func computePaymentReference(intentId, salt, destination string) string {
    combined := strings.ToLower(intentId + salt + destination)
    hash := crypto.Keccak256([]byte(combined))
    // last 8 bytes = last 16 hex chars
    return "0x" + hex.EncodeToString(hash[24:])
}
```

### TransferWithReferenceAndFee event topic

```
0xbc8b749850bdc0c5e4a49d50f87ec40dca2acdb70a84e7c6823ccdc7b3b3d4b3
```

The `paymentReference` is the **4th topic** (indexed `bytes` parameter). Read it from `log.Topics[1]` (after the event signature topic). It is ABI-encoded as a keccak256 hash of the bytes value when indexed — verify against a known BSC transaction before trusting.

### eth_getLogs chunking

Most chains cap at 2000–5000 blocks per call. Chunk in 2000-block windows:

```go
for from := checkpoint; from <= head; from += 2000 {
    to := min(from+1999, head)
    logs := ethGetLogs(rpc, proxyAddr, from, to)
    process(logs)
    saveCheckpoint(chainId, to)
}
```

### Webhook retry

On HTTP error or non-2xx response, retry with exponential backoff: 5s, 30s, 2min, 10min, 1h. After 5 failures, mark intent `webhook_failed` and log — do not lose the confirmed state.

### Idempotency

Index on `(txHash, logIndex)` in the intent store. On duplicate, skip processing — do not re-deliver the webhook.

### Scanner lag alert

If `chainHead - lastScannedBlock > 100` (about 5 minutes on BSC), log at `WARN` level. The Node.js backend's admin status route can surface this to Telegram notifications.

---

## Out of Scope for This PRD

- Trezor-signed sweep (Task #11) — required to make derived addresses as real destinations viable end-to-end
- AML screening on matched payments (Task #10)
- Per-chain confirmation threshold admin UI (Task #9) — scanner reads thresholds from config; the Node.js admin UI is Task #9
- TON / non-EVM chain support — scanner is EVM-only
- Productizing the scanner as a standalone SaaS — this PRD is for internal use only