diff --git a/PRD - Retire Request Network — In-House Payment Scanner.md b/PRD - Retire Request Network — In-House Payment Scanner.md index 6196cf2..46a7ea8 100644 --- a/PRD - Retire Request Network — In-House Payment Scanner.md +++ b/PRD - Retire Request Network — In-House Payment Scanner.md @@ -1,9 +1,9 @@ -# PRD — Retire Request Network: In-House Payment Scanner +# PRD — Retire Request Network: AMN Pay Scanner (Standalone Microservice) > Status: **Ready for implementation** > Task: #13 (new) > Priority: High -> Effort estimate: ~3–4 days (backend scanner + frontend checkout adjustment) +> Effort estimate: ~4–6 days (Rust/Go scanner service + Node.js adapter swap) > Depends on: Task #8 (done), Task #9 (confirmation thresholds), Task #11 (Trezor sweep — parallel, not blocking) --- @@ -18,193 +18,354 @@ The platform currently uses Request Network (RN) as a payment infrastructure mid The underlying payment mechanism — the `ERC20FeeProxy` smart contract — is **not proprietary to RN**. It is open-source, deployed on all five supported chains (BSC, Arbitrum, Ethereum, Polygon, Base), and already integrated into our in-house checkout. We call it directly today for payment execution. -This PRD describes replacing RN's API dependency with a self-contained scanner and local reference generator, while continuing to use the same `ERC20FeeProxy` contracts. +RN's critical constraint: it validates the payment `destination` against the registered merchant wallet (`0x05E280...`). This makes it impossible to route payments to HD-derived per-(buyer, seller) addresses through RN's API. + +This PRD describes replacing RN with **AMN Pay Scanner** — a standalone, lightweight microservice that exposes the same provider interface as RN but without the destination restriction and without any external dependency. --- -## What Changes and What Stays the Same +## Architecture Overview -### Stays the same +``` +┌─────────────────────────────────────────────────────────────────┐ +│ Node.js Backend │ +│ │ +│ POST /api/payment/intents │ +│ → amnPayAdapter.createIntent(...) │ +│ ↓ HTTP POST /intents │ +│ ┌──────────────────────────────┐ │ +│ │ AMN Pay Scanner │ ← replaces RN API │ +│ │ (Rust or Go binary) │ │ +│ │ │ │ +│ │ • generates salt + ref │ │ +│ │ • eth_getLogs per chain │ │ +│ │ • any destination address │ │ +│ │ • webhooks backend on match │ │ +│ └──────────────────────────────┘ │ +│ ↑ POST /webhook (confirmed) │ +│ → payment.status = 'confirmed' │ +│ → emit socket event to buyer │ +└─────────────────────────────────────────────────────────────────┘ +``` -- `ERC20FeeProxy` contracts on all five chains — no new contract, no deployment, no audit -- In-house checkout UI (frontend `rn-in-house-checkout-view.tsx`) — already calls the proxy directly -- HD wallet derivation per (buyer, sellerOffer) pair — continues as-is -- Payment model, status machine, webhook fanout — unchanged - -### Removed - -- `requestNetworkPaymentAdapter` call to RN's API (`POST /v2/secure-payments` or `/v2/request`) -- RN webhook receiver and its signature verification -- Dependency on RN's salt/requestId for paymentReference derivation -- `REQUEST_NETWORK_API_KEY` — no longer needed - -### Added - -- **Local salt generator** — 8 random bytes, replaces RN's `requestId` -- **`paymentReference` generated locally** using the existing `computeOnChainPaymentReference()` formula (already implemented in `paymentReference.ts`) -- **Chain scanner service** — background poller per chain; reads `eth_getLogs` for `TransferWithReferenceAndFee` events on the `ERC20FeeProxy` contract; matches against pending payment references in MongoDB -- **`ScannerCheckpoint` collection** — one document per chainId, tracks `lastScannedBlock` for crash-safe resume +The Node.js backend treats AMN Pay Scanner as just another payment provider. The existing adapter pattern (`requestNetworkAdapter.ts`) is replaced with `amnPayAdapter.ts`. Everything upstream (checkout UI, payment model, status machine, socket events) is unchanged. --- -## Architecture +## What AMN Pay Scanner Is -### Payment creation (replaces RN API call) +A standalone HTTP service that: -``` -POST /api/payment/request-network/intents - → generate salt locally (crypto.randomBytes(8).hex) - → paymentReference = computeOnChainPaymentReference(orderId, salt, destination) - → store payment with { salt, paymentReference, status: 'pending' } - → return checkout block to frontend (same shape as today) +1. **Accepts payment intents** — `POST /intents` with `{ chainId, tokenAddress, destination, amount, paymentReference?, callbackUrl }`. Returns `{ intentId, paymentReference, checkoutBlock }`. +2. **Generates paymentReference locally** if not provided — `last8bytes(keccak256(intentId + salt + destination))`, identical to RN's formula. +3. **Scans chains** — one async loop per chain, `eth_getLogs` on `ERC20FeeProxy` for `TransferWithReferenceAndFee` events matching pending intents. +4. **Delivers webhook** — `POST callbackUrl` with `{ intentId, paymentReference, txHash, blockNumber, amount, token, chainId, status: "confirmed" }` once confirmations ≥ threshold. +5. **Exposes status API** — `GET /intents/:intentId` returns current state. +6. **Exposes health/admin API** — `GET /health`, `GET /scanner/status` (per-chain: lastScannedBlock, chainHead, lag, pendingCount). + +It holds its own state (SQLite or embedded key-value store — no MongoDB dependency). It is **stateless from the backend's perspective**: if it restarts, it replays from its checkpoint. + +--- + +## Language Choice: Rust or Go + +Both are appropriate. The choice affects implementation speed vs long-term performance. + +### Go + +- Faster to write for a networked service (goroutines, `net/http`, `encoding/json` all stdlib) +- Easier for a generalist agent to implement correctly +- Good RPC JSON libraries available +- Produces a small static binary (~5MB) +- **Recommended for first version** + +### Rust + +- Marginally faster at sustained high-throughput block scanning (thousands of chains, millions of events) +- Steeper implementation effort, especially async (`tokio`, `ethers-rs` or `alloy`) +- Better choice if the service becomes a platform product +- **Recommended if scaling beyond 5 chains or productizing** + +**Decision for this PRD: Go for v1.** Rewrite to Rust if/when volume justifies it. The API surface is small; rewriting is low cost. + +--- + +## Service API + +### `POST /intents` + +```json +Request: +{ + "intentId": "6847abc123...", // caller-provided idempotency key (MongoDB Payment _id) + "chainId": 56, + "tokenAddress": "0x55d398...", // USDT on BSC + "destination": "0xDERIVED...", // any address — no restriction + "amount": "10000000000000000000", // wei + "callbackUrl": "https://api.amn.gg/api/payment/amn-scanner/webhook", + "callbackSecret": "hmac_secret", // used to sign webhook payload + "confirmations": 12 // optional override; defaults to chain threshold +} + +Response: +{ + "intentId": "6847abc123...", + "paymentReference": "0x1a2b3c4d5e6f7a8b", + "checkoutBlock": { + "destination": "0xDERIVED...", + "tokenAddress": "0x55d398...", + "tokenSymbol": "USDT", + "decimals": 18, + "chainId": 56, + "proxyAddress": "0x0DfbEe...", + "paymentReference": "0x1a2b3c4d5e6f7a8b", + "feeAmount": "0", + "feeAddress": "0x000000000000000000000000000000000000dEaD", + "amountWei": "10000000000000000000" + } +} ``` -The `orderId` used as the `requestId` substitute can be the MongoDB Payment `_id` (hex string). The formula is identical to what RN uses and what `paymentReference.ts` already implements. +### `GET /intents/:intentId` -### On-chain detection (replaces RN webhook) - -``` -ChainScanner (per chain, runs every 10–30s) - → eth_getLogs({ - address: ERC20FeeProxy[chainId], - topics: [TransferWithReferenceAndFee_topic], - fromBlock: checkpoint.lastScannedBlock, - toBlock: 'latest' - }) - → for each log: match paymentReference against pending payments in MongoDB - → if matched and confirmations >= threshold: mark payment 'confirmed' - → update checkpoint.lastScannedBlock = toBlock +```json +{ + "intentId": "...", + "status": "pending" | "confirmed" | "expired", + "paymentReference": "0x...", + "txHash": "0x..." | null, + "blockNumber": 12345678 | null, + "confirmations": 3, + "requiredConfirmations": 12 +} ``` -On startup, replay from `lastScannedBlock - 500` (about 25 minutes on BSC) to catch events missed during downtime. +### `POST /webhook` (outbound — scanner → backend) -### Destination: master wallet vs derived addresses +```json +{ + "intentId": "6847abc123...", + "paymentReference": "0x1a2b3c4d5e6f7a8b", + "txHash": "0xabc...", + "blockNumber": 39105482, + "amount": "10000000000000000000", + "token": "0x55d398...", + "chainId": 56, + "status": "confirmed" +} +// Header: X-AMN-Signature: hmac_sha256(body, callbackSecret) +``` -Two options, which can be decided independently of this PRD: +### `GET /scanner/status` -**Option A — Master wallet (simpler, ship now):** -Destination in the EVM call remains `0x05E280...` (current behaviour). Scanner matches on `paymentReference`. Funds land in a single wallet. No sweep needed. `derivedDestination` remains metadata-only (for future use). This is the conservative path — zero custody change. +```json +{ + "chains": [ + { "chainId": 56, "name": "BSC", "lastScannedBlock": 39105490, "chainHead": 39105492, "lag": 2, "pendingIntents": 1 }, + { "chainId": 42161, "name": "Arbitrum", "lastScannedBlock": 185234101, "chainHead": 185234104, "lag": 3, "pendingIntents": 0 } + ] +} +``` + +--- + +## Node.js Backend Changes + +### New adapter: `amnPayAdapter.ts` + +Replaces `requestNetworkAdapter.ts`. Calls `AMN_SCANNER_URL` (env var) instead of the RN API. The adapter interface (`CreatePayInIntentInput` → `PayInIntentResult`) is identical — the rest of the backend is untouched. + +### New webhook receiver: `POST /api/payment/amn-scanner/webhook` + +Replaces the existing RN webhook handler. Verifies `X-AMN-Signature`, finds the Payment by `intentId`, advances status to `confirmed`, emits socket event. ~50 lines, same pattern as existing RN webhook handler. + +### Adapter swap: `providerConfig.ts` + +```typescript +// Before: +provider: 'request.network' + +// After (new env var): +provider: process.env.AMN_SCANNER_URL ? 'amn.scanner' : 'request.network' +``` + +Allows parallel run: RN stays active for existing in-flight payments; new intents go to the scanner. + +### Environment variables + +| Variable | Where | Purpose | +|---|---|---| +| `AMN_SCANNER_URL` | Backend `.env` | Base URL of the scanner service (e.g. `http://amn-scanner:8080`) | +| `AMN_SCANNER_WEBHOOK_SECRET` | Backend `.env` | HMAC secret for incoming webhook verification | +| `AMN_SCANNER_CHAINS` | Scanner config | JSON array of chain configs (loaded from `supportedChains.json`) | +| `AMN_SCANNER_RPC_*` | Scanner config | Per-chain RPC URL overrides | + +--- + +## Destination: master wallet vs derived addresses + +With AMN Pay Scanner, **both options are available from day one** — there is no registered-wallet restriction. + +**Option A — Master wallet (conservative, ship immediately):** +`destination = 0x05E280...`. Funds land in single wallet. No sweep needed. `derivedDestination` stays as metadata. Zero operational change. **Option B — Derived HD addresses (full custody separation):** -Destination becomes the derived address per (buyer, sellerOffer) pair. Funds land in unique addresses. Scanner still matches on `paymentReference` (destination does not affect the reference formula). Requires Trezor-signed sweep (Task #11) to consolidate funds. This is the correct long-term architecture but introduces sweep complexity. +`destination = derivedDestination.address` per (buyer, sellerOffer) pair. Funds land in unique addresses. Requires Trezor-signed sweep (Task #11). Full per-buyer custody separation. -**Recommendation for initial ship: Option A.** Enable Option B after Task #11 (Trezor) is complete. +Recommendation: ship with Option A; flip to Option B after Task #11 (Trezor) is merged — it is a one-line change in `amnPayAdapter.ts`. --- ## Benefits -- **No RN API dependency** — no API key rotation, no RN rate limits, no RN downtime affecting checkouts -- **PaymentReference generated locally** — deterministic, auditable, no round-trip to external service before the buyer sees the checkout -- **Faster checkout initiation** — removes one external HTTP call from the critical path (RN API call took 500–2000ms in tests) -- **HD derived addresses usable as real destinations** — once Trezor sweep is in place, per-(buyer, seller) wallet separation is fully realized -- **Full observability** — scanner logs every block scanned, every match, every confirmation count; no black-box webhook -- **Cost reduction** — removes RN API subscription cost (if on a paid plan); replaces with RPC cost (~$0–50/month at current transaction volume) -- **Reuses audited contracts** — ERC20FeeProxy is RN's own open-source contract, already deployed and used in production; no new smart contract risk +- **No RN API dependency** — no API key, no RN rate limits, no RN downtime affecting checkouts +- **Any destination address** — removes the core RN constraint; derived HD addresses become real payment destinations +- **Faster checkout initiation** — `paymentReference` generated in the scanner in microseconds; no external HTTP round-trip on the critical path +- **Separation of concerns** — scanner is a dedicated process; a crash does not affect the Node.js backend's availability +- **Language-level performance** — Go/Rust binary uses far less memory than a Node.js process for sustained polling; more appropriate for an always-on IO-bound loop +- **Portable** — scanner can be deployed alongside the backend or on a separate host; scales independently +- **Reuses audited contracts** — same `ERC20FeeProxy` contracts; no new smart contract risk +- **Cost reduction** — removes RN subscription; RPC cost only (~$0–50/month at current volume) +- **Full observability** — every block, every log, every match is logged locally; no black-box --- ## Risks -- **Scanner downtime = delayed confirmation** — if the scanner process crashes, payments are not confirmed until it restarts. Mitigated by: checkpoint resume, restart policy in Docker (`restart: always`), alerting on scanner lag. -- **RPC reliability** — a flaky public RPC can cause missed blocks. Mitigated by: two RPCs per chain already in `supportedChains.json`, automatic fallback in scanner, and alerting on `eth_getLogs` errors. -- **Block reorganisations** — a shallow reorg could temporarily show a payment as confirmed. Mitigated by: confirmation thresholds (Task #9) set conservatively (12 blocks on BSC ≈ 36s). -- **Lost RN-hosted-page fallback** — some integrations may still generate `requestNetworkSecurePaymentUrl`. After migration, those URLs are no longer meaningful for new payments. Mitigated by: feature-flag the scanner; keep RN adapter runnable (just disabled) for 30 days post-cutover. -- **PaymentReference collision** — 8 bytes = 1-in-18-quintillion per pair. Not a practical risk. -- **Existing in-flight RN payments** — payments created before migration have RN-generated references and will not be detected by our scanner unless we also watch RN's webhook during the transition window. Mitigated by: drain existing pending payments before hard cutover, or run both paths in parallel for 24 hours. +- **Additional service to operate** — one more Docker container, one more process to monitor, one more thing that can be misconfigured. Mitigated by: `restart: always`, health endpoint, admin dashboard scanner-lag column. +- **Scanner downtime = delayed confirmation** — payments are not marked confirmed until the scanner catches up. Mitigated by: checkpoint resume (replays from `lastScannedBlock - 500` on startup), alerting on lag > 60s. +- **RPC reliability** — flaky RPC causes missed blocks. Mitigated by: two RPCs per chain with automatic fallback (already in `supportedChains.json`). +- **Block reorganisations** — shallow reorgs can temporarily confirm a payment that is later invalidated. Mitigated by: confirmation threshold (12 blocks on BSC ≈ 36s). +- **New codebase to maintain** — a Go/Rust service adds a new language to the stack. Mitigated by: the service is small (~500–800 lines), well-scoped, and unlikely to change frequently after initial ship. +- **Existing in-flight RN payments** — payments created before cutover have RN-generated references; only the RN webhook can detect them. Mitigated by: parallel-run window (both providers active); drain RN-pending payments before removing RN adapter. +- **HMAC secret rotation** — if `AMN_SCANNER_WEBHOOK_SECRET` leaks, an attacker can forge confirmations. Mitigated by: secret stored in deploy vault, not in source; webhook handler also checks `intentId` exists in DB before marking confirmed. --- ## Neutral Assessment -| Dimension | RN-hosted | In-house scanner | +| Dimension | RN-hosted | AMN Pay Scanner | |---|---|---| -| External dependency | RN API + webhook | RPC providers (two per chain) | -| Time to confirmation notification | RN webhook latency (seconds to minutes, opaque) | Scanner poll interval (15–30s, deterministic) | -| Cost | RN subscription (if paid) + RPC | RPC only ($0–50/month) | -| Operational complexity | Low (RN handles detection) | Medium (scanner process to run + monitor) | -| Custody flexibility | Locked to registered merchant wallet | Any address (derived wallets possible) | -| Auditability | Depends on RN logs | Full — every block, every match logged locally | +| External dependency | RN API + RN webhook | RPC providers (two per chain, already present) | +| Destination restriction | Registered merchant wallet only | Any EVM address | +| Confirmation latency | RN webhook (seconds–minutes, opaque) | Scanner poll interval (15–30s, deterministic) | +| Cost | RN subscription + RPC | RPC only ($0–50/month) | +| Operational complexity | Low — RN runs it | Medium — one extra Docker container | +| Languages in stack | Node.js only | Node.js + Go (or Rust) | +| Custody flexibility | None | Full (any address, incl. derived wallets) | +| Auditability | RN-side logs only | Full local logs: every block, every match | | Smart contract risk | RN's contracts (audited) | Same contracts — unchanged | -| Development effort | Zero (already integrated) | ~3–4 days | +| Dev effort to ship | Zero | ~4–6 days | -Neither approach is categorically superior. The in-house scanner trades operational ownership for custody flexibility and removes an external dependency. The tradeoff is worth taking if: (a) derived HD addresses are a priority, or (b) RN API reliability or cost is a concern, or (c) the team wants full control over the confirmation pipeline. +The primary driver for choosing AMN Pay Scanner over the embedded-in-Node approach (previous version of this PRD) is **process isolation**: an always-on block-scanning loop is a better fit for a compiled binary than for the Node.js event loop, and a crash in the scanner cannot take down the API server. --- ## Acceptance Criteria -1. A new Payment created via `/api/payment/request-network/intents` does **not** call the RN API — `paymentReference` is generated locally and stored on the Payment record at creation time. -2. The frontend checkout block is returned within 300ms of the intent request (no external HTTP dependency). +1. `POST /api/payment/intents` (or `/request-network/intents`) returns a checkout block within 300ms with no call to RN's API. +2. `paymentReference` is generated by the scanner and stored on the Payment record before the response is returned. 3. Scanner detects a `TransferWithReferenceAndFee` event on BSC within two poll cycles (≤30s) of the transaction being mined. -4. Matched payment is marked `confirmed` once `confirmations >= confirmationThreshold` for the chain. -5. Scanner resumes from `lastScannedBlock` after a process restart; no event is processed twice (idempotent on `txHash + logIndex`). -6. Existing in-flight payments (RN-originated) continue to be processed by the old path for 30 days (parallel run) or until manually drained. -7. `REQUEST_NETWORK_API_KEY` is no longer required at runtime; removing it does not break startup. -8. Admin dashboard shows scanner lag (current block vs last scanned block) per chain. +4. Matched payment is marked `confirmed` once `confirmations >= threshold`; socket event is emitted to the buyer. +5. Scanner resumes from `lastScannedBlock` after restart; no event is processed twice (idempotent on `txHash + logIndex`). +6. Webhook payload is rejected if `X-AMN-Signature` does not match. +7. Existing RN-originated payments continue to be detected via RN webhook during the parallel-run window. +8. `GET /api/admin/rn/scanner/status` returns per-chain lag; admin networks page shows lag column. +9. `REQUEST_NETWORK_API_KEY` can be removed from env without breaking startup. --- ## Files to Create / Modify +### Scanner service (new repo or `scanner/` directory in monorepo) + +| File | Purpose | +|---|---| +| `scanner/main.go` | Entry point: load config, start chain workers, start HTTP server | +| `scanner/intent.go` | Intent store: create, lookup, update status (SQLite via `database/sql`) | +| `scanner/chain.go` | Per-chain scanner loop: `eth_getLogs`, match, confirm, webhook delivery | +| `scanner/reference.go` | `computePaymentReference(intentId, salt, destination)` — mirrors `paymentReference.ts` formula | +| `scanner/webhook.go` | Outbound webhook POST with HMAC signing and retry | +| `scanner/api.go` | HTTP handlers: `POST /intents`, `GET /intents/:id`, `GET /scanner/status`, `GET /health` | +| `scanner/config.go` | Load chains from `supported-chains.json` (copy of backend's file) | +| `scanner/Dockerfile` | Multi-stage build: `golang:1.22-alpine` → `alpine:3.19` | +| `scanner/supported-chains.json` | Copy (or symlink) of backend's `supportedChains.json` | + +### Node.js backend + | File | Change | |---|---| -| `backend/src/services/payment/requestNetwork/requestNetworkPayInService.ts` | Replace RN adapter call with local salt + paymentReference generation | -| `backend/src/services/payment/scanner/chainScanner.ts` | **CREATE** — `eth_getLogs` polling loop, one instance per chain | -| `backend/src/services/payment/scanner/scannerCheckpoint.ts` | **CREATE** — MongoDB model + helpers for `lastScannedBlock` per chain | -| `backend/src/services/payment/scanner/index.ts` | **CREATE** — start all chain scanners on app boot | -| `backend/src/app.ts` | Call `startAllScanners()` on startup; add `GET /api/admin/rn/scanner/status` route | -| `backend/src/models/ScannerCheckpoint.ts` | **CREATE** — Mongoose schema: `{ chainId, lastScannedBlock, lastScannedAt, lastMatchAt }` | -| `frontend/src/sections/admin/networks/networks-list-view.tsx` | Add scanner lag column (current block vs checkpoint) per chain | +| `backend/src/services/payment/adapters/amnPayAdapter.ts` | **CREATE** — HTTP client for AMN Pay Scanner; same interface as `requestNetworkAdapter.ts` | +| `backend/src/services/payment/requestNetwork/requestNetworkPayInService.ts` | Add branch: if provider is `amn.scanner`, call `amnPayAdapter` instead of `requestNetworkAdapter` | +| `backend/src/routes/amnScannerWebhookRoutes.ts` | **CREATE** — `POST /api/payment/amn-scanner/webhook`; verify HMAC, update payment status | +| `backend/src/app.ts` | Mount `amnScannerWebhookRoutes`; add `GET /api/admin/rn/scanner/status` proxy | +| `backend/src/services/payment/providerConfig.ts` | Register `amn.scanner` as a provider; read `AMN_SCANNER_URL` env var | +| `frontend/src/sections/admin/networks/networks-list-view.tsx` | Add scanner lag column per chain | + +### Deployment + +| File | Change | +|---|---| +| `docker-compose.dev.yml` | Add `amn-scanner` service: build from `./scanner`, port 8080, env from `.env` | +| `docker-compose.production.yml` | Same | +| `.env.example` (backend) | Add `AMN_SCANNER_URL`, `AMN_SCANNER_WEBHOOK_SECRET` | +| `scanner/.env.example` | `PORT`, `CHAINS_JSON_PATH`, `RPC_BSC`, `RPC_ARB`, `RPC_ETH`, `RPC_POLYGON`, `RPC_BASE`, `DB_PATH` | --- -## Implementation Notes for Agent +## Implementation Notes for Agent (Kimi) -### Local paymentReference generation +### paymentReference formula (Go) -```typescript -// In requestNetworkPayInService.ts, replace the adapter call block with: -import crypto from 'crypto'; -import { computeOnChainPaymentReference } from './paymentReference'; - -const salt = crypto.randomBytes(8).toString('hex'); -const paymentId = new mongoose.Types.ObjectId().toHexString(); -const destination = derivedDestination?.address || process.env.REQUEST_NETWORK_RECIPIENT_ADDRESS; -const paymentReference = computeOnChainPaymentReference(paymentId, salt, destination); - -// Store on the Payment document: -// metadata.salt, metadata.paymentReference (the 8-byte hex) -// providerPaymentId = paymentId (the generated hex ID) +```go +// Mirrors backend/src/services/payment/requestNetwork/paymentReference.ts +func computePaymentReference(intentId, salt, destination string) string { + combined := strings.ToLower(intentId + salt + destination) + hash := crypto.Keccak256([]byte(combined)) + // last 8 bytes = last 16 hex chars + return "0x" + hex.EncodeToString(hash[24:]) +} ``` -### Scanner event signature +### TransferWithReferenceAndFee event topic -```typescript -// TransferWithReferenceAndFee(address,address,uint256,bytes,uint256,address) -const TOPIC = '0xbc8b749850bdc0c5e4a49d50f87ec40dca2acdb70a84e7c6823ccdc7b3b3d4b3'; - -// Log decode (ethers v6): -const iface = new ethers.Interface([ - 'event TransferWithReferenceAndFee(address token, address to, uint256 amount, bytes indexed paymentReference, uint256 feeAmount, address feeAddress)' -]); -const decoded = iface.parseLog(log); -const ref = decoded.args.paymentReference; // bytes → hex string +``` +0xbc8b749850bdc0c5e4a49d50f87ec40dca2acdb70a84e7c6823ccdc7b3b3d4b3 ``` -### Scanner idempotency key +The `paymentReference` is the **4th topic** (indexed `bytes` parameter). Read it from `log.Topics[1]` (after the event signature topic). It is ABI-encoded as a keccak256 hash of the bytes value when indexed — verify against a known BSC transaction before trusting. -Match on `{ 'metadata.paymentReference': ref }` in MongoDB. When marking confirmed, use `findOneAndUpdate` with `{ $set: { status: 'confirmed' } }` — safe to call twice; second call is a no-op because status is already confirmed. +### eth_getLogs chunking -### RPC eth_getLogs batch limit +Most chains cap at 2000–5000 blocks per call. Chunk in 2000-block windows: -BSC and most chains cap `eth_getLogs` to 2000–5000 blocks per call. Scan in 2000-block chunks if `toBlock - fromBlock > 2000`. Always store `lastScannedBlock` as `toBlock` of the last successful chunk, not the block of the last match. +```go +for from := checkpoint; from <= head; from += 2000 { + to := min(from+1999, head) + logs := ethGetLogs(rpc, proxyAddr, from, to) + process(logs) + saveCheckpoint(chainId, to) +} +``` + +### Webhook retry + +On HTTP error or non-2xx response, retry with exponential backoff: 5s, 30s, 2min, 10min, 1h. After 5 failures, mark intent `webhook_failed` and log — do not lose the confirmed state. + +### Idempotency + +Index on `(txHash, logIndex)` in the intent store. On duplicate, skip processing — do not re-deliver the webhook. + +### Scanner lag alert + +If `chainHead - lastScannedBlock > 100` (about 5 minutes on BSC), log at `WARN` level. The Node.js backend's admin status route can surface this to Telegram notifications. --- ## Out of Scope for This PRD -- Trezor-signed sweep (Task #11) — required to make Option B (derived addresses as real destinations) viable -- AML screening on scanner matches (Task #10) -- Per-chain confirmation threshold admin UI (Task #9) — scanner reads thresholds from `supportedChains.json`; the UI to edit them is Task #9 -- TON / non-EVM chain support — scanner is EVM-only; separate work if needed +- Trezor-signed sweep (Task #11) — required to make derived addresses as real destinations viable end-to-end +- AML screening on matched payments (Task #10) +- Per-chain confirmation threshold admin UI (Task #9) — scanner reads thresholds from config; the Node.js admin UI is Task #9 +- TON / non-EVM chain support — scanner is EVM-only +- Productizing the scanner as a standalone SaaS — this PRD is for internal use only