nick-doc/PRD - Retire Request Network — In-House Payment Scanner.md

PRD — Retire Request Network: In-House Payment Scanner

Status: Ready for implementation Task: #13 (new) Priority: High Effort estimate: ~3–4 days (backend scanner + frontend checkout adjustment) Depends on: Task #8 (done), Task #9 (confirmation thresholds), Task #11 (Trezor sweep — parallel, not blocking)

---

Context

The platform currently uses Request Network (RN) as a payment infrastructure middleware. RN's API is called to:

1. Create a payment request (returns a requestId + salt used to derive an on-chain paymentReference)
2. Provide a hosted checkout page (already bypassed by our in-house checkout)
3. Deliver a webhook callback when payment is detected on-chain

The underlying payment mechanism — the ERC20FeeProxy smart contract — is not proprietary to RN. It is open-source, deployed on all five supported chains (BSC, Arbitrum, Ethereum, Polygon, Base), and already integrated into our in-house checkout. We call it directly today for payment execution.

This PRD describes replacing RN's API dependency with a self-contained scanner and local reference generator, while continuing to use the same ERC20FeeProxy contracts.

---

What Changes and What Stays the Same

Stays the same

• ERC20FeeProxy contracts on all five chains — no new contract, no deployment, no audit
• In-house checkout UI (frontend rn-in-house-checkout-view.tsx) — already calls the proxy directly
• HD wallet derivation per (buyer, sellerOffer) pair — continues as-is
• Payment model, status machine, webhook fanout — unchanged

Removed

• requestNetworkPaymentAdapter call to RN's API (POST /v2/secure-payments or /v2/request)
• RN webhook receiver and its signature verification
• Dependency on RN's salt/requestId for paymentReference derivation
• REQUEST_NETWORK_API_KEY — no longer needed

Added

• Local salt generator — 8 random bytes, replaces RN's requestId
• paymentReference generated locally using the existing computeOnChainPaymentReference() formula (already implemented in paymentReference.ts)
• Chain scanner service — background poller per chain; reads eth_getLogs for TransferWithReferenceAndFee events on the ERC20FeeProxy contract; matches against pending payment references in MongoDB
• ScannerCheckpoint collection — one document per chainId, tracks lastScannedBlock for crash-safe resume

---

Architecture

Payment creation (replaces RN API call)

POST /api/payment/request-network/intents
  → generate salt locally (crypto.randomBytes(8).hex)
  → paymentReference = computeOnChainPaymentReference(orderId, salt, destination)
  → store payment with { salt, paymentReference, status: 'pending' }
  → return checkout block to frontend (same shape as today)

The orderId used as the requestId substitute can be the MongoDB Payment _id (hex string). The formula is identical to what RN uses and what paymentReference.ts already implements.

On-chain detection (replaces RN webhook)

ChainScanner (per chain, runs every 10–30s)
  → eth_getLogs({
       address: ERC20FeeProxy[chainId],
       topics: [TransferWithReferenceAndFee_topic],
       fromBlock: checkpoint.lastScannedBlock,
       toBlock: 'latest'
     })
  → for each log: match paymentReference against pending payments in MongoDB
  → if matched and confirmations >= threshold: mark payment 'confirmed'
  → update checkpoint.lastScannedBlock = toBlock

On startup, replay from lastScannedBlock - 500 (about 25 minutes on BSC) to catch events missed during downtime.

Destination: master wallet vs derived addresses

Two options, which can be decided independently of this PRD:

Option A — Master wallet (simpler, ship now): Destination in the EVM call remains 0x05E280... (current behaviour). Scanner matches on paymentReference. Funds land in a single wallet. No sweep needed. derivedDestination remains metadata-only (for future use). This is the conservative path — zero custody change.

Option B — Derived HD addresses (full custody separation): Destination becomes the derived address per (buyer, sellerOffer) pair. Funds land in unique addresses. Scanner still matches on paymentReference (destination does not affect the reference formula). Requires Trezor-signed sweep (Task #11) to consolidate funds. This is the correct long-term architecture but introduces sweep complexity.

Recommendation for initial ship: Option A. Enable Option B after Task #11 (Trezor) is complete.

---

Benefits

• No RN API dependency — no API key rotation, no RN rate limits, no RN downtime affecting checkouts
• PaymentReference generated locally — deterministic, auditable, no round-trip to external service before the buyer sees the checkout
• Faster checkout initiation — removes one external HTTP call from the critical path (RN API call took 500–2000ms in tests)
• HD derived addresses usable as real destinations — once Trezor sweep is in place, per-(buyer, seller) wallet separation is fully realized
• Full observability — scanner logs every block scanned, every match, every confirmation count; no black-box webhook
• Cost reduction — removes RN API subscription cost (if on a paid plan); replaces with RPC cost (~$0–50/month at current transaction volume)
• Reuses audited contracts — ERC20FeeProxy is RN's own open-source contract, already deployed and used in production; no new smart contract risk

---

Risks

• Scanner downtime = delayed confirmation — if the scanner process crashes, payments are not confirmed until it restarts. Mitigated by: checkpoint resume, restart policy in Docker (restart: always), alerting on scanner lag.
• RPC reliability — a flaky public RPC can cause missed blocks. Mitigated by: two RPCs per chain already in supportedChains.json, automatic fallback in scanner, and alerting on eth_getLogs errors.
• Block reorganisations — a shallow reorg could temporarily show a payment as confirmed. Mitigated by: confirmation thresholds (Task #9) set conservatively (12 blocks on BSC ≈ 36s).
• Lost RN-hosted-page fallback — some integrations may still generate requestNetworkSecurePaymentUrl. After migration, those URLs are no longer meaningful for new payments. Mitigated by: feature-flag the scanner; keep RN adapter runnable (just disabled) for 30 days post-cutover.
• PaymentReference collision — 8 bytes = 1-in-18-quintillion per pair. Not a practical risk.
• Existing in-flight RN payments — payments created before migration have RN-generated references and will not be detected by our scanner unless we also watch RN's webhook during the transition window. Mitigated by: drain existing pending payments before hard cutover, or run both paths in parallel for 24 hours.

---

Neutral Assessment

Dimension	RN-hosted	In-house scanner
External dependency	RN API + webhook	RPC providers (two per chain)
Time to confirmation notification	RN webhook latency (seconds to minutes, opaque)	Scanner poll interval (15–30s, deterministic)
Cost	RN subscription (if paid) + RPC	RPC only ($0–50/month)
Operational complexity	Low (RN handles detection)	Medium (scanner process to run + monitor)
Custody flexibility	Locked to registered merchant wallet	Any address (derived wallets possible)
Auditability	Depends on RN logs	Full — every block, every match logged locally
Smart contract risk	RN's contracts (audited)	Same contracts — unchanged
Development effort	Zero (already integrated)	~3–4 days

Neither approach is categorically superior. The in-house scanner trades operational ownership for custody flexibility and removes an external dependency. The tradeoff is worth taking if: (a) derived HD addresses are a priority, or (b) RN API reliability or cost is a concern, or (c) the team wants full control over the confirmation pipeline.

---

Acceptance Criteria

1. A new Payment created via /api/payment/request-network/intents does not call the RN API — paymentReference is generated locally and stored on the Payment record at creation time.
2. The frontend checkout block is returned within 300ms of the intent request (no external HTTP dependency).
3. Scanner detects a TransferWithReferenceAndFee event on BSC within two poll cycles (≤30s) of the transaction being mined.
4. Matched payment is marked confirmed once confirmations >= confirmationThreshold for the chain.
5. Scanner resumes from lastScannedBlock after a process restart; no event is processed twice (idempotent on txHash + logIndex).
6. Existing in-flight payments (RN-originated) continue to be processed by the old path for 30 days (parallel run) or until manually drained.
7. REQUEST_NETWORK_API_KEY is no longer required at runtime; removing it does not break startup.
8. Admin dashboard shows scanner lag (current block vs last scanned block) per chain.

---

Files to Create / Modify

File	Change
backend/src/services/payment/requestNetwork/requestNetworkPayInService.ts	Replace RN adapter call with local salt + paymentReference generation
backend/src/services/payment/scanner/chainScanner.ts	CREATE — eth_getLogs polling loop, one instance per chain
backend/src/services/payment/scanner/scannerCheckpoint.ts	CREATE — MongoDB model + helpers for lastScannedBlock per chain
backend/src/services/payment/scanner/index.ts	CREATE — start all chain scanners on app boot
backend/src/app.ts	Call startAllScanners() on startup; add GET /api/admin/rn/scanner/status route
backend/src/models/ScannerCheckpoint.ts	CREATE — Mongoose schema: { chainId, lastScannedBlock, lastScannedAt, lastMatchAt }
frontend/src/sections/admin/networks/networks-list-view.tsx	Add scanner lag column (current block vs checkpoint) per chain

---

Implementation Notes for Agent

Local paymentReference generation

// In requestNetworkPayInService.ts, replace the adapter call block with:
import crypto from 'crypto';
import { computeOnChainPaymentReference } from './paymentReference';

const salt = crypto.randomBytes(8).toString('hex');
const paymentId = new mongoose.Types.ObjectId().toHexString();
const destination = derivedDestination?.address || process.env.REQUEST_NETWORK_RECIPIENT_ADDRESS;
const paymentReference = computeOnChainPaymentReference(paymentId, salt, destination);

// Store on the Payment document:
// metadata.salt, metadata.paymentReference (the 8-byte hex)
// providerPaymentId = paymentId (the generated hex ID)

Scanner event signature

// TransferWithReferenceAndFee(address,address,uint256,bytes,uint256,address)
const TOPIC = '0xbc8b749850bdc0c5e4a49d50f87ec40dca2acdb70a84e7c6823ccdc7b3b3d4b3';

// Log decode (ethers v6):
const iface = new ethers.Interface([
  'event TransferWithReferenceAndFee(address token, address to, uint256 amount, bytes indexed paymentReference, uint256 feeAmount, address feeAddress)'
]);
const decoded = iface.parseLog(log);
const ref = decoded.args.paymentReference; // bytes → hex string

Scanner idempotency key

Match on { 'metadata.paymentReference': ref } in MongoDB. When marking confirmed, use findOneAndUpdate with { $set: { status: 'confirmed' } } — safe to call twice; second call is a no-op because status is already confirmed.

RPC eth_getLogs batch limit

BSC and most chains cap eth_getLogs to 2000–5000 blocks per call. Scan in 2000-block chunks if toBlock - fromBlock > 2000. Always store lastScannedBlock as toBlock of the last successful chunk, not the block of the last match.

---

Out of Scope for This PRD

• Trezor-signed sweep (Task #11) — required to make Option B (derived addresses as real destinations) viable
• AML screening on scanner matches (Task #10)
• Per-chain confirmation threshold admin UI (Task #9) — scanner reads thresholds from supportedChains.json; the UI to edit them is Task #9
• TON / non-EVM chain support — scanner is EVM-only; separate work if needed