From 4f09b1356edac86cf0b9294ab1068a0dc69f1947 Mon Sep 17 00:00:00 2001 From: Siavash Sameni Date: Fri, 29 May 2026 12:26:51 +0400 Subject: [PATCH] docs: PRD for retiring RN API with in-house payment scanner (task #13) --- ...uest Network — In-House Payment Scanner.md | 210 ++++++++++++++++++ 1 file changed, 210 insertions(+) create mode 100644 PRD - Retire Request Network — In-House Payment Scanner.md diff --git a/PRD - Retire Request Network — In-House Payment Scanner.md b/PRD - Retire Request Network — In-House Payment Scanner.md new file mode 100644 index 0000000..6196cf2 --- /dev/null +++ b/PRD - Retire Request Network — In-House Payment Scanner.md @@ -0,0 +1,210 @@ +# PRD — Retire Request Network: In-House Payment Scanner + +> Status: **Ready for implementation** +> Task: #13 (new) +> Priority: High +> Effort estimate: ~3–4 days (backend scanner + frontend checkout adjustment) +> Depends on: Task #8 (done), Task #9 (confirmation thresholds), Task #11 (Trezor sweep — parallel, not blocking) + +--- + +## Context + +The platform currently uses Request Network (RN) as a payment infrastructure middleware. RN's API is called to: + +1. Create a payment request (returns a `requestId` + `salt` used to derive an on-chain `paymentReference`) +2. Provide a hosted checkout page (already bypassed by our in-house checkout) +3. Deliver a webhook callback when payment is detected on-chain + +The underlying payment mechanism — the `ERC20FeeProxy` smart contract — is **not proprietary to RN**. It is open-source, deployed on all five supported chains (BSC, Arbitrum, Ethereum, Polygon, Base), and already integrated into our in-house checkout. We call it directly today for payment execution. + +This PRD describes replacing RN's API dependency with a self-contained scanner and local reference generator, while continuing to use the same `ERC20FeeProxy` contracts. + +--- + +## What Changes and What Stays the Same + +### Stays the same + +- `ERC20FeeProxy` contracts on all five chains — no new contract, no deployment, no audit +- In-house checkout UI (frontend `rn-in-house-checkout-view.tsx`) — already calls the proxy directly +- HD wallet derivation per (buyer, sellerOffer) pair — continues as-is +- Payment model, status machine, webhook fanout — unchanged + +### Removed + +- `requestNetworkPaymentAdapter` call to RN's API (`POST /v2/secure-payments` or `/v2/request`) +- RN webhook receiver and its signature verification +- Dependency on RN's salt/requestId for paymentReference derivation +- `REQUEST_NETWORK_API_KEY` — no longer needed + +### Added + +- **Local salt generator** — 8 random bytes, replaces RN's `requestId` +- **`paymentReference` generated locally** using the existing `computeOnChainPaymentReference()` formula (already implemented in `paymentReference.ts`) +- **Chain scanner service** — background poller per chain; reads `eth_getLogs` for `TransferWithReferenceAndFee` events on the `ERC20FeeProxy` contract; matches against pending payment references in MongoDB +- **`ScannerCheckpoint` collection** — one document per chainId, tracks `lastScannedBlock` for crash-safe resume + +--- + +## Architecture + +### Payment creation (replaces RN API call) + +``` +POST /api/payment/request-network/intents + → generate salt locally (crypto.randomBytes(8).hex) + → paymentReference = computeOnChainPaymentReference(orderId, salt, destination) + → store payment with { salt, paymentReference, status: 'pending' } + → return checkout block to frontend (same shape as today) +``` + +The `orderId` used as the `requestId` substitute can be the MongoDB Payment `_id` (hex string). The formula is identical to what RN uses and what `paymentReference.ts` already implements. + +### On-chain detection (replaces RN webhook) + +``` +ChainScanner (per chain, runs every 10–30s) + → eth_getLogs({ + address: ERC20FeeProxy[chainId], + topics: [TransferWithReferenceAndFee_topic], + fromBlock: checkpoint.lastScannedBlock, + toBlock: 'latest' + }) + → for each log: match paymentReference against pending payments in MongoDB + → if matched and confirmations >= threshold: mark payment 'confirmed' + → update checkpoint.lastScannedBlock = toBlock +``` + +On startup, replay from `lastScannedBlock - 500` (about 25 minutes on BSC) to catch events missed during downtime. + +### Destination: master wallet vs derived addresses + +Two options, which can be decided independently of this PRD: + +**Option A — Master wallet (simpler, ship now):** +Destination in the EVM call remains `0x05E280...` (current behaviour). Scanner matches on `paymentReference`. Funds land in a single wallet. No sweep needed. `derivedDestination` remains metadata-only (for future use). This is the conservative path — zero custody change. + +**Option B — Derived HD addresses (full custody separation):** +Destination becomes the derived address per (buyer, sellerOffer) pair. Funds land in unique addresses. Scanner still matches on `paymentReference` (destination does not affect the reference formula). Requires Trezor-signed sweep (Task #11) to consolidate funds. This is the correct long-term architecture but introduces sweep complexity. + +**Recommendation for initial ship: Option A.** Enable Option B after Task #11 (Trezor) is complete. + +--- + +## Benefits + +- **No RN API dependency** — no API key rotation, no RN rate limits, no RN downtime affecting checkouts +- **PaymentReference generated locally** — deterministic, auditable, no round-trip to external service before the buyer sees the checkout +- **Faster checkout initiation** — removes one external HTTP call from the critical path (RN API call took 500–2000ms in tests) +- **HD derived addresses usable as real destinations** — once Trezor sweep is in place, per-(buyer, seller) wallet separation is fully realized +- **Full observability** — scanner logs every block scanned, every match, every confirmation count; no black-box webhook +- **Cost reduction** — removes RN API subscription cost (if on a paid plan); replaces with RPC cost (~$0–50/month at current transaction volume) +- **Reuses audited contracts** — ERC20FeeProxy is RN's own open-source contract, already deployed and used in production; no new smart contract risk + +--- + +## Risks + +- **Scanner downtime = delayed confirmation** — if the scanner process crashes, payments are not confirmed until it restarts. Mitigated by: checkpoint resume, restart policy in Docker (`restart: always`), alerting on scanner lag. +- **RPC reliability** — a flaky public RPC can cause missed blocks. Mitigated by: two RPCs per chain already in `supportedChains.json`, automatic fallback in scanner, and alerting on `eth_getLogs` errors. +- **Block reorganisations** — a shallow reorg could temporarily show a payment as confirmed. Mitigated by: confirmation thresholds (Task #9) set conservatively (12 blocks on BSC ≈ 36s). +- **Lost RN-hosted-page fallback** — some integrations may still generate `requestNetworkSecurePaymentUrl`. After migration, those URLs are no longer meaningful for new payments. Mitigated by: feature-flag the scanner; keep RN adapter runnable (just disabled) for 30 days post-cutover. +- **PaymentReference collision** — 8 bytes = 1-in-18-quintillion per pair. Not a practical risk. +- **Existing in-flight RN payments** — payments created before migration have RN-generated references and will not be detected by our scanner unless we also watch RN's webhook during the transition window. Mitigated by: drain existing pending payments before hard cutover, or run both paths in parallel for 24 hours. + +--- + +## Neutral Assessment + +| Dimension | RN-hosted | In-house scanner | +|---|---|---| +| External dependency | RN API + webhook | RPC providers (two per chain) | +| Time to confirmation notification | RN webhook latency (seconds to minutes, opaque) | Scanner poll interval (15–30s, deterministic) | +| Cost | RN subscription (if paid) + RPC | RPC only ($0–50/month) | +| Operational complexity | Low (RN handles detection) | Medium (scanner process to run + monitor) | +| Custody flexibility | Locked to registered merchant wallet | Any address (derived wallets possible) | +| Auditability | Depends on RN logs | Full — every block, every match logged locally | +| Smart contract risk | RN's contracts (audited) | Same contracts — unchanged | +| Development effort | Zero (already integrated) | ~3–4 days | + +Neither approach is categorically superior. The in-house scanner trades operational ownership for custody flexibility and removes an external dependency. The tradeoff is worth taking if: (a) derived HD addresses are a priority, or (b) RN API reliability or cost is a concern, or (c) the team wants full control over the confirmation pipeline. + +--- + +## Acceptance Criteria + +1. A new Payment created via `/api/payment/request-network/intents` does **not** call the RN API — `paymentReference` is generated locally and stored on the Payment record at creation time. +2. The frontend checkout block is returned within 300ms of the intent request (no external HTTP dependency). +3. Scanner detects a `TransferWithReferenceAndFee` event on BSC within two poll cycles (≤30s) of the transaction being mined. +4. Matched payment is marked `confirmed` once `confirmations >= confirmationThreshold` for the chain. +5. Scanner resumes from `lastScannedBlock` after a process restart; no event is processed twice (idempotent on `txHash + logIndex`). +6. Existing in-flight payments (RN-originated) continue to be processed by the old path for 30 days (parallel run) or until manually drained. +7. `REQUEST_NETWORK_API_KEY` is no longer required at runtime; removing it does not break startup. +8. Admin dashboard shows scanner lag (current block vs last scanned block) per chain. + +--- + +## Files to Create / Modify + +| File | Change | +|---|---| +| `backend/src/services/payment/requestNetwork/requestNetworkPayInService.ts` | Replace RN adapter call with local salt + paymentReference generation | +| `backend/src/services/payment/scanner/chainScanner.ts` | **CREATE** — `eth_getLogs` polling loop, one instance per chain | +| `backend/src/services/payment/scanner/scannerCheckpoint.ts` | **CREATE** — MongoDB model + helpers for `lastScannedBlock` per chain | +| `backend/src/services/payment/scanner/index.ts` | **CREATE** — start all chain scanners on app boot | +| `backend/src/app.ts` | Call `startAllScanners()` on startup; add `GET /api/admin/rn/scanner/status` route | +| `backend/src/models/ScannerCheckpoint.ts` | **CREATE** — Mongoose schema: `{ chainId, lastScannedBlock, lastScannedAt, lastMatchAt }` | +| `frontend/src/sections/admin/networks/networks-list-view.tsx` | Add scanner lag column (current block vs checkpoint) per chain | + +--- + +## Implementation Notes for Agent + +### Local paymentReference generation + +```typescript +// In requestNetworkPayInService.ts, replace the adapter call block with: +import crypto from 'crypto'; +import { computeOnChainPaymentReference } from './paymentReference'; + +const salt = crypto.randomBytes(8).toString('hex'); +const paymentId = new mongoose.Types.ObjectId().toHexString(); +const destination = derivedDestination?.address || process.env.REQUEST_NETWORK_RECIPIENT_ADDRESS; +const paymentReference = computeOnChainPaymentReference(paymentId, salt, destination); + +// Store on the Payment document: +// metadata.salt, metadata.paymentReference (the 8-byte hex) +// providerPaymentId = paymentId (the generated hex ID) +``` + +### Scanner event signature + +```typescript +// TransferWithReferenceAndFee(address,address,uint256,bytes,uint256,address) +const TOPIC = '0xbc8b749850bdc0c5e4a49d50f87ec40dca2acdb70a84e7c6823ccdc7b3b3d4b3'; + +// Log decode (ethers v6): +const iface = new ethers.Interface([ + 'event TransferWithReferenceAndFee(address token, address to, uint256 amount, bytes indexed paymentReference, uint256 feeAmount, address feeAddress)' +]); +const decoded = iface.parseLog(log); +const ref = decoded.args.paymentReference; // bytes → hex string +``` + +### Scanner idempotency key + +Match on `{ 'metadata.paymentReference': ref }` in MongoDB. When marking confirmed, use `findOneAndUpdate` with `{ $set: { status: 'confirmed' } }` — safe to call twice; second call is a no-op because status is already confirmed. + +### RPC eth_getLogs batch limit + +BSC and most chains cap `eth_getLogs` to 2000–5000 blocks per call. Scan in 2000-block chunks if `toBlock - fromBlock > 2000`. Always store `lastScannedBlock` as `toBlock` of the last successful chunk, not the block of the last match. + +--- + +## Out of Scope for This PRD + +- Trezor-signed sweep (Task #11) — required to make Option B (derived addresses as real destinations) viable +- AML screening on scanner matches (Task #10) +- Per-chain confirmation threshold admin UI (Task #9) — scanner reads thresholds from `supportedChains.json`; the UI to edit them is Task #9 +- TON / non-EVM chain support — scanner is EVM-only; separate work if needed