docs: PRD for retiring RN API with in-house payment scanner (task #13)
This commit is contained in:
Siavash Sameni
210
PRD - Retire Request Network — In-House Payment Scanner.md
Normal file
210
PRD - Retire Request Network — In-House Payment Scanner.md
Normal file
@@ -0,0 +1,210 @@
|
|||||||
|
# PRD — Retire Request Network: In-House Payment Scanner
|
||||||
|
|
||||||
|
> Status: **Ready for implementation**
|
||||||
|
> Task: #13 (new)
|
||||||
|
> Priority: High
|
||||||
|
> Effort estimate: ~3–4 days (backend scanner + frontend checkout adjustment)
|
||||||
|
> Depends on: Task #8 (done), Task #9 (confirmation thresholds), Task #11 (Trezor sweep — parallel, not blocking)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
The platform currently uses Request Network (RN) as a payment infrastructure middleware. RN's API is called to:
|
||||||
|
|
||||||
|
1. Create a payment request (returns a `requestId` + `salt` used to derive an on-chain `paymentReference`)
|
||||||
|
2. Provide a hosted checkout page (already bypassed by our in-house checkout)
|
||||||
|
3. Deliver a webhook callback when payment is detected on-chain
|
||||||
|
|
||||||
|
The underlying payment mechanism — the `ERC20FeeProxy` smart contract — is **not proprietary to RN**. It is open-source, deployed on all five supported chains (BSC, Arbitrum, Ethereum, Polygon, Base), and already integrated into our in-house checkout. We call it directly today for payment execution.
|
||||||
|
|
||||||
|
This PRD describes replacing RN's API dependency with a self-contained scanner and local reference generator, while continuing to use the same `ERC20FeeProxy` contracts.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## What Changes and What Stays the Same
|
||||||
|
|
||||||
|
### Stays the same
|
||||||
|
|
||||||
|
- `ERC20FeeProxy` contracts on all five chains — no new contract, no deployment, no audit
|
||||||
|
- In-house checkout UI (frontend `rn-in-house-checkout-view.tsx`) — already calls the proxy directly
|
||||||
|
- HD wallet derivation per (buyer, sellerOffer) pair — continues as-is
|
||||||
|
- Payment model, status machine, webhook fanout — unchanged
|
||||||
|
|
||||||
|
### Removed
|
||||||
|
|
||||||
|
- `requestNetworkPaymentAdapter` call to RN's API (`POST /v2/secure-payments` or `/v2/request`)
|
||||||
|
- RN webhook receiver and its signature verification
|
||||||
|
- Dependency on RN's salt/requestId for paymentReference derivation
|
||||||
|
- `REQUEST_NETWORK_API_KEY` — no longer needed
|
||||||
|
|
||||||
|
### Added
|
||||||
|
|
||||||
|
- **Local salt generator** — 8 random bytes, replaces RN's `requestId`
|
||||||
|
- **`paymentReference` generated locally** using the existing `computeOnChainPaymentReference()` formula (already implemented in `paymentReference.ts`)
|
||||||
|
- **Chain scanner service** — background poller per chain; reads `eth_getLogs` for `TransferWithReferenceAndFee` events on the `ERC20FeeProxy` contract; matches against pending payment references in MongoDB
|
||||||
|
- **`ScannerCheckpoint` collection** — one document per chainId, tracks `lastScannedBlock` for crash-safe resume
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
### Payment creation (replaces RN API call)
|
||||||
|
|
||||||
|
```
|
||||||
|
POST /api/payment/request-network/intents
|
||||||
|
→ generate salt locally (crypto.randomBytes(8).hex)
|
||||||
|
→ paymentReference = computeOnChainPaymentReference(orderId, salt, destination)
|
||||||
|
→ store payment with { salt, paymentReference, status: 'pending' }
|
||||||
|
→ return checkout block to frontend (same shape as today)
|
||||||
|
```
|
||||||
|
|
||||||
|
The `orderId` used as the `requestId` substitute can be the MongoDB Payment `_id` (hex string). The formula is identical to what RN uses and what `paymentReference.ts` already implements.
|
||||||
|
|
||||||
|
### On-chain detection (replaces RN webhook)
|
||||||
|
|
||||||
|
```
|
||||||
|
ChainScanner (per chain, runs every 10–30s)
|
||||||
|
→ eth_getLogs({
|
||||||
|
address: ERC20FeeProxy[chainId],
|
||||||
|
topics: [TransferWithReferenceAndFee_topic],
|
||||||
|
fromBlock: checkpoint.lastScannedBlock,
|
||||||
|
toBlock: 'latest'
|
||||||
|
})
|
||||||
|
→ for each log: match paymentReference against pending payments in MongoDB
|
||||||
|
→ if matched and confirmations >= threshold: mark payment 'confirmed'
|
||||||
|
→ update checkpoint.lastScannedBlock = toBlock
|
||||||
|
```
|
||||||
|
|
||||||
|
On startup, replay from `lastScannedBlock - 500` (about 25 minutes on BSC) to catch events missed during downtime.
|
||||||
|
|
||||||
|
### Destination: master wallet vs derived addresses
|
||||||
|
|
||||||
|
Two options, which can be decided independently of this PRD:
|
||||||
|
|
||||||
|
**Option A — Master wallet (simpler, ship now):**
|
||||||
|
Destination in the EVM call remains `0x05E280...` (current behaviour). Scanner matches on `paymentReference`. Funds land in a single wallet. No sweep needed. `derivedDestination` remains metadata-only (for future use). This is the conservative path — zero custody change.
|
||||||
|
|
||||||
|
**Option B — Derived HD addresses (full custody separation):**
|
||||||
|
Destination becomes the derived address per (buyer, sellerOffer) pair. Funds land in unique addresses. Scanner still matches on `paymentReference` (destination does not affect the reference formula). Requires Trezor-signed sweep (Task #11) to consolidate funds. This is the correct long-term architecture but introduces sweep complexity.
|
||||||
|
|
||||||
|
**Recommendation for initial ship: Option A.** Enable Option B after Task #11 (Trezor) is complete.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Benefits
|
||||||
|
|
||||||
|
- **No RN API dependency** — no API key rotation, no RN rate limits, no RN downtime affecting checkouts
|
||||||
|
- **PaymentReference generated locally** — deterministic, auditable, no round-trip to external service before the buyer sees the checkout
|
||||||
|
- **Faster checkout initiation** — removes one external HTTP call from the critical path (RN API call took 500–2000ms in tests)
|
||||||
|
- **HD derived addresses usable as real destinations** — once Trezor sweep is in place, per-(buyer, seller) wallet separation is fully realized
|
||||||
|
- **Full observability** — scanner logs every block scanned, every match, every confirmation count; no black-box webhook
|
||||||
|
- **Cost reduction** — removes RN API subscription cost (if on a paid plan); replaces with RPC cost (~$0–50/month at current transaction volume)
|
||||||
|
- **Reuses audited contracts** — ERC20FeeProxy is RN's own open-source contract, already deployed and used in production; no new smart contract risk
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Risks
|
||||||
|
|
||||||
|
- **Scanner downtime = delayed confirmation** — if the scanner process crashes, payments are not confirmed until it restarts. Mitigated by: checkpoint resume, restart policy in Docker (`restart: always`), alerting on scanner lag.
|
||||||
|
- **RPC reliability** — a flaky public RPC can cause missed blocks. Mitigated by: two RPCs per chain already in `supportedChains.json`, automatic fallback in scanner, and alerting on `eth_getLogs` errors.
|
||||||
|
- **Block reorganisations** — a shallow reorg could temporarily show a payment as confirmed. Mitigated by: confirmation thresholds (Task #9) set conservatively (12 blocks on BSC ≈ 36s).
|
||||||
|
- **Lost RN-hosted-page fallback** — some integrations may still generate `requestNetworkSecurePaymentUrl`. After migration, those URLs are no longer meaningful for new payments. Mitigated by: feature-flag the scanner; keep RN adapter runnable (just disabled) for 30 days post-cutover.
|
||||||
|
- **PaymentReference collision** — 8 bytes = 1-in-18-quintillion per pair. Not a practical risk.
|
||||||
|
- **Existing in-flight RN payments** — payments created before migration have RN-generated references and will not be detected by our scanner unless we also watch RN's webhook during the transition window. Mitigated by: drain existing pending payments before hard cutover, or run both paths in parallel for 24 hours.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Neutral Assessment
|
||||||
|
|
||||||
|
| Dimension | RN-hosted | In-house scanner |
|
||||||
|
|---|---|---|
|
||||||
|
| External dependency | RN API + webhook | RPC providers (two per chain) |
|
||||||
|
| Time to confirmation notification | RN webhook latency (seconds to minutes, opaque) | Scanner poll interval (15–30s, deterministic) |
|
||||||
|
| Cost | RN subscription (if paid) + RPC | RPC only ($0–50/month) |
|
||||||
|
| Operational complexity | Low (RN handles detection) | Medium (scanner process to run + monitor) |
|
||||||
|
| Custody flexibility | Locked to registered merchant wallet | Any address (derived wallets possible) |
|
||||||
|
| Auditability | Depends on RN logs | Full — every block, every match logged locally |
|
||||||
|
| Smart contract risk | RN's contracts (audited) | Same contracts — unchanged |
|
||||||
|
| Development effort | Zero (already integrated) | ~3–4 days |
|
||||||
|
|
||||||
|
Neither approach is categorically superior. The in-house scanner trades operational ownership for custody flexibility and removes an external dependency. The tradeoff is worth taking if: (a) derived HD addresses are a priority, or (b) RN API reliability or cost is a concern, or (c) the team wants full control over the confirmation pipeline.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Acceptance Criteria
|
||||||
|
|
||||||
|
1. A new Payment created via `/api/payment/request-network/intents` does **not** call the RN API — `paymentReference` is generated locally and stored on the Payment record at creation time.
|
||||||
|
2. The frontend checkout block is returned within 300ms of the intent request (no external HTTP dependency).
|
||||||
|
3. Scanner detects a `TransferWithReferenceAndFee` event on BSC within two poll cycles (≤30s) of the transaction being mined.
|
||||||
|
4. Matched payment is marked `confirmed` once `confirmations >= confirmationThreshold` for the chain.
|
||||||
|
5. Scanner resumes from `lastScannedBlock` after a process restart; no event is processed twice (idempotent on `txHash + logIndex`).
|
||||||
|
6. Existing in-flight payments (RN-originated) continue to be processed by the old path for 30 days (parallel run) or until manually drained.
|
||||||
|
7. `REQUEST_NETWORK_API_KEY` is no longer required at runtime; removing it does not break startup.
|
||||||
|
8. Admin dashboard shows scanner lag (current block vs last scanned block) per chain.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Files to Create / Modify
|
||||||
|
|
||||||
|
| File | Change |
|
||||||
|
|---|---|
|
||||||
|
| `backend/src/services/payment/requestNetwork/requestNetworkPayInService.ts` | Replace RN adapter call with local salt + paymentReference generation |
|
||||||
|
| `backend/src/services/payment/scanner/chainScanner.ts` | **CREATE** — `eth_getLogs` polling loop, one instance per chain |
|
||||||
|
| `backend/src/services/payment/scanner/scannerCheckpoint.ts` | **CREATE** — MongoDB model + helpers for `lastScannedBlock` per chain |
|
||||||
|
| `backend/src/services/payment/scanner/index.ts` | **CREATE** — start all chain scanners on app boot |
|
||||||
|
| `backend/src/app.ts` | Call `startAllScanners()` on startup; add `GET /api/admin/rn/scanner/status` route |
|
||||||
|
| `backend/src/models/ScannerCheckpoint.ts` | **CREATE** — Mongoose schema: `{ chainId, lastScannedBlock, lastScannedAt, lastMatchAt }` |
|
||||||
|
| `frontend/src/sections/admin/networks/networks-list-view.tsx` | Add scanner lag column (current block vs checkpoint) per chain |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Implementation Notes for Agent
|
||||||
|
|
||||||
|
### Local paymentReference generation
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// In requestNetworkPayInService.ts, replace the adapter call block with:
|
||||||
|
import crypto from 'crypto';
|
||||||
|
import { computeOnChainPaymentReference } from './paymentReference';
|
||||||
|
|
||||||
|
const salt = crypto.randomBytes(8).toString('hex');
|
||||||
|
const paymentId = new mongoose.Types.ObjectId().toHexString();
|
||||||
|
const destination = derivedDestination?.address || process.env.REQUEST_NETWORK_RECIPIENT_ADDRESS;
|
||||||
|
const paymentReference = computeOnChainPaymentReference(paymentId, salt, destination);
|
||||||
|
|
||||||
|
// Store on the Payment document:
|
||||||
|
// metadata.salt, metadata.paymentReference (the 8-byte hex)
|
||||||
|
// providerPaymentId = paymentId (the generated hex ID)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Scanner event signature
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
// TransferWithReferenceAndFee(address,address,uint256,bytes,uint256,address)
|
||||||
|
const TOPIC = '0xbc8b749850bdc0c5e4a49d50f87ec40dca2acdb70a84e7c6823ccdc7b3b3d4b3';
|
||||||
|
|
||||||
|
// Log decode (ethers v6):
|
||||||
|
const iface = new ethers.Interface([
|
||||||
|
'event TransferWithReferenceAndFee(address token, address to, uint256 amount, bytes indexed paymentReference, uint256 feeAmount, address feeAddress)'
|
||||||
|
]);
|
||||||
|
const decoded = iface.parseLog(log);
|
||||||
|
const ref = decoded.args.paymentReference; // bytes → hex string
|
||||||
|
```
|
||||||
|
|
||||||
|
### Scanner idempotency key
|
||||||
|
|
||||||
|
Match on `{ 'metadata.paymentReference': ref }` in MongoDB. When marking confirmed, use `findOneAndUpdate` with `{ $set: { status: 'confirmed' } }` — safe to call twice; second call is a no-op because status is already confirmed.
|
||||||
|
|
||||||
|
### RPC eth_getLogs batch limit
|
||||||
|
|
||||||
|
BSC and most chains cap `eth_getLogs` to 2000–5000 blocks per call. Scan in 2000-block chunks if `toBlock - fromBlock > 2000`. Always store `lastScannedBlock` as `toBlock` of the last successful chunk, not the block of the last match.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Out of Scope for This PRD
|
||||||
|
|
||||||
|
- Trezor-signed sweep (Task #11) — required to make Option B (derived addresses as real destinations) viable
|
||||||
|
- AML screening on scanner matches (Task #10)
|
||||||
|
- Per-chain confirmation threshold admin UI (Task #9) — scanner reads thresholds from `supportedChains.json`; the UI to edit them is Task #9
|
||||||
|
- TON / non-EVM chain support — scanner is EVM-only; separate work if needed
|
||||||
Reference in New Issue
Block a user