--- title: Payment Flow - SHKeeper tags: [flow, payment, shkeeper, crypto, escrow, webhook] related_models: ["[[Payment]]", "[[PurchaseRequest]]", "[[SellerOffer]]"] related_apis: ["POST /api/payment/shkeeper/intents", "POST /api/payment/shkeeper/webhook", "POST /api/payment/:id/release", "POST /api/payment/:id/refund"] --- > **Last updated:** 2026-05-29 — aligned with code (see Doc vs Code Audit Report) > [!caution] Audit — 2026-05-29 > This document was reviewed against the live codebase. **3 corrections applied**: (1) the non-existent HTTP polling endpoint has been removed (status updates arrive via socket only), (2) the release/refund/confirm paths have been corrected to remove the erroneous `/shkeeper/` segment, and (3) the intent-creation endpoint corrected from `/shkeeper/create` to `/shkeeper/intents` and parallel stats/export paths documented. # Payment Flow — SHKeeper (Crypto Pay-In) > [!warning] Historical migration document > This page describes the older SHKeeper pay-in rail. It is retained for migration/reconciliation context only. The current primary pay-in path is [[PRD - Request Network In-House Checkout]], and the current escrow/custody model is [[Escrow Flow]] plus [[PRD - Decentralized Custody and Smart-Contract Escrow Roadmap]]. End-to-end **crypto pay-in** via the self-hosted [SHKeeper](https://github.com/vsys-host/shkeeper.io) gateway at `pay.amn.gg`. The buyer pays in stablecoins/crypto; SHKeeper monitors the blockchain, sends webhooks back, and the backend marks the escrow funded. ## Supported assets Pulled from env: `SHKEEPER_NETWORKS` and `SHKEEPER_ALLOWED_TOKENS` (`shkeeperService.ts:97-98`). - **Networks**: `bsc`, `ethereum` (default in code). - **Tokens**: `USDT`, `USDC` (default). The endpoint URL is built as `https://pay.amn.gg/api/v1/{NETWORK_PREFIX}-{TOKEN}/payment_request` (`shkeeperService.ts:413-417`). - BSC → `BNB-USDT`, `BNB-USDC` (i.e. BEP-20). - Ethereum → `ETH-USDT`, `ETH-USDC` (ERC-20). - TRC-20 (`USDT-TRC20`) and native `BTC` are mentioned in the task brief but **not currently wired** in `shkeeperService.ts` — only BSC/ETH variants are produced from the code path. Verify SHKeeper-side configuration if those are required. ## Actors - **Buyer** — pays. - **Seller** — passive in this flow; gets notified on success. - **Frontend** — checkout components under `frontend/src/sections/request/components/buyer-steps/step-3-components/` and `frontend/src/sections/payment/`. - **Backend** — `shkeeperService.createPayInIntent` (`backend/src/services/payment/shkeeper/shkeeperService.ts:48-533`) and `shkeeperWebhook.handleShkeeperWebhook` (`backend/src/services/payment/shkeeper/shkeeperWebhook.ts`). - **SHKeeper gateway** (`https://pay.amn.gg`) — issues per-payment deposit addresses, watches the chain, sends webhooks. - **Blockchain** — BSC / Ethereum. - **PaymentCoordinator** (`backend/src/services/payment/paymentCoordinator.ts`) — serialises concurrent payment-status updates from multiple sources (webhook, wallet monitor, manual confirm). - **MongoDB** — `payments`, `purchaserequests`, `selleroffers`, `chats`, `notifications`. - **Redis** — `paymentRedisService` (wallet-address cache, 2 h TTL). - **Socket.IO** — `payment-created`, `payment-update`, `template-checkout-payment-confirmed`, `seller-offer-update`, `purchase-request-update`. ## Preconditions - Buyer has selected an offer (or is using the template-checkout shortcut). - Backend env: `SHKEEPER_API_URL`, `SHKEEPER_API_KEY`, `SHKEEPER_WEBHOOK_SECRET`, `API_URL` (for the callback URL). - Redis is reachable (graceful degradation if not — see `app.ts:361-367`). - SHKeeper has wallets provisioned for each `crypto_name`. ## Payment state machine ```mermaid stateDiagram-v2 [*] --> pending: createPayInIntent\n(Payment.status="pending") pending --> pending_partial: webhook PARTIAL\nescrowState="partial" pending --> completed: webhook PAID/OVERPAID\nescrowState="funded" pending --> failed: webhook EXPIRED/CANCELLED\nescrowState="cancelled" pending_partial --> completed: top-up arrives, total ≥ amount pending_partial --> failed: expires completed --> released: admin release → seller payout\n[[Payout Flow]] completed --> refunded: dispute resolution → buyer refund refunded --> [*] released --> [*] failed --> [*] ``` ## Step-by-step narrative ### Phase 1 — Create intent 1. Buyer clicks "Pay" on the chosen offer (`/dashboard/buyer/requests/{id}` → step-3-select-and-pay). 2. Frontend POSTs `POST /api/payment/shkeeper/intents` with `{ purchaseRequestId, sellerOfferId, amount, token?, network? }`. 3. Backend `createPayInIntent`: - Validates ObjectIds (`shkeeperService.ts:55-71`). Special path for **template checkout** (string IDs starting with `template-checkout-`). - **Duplicate-guard #1** (`:75-116`): if there is already an active `Payment` (`status ∈ {pending, processing}`) for the same `{purchaseRequestId, sellerOfferId, buyerId}`, the existing record is reused — same `paymentUrl`, no new wallet allocation. - **Duplicate-guard #2 (template checkout)** (`:131-198`): if a recent `completed`/`confirmed` payment exists for the same buyer + template session, reuse it; otherwise dedupe pending records within the last 5 minutes. - **Upsert** (`:218-249`): atomic `Payment.findOneAndUpdate(filter, {$setOnInsert: {...}}, {upsert: true, new: true})` keyed by `{buyerId, purchaseRequestId, provider, direction:'in', status:'pending'}` — prevents race-condition duplicates. - Sets `Payment.providerPaymentId = externalId`. For template checkouts, `externalId = template-{ts}-{rand}`; otherwise it's the `Payment._id`. - **Wallet cache lookup** (`:421-450`): `paymentRedisService.getCachedWallet(cacheKey)` — if a wallet was allocated for the same `(amount, token, network, requestId)` within the last 2 h, reuse it. Avoids hammering SHKeeper for the same checkout. - **Call SHKeeper API** (`:453-475`): `POST https://pay.amn.gg/api/v1/{cryptoName}/payment_request` with header `X-Shkeeper-Api-Key`. Body: `{ external_id, fiat: 'USD', amount: '12.34', callback_url: ${API_URL}/api/payment/shkeeper/webhook }`. The HTTP call is wrapped by `shkeeperFetch` (`shkeeperHealthCheck.ts`) which trips the breaker on repeated failures. - **Persist response** (`:484-503`): updates `Payment.metadata.{shkeeperInvoiceId, shkeeperData, cryptoName, walletAddress}`; caches the wallet for 2 h; calls `walletMonitor.addWallet(...)` so the on-chain watcher can confirm independently (belt-and-braces against missed webhooks). 4. Returns `{ paymentId, paymentUrl, shkeeperInvoiceId, walletAddress, amount, exchangeRate, displayName, cryptoName }`. 5. Emits **`payment-created`** globally via `emitGlobalEvent` (`shkeeperService.ts:277-287`) so the admin dashboard sees the new pending payment in real time. ### Phase 2 — Buyer pays 6. Frontend renders a **QR code** for `${walletAddress}?amount=${amount}&token=...` and shows the exchange-rate-locked USDT amount, recalculate-after timer (`recalculate_after` from SHKeeper, typically 15 min), and a copy-to-clipboard button. 7. Buyer scans with MetaMask / Trust Wallet / Binance App and sends the on-chain transfer. 8. SHKeeper polls the chain, detects the deposit. When confirmations reach the threshold it marks the invoice `PAID` (or `OVERPAID` if the buyer sent extra). ### Phase 3 — Webhook 9. SHKeeper POSTs to `${API_URL}/api/payment/shkeeper/webhook` with the `ShkeeperWebhookPayload` shape (`shkeeperWebhook.ts:14-37`): ``` { external_id, crypto: "BNB-USDT", addr, fiat: "USD", balance_fiat, balance_crypto, paid: true, status: "PAID" | "PENDING" | "EXPIRED" | "CANCELLED" | "PARTIAL" | "OVERPAID", transactions: [{ txid, date, amount_crypto, amount_fiat, trigger: true, ... }], fee_percent, fee_fixed, fee_policy, overpaid_fiat } ``` 10. **Signature verification** (`shkeeperWebhook.ts:84-120`): HMAC-SHA256 of the raw body with `SHKEEPER_WEBHOOK_SECRET`, header `x-shkeeper-signature` (also accepts `x-signature`, `signature`, `x-hub-signature`, `x-hub-signature-256`). Mismatch → `401` in production, allowed in dev. Length-mismatched signature → `401` (avoids `timingSafeEqual` crash). 11. **Fallback auth** (`:122-141`): if no signature header but env requires it, the route accepts `X-Shkeeper-Api-Key` matching `SHKEEPER_API_KEY`. Otherwise returns `202` to **stop SHKeeper retries** even if rejected (idempotency principle: always 2xx unless the request itself is mangled). 12. **DB reconnect** (`:143-155`): if Mongoose is disconnected, attempt reconnection. On failure → `202 OK` to avoid retry loop, log for investigation. 13. **Payment lookup**: `Payment.findOne({ providerPaymentId: payload.external_id })`. If not found and the external_id looks like a template checkout, hand off to `handleTemplateCheckoutWebhook` (`templateCheckoutWebhook.ts`). Otherwise → `202 OK` with a rate-limited log. 14. **Duplicate-webhook detection** (`:249-296`): if `metadata.shkeeperStatus`, `balance_fiat`, `paid` are identical to the previous webhook **and** less than 10 seconds have passed → return `202` (idempotent). Logged once per minute per payment. 15. **Map SHKeeper status → internal status** (`:387-411`): | SHKeeper | Internal `status` | `escrowState` | |---|---|---| | `PAID` | `completed` | `funded` | | `OVERPAID` | `completed` | `funded` | | `EXPIRED`, `CANCELLED` | `failed` | `cancelled` | | `PARTIAL` | `pending` | `partial` | | `PENDING` | `pending` | — | 16. **Extract `transactionHash`** (`:311-385`) — prefers the transaction with `trigger === true`, then falls back to the latest, then to fetching from SHKeeper's invoice endpoint if the webhook somehow arrived without transactions. 17. **PaymentCoordinator** (`:482-507`) — `coordinatePaymentUpdate` returns false if another worker already started processing this state change, otherwise `executePaymentUpdate` writes the new status/escrowState/txHash atomically with metadata. 18. **Cascade on PAID/OVERPAID** (`:543-714`): - Load `PurchaseRequest` and `SellerOffer` for this payment. - **Mark winning offer accepted**: `selectedOffer.status = 'accepted'; save()`. - **Reject all other offers**: `SellerOffer.updateMany({ purchaseRequestId, _id: { $ne: sellerOfferId } }, { status: 'rejected' })`. - **Promote request**: `status = 'payment'; selectedOfferId = sellerOfferId`. - **Create direct chat** (`chatService.createChat`, see [[Chat Flow]]). - **Notifications**: `notifyPaymentConfirmed` (to both parties), `notifyOfferAccepted` (winner), `notifyRequestStatusChanged` (`received_offers → payment`). - **Socket fan-out**: `seller-offer-update` `'payment-completed'` to winner, `'offer-rejected'` to losers (each carries `offerId`, `reason`). 19. **Cleanup**: `simpleAutoWebhook.removePayment(external_id)` stops the simple polling fallback. 20. Always respond `202 Accepted` (SHKeeper retries on non-2xx). `200` would cause infinite retries because SHKeeper expects `202` per its convention. ### Phase 4 — Frontend reaction 21. The buyer's checkout page subscribes to socket events (`payment-update`, `template-checkout-payment-confirmed`). When the status flips to `completed`, the UI transitions to "Payment received" and unlocks the next buyer step (await delivery). > [!warning] No HTTP polling endpoint — socket events only > `GET /api/payment/shkeeper/status/:paymentId` **does not exist** — there is no polling route in `shkeeperRoutes.ts`. Status transitions must be observed via Socket.IO events (`payment-update`, `template-checkout-payment-confirmed`). Any frontend code path that polls this URL will always receive 404. Remove HTTP polling and rely solely on the socket subscription. 22. The seller's dashboard receives `seller-offer-update` `payment-completed` and surfaces the green "Order paid — start preparing" banner. ## Sequence diagram ```mermaid sequenceDiagram autonumber actor B as Buyer participant FE as Frontend participant BE as Backend participant R as Redis participant SK as SHKeeper (pay.amn.gg) participant BC as Blockchain participant DB as MongoDB participant IO as Socket.IO actor S as Seller B->>FE: Choose offer, click "Pay" FE->>BE: POST /api/payment/shkeeper/intents BE->>DB: dedupe / upsert Payment(status:"pending") BE->>R: getCachedWallet(amount, token, network, requestId) alt cache hit R-->>BE: cached wallet else cache miss BE->>SK: POST /api/v1/{cryptoName}/payment_request\nX-Shkeeper-Api-Key SK-->>BE: { id, wallet, amount, exchange_rate, ... } BE->>R: setCachedWallet (TTL 2h) BE->>BE: walletMonitor.addWallet (chain watcher) end BE->>IO: emit 'payment-created' (admin) BE-->>FE: { paymentId, walletAddress, amount, QR-ready data } FE-->>B: Render QR + countdown + copy address B->>BC: Send USDT/USDC to walletAddress BC-->>SK: deposit confirmed SK->>BE: POST /api/payment/shkeeper/webhook\nx-shkeeper-signature BE->>BE: HMAC verify BE->>DB: Payment.findOne({providerPaymentId}) BE->>BE: duplicate-webhook check BE->>BE: PaymentCoordinator.coordinate + execute BE->>DB: Payment.status="completed"\nescrowState="funded"\nblockchain.transactionHash=... BE->>DB: SellerOffer.status="accepted" (others "rejected") BE->>DB: PurchaseRequest.status="payment", selectedOfferId BE->>DB: Chat.create (buyer + winning seller) BE->>IO: emit seller-{winner} 'payment-completed' BE->>IO: emit seller-{loser_i} 'offer-rejected' BE-->>SK: 202 OK IO-->>FE: payment-update / status updated IO-->>S: dashboard updates FE-->>B: "Payment received ✓" ``` ## API calls | Method | Endpoint | Purpose | Auth | Source | |---|---|---|---|---| | `POST` | `/api/payment/shkeeper/intents` | Create pay-in intent | Bearer JWT (buyer) | `shkeeperRoutes.ts` → `shkeeperService.createPayInIntent` | | `POST` | `/api/payment/shkeeper/webhook` | Receive SHKeeper webhook | HMAC / API key | `shkeeperWebhook.handleShkeeperWebhook` | | `POST` | `/api/payment/:id/release` | Release escrow to seller | Bearer JWT | `paymentRoutes.ts` | | `POST` | `/api/payment/:id/release/confirm` | Confirm escrow release | Bearer JWT | `paymentRoutes.ts` | | `POST` | `/api/payment/:id/refund` | Refund to buyer | Bearer JWT | `paymentRoutes.ts` | | `POST` | `/api/payment/:id/refund/confirm` | Confirm buyer refund | Bearer JWT | `paymentRoutes.ts` | | `POST` | `/api/payment/payments/:id/fetch-tx` | Manual transaction lookup | Bearer JWT | `paymentRoutes.ts` | | `GET` | `/api/payment/payments/stats` | Payment statistics (admin-gated strict) | Bearer JWT + admin role | `paymentRoutes.ts` | | `GET` | `/api/payment/stats` | Payment statistics (no admin guard) | Bearer JWT | `paymentRoutes.ts` | | ~~`GET /api/payment/shkeeper/status/:paymentId`~~ | | **404 — does not exist.** Use socket events instead. | — | — | > [!note] Two parallel stats paths > Two separate stats endpoints exist with different auth levels: > - `GET /api/payment/payments/stats` — admin-gated (strict role check); intended for admin dashboard. > - `GET /api/payment/stats` — authenticated but no admin guard; accessible to any logged-in user. > Similarly, export endpoints exist at two paths with different auth levels. Confirm which is appropriate for each consumer before wiring the frontend. > [!warning] Release/refund path correction > Previously documented paths included a `/shkeeper/` segment that does **not** exist in the router: > - ~~`POST /api/payment/shkeeper/:id/release`~~ → correct: `POST /api/payment/:id/release` > - ~~`POST /api/payment/shkeeper/:id/release/confirm`~~ → correct: `POST /api/payment/:id/release/confirm` > - ~~`POST /api/payment/shkeeper/:id/refund`~~ → correct: `POST /api/payment/:id/refund` > - ~~`POST /api/payment/shkeeper/:id/refund/confirm`~~ → correct: `POST /api/payment/:id/refund/confirm` > > The `/shkeeper/` infix never existed on release/refund routes. These are generic payment lifecycle endpoints shared across all providers. ## Database writes - **`payments`**: insert on intent creation (`status: 'pending'`); update on each webhook (status, escrowState, blockchain.transactionHash, metadata). - **`payments`** (unique index `uniq_pending_shkeeper_by_buyer_session`, see `Payment.ts:181-188`): partial unique on `{buyerId, purchaseRequestId, provider:'shkeeper', direction:'in', status:'pending'}` prevents duplicate pending pay-ins. - **`selleroffers`**: `status` flipped (`accepted` / `rejected`) by the webhook cascade. - **`purchaserequests`**: `status` → `payment`, `selectedOfferId` set. - **`chats`**: a new `direct` chat (or reuse) — find-or-create via `ChatService.createChat`. - **`notifications`**: 2–N entries depending on parties. ## Socket events emitted - **`payment-created`** (global) — broadcast on intent creation. - **`payment-update`** — status change notifications to the buyer's checkout page. - **`template-checkout-payment-confirmed`** — for template checkout flows. - **`seller-offer-update`** with `eventType: 'payment-completed'` → winning seller. - **`seller-offer-update`** with `eventType: 'offer-rejected'` → each losing seller. - **`purchase-request-update`** with `eventType: 'status-changed'` (via `PurchaseRequestService`) → `request-{id}`. - **`new-notification`** → both buyer and seller. ## Side effects - **Redis**: `walletCache:{amount}_{token}_{network}_{requestId}` set for 2 hours. - **Wallet monitor**: `walletMonitor.addWallet(addr, amount, paymentId, token, network)` — the on-chain watcher polls BSC/ETH RPCs directly; if SHKeeper's webhook is lost, the monitor still flips the payment to `completed`. This is the redundancy mechanism noted in the comments. - **simpleAutoWebhook** (`shkeeperSimpleAuto.ts`): a poll-based fallback that asks SHKeeper for invoice status; cleaned up once a real webhook arrives. - **Webhook stats**: `webhookStats.recordWebhook(...)` updates an in-memory ring buffer surfaced via `webhookStats.ts` admin endpoint. ## Error / edge cases - **Duplicate intent submission** → reuse the existing pending payment (no new wallet). UX-safe. - **SHKeeper API unreachable** → `shkeeperFetch` (with circuit breaker) throws; controller returns a **demo fallback** URL (`shkeeperService.ts:520-532`). In production this is observed as a Sentry error. - **Webhook signature mismatch in prod** → `401`, SHKeeper retries — usually the secret has rotated; fix env and they catch up. - **Webhook missing both signature and API key in prod** → `202 OK` (no-op) to prevent retry storm. - **DB disconnected during webhook** → reconnect; on failure `202 OK` + log (consider DLQ). - **`PARTIAL` payment** → state held as `pending/partial`; further deposits to the same address are aggregated by SHKeeper and a new webhook arrives. - **`OVERPAID`** → treated as `completed/funded`; the overage stays with the platform unless an admin manually refunds (no automatic refund of overpayment today). - **`EXPIRED`** → `failed/cancelled`. Buyer can re-initiate; the duplicate-guard will create a fresh intent because the old one is no longer `pending`. - **External_id not found** → `202` with rate-limited log; common for orphaned webhooks from old tests. - **Webhook arrives twice within 10 s with same data** → idempotency skip → `202`. - **`PaymentCoordinator` deferral** → `202` with a "coordinator skipped update" log; the in-flight worker will finish the state change. - **Wallet address reuse** — cached for 2 h means two parallel checkouts for the same `(amount, token, network, requestId)` share one address; whichever pays first wins (acceptable since the duplicate-guard reuses the same `Payment` doc anyway). - **`crypto_name` mismatch** — only `BNB-*` and `ETH-*` are produced; for TRC-20, additional logic is needed in `shkeeperService.ts:415`. > [!warning] Webhook returns 202 even on errors > The handler always responds 2xx to avoid SHKeeper's retry storm — even for unknown payments, signature failures (in non-production paths), DB errors, and unexpected exceptions. Operationally this means failed-to-process webhooks are silently swallowed unless someone tails the logs. Hook the catch-all into Sentry severity = `error` and alert on `webhookStats.errorCount`. > [!tip] Manual reconciliation > Use `fix-transaction-hashes.js` at repo root to backfill `blockchain.transactionHash` for payments where the webhook arrived without transactions. See [[Payout Flow]] for the parallel payout-side script usage. ## Linked flows - [[Purchase Request Flow]] — supplies the request being paid for. - [[Seller Offer Flow]] — supplies the offer being accepted. - [[Payment Flow - DePay & Web3]] — alternative direct-wallet route. - [[Escrow Flow]] — what `escrowState=funded` means downstream. - [[Chat Flow]] — auto-created on success. - [[Notification Flow]] — both parties pinged. - [[Payout Flow]] — pays the seller from the funded escrow. - [[Dispute Flow]] — escape if the order goes wrong. ## Source files - Backend: `backend/src/services/payment/shkeeper/shkeeperService.ts` (intent creation, ~650 lines) - Backend: `backend/src/services/payment/shkeeper/shkeeperWebhook.ts` (webhook handler, ~750 lines) - Backend: `backend/src/services/payment/shkeeper/shkeeperHealthCheck.ts` (circuit breaker) - Backend: `backend/src/services/payment/shkeeper/shkeeperRoutes.ts` - Backend: `backend/src/services/payment/paymentCoordinator.ts` - Backend: `backend/src/services/payment/cleanupPendingPayments.ts` (periodic GC) - Backend: `backend/src/services/blockchain/walletMonitor.ts` (chain watcher) - Backend: `backend/src/services/redis/paymentRedisService.ts` (wallet cache) - Backend: `backend/src/models/Payment.ts` - Frontend: `frontend/src/sections/request/components/buyer-steps/step-3-components/step-3-payment.tsx` - Frontend: `frontend/src/sections/payment/`