nick-doc/04 - Flows/Payment Flow - SHKeeper.md

title, tags, related_models, related_apis

title	tags	related_models	related_apis
Payment Flow - SHKeeper	flow payment shkeeper crypto escrow webhook	Payment PurchaseRequest SellerOffer	POST /api/payment/shkeeper/create POST /api/payment/shkeeper/webhook GET /api/payment/shkeeper/status/:id

Payment Flow — SHKeeper (Crypto Pay-In)

[!warning] Historical migration document This page describes the older SHKeeper pay-in rail. It is retained for migration/reconciliation context only. The current primary pay-in path is PRD - Request Network In-House Checkout, and the current escrow/custody model is Escrow Flow plus PRD - Decentralized Custody and Smart-Contract Escrow Roadmap.

End-to-end crypto pay-in via the self-hosted SHKeeper gateway at pay.amn.gg. The buyer pays in stablecoins/crypto; SHKeeper monitors the blockchain, sends webhooks back, and the backend marks the escrow funded.

Supported assets

Pulled from env: SHKEEPER_NETWORKS and SHKEEPER_ALLOWED_TOKENS (shkeeperService.ts:97-98).

• Networks: bsc, ethereum (default in code).
• Tokens: USDT, USDC (default). The endpoint URL is built as https://pay.amn.gg/api/v1/{NETWORK_PREFIX}-{TOKEN}/payment_request (shkeeperService.ts:413-417).
  • BSC → BNB-USDT, BNB-USDC (i.e. BEP-20).
  • Ethereum → ETH-USDT, ETH-USDC (ERC-20).
• TRC-20 (USDT-TRC20) and native BTC are mentioned in the task brief but not currently wired in shkeeperService.ts — only BSC/ETH variants are produced from the code path. Verify SHKeeper-side configuration if those are required.

Actors

• Buyer — pays.
• Seller — passive in this flow; gets notified on success.
• Frontend — checkout components under frontend/src/sections/request/components/buyer-steps/step-3-components/ and frontend/src/sections/payment/.
• Backend — shkeeperService.createPayInIntent (backend/src/services/payment/shkeeper/shkeeperService.ts:48-533) and shkeeperWebhook.handleShkeeperWebhook (backend/src/services/payment/shkeeper/shkeeperWebhook.ts).
• SHKeeper gateway (https://pay.amn.gg) — issues per-payment deposit addresses, watches the chain, sends webhooks.
• Blockchain — BSC / Ethereum.
• PaymentCoordinator (backend/src/services/payment/paymentCoordinator.ts) — serialises concurrent payment-status updates from multiple sources (webhook, wallet monitor, manual confirm).
• MongoDB — payments, purchaserequests, selleroffers, chats, notifications.
• Redis — paymentRedisService (wallet-address cache, 2 h TTL).
• Socket.IO — payment-created, seller-offer-update, purchase-request-update.

Preconditions

• Buyer has selected an offer (or is using the template-checkout shortcut).
• Backend env: SHKEEPER_API_URL, SHKEEPER_API_KEY, SHKEEPER_WEBHOOK_SECRET, API_URL (for the callback URL).
• Redis is reachable (graceful degradation if not — see app.ts:361-367).
• SHKeeper has wallets provisioned for each crypto_name.

Payment state machine

stateDiagram-v2
    [*] --> pending: createPayInIntent\n(Payment.status="pending")
    pending --> pending_partial: webhook PARTIAL\nescrowState="partial"
    pending --> completed: webhook PAID/OVERPAID\nescrowState="funded"
    pending --> failed: webhook EXPIRED/CANCELLED\nescrowState="cancelled"
    pending_partial --> completed: top-up arrives, total ≥ amount
    pending_partial --> failed: expires
    completed --> released: admin release → seller payout\n[[Payout Flow]]
    completed --> refunded: dispute resolution → buyer refund
    refunded --> [*]
    released --> [*]
    failed --> [*]

Step-by-step narrative

Phase 1 — Create intent

1. Buyer clicks "Pay" on the chosen offer (/dashboard/buyer/requests/{id} → step-3-select-and-pay).
2. Frontend POSTs POST /api/payment/shkeeper/create with { purchaseRequestId, sellerOfferId, amount, token?, network? }.
3. Backend createPayInIntent:
   • Validates ObjectIds (shkeeperService.ts:55-71). Special path for template checkout (string IDs starting with template-checkout-).
   • Duplicate-guard #1 (:75-116): if there is already an active Payment (status ∈ {pending, processing}) for the same {purchaseRequestId, sellerOfferId, buyerId}, the existing record is reused — same paymentUrl, no new wallet allocation.
   • Duplicate-guard #2 (template checkout) (:131-198): if a recent completed/confirmed payment exists for the same buyer + template session, reuse it; otherwise dedupe pending records within the last 5 minutes.
   • Upsert (:218-249): atomic Payment.findOneAndUpdate(filter, {$setOnInsert: {...}}, {upsert: true, new: true}) keyed by {buyerId, purchaseRequestId, provider, direction:'in', status:'pending'} — prevents race-condition duplicates.
   • Sets Payment.providerPaymentId = externalId. For template checkouts, externalId = template-{ts}-{rand}; otherwise it's the Payment._id.
   • Wallet cache lookup (:421-450): paymentRedisService.getCachedWallet(cacheKey) — if a wallet was allocated for the same (amount, token, network, requestId) within the last 2 h, reuse it. Avoids hammering SHKeeper for the same checkout.
   • Call SHKeeper API (:453-475): POST https://pay.amn.gg/api/v1/{cryptoName}/payment_request with header X-Shkeeper-Api-Key. Body: { external_id, fiat: 'USD', amount: '12.34', callback_url: ${API_URL}/api/payment/shkeeper/webhook }. The HTTP call is wrapped by shkeeperFetch (shkeeperHealthCheck.ts) which trips the breaker on repeated failures.
   • Persist response (:484-503): updates Payment.metadata.{shkeeperInvoiceId, shkeeperData, cryptoName, walletAddress}; caches the wallet for 2 h; calls walletMonitor.addWallet(...) so the on-chain watcher can confirm independently (belt-and-braces against missed webhooks).
4. Returns { paymentId, paymentUrl, shkeeperInvoiceId, walletAddress, amount, exchangeRate, displayName, cryptoName }.
5. Emits payment-created globally via emitGlobalEvent (shkeeperService.ts:277-287) so the admin dashboard sees the new pending payment in real time.

Phase 2 — Buyer pays

6. Frontend renders a QR code for ${walletAddress}?amount=${amount}&token=... and shows the exchange-rate-locked USDT amount, recalculate-after timer (recalculate_after from SHKeeper, typically 15 min), and a copy-to-clipboard button.
7. Buyer scans with MetaMask / Trust Wallet / Binance App and sends the on-chain transfer.
8. SHKeeper polls the chain, detects the deposit. When confirmations reach the threshold it marks the invoice PAID (or OVERPAID if the buyer sent extra).

Phase 3 — Webhook

9. SHKeeper POSTs to ${API_URL}/api/payment/shkeeper/webhook with the ShkeeperWebhookPayload shape (shkeeperWebhook.ts:14-37):

   {
     external_id, crypto: "BNB-USDT", addr, fiat: "USD",
     balance_fiat, balance_crypto, paid: true,
     status: "PAID" | "PENDING" | "EXPIRED" | "CANCELLED" | "PARTIAL" | "OVERPAID",
     transactions: [{ txid, date, amount_crypto, amount_fiat, trigger: true, ... }],
     fee_percent, fee_fixed, fee_policy, overpaid_fiat
   }
   

10. Signature verification (shkeeperWebhook.ts:84-120): HMAC-SHA256 of the raw body with SHKEEPER_WEBHOOK_SECRET, header x-shkeeper-signature (also accepts x-signature, signature, x-hub-signature, x-hub-signature-256). Mismatch → 401 in production, allowed in dev. Length-mismatched signature → 401 (avoids timingSafeEqual crash).
11. Fallback auth (:122-141): if no signature header but env requires it, the route accepts X-Shkeeper-Api-Key matching SHKEEPER_API_KEY. Otherwise returns 202 to stop SHKeeper retries even if rejected (idempotency principle: always 2xx unless the request itself is mangled).
12. DB reconnect (:143-155): if Mongoose is disconnected, attempt reconnection. On failure → 202 OK to avoid retry loop, log for investigation.
13. Payment lookup: Payment.findOne({ providerPaymentId: payload.external_id }). If not found and the external_id looks like a template checkout, hand off to handleTemplateCheckoutWebhook (templateCheckoutWebhook.ts). Otherwise → 202 OK with a rate-limited log.
14. Duplicate-webhook detection (:249-296): if metadata.shkeeperStatus, balance_fiat, paid are identical to the previous webhook and less than 10 seconds have passed → return 202 (idempotent). Logged once per minute per payment.
15. Map SHKeeper status → internal status (:387-411):

    SHKeeper	Internal status	escrowState
    PAID	completed	funded
    OVERPAID	completed	funded
    EXPIRED, CANCELLED	failed	cancelled
    PARTIAL	pending	partial
    PENDING	pending	—

16. Extract transactionHash (:311-385) — prefers the transaction with trigger === true, then falls back to the latest, then to fetching from SHKeeper's invoice endpoint if the webhook somehow arrived without transactions.
17. PaymentCoordinator (:482-507) — coordinatePaymentUpdate returns false if another worker already started processing this state change, otherwise executePaymentUpdate writes the new status/escrowState/txHash atomically with metadata.
18. Cascade on PAID/OVERPAID (:543-714):
    • Load PurchaseRequest and SellerOffer for this payment.
    • Mark winning offer accepted: selectedOffer.status = 'accepted'; save().
    • Reject all other offers: SellerOffer.updateMany({ purchaseRequestId, _id: { $ne: sellerOfferId } }, { status: 'rejected' }).
    • Promote request: status = 'payment'; selectedOfferId = sellerOfferId.
    • Create direct chat (chatService.createChat, see Chat Flow).
    • Notifications: notifyPaymentConfirmed (to both parties), notifyOfferAccepted (winner), notifyRequestStatusChanged (received_offers → payment).
    • Socket fan-out: seller-offer-update 'payment-completed' to winner, 'offer-rejected' to losers (each carries offerId, reason).
19. Cleanup: simpleAutoWebhook.removePayment(external_id) stops the simple polling fallback.
20. Always respond 202 Accepted (SHKeeper retries on non-2xx). 200 would cause infinite retries because SHKeeper expects 202 per its convention.

Phase 4 — Frontend reaction

21. The buyer's checkout page subscribes to socket events and polls GET /api/payment/shkeeper/status/{paymentId}. When status flips to completed, the UI transitions to "Payment received" and unlocks the next buyer step (await delivery).
22. The seller's dashboard receives seller-offer-update payment-completed and surfaces the green "Order paid — start preparing" banner.

Sequence diagram

sequenceDiagram
    autonumber
    actor B as Buyer
    participant FE as Frontend
    participant BE as Backend
    participant R as Redis
    participant SK as SHKeeper (pay.amn.gg)
    participant BC as Blockchain
    participant DB as MongoDB
    participant IO as Socket.IO
    actor S as Seller

    B->>FE: Choose offer, click "Pay"
    FE->>BE: POST /api/payment/shkeeper/create
    BE->>DB: dedupe / upsert Payment(status:"pending")
    BE->>R: getCachedWallet(amount, token, network, requestId)
    alt cache hit
        R-->>BE: cached wallet
    else cache miss
        BE->>SK: POST /api/v1/{cryptoName}/payment_request\nX-Shkeeper-Api-Key
        SK-->>BE: { id, wallet, amount, exchange_rate, ... }
        BE->>R: setCachedWallet (TTL 2h)
        BE->>BE: walletMonitor.addWallet (chain watcher)
    end
    BE->>IO: emit 'payment-created' (admin)
    BE-->>FE: { paymentId, walletAddress, amount, QR-ready data }
    FE-->>B: Render QR + countdown + copy address
    B->>BC: Send USDT/USDC to walletAddress
    BC-->>SK: deposit confirmed
    SK->>BE: POST /api/payment/shkeeper/webhook\nx-shkeeper-signature
    BE->>BE: HMAC verify
    BE->>DB: Payment.findOne({providerPaymentId})
    BE->>BE: duplicate-webhook check
    BE->>BE: PaymentCoordinator.coordinate + execute
    BE->>DB: Payment.status="completed"\nescrowState="funded"\nblockchain.transactionHash=...
    BE->>DB: SellerOffer.status="accepted" (others "rejected")
    BE->>DB: PurchaseRequest.status="payment", selectedOfferId
    BE->>DB: Chat.create (buyer + winning seller)
    BE->>IO: emit seller-{winner} 'payment-completed'
    BE->>IO: emit seller-{loser_i} 'offer-rejected'
    BE-->>SK: 202 OK
    IO-->>FE: status updated
    IO-->>S: dashboard updates
    FE-->>B: "Payment received ✓"

API calls

Method	Endpoint	Purpose	Source
POST	/api/payment/shkeeper/create	Create pay-in intent	shkeeperRoutes.ts → shkeeperService.createPayInIntent
POST	/api/payment/shkeeper/webhook	Receive SHKeeper webhook	shkeeperWebhook.handleShkeeperWebhook
GET	/api/payment/shkeeper/status/:paymentId	Frontend polling	shkeeperRoutes.ts
GET	/api/payment/fetch-tx/:paymentId	Manual transaction lookup	paymentRoutes.ts

Database writes

• payments: insert on intent creation (status: 'pending'); update on each webhook (status, escrowState, blockchain.transactionHash, metadata).
• payments (unique index uniq_pending_shkeeper_by_buyer_session, see Payment.ts:181-188): partial unique on {buyerId, purchaseRequestId, provider:'shkeeper', direction:'in', status:'pending'} prevents duplicate pending pay-ins.
• selleroffers: status flipped (accepted / rejected) by the webhook cascade.
• purchaserequests: status → payment, selectedOfferId set.
• chats: a new direct chat (or reuse) — find-or-create via ChatService.createChat.
• notifications: 2–N entries depending on parties.

Socket events emitted

• payment-created (global) — broadcast on intent creation.
• seller-offer-update with eventType: 'payment-completed' → winning seller.
• seller-offer-update with eventType: 'offer-rejected' → each losing seller.
• purchase-request-update with eventType: 'status-changed' (via PurchaseRequestService) → request-{id}.
• new-notification → both buyer and seller.

Side effects

• Redis: walletCache:{amount}_{token}_{network}_{requestId} set for 2 hours.
• Wallet monitor: walletMonitor.addWallet(addr, amount, paymentId, token, network) — the on-chain watcher polls BSC/ETH RPCs directly; if SHKeeper's webhook is lost, the monitor still flips the payment to completed. This is the redundancy mechanism noted in the comments.
• simpleAutoWebhook (shkeeperSimpleAuto.ts): a poll-based fallback that asks SHKeeper for invoice status; cleaned up once a real webhook arrives.
• Webhook stats: webhookStats.recordWebhook(...) updates an in-memory ring buffer surfaced via webhookStats.ts admin endpoint.

Error / edge cases

• Duplicate intent submission → reuse the existing pending payment (no new wallet). UX-safe.
• SHKeeper API unreachable → shkeeperFetch (with circuit breaker) throws; controller returns a demo fallback URL (shkeeperService.ts:520-532). In production this is observed as a Sentry error.
• Webhook signature mismatch in prod → 401, SHKeeper retries — usually the secret has rotated; fix env and they catch up.
• Webhook missing both signature and API key in prod → 202 OK (no-op) to prevent retry storm.
• DB disconnected during webhook → reconnect; on failure 202 OK + log (consider DLQ).
• PARTIAL payment → state held as pending/partial; further deposits to the same address are aggregated by SHKeeper and a new webhook arrives.
• OVERPAID → treated as completed/funded; the overage stays with the platform unless an admin manually refunds (no automatic refund of overpayment today).
• EXPIRED → failed/cancelled. Buyer can re-initiate; the duplicate-guard will create a fresh intent because the old one is no longer pending.
• External_id not found → 202 with rate-limited log; common for orphaned webhooks from old tests.
• Webhook arrives twice within 10 s with same data → idempotency skip → 202.
• PaymentCoordinator deferral → 202 with a "coordinator skipped update" log; the in-flight worker will finish the state change.
• Wallet address reuse — cached for 2 h means two parallel checkouts for the same (amount, token, network, requestId) share one address; whichever pays first wins (acceptable since the duplicate-guard reuses the same Payment doc anyway).
• crypto_name mismatch — only BNB-* and ETH-* are produced; for TRC-20, additional logic is needed in shkeeperService.ts:415.

[!warning] Webhook returns 202 even on errors The handler always responds 2xx to avoid SHKeeper's retry storm — even for unknown payments, signature failures (in non-production paths), DB errors, and unexpected exceptions. Operationally this means failed-to-process webhooks are silently swallowed unless someone tails the logs. Hook the catch-all into Sentry severity = error and alert on webhookStats.errorCount.

[!tip] Manual reconciliation Use fix-transaction-hashes.js at repo root to backfill blockchain.transactionHash for payments where the webhook arrived without transactions. See Payout Flow for the parallel payout-side script usage.

Linked flows

• Purchase Request Flow — supplies the request being paid for.
• Seller Offer Flow — supplies the offer being accepted.
• Payment Flow - DePay & Web3 — alternative direct-wallet route.
• Escrow Flow — what escrowState=funded means downstream.
• Chat Flow — auto-created on success.
• Notification Flow — both parties pinged.
• Payout Flow — pays the seller from the funded escrow.
• Dispute Flow — escape if the order goes wrong.

Source files

• Backend: backend/src/services/payment/shkeeper/shkeeperService.ts (intent creation, ~650 lines)
• Backend: backend/src/services/payment/shkeeper/shkeeperWebhook.ts (webhook handler, ~750 lines)
• Backend: backend/src/services/payment/shkeeper/shkeeperHealthCheck.ts (circuit breaker)
• Backend: backend/src/services/payment/shkeeper/shkeeperRoutes.ts
• Backend: backend/src/services/payment/paymentCoordinator.ts
• Backend: backend/src/services/payment/cleanupPendingPayments.ts (periodic GC)
• Backend: backend/src/services/blockchain/walletMonitor.ts (chain watcher)
• Backend: backend/src/services/redis/paymentRedisService.ts (wallet cache)
• Backend: backend/src/models/Payment.ts
• Frontend: frontend/src/sections/request/components/buyer-steps/step-3-components/step-3-payment.tsx
• Frontend: frontend/src/sections/payment/