From bee91dd01fd217c51a3c71e79a3d8aeb01a68c05 Mon Sep 17 00:00:00 2001 From: Siavash Sameni Date: Sat, 6 Jun 2026 10:57:44 +0400 Subject: [PATCH] docs: add notification and concurrency test procedures --- .../Concurrency and Performance Profile.md | 270 ++++++++++++++++++ .../Escrow Marketplace E2E Procedure.md | 66 ++++- .../Notification Assertion Procedure.md | 160 +++++++++++ 11 - Testing/Test Scenario Catalog.md | 20 +- 11 - Testing/Testing Expansion Backlog.md | 45 ++- 11 - Testing/Testing Overview.md | 7 +- 6 files changed, 552 insertions(+), 16 deletions(-) create mode 100644 11 - Testing/Concurrency and Performance Profile.md create mode 100644 11 - Testing/Notification Assertion Procedure.md diff --git a/11 - Testing/Concurrency and Performance Profile.md b/11 - Testing/Concurrency and Performance Profile.md new file mode 100644 index 0000000..b09f2e6 --- /dev/null +++ b/11 - Testing/Concurrency and Performance Profile.md @@ -0,0 +1,270 @@ +--- +title: Concurrency and Performance Profile +tags: [testing, performance, concurrency, profiling, e2e] +created: 2026-06-06 +--- + +# Concurrency and Performance Profile + +This procedure defines the ramp test for simultaneous escrow E2E flows and the +report format for performance characteristics. + +The purpose is not only load generation. It must prove that business behavior +remains correct under concurrency: payments confirm once, notifications are +issued to the right users, and no request/offer/payment state leaks across +parallel workers. + +## Test Shape + +One worker is one complete isolated E2E flow: + +```text +buyer + sellers -> request -> bids -> accept -> payment intent -> tUSDT payment +-> scanner confirmation -> seller delivery -> buyer confirmation +``` + +Each worker must use unique: + +- run id suffix; +- buyer and seller users; +- purchase request; +- selected offer; +- payment id; +- scanner destination/baseline; +- tx hash or simulated payment fixture, depending on mode. + +Notifications are mandatory inside every worker. See +[[Notification Assertion Procedure]]. + +## Ramp Plan + +Start with one simultaneous worker and double until a stop condition is reached: + +| Stage | Simultaneous workers | Purpose | +|---|---:|---| +| C1 | `1` | Baseline correctness and latency. | +| C2 | `2` | Detect simple race conditions. | +| C4 | `4` | Validate small parallel seller/payment load. | +| C8 | `8` | First meaningful contention check. | +| C16 | `16` | Stress DB/API/socket fanout. | +| C32 | `32` | Upper dev-stack target before release planning. | +| C64+ | `64+` | Only if C32 passes and infrastructure headroom is clear. | + +Hold each stage long enough to complete at least one full E2E round per worker. +For API-only profiling, also support a fixed-duration mode such as 5 minutes per +stage. + +## Modes + +| Mode | Payment behavior | Use | +|---|---|---| +| Live-chain mode | Real BSC Testnet tUSDT transfers | Final confidence at low concurrency; expensive/slower; consumes gas. | +| Scanner fixture mode | Deterministic scanner/balance fixture or controlled test endpoint | High concurrency without chain bottleneck. Must not be enabled in production. | +| API-only dry run | Runs request/offer/delivery and skips payment finalization | Marketplace/notification profiling without chain variables. | + +Live-chain mode should usually stop at low concurrency unless there is enough +tBNB/tUSDT and the chain/RPC is reliable. Higher stages should use scanner +fixture mode once implemented. + +## Metrics To Collect + +### Business correctness + +| Metric | Target | +|---|---| +| completed worker success rate | `100%` for C1-C8, `>= 99%` for C16+ after retries are classified | +| duplicate payment credit count | `0` | +| wrong-recipient notification count | `0` | +| cross-worker state leak count | `0` | +| non-buyer delivery confirmation success | `0` | +| ledger inconsistency count | `0` | + +### API latency + +Initial performance goals for dev profiling: + +| Operation | p50 goal | p95 goal | p99 watch | +|---|---:|---:|---:| +| login | `< 300 ms` | `< 1 s` | `< 2 s` | +| create request | `< 400 ms` | `< 1.5 s` | `< 3 s` | +| create offer | `< 400 ms` | `< 1.5 s` | `< 3 s` | +| accept offer | `< 500 ms` | `< 2 s` | `< 4 s` | +| create payment intent | `< 750 ms` | `< 3 s` | `< 6 s` | +| scanner balance check | `< 1 s` | `< 5 s` | `< 10 s` | +| seller delivery | `< 500 ms` | `< 2 s` | `< 4 s` | +| buyer delivery confirmation | `< 500 ms` | `< 2 s` | `< 4 s` | +| notification visibility | `< 1 s` | `< 5 s` | `< 10 s` | + +These are starting goals, not final SLOs. The first complete C1-C32 run should +produce a baseline report and then adjust targets with evidence. + +### Infrastructure + +Collect per stage: + +- backend CPU and memory; +- frontend CPU and memory; +- scanner CPU and memory; +- MongoDB CPU, memory, connections, slow queries; +- Postgres CPU, memory, connections, locks; +- Redis CPU, memory, connected clients; +- container restarts; +- Docker image/version; +- BSC Testnet RPC latency/error rate; +- Socket.IO connected clients and emitted event count; +- notification insert count and error count. + +Suggested host commands: + +```bash +docker stats --no-stream +docker ps --format '{{.Names}}\t{{.Image}}\t{{.Status}}' +docker logs --since 5m escrow-backend +docker logs --since 5m escrow-scanner +``` + +Do not paste secrets from environment output into reports. + +## Stop Conditions + +Stop the ramp immediately if any P0 condition appears: + +- payment marked paid without correct chain/token/destination/amount evidence; +- duplicate ledger credit; +- notification delivered to wrong user; +- expected notification missing for a step without approved known-gap classification; +- backend, scanner, Mongo, Postgres, or Redis container restarts; +- sustained HTTP 5xx rate above `1%`; +- p95 create payment intent exceeds `10 s` for two consecutive stages; +- scanner confirmation/check p95 exceeds `30 s` outside known BSC Testnet RPC issues; +- queue/backlog grows without draining after the stage ends; +- host CPU remains above `85%` or memory above `90%` after cooldown. + +## Stage Procedure + +For each stage: + +1. Verify dev stack health. +2. Capture container stats baseline. +3. Create isolated worker test data. +4. Start all workers at a barrier time. +5. For every worker, execute full E2E and notification assertions. +6. Capture per-operation timings. +7. Capture infrastructure metrics during run. +8. Wait for queues/notifications to settle. +9. Capture cooldown metrics. +10. Classify failures: + - product bug; + - test data/setup bug; + - BSC Testnet/RPC external issue; + - infrastructure capacity issue; + - known product gap. +11. Decide whether to proceed to the next stage. + +## Worker Result Schema + +Each worker should produce a JSON result: + +```json +{ + "workerId": "C8-W03", + "stage": 8, + "runId": "20260606-perf-C8-W03", + "status": "pass", + "buyerUserId": "", + "sellerUserIds": ["", "", ""], + "purchaseRequestId": "", + "selectedOfferId": "", + "paymentId": "", + "txHash": "0x...", + "timingsMs": { + "login": 180, + "createRequest": 420, + "createOffers": 910, + "acceptOffer": 330, + "createPaymentIntent": 850, + "scannerConfirm": 4200, + "sellerDelivery": 380, + "buyerConfirmDelivery": 410, + "total": 12100 + }, + "notifications": [ + { + "step": "seller_offer_created", + "recipient": "buyer", + "observed": true, + "latencyMs": 640 + } + ], + "errors": [] +} +``` + +## Report Template + +Create one report per full ramp: + +```markdown +# Performance Profile Report - + +## Summary + +- Target: +- Backend/frontend/scanner versions: +- Commit SHAs: +- Payment mode: live-chain / scanner fixture / API-only +- Ramp stages completed: +- Overall result: + +## Key Findings + +| Finding | Severity | Evidence | Next action | +|---|---|---|---| + +## Stage Results + +| Stage | Workers | Pass | Fail | p95 total | p95 payment intent | p95 scanner | p95 notification | 5xx rate | +|---|---:|---:|---:|---:|---:|---:|---:|---:| + +## Notification Results + +| Step | Expected | Observed | Missing | Wrong recipient | p95 latency | +|---|---:|---:|---:|---:|---:| + +## Infrastructure + +| Stage | Backend CPU/mem | Scanner CPU/mem | Mongo | Postgres | Redis | Restarts | +|---|---|---|---|---|---|---:| + +## Payment Correctness + +- Duplicate credits: +- Under/overpayment anomalies: +- Scanner mismatches: +- Ledger mismatches: + +## Bottlenecks + +- API: +- Database: +- Scanner: +- Socket/notifications: +- RPC/chain: + +## Decisions + +- Current safe dev concurrency: +- Recommended production target: +- Required fixes before next ramp: +``` + +## Initial Performance Characteristic Hypotheses + +These are the expectations to validate: + +- Request/offer APIs should scale mostly with Mongo/Postgres write throughput. +- Notification latency will become a visible bottleneck before raw API latency if every offer/status change creates individual Mongo inserts and socket emits. +- Scanner live-chain checks are likely bounded by BSC Testnet RPC latency and should be separated from API-only profiling. +- Payment intent creation may become slower if destination derivation, token registry lookup, and scanner registration are serial. +- Socket fanout should be watched at C16+ because each worker has multiple actors and multiple tabs/devices may multiply room membership. + diff --git a/11 - Testing/Escrow Marketplace E2E Procedure.md b/11 - Testing/Escrow Marketplace E2E Procedure.md index 3106c62..375c4b2 100644 --- a/11 - Testing/Escrow Marketplace E2E Procedure.md +++ b/11 - Testing/Escrow Marketplace E2E Procedure.md @@ -18,6 +18,7 @@ or scanner changes. - BSC Testnet wallet has enough tBNB for gas and enough canonical tUSDT. - Backend/scanner chain 97 registry points to `0x109F54Dab34426D5477986b0460aE5dFBA65f022`. - Local wallet private key or mnemonic is stored only in ignored `.env`. +- Notification checks are enabled in the runner. See [[Notification Assertion Procedure]]. ## Actors @@ -31,20 +32,28 @@ or scanner changes. 1. Generate a run id. 2. Admin creates one buyer and at least two sellers. -3. Buyer creates purchase request with a min/max USDT budget. -4. Sellers submit bids inside the budget range. -5. Randomize or vary: +3. Assert initial notification baselines for buyer and sellers. +4. Buyer creates purchase request with a min/max USDT budget. +5. Assert purchase-request notifications for expected sellers. +6. Sellers submit bids inside the budget range. +7. After each seller bid, assert offer notifications for the buyer. +8. Randomize or vary: - bid amount, - delivery timing, - seller note, - delivery method. -6. Buyer selects a bid. -7. Backend creates scanner payment intent. -8. Buyer pays with BSC Testnet tUSDT. -9. Scanner confirms the payment. -10. Seller marks delivery. -11. Buyer confirms delivery. -12. Record current block: flow pauses before automated release policy. +9. Buyer selects a bid. +10. Assert offer-accepted notification for selected seller and offer-rejected/updated notifications for non-selected sellers when implemented. +11. Backend creates scanner payment intent. +12. Assert payment-pending or payment-started notification where implemented; if missing, record as notification coverage gap. +13. Buyer pays with BSC Testnet tUSDT. +14. Scanner confirms the payment. +15. Assert payment-confirmed notifications for buyer and selected seller. +16. Seller marks delivery. +17. Assert delivery notification for buyer. +18. Buyer confirms delivery. +19. Assert delivery-confirmed notification for selected seller and buyer status notification if implemented. +20. Record current block: flow pauses before automated release policy. ## Expected State Transitions @@ -58,6 +67,40 @@ or scanner changes. | Seller delivers | `delivery` | escrow remains held | | Buyer confirms | `delivered`, `deliveryConfirmed=true` | release policy not automatic yet | +## Notification Assertions After Every Step + +Notification verification is mandatory after every state-changing step. A step +is not complete until the test runner has either: + +1. observed the expected notification for every recipient; or +2. recorded a known coverage gap with route/action, expected recipient, and linked issue/backlog item. + +Use [[Notification Assertion Procedure]] for the exact API/socket checks. + +| Step | Expected recipients | Required notification assertion | +|---|---|---| +| Baseline after user creation | buyer, every seller | Capture unread count and latest notification id for each actor before business mutations. | +| Buyer creates request | eligible sellers | Sellers receive a request/new-opportunity notification, or the gap is recorded. | +| Seller submits bid | buyer | Buyer receives a new-offer notification with `relatedId` or action URL pointing to the request/offer. | +| Buyer selects bid | selected seller; non-selected sellers if rejection notices are implemented | Selected seller receives acceptance notification. Non-selected sellers receive rejection/update notification where supported. | +| Payment intent created | buyer | Buyer receives payment-started/pending notification where supported. Missing `pending_payment` coverage is a known notification gap and must be recorded. | +| Scanner confirms payment | buyer, selected seller | Buyer and seller receive payment-confirmed/funded notification. | +| Seller marks delivery | buyer | Buyer receives delivery/proof submitted notification. | +| Buyer confirms delivery | selected seller; buyer if status-change notices are implemented | Seller receives delivery-confirmed notification. | +| Dispute raised, when added | buyer, seller, admin/mediator | All parties receive dispute-created/hold notification. | +| Release/refund, when added | buyer, seller, admin | Funds movement notification is persisted and pushed. | + +Minimum notification evidence per assertion: + +| Evidence | Source | +|---|---| +| unread count before and after | `GET /api/notifications/unread-count` | +| latest notification payload | `GET /api/notifications?limit=5` | +| recipient id/email | authenticated actor profile or test user record | +| notification category/type/title/actionUrl | notification payload | +| socket event, if runner supports it | `new-notification` on `user-` | +| elapsed time from action to notification visibility | runner timestamp | + ## Buyer Request Template Use unique values so live data can be filtered later: @@ -142,6 +185,8 @@ Minimum assertions per round: | Seller delivery returns HTTP 200 | yes | | Buyer confirm delivery returns HTTP 200 | yes | | Final request status is `delivered` | yes | +| Notification assertion executed after each state-changing step | yes | +| Notification gaps are recorded with expected recipient and route/action | yes | ## Reference Execution - 2026-06-06 @@ -204,4 +249,3 @@ Release policy is not complete: - dispute-to-release/refund automation is not complete. Use [[Testing Expansion Backlog]] before extending this scenario into fund release. - diff --git a/11 - Testing/Notification Assertion Procedure.md b/11 - Testing/Notification Assertion Procedure.md new file mode 100644 index 0000000..fc9871c --- /dev/null +++ b/11 - Testing/Notification Assertion Procedure.md @@ -0,0 +1,160 @@ +--- +title: Notification Assertion Procedure +tags: [testing, notifications, e2e, socket-io] +created: 2026-06-06 +--- + +# Notification Assertion Procedure + +Every E2E business step must verify notifications. A test step is incomplete +until notification persistence and, where practical, real-time delivery are +validated. + +Related docs: + +- [[04 - Flows/Notification Flow]] +- [[03 - API Reference/Notification API]] +- [[Escrow Marketplace E2E Procedure]] + +## Principle + +For each state-changing action, assert notifications for the expected recipients. +If the product currently does not emit a notification for that action, the E2E +must record a notification coverage gap. Silent missing notifications should not +be treated as a pass. + +## Notification Channels To Check + +| Channel | Required? | How | +|---|---|---| +| Persistence | Always | `GET /api/notifications`, `GET /api/notifications/unread-count` | +| Socket push | Required when runner has sockets enabled | Listen for `new-notification` on `user-` | +| Badge count sync | Required for read/mark-read steps | `unread-count-update` socket event or count endpoint | +| Email/push | Not required today | Planned digest/push work is not implemented | + +## Baseline Before a Scenario + +For every actor: + +1. Authenticate. +2. Read unread count: + ```http + GET /api/notifications/unread-count + ``` +3. Read latest notifications: + ```http + GET /api/notifications?page=1&limit=5 + ``` +4. Store: + - `baselineUnreadCount`; + - latest notification id; + - timestamp. + +Do this before creating purchase requests or offers so later assertions can +distinguish new notifications from old ones. + +## Per-Step Assertion Algorithm + +After each action: + +1. Identify expected recipients. +2. Poll each recipient's notifications for up to the notification SLA window. +3. Assert at least one new notification matching the action. +4. Assert unread count increased by the expected number, unless the notification + is intentionally auto-read. +5. Assert `actionUrl` or `relatedId` points to the correct request/payment/offer + where the notification type supports it. +6. If socket instrumentation is active, assert the same notification arrived via + `new-notification`. +7. Record `notificationLatencyMs = notification.createdAt - actionCompletedAt` + or runner-observed first-seen time. + +Suggested polling: + +| Target | Value | +|---|---:| +| poll interval | `500 ms` | +| timeout | `10 s` for local/dev API notifications | +| hard failure threshold | `30 s` | + +## Expected Marketplace Notifications + +| Action | Expected recipients | Expected category/type | +|---|---|---| +| Buyer creates request | eligible sellers | `purchase_request` / new request or opportunity | +| Seller submits offer | buyer | `offer` / new offer | +| Buyer accepts offer | selected seller | `offer` / accepted | +| Buyer accepts offer | non-selected sellers | `offer` / rejected or no-longer-selected, when implemented | +| Payment intent created | buyer | `payment` / pending or started, when implemented | +| Scanner confirms payment | buyer, selected seller | `payment` / confirmed or funded | +| Seller marks delivery | buyer | `delivery` / delivery submitted | +| Buyer confirms delivery | selected seller | `delivery` / delivery confirmed | +| Dispute raised | buyer, seller, admin/mediator | `delivery`/`payment`/`system` dispute hold, when implemented | +| Release/refund | buyer, seller | `payment` / release or refund | + +Known current gaps from existing docs: + +- `pending_payment` and `seller_paid` status changes are not covered by + `NotificationService.notifyRequestStatusChanged`. +- Dispute service notification emits are TODO stubs in the dashboard dispute path. + +These known gaps should still be reported by the E2E runner as gaps, not ignored. + +## Evidence Format + +For each step, append a notification assertion record: + +```json +{ + "step": "seller_offer_created", + "recipient": "buyer", + "recipientUserId": "", + "expected": true, + "observed": true, + "latencyMs": 742, + "unreadBefore": 3, + "unreadAfter": 4, + "notificationId": "", + "category": "offer", + "actionUrl": "/dashboard/request/", + "socketObserved": true, + "gap": null +} +``` + +For a known gap: + +```json +{ + "step": "payment_intent_created", + "recipient": "buyer", + "expected": true, + "observed": false, + "gap": "No notification emitted for pending_payment status", + "linkedDoc": "Notification Flow#Purchase request status coverage gap" +} +``` + +## Pass/Fail Rules + +| Condition | Result | +|---|---| +| Expected notification appears within SLA | pass | +| Expected notification appears after SLA but before hard timeout | warning | +| Expected notification never appears and no approved gap exists | fail | +| Notification appears for wrong user | fail | +| Notification exists but has wrong request/payment/offer id | fail | +| Socket missing but persistence exists | warning unless socket coverage is the target | +| Unread count inconsistent with persisted notification | fail | + +## Performance Metrics + +Notification metrics must be included in concurrency profiles: + +- notification persistence latency p50/p95/p99; +- socket delivery latency p50/p95/p99; +- notification failures by action and recipient; +- unread count mismatch rate; +- duplicate notification rate; +- Mongo notification insert error rate. + diff --git a/11 - Testing/Test Scenario Catalog.md b/11 - Testing/Test Scenario Catalog.md index 08c7b8b..34b449f 100644 --- a/11 - Testing/Test Scenario Catalog.md +++ b/11 - Testing/Test Scenario Catalog.md @@ -38,6 +38,8 @@ have an automation owner, a smoke command, and a UAT checklist. | DISPUTE-003 | Admin resolves dispute for buyer and refund path is unblocked | P0 | Not complete | [[Testing Expansion Backlog]] | | CI-001 | Code push builds and deploys backend/frontend/scanner via Woodpecker | P0 | Live used | [[Smoke and Regression Procedure]] | | UI-CHAIN-001 | Checkout shows BSC Testnet, tUSDT contract, and testnet explorer links | P1 | Implemented in frontend `2.8.118` | [[Scanner BSC Testnet Payment Procedure]] | +| NOTIF-E2E-001 | Every E2E state-changing step issues expected notifications | P0 | Designed, needs automation | [[Notification Assertion Procedure]] | +| PERF-CONC-001 | Ramp simultaneous full escrow E2E workers: 1, 2, 4, 8, 16, 32+ | P0 | Designed, needs automation and baseline report | [[Concurrency and Performance Profile]] | | AUTH-001 | Admin-created generated users can log in and execute role actions | P0 | Live used | [[Escrow Marketplace E2E Procedure]] | | CLEANUP-001 | Test users/requests can be identified and excluded from reports | P2 | Designed | [[Test Environment and Data]] | @@ -52,9 +54,24 @@ flowchart LR E --> F["Scanner confirms payment"] F --> G["Seller delivers"] G --> H["Buyer confirms delivery"] - H --> I["Policy boundary: grace, release, or dispute"] + H --> I["Assert notifications after every step"] + I --> J["Policy boundary: grace, release, or dispute"] ``` +## Concurrency Scenario Family + +The concurrency profile runs the same full escrow worker in parallel and doubles +the worker count until a stop condition is reached: + +```text +1 -> 2 -> 4 -> 8 -> 16 -> 32 -> 64 ... +``` + +Each worker executes one isolated buyer/seller/payment/delivery flow with unique +test users, request ids, destination addresses, and payment ids. Notification +assertions remain mandatory inside each worker. See +[[Concurrency and Performance Profile]]. + ## Payment Negative Scenario Families | Family | What to mutate | Expected result | @@ -88,4 +105,3 @@ A scenario is complete when all are true: 4. It has at least one automated test or smoke script where practical. 5. It has a documented live-dev verification result. 6. It names residual risk or product gaps. - diff --git a/11 - Testing/Testing Expansion Backlog.md b/11 - Testing/Testing Expansion Backlog.md index 004e889..6dd4f2e 100644 --- a/11 - Testing/Testing Expansion Backlog.md +++ b/11 - Testing/Testing Expansion Backlog.md @@ -102,6 +102,50 @@ Add Playwright tests for: - payment submitted screen waits for scanner confirmation; - paid socket/poll fallback transitions to success. +## P1 - Notification Assertions + +Turn [[Notification Assertion Procedure]] into reusable E2E helpers: + +- capture per-user notification baseline; +- poll `GET /api/notifications` and `GET /api/notifications/unread-count`; +- optionally attach Socket.IO clients for buyer/seller/admin actors; +- fail on wrong-recipient notifications; +- record known notification gaps with route/action and expected recipient; +- include notification latency in every E2E report. + +Acceptance tests: + +| ID | Expected | +|---|---| +| NOTIF-E2E-001 | New seller offer creates a buyer notification and unread-count increment. | +| NOTIF-E2E-002 | Buyer offer acceptance creates selected seller notification. | +| NOTIF-E2E-003 | Seller delivery creates buyer notification. | +| NOTIF-E2E-004 | Buyer delivery confirmation creates seller notification. | +| NOTIF-E2E-005 | Missing `pending_payment` notification is reported as a known gap, not a silent pass. | + +## P1 - Concurrency and Performance Profile + +Implement [[Concurrency and Performance Profile]] as an executable runner and +report generator: + +- worker abstraction for one isolated buyer/sellers/payment/delivery flow; +- barrier start for simultaneous workers; +- ramp stages `1, 2, 4, 8, 16, 32`; +- JSON result per worker; +- markdown performance profile report per run; +- infrastructure snapshots before/during/after each stage; +- stop-condition enforcement. + +Acceptance tests: + +| ID | Expected | +|---|---| +| PERF-CONC-001 | C1 completes with all notification assertions. | +| PERF-CONC-002 | C2/C4 detect no cross-worker state leakage. | +| PERF-CONC-003 | Report includes p50/p95/p99 API, scanner, notification, and total timings. | +| PERF-CONC-004 | Runner stops the ramp on P0 notification/payment/ledger correctness failure. | +| PERF-CONC-005 | Runner separates live-chain mode from scanner-fixture/API-only modes. | + ## P1 - CI and Deployment Add a deployment smoke checklist artifact after every pipeline: @@ -142,4 +186,3 @@ Add dashboards/log queries for: 5. Add release-policy tests. 6. Resolve dispute route/policy gaps. 7. Add dispute-to-release/refund E2E tests. - diff --git a/11 - Testing/Testing Overview.md b/11 - Testing/Testing Overview.md index e64a4ec..b96ae55 100644 --- a/11 - Testing/Testing Overview.md +++ b/11 - Testing/Testing Overview.md @@ -21,6 +21,8 @@ seller, payment scanner, delivery, payout, dispute, and deployment behavior. | [[Test Scenario Catalog]] | Canonical scenarios we have designed or need to extend. | | [[Escrow Marketplace E2E Procedure]] | Buyer/seller/request/bid/delivery procedure, including the current two-round flow. | | [[Scanner BSC Testnet Payment Procedure]] | BSC Testnet tUSDT scanner payment procedure and failure modes. | +| [[Notification Assertion Procedure]] | Required notification checks after every E2E business step. | +| [[Concurrency and Performance Profile]] | Ramp test design, profiling targets, metrics, and report template. | | [[Smoke and Regression Procedure]] | CLI, CI, and post-deploy smoke checks. | | [[Testing Expansion Backlog]] | Gaps to cover before broader release confidence. | @@ -33,6 +35,7 @@ seller, payment scanner, delivery, payout, dispute, and deployment behavior. | Smoke tests | Fast checks for one deployment target, usually via `BASE_URL`. | Backend scripts | | Browser E2E | Validate user-visible web flows. | Frontend Playwright | | Live dev E2E | Validate real dev deployment, CI image, scanner service, BSC Testnet, and test tokens. | QA/operator | +| Concurrency profile | Ramp simultaneous full-flow workers and measure API, DB, scanner, notification, and chain behavior. | QA/operator + backend | | UAT | Validate product scenarios and acceptance criteria. | Product + QA | ## Golden Path Coverage @@ -47,7 +50,8 @@ The minimum live-dev confidence path is: 6. Scanner confirms the token transfer. 7. Seller delivers. 8. Buyer confirms delivery. -9. Flow pauses at the current product-policy boundary: release/grace/dispute automation is not complete. +9. After every state-changing step, assert notifications for every expected recipient. +10. Flow pauses at the current product-policy boundary: release/grace/dispute automation is not complete. The current live tested version of this path is documented in [[Escrow Marketplace E2E Procedure#Reference execution - 2026-06-06]]. @@ -70,4 +74,3 @@ The current live tested version of this path is documented in - Record enough evidence to reproduce a failure: request id, offer id, payment id, tx hash, chain id, token address, HTTP status, and CI build number. - Treat a passing local test as necessary but not sufficient for scanner/payment work. Payment changes must also be verified against dev after deploy. - Every live-dev test should state what remains untested or blocked. -