docs: add notification and concurrency test procedures
This commit is contained in:
270
11 - Testing/Concurrency and Performance Profile.md
Normal file
270
11 - Testing/Concurrency and Performance Profile.md
Normal file
@@ -0,0 +1,270 @@
|
||||
---
|
||||
title: Concurrency and Performance Profile
|
||||
tags: [testing, performance, concurrency, profiling, e2e]
|
||||
created: 2026-06-06
|
||||
---
|
||||
|
||||
# Concurrency and Performance Profile
|
||||
|
||||
This procedure defines the ramp test for simultaneous escrow E2E flows and the
|
||||
report format for performance characteristics.
|
||||
|
||||
The purpose is not only load generation. It must prove that business behavior
|
||||
remains correct under concurrency: payments confirm once, notifications are
|
||||
issued to the right users, and no request/offer/payment state leaks across
|
||||
parallel workers.
|
||||
|
||||
## Test Shape
|
||||
|
||||
One worker is one complete isolated E2E flow:
|
||||
|
||||
```text
|
||||
buyer + sellers -> request -> bids -> accept -> payment intent -> tUSDT payment
|
||||
-> scanner confirmation -> seller delivery -> buyer confirmation
|
||||
```
|
||||
|
||||
Each worker must use unique:
|
||||
|
||||
- run id suffix;
|
||||
- buyer and seller users;
|
||||
- purchase request;
|
||||
- selected offer;
|
||||
- payment id;
|
||||
- scanner destination/baseline;
|
||||
- tx hash or simulated payment fixture, depending on mode.
|
||||
|
||||
Notifications are mandatory inside every worker. See
|
||||
[[Notification Assertion Procedure]].
|
||||
|
||||
## Ramp Plan
|
||||
|
||||
Start with one simultaneous worker and double until a stop condition is reached:
|
||||
|
||||
| Stage | Simultaneous workers | Purpose |
|
||||
|---|---:|---|
|
||||
| C1 | `1` | Baseline correctness and latency. |
|
||||
| C2 | `2` | Detect simple race conditions. |
|
||||
| C4 | `4` | Validate small parallel seller/payment load. |
|
||||
| C8 | `8` | First meaningful contention check. |
|
||||
| C16 | `16` | Stress DB/API/socket fanout. |
|
||||
| C32 | `32` | Upper dev-stack target before release planning. |
|
||||
| C64+ | `64+` | Only if C32 passes and infrastructure headroom is clear. |
|
||||
|
||||
Hold each stage long enough to complete at least one full E2E round per worker.
|
||||
For API-only profiling, also support a fixed-duration mode such as 5 minutes per
|
||||
stage.
|
||||
|
||||
## Modes
|
||||
|
||||
| Mode | Payment behavior | Use |
|
||||
|---|---|---|
|
||||
| Live-chain mode | Real BSC Testnet tUSDT transfers | Final confidence at low concurrency; expensive/slower; consumes gas. |
|
||||
| Scanner fixture mode | Deterministic scanner/balance fixture or controlled test endpoint | High concurrency without chain bottleneck. Must not be enabled in production. |
|
||||
| API-only dry run | Runs request/offer/delivery and skips payment finalization | Marketplace/notification profiling without chain variables. |
|
||||
|
||||
Live-chain mode should usually stop at low concurrency unless there is enough
|
||||
tBNB/tUSDT and the chain/RPC is reliable. Higher stages should use scanner
|
||||
fixture mode once implemented.
|
||||
|
||||
## Metrics To Collect
|
||||
|
||||
### Business correctness
|
||||
|
||||
| Metric | Target |
|
||||
|---|---|
|
||||
| completed worker success rate | `100%` for C1-C8, `>= 99%` for C16+ after retries are classified |
|
||||
| duplicate payment credit count | `0` |
|
||||
| wrong-recipient notification count | `0` |
|
||||
| cross-worker state leak count | `0` |
|
||||
| non-buyer delivery confirmation success | `0` |
|
||||
| ledger inconsistency count | `0` |
|
||||
|
||||
### API latency
|
||||
|
||||
Initial performance goals for dev profiling:
|
||||
|
||||
| Operation | p50 goal | p95 goal | p99 watch |
|
||||
|---|---:|---:|---:|
|
||||
| login | `< 300 ms` | `< 1 s` | `< 2 s` |
|
||||
| create request | `< 400 ms` | `< 1.5 s` | `< 3 s` |
|
||||
| create offer | `< 400 ms` | `< 1.5 s` | `< 3 s` |
|
||||
| accept offer | `< 500 ms` | `< 2 s` | `< 4 s` |
|
||||
| create payment intent | `< 750 ms` | `< 3 s` | `< 6 s` |
|
||||
| scanner balance check | `< 1 s` | `< 5 s` | `< 10 s` |
|
||||
| seller delivery | `< 500 ms` | `< 2 s` | `< 4 s` |
|
||||
| buyer delivery confirmation | `< 500 ms` | `< 2 s` | `< 4 s` |
|
||||
| notification visibility | `< 1 s` | `< 5 s` | `< 10 s` |
|
||||
|
||||
These are starting goals, not final SLOs. The first complete C1-C32 run should
|
||||
produce a baseline report and then adjust targets with evidence.
|
||||
|
||||
### Infrastructure
|
||||
|
||||
Collect per stage:
|
||||
|
||||
- backend CPU and memory;
|
||||
- frontend CPU and memory;
|
||||
- scanner CPU and memory;
|
||||
- MongoDB CPU, memory, connections, slow queries;
|
||||
- Postgres CPU, memory, connections, locks;
|
||||
- Redis CPU, memory, connected clients;
|
||||
- container restarts;
|
||||
- Docker image/version;
|
||||
- BSC Testnet RPC latency/error rate;
|
||||
- Socket.IO connected clients and emitted event count;
|
||||
- notification insert count and error count.
|
||||
|
||||
Suggested host commands:
|
||||
|
||||
```bash
|
||||
docker stats --no-stream
|
||||
docker ps --format '{{.Names}}\t{{.Image}}\t{{.Status}}'
|
||||
docker logs --since 5m escrow-backend
|
||||
docker logs --since 5m escrow-scanner
|
||||
```
|
||||
|
||||
Do not paste secrets from environment output into reports.
|
||||
|
||||
## Stop Conditions
|
||||
|
||||
Stop the ramp immediately if any P0 condition appears:
|
||||
|
||||
- payment marked paid without correct chain/token/destination/amount evidence;
|
||||
- duplicate ledger credit;
|
||||
- notification delivered to wrong user;
|
||||
- expected notification missing for a step without approved known-gap classification;
|
||||
- backend, scanner, Mongo, Postgres, or Redis container restarts;
|
||||
- sustained HTTP 5xx rate above `1%`;
|
||||
- p95 create payment intent exceeds `10 s` for two consecutive stages;
|
||||
- scanner confirmation/check p95 exceeds `30 s` outside known BSC Testnet RPC issues;
|
||||
- queue/backlog grows without draining after the stage ends;
|
||||
- host CPU remains above `85%` or memory above `90%` after cooldown.
|
||||
|
||||
## Stage Procedure
|
||||
|
||||
For each stage:
|
||||
|
||||
1. Verify dev stack health.
|
||||
2. Capture container stats baseline.
|
||||
3. Create isolated worker test data.
|
||||
4. Start all workers at a barrier time.
|
||||
5. For every worker, execute full E2E and notification assertions.
|
||||
6. Capture per-operation timings.
|
||||
7. Capture infrastructure metrics during run.
|
||||
8. Wait for queues/notifications to settle.
|
||||
9. Capture cooldown metrics.
|
||||
10. Classify failures:
|
||||
- product bug;
|
||||
- test data/setup bug;
|
||||
- BSC Testnet/RPC external issue;
|
||||
- infrastructure capacity issue;
|
||||
- known product gap.
|
||||
11. Decide whether to proceed to the next stage.
|
||||
|
||||
## Worker Result Schema
|
||||
|
||||
Each worker should produce a JSON result:
|
||||
|
||||
```json
|
||||
{
|
||||
"workerId": "C8-W03",
|
||||
"stage": 8,
|
||||
"runId": "20260606-perf-C8-W03",
|
||||
"status": "pass",
|
||||
"buyerUserId": "<id>",
|
||||
"sellerUserIds": ["<id>", "<id>", "<id>"],
|
||||
"purchaseRequestId": "<uuid>",
|
||||
"selectedOfferId": "<uuid>",
|
||||
"paymentId": "<uuid>",
|
||||
"txHash": "0x...",
|
||||
"timingsMs": {
|
||||
"login": 180,
|
||||
"createRequest": 420,
|
||||
"createOffers": 910,
|
||||
"acceptOffer": 330,
|
||||
"createPaymentIntent": 850,
|
||||
"scannerConfirm": 4200,
|
||||
"sellerDelivery": 380,
|
||||
"buyerConfirmDelivery": 410,
|
||||
"total": 12100
|
||||
},
|
||||
"notifications": [
|
||||
{
|
||||
"step": "seller_offer_created",
|
||||
"recipient": "buyer",
|
||||
"observed": true,
|
||||
"latencyMs": 640
|
||||
}
|
||||
],
|
||||
"errors": []
|
||||
}
|
||||
```
|
||||
|
||||
## Report Template
|
||||
|
||||
Create one report per full ramp:
|
||||
|
||||
```markdown
|
||||
# Performance Profile Report - <date>
|
||||
|
||||
## Summary
|
||||
|
||||
- Target:
|
||||
- Backend/frontend/scanner versions:
|
||||
- Commit SHAs:
|
||||
- Payment mode: live-chain / scanner fixture / API-only
|
||||
- Ramp stages completed:
|
||||
- Overall result:
|
||||
|
||||
## Key Findings
|
||||
|
||||
| Finding | Severity | Evidence | Next action |
|
||||
|---|---|---|---|
|
||||
|
||||
## Stage Results
|
||||
|
||||
| Stage | Workers | Pass | Fail | p95 total | p95 payment intent | p95 scanner | p95 notification | 5xx rate |
|
||||
|---|---:|---:|---:|---:|---:|---:|---:|---:|
|
||||
|
||||
## Notification Results
|
||||
|
||||
| Step | Expected | Observed | Missing | Wrong recipient | p95 latency |
|
||||
|---|---:|---:|---:|---:|---:|
|
||||
|
||||
## Infrastructure
|
||||
|
||||
| Stage | Backend CPU/mem | Scanner CPU/mem | Mongo | Postgres | Redis | Restarts |
|
||||
|---|---|---|---|---|---|---:|
|
||||
|
||||
## Payment Correctness
|
||||
|
||||
- Duplicate credits:
|
||||
- Under/overpayment anomalies:
|
||||
- Scanner mismatches:
|
||||
- Ledger mismatches:
|
||||
|
||||
## Bottlenecks
|
||||
|
||||
- API:
|
||||
- Database:
|
||||
- Scanner:
|
||||
- Socket/notifications:
|
||||
- RPC/chain:
|
||||
|
||||
## Decisions
|
||||
|
||||
- Current safe dev concurrency:
|
||||
- Recommended production target:
|
||||
- Required fixes before next ramp:
|
||||
```
|
||||
|
||||
## Initial Performance Characteristic Hypotheses
|
||||
|
||||
These are the expectations to validate:
|
||||
|
||||
- Request/offer APIs should scale mostly with Mongo/Postgres write throughput.
|
||||
- Notification latency will become a visible bottleneck before raw API latency if every offer/status change creates individual Mongo inserts and socket emits.
|
||||
- Scanner live-chain checks are likely bounded by BSC Testnet RPC latency and should be separated from API-only profiling.
|
||||
- Payment intent creation may become slower if destination derivation, token registry lookup, and scanner registration are serial.
|
||||
- Socket fanout should be watched at C16+ because each worker has multiple actors and multiple tabs/devices may multiply room membership.
|
||||
|
||||
@@ -18,6 +18,7 @@ or scanner changes.
|
||||
- BSC Testnet wallet has enough tBNB for gas and enough canonical tUSDT.
|
||||
- Backend/scanner chain 97 registry points to `0x109F54Dab34426D5477986b0460aE5dFBA65f022`.
|
||||
- Local wallet private key or mnemonic is stored only in ignored `.env`.
|
||||
- Notification checks are enabled in the runner. See [[Notification Assertion Procedure]].
|
||||
|
||||
## Actors
|
||||
|
||||
@@ -31,20 +32,28 @@ or scanner changes.
|
||||
|
||||
1. Generate a run id.
|
||||
2. Admin creates one buyer and at least two sellers.
|
||||
3. Buyer creates purchase request with a min/max USDT budget.
|
||||
4. Sellers submit bids inside the budget range.
|
||||
5. Randomize or vary:
|
||||
3. Assert initial notification baselines for buyer and sellers.
|
||||
4. Buyer creates purchase request with a min/max USDT budget.
|
||||
5. Assert purchase-request notifications for expected sellers.
|
||||
6. Sellers submit bids inside the budget range.
|
||||
7. After each seller bid, assert offer notifications for the buyer.
|
||||
8. Randomize or vary:
|
||||
- bid amount,
|
||||
- delivery timing,
|
||||
- seller note,
|
||||
- delivery method.
|
||||
6. Buyer selects a bid.
|
||||
7. Backend creates scanner payment intent.
|
||||
8. Buyer pays with BSC Testnet tUSDT.
|
||||
9. Scanner confirms the payment.
|
||||
10. Seller marks delivery.
|
||||
11. Buyer confirms delivery.
|
||||
12. Record current block: flow pauses before automated release policy.
|
||||
9. Buyer selects a bid.
|
||||
10. Assert offer-accepted notification for selected seller and offer-rejected/updated notifications for non-selected sellers when implemented.
|
||||
11. Backend creates scanner payment intent.
|
||||
12. Assert payment-pending or payment-started notification where implemented; if missing, record as notification coverage gap.
|
||||
13. Buyer pays with BSC Testnet tUSDT.
|
||||
14. Scanner confirms the payment.
|
||||
15. Assert payment-confirmed notifications for buyer and selected seller.
|
||||
16. Seller marks delivery.
|
||||
17. Assert delivery notification for buyer.
|
||||
18. Buyer confirms delivery.
|
||||
19. Assert delivery-confirmed notification for selected seller and buyer status notification if implemented.
|
||||
20. Record current block: flow pauses before automated release policy.
|
||||
|
||||
## Expected State Transitions
|
||||
|
||||
@@ -58,6 +67,40 @@ or scanner changes.
|
||||
| Seller delivers | `delivery` | escrow remains held |
|
||||
| Buyer confirms | `delivered`, `deliveryConfirmed=true` | release policy not automatic yet |
|
||||
|
||||
## Notification Assertions After Every Step
|
||||
|
||||
Notification verification is mandatory after every state-changing step. A step
|
||||
is not complete until the test runner has either:
|
||||
|
||||
1. observed the expected notification for every recipient; or
|
||||
2. recorded a known coverage gap with route/action, expected recipient, and linked issue/backlog item.
|
||||
|
||||
Use [[Notification Assertion Procedure]] for the exact API/socket checks.
|
||||
|
||||
| Step | Expected recipients | Required notification assertion |
|
||||
|---|---|---|
|
||||
| Baseline after user creation | buyer, every seller | Capture unread count and latest notification id for each actor before business mutations. |
|
||||
| Buyer creates request | eligible sellers | Sellers receive a request/new-opportunity notification, or the gap is recorded. |
|
||||
| Seller submits bid | buyer | Buyer receives a new-offer notification with `relatedId` or action URL pointing to the request/offer. |
|
||||
| Buyer selects bid | selected seller; non-selected sellers if rejection notices are implemented | Selected seller receives acceptance notification. Non-selected sellers receive rejection/update notification where supported. |
|
||||
| Payment intent created | buyer | Buyer receives payment-started/pending notification where supported. Missing `pending_payment` coverage is a known notification gap and must be recorded. |
|
||||
| Scanner confirms payment | buyer, selected seller | Buyer and seller receive payment-confirmed/funded notification. |
|
||||
| Seller marks delivery | buyer | Buyer receives delivery/proof submitted notification. |
|
||||
| Buyer confirms delivery | selected seller; buyer if status-change notices are implemented | Seller receives delivery-confirmed notification. |
|
||||
| Dispute raised, when added | buyer, seller, admin/mediator | All parties receive dispute-created/hold notification. |
|
||||
| Release/refund, when added | buyer, seller, admin | Funds movement notification is persisted and pushed. |
|
||||
|
||||
Minimum notification evidence per assertion:
|
||||
|
||||
| Evidence | Source |
|
||||
|---|---|
|
||||
| unread count before and after | `GET /api/notifications/unread-count` |
|
||||
| latest notification payload | `GET /api/notifications?limit=5` |
|
||||
| recipient id/email | authenticated actor profile or test user record |
|
||||
| notification category/type/title/actionUrl | notification payload |
|
||||
| socket event, if runner supports it | `new-notification` on `user-<userId>` |
|
||||
| elapsed time from action to notification visibility | runner timestamp |
|
||||
|
||||
## Buyer Request Template
|
||||
|
||||
Use unique values so live data can be filtered later:
|
||||
@@ -142,6 +185,8 @@ Minimum assertions per round:
|
||||
| Seller delivery returns HTTP 200 | yes |
|
||||
| Buyer confirm delivery returns HTTP 200 | yes |
|
||||
| Final request status is `delivered` | yes |
|
||||
| Notification assertion executed after each state-changing step | yes |
|
||||
| Notification gaps are recorded with expected recipient and route/action | yes |
|
||||
|
||||
## Reference Execution - 2026-06-06
|
||||
|
||||
@@ -204,4 +249,3 @@ Release policy is not complete:
|
||||
- dispute-to-release/refund automation is not complete.
|
||||
|
||||
Use [[Testing Expansion Backlog]] before extending this scenario into fund release.
|
||||
|
||||
|
||||
160
11 - Testing/Notification Assertion Procedure.md
Normal file
160
11 - Testing/Notification Assertion Procedure.md
Normal file
@@ -0,0 +1,160 @@
|
||||
---
|
||||
title: Notification Assertion Procedure
|
||||
tags: [testing, notifications, e2e, socket-io]
|
||||
created: 2026-06-06
|
||||
---
|
||||
|
||||
# Notification Assertion Procedure
|
||||
|
||||
Every E2E business step must verify notifications. A test step is incomplete
|
||||
until notification persistence and, where practical, real-time delivery are
|
||||
validated.
|
||||
|
||||
Related docs:
|
||||
|
||||
- [[04 - Flows/Notification Flow]]
|
||||
- [[03 - API Reference/Notification API]]
|
||||
- [[Escrow Marketplace E2E Procedure]]
|
||||
|
||||
## Principle
|
||||
|
||||
For each state-changing action, assert notifications for the expected recipients.
|
||||
If the product currently does not emit a notification for that action, the E2E
|
||||
must record a notification coverage gap. Silent missing notifications should not
|
||||
be treated as a pass.
|
||||
|
||||
## Notification Channels To Check
|
||||
|
||||
| Channel | Required? | How |
|
||||
|---|---|---|
|
||||
| Persistence | Always | `GET /api/notifications`, `GET /api/notifications/unread-count` |
|
||||
| Socket push | Required when runner has sockets enabled | Listen for `new-notification` on `user-<userId>` |
|
||||
| Badge count sync | Required for read/mark-read steps | `unread-count-update` socket event or count endpoint |
|
||||
| Email/push | Not required today | Planned digest/push work is not implemented |
|
||||
|
||||
## Baseline Before a Scenario
|
||||
|
||||
For every actor:
|
||||
|
||||
1. Authenticate.
|
||||
2. Read unread count:
|
||||
```http
|
||||
GET /api/notifications/unread-count
|
||||
```
|
||||
3. Read latest notifications:
|
||||
```http
|
||||
GET /api/notifications?page=1&limit=5
|
||||
```
|
||||
4. Store:
|
||||
- `baselineUnreadCount`;
|
||||
- latest notification id;
|
||||
- timestamp.
|
||||
|
||||
Do this before creating purchase requests or offers so later assertions can
|
||||
distinguish new notifications from old ones.
|
||||
|
||||
## Per-Step Assertion Algorithm
|
||||
|
||||
After each action:
|
||||
|
||||
1. Identify expected recipients.
|
||||
2. Poll each recipient's notifications for up to the notification SLA window.
|
||||
3. Assert at least one new notification matching the action.
|
||||
4. Assert unread count increased by the expected number, unless the notification
|
||||
is intentionally auto-read.
|
||||
5. Assert `actionUrl` or `relatedId` points to the correct request/payment/offer
|
||||
where the notification type supports it.
|
||||
6. If socket instrumentation is active, assert the same notification arrived via
|
||||
`new-notification`.
|
||||
7. Record `notificationLatencyMs = notification.createdAt - actionCompletedAt`
|
||||
or runner-observed first-seen time.
|
||||
|
||||
Suggested polling:
|
||||
|
||||
| Target | Value |
|
||||
|---|---:|
|
||||
| poll interval | `500 ms` |
|
||||
| timeout | `10 s` for local/dev API notifications |
|
||||
| hard failure threshold | `30 s` |
|
||||
|
||||
## Expected Marketplace Notifications
|
||||
|
||||
| Action | Expected recipients | Expected category/type |
|
||||
|---|---|---|
|
||||
| Buyer creates request | eligible sellers | `purchase_request` / new request or opportunity |
|
||||
| Seller submits offer | buyer | `offer` / new offer |
|
||||
| Buyer accepts offer | selected seller | `offer` / accepted |
|
||||
| Buyer accepts offer | non-selected sellers | `offer` / rejected or no-longer-selected, when implemented |
|
||||
| Payment intent created | buyer | `payment` / pending or started, when implemented |
|
||||
| Scanner confirms payment | buyer, selected seller | `payment` / confirmed or funded |
|
||||
| Seller marks delivery | buyer | `delivery` / delivery submitted |
|
||||
| Buyer confirms delivery | selected seller | `delivery` / delivery confirmed |
|
||||
| Dispute raised | buyer, seller, admin/mediator | `delivery`/`payment`/`system` dispute hold, when implemented |
|
||||
| Release/refund | buyer, seller | `payment` / release or refund |
|
||||
|
||||
Known current gaps from existing docs:
|
||||
|
||||
- `pending_payment` and `seller_paid` status changes are not covered by
|
||||
`NotificationService.notifyRequestStatusChanged`.
|
||||
- Dispute service notification emits are TODO stubs in the dashboard dispute path.
|
||||
|
||||
These known gaps should still be reported by the E2E runner as gaps, not ignored.
|
||||
|
||||
## Evidence Format
|
||||
|
||||
For each step, append a notification assertion record:
|
||||
|
||||
```json
|
||||
{
|
||||
"step": "seller_offer_created",
|
||||
"recipient": "buyer",
|
||||
"recipientUserId": "<id>",
|
||||
"expected": true,
|
||||
"observed": true,
|
||||
"latencyMs": 742,
|
||||
"unreadBefore": 3,
|
||||
"unreadAfter": 4,
|
||||
"notificationId": "<id>",
|
||||
"category": "offer",
|
||||
"actionUrl": "/dashboard/request/<requestId>",
|
||||
"socketObserved": true,
|
||||
"gap": null
|
||||
}
|
||||
```
|
||||
|
||||
For a known gap:
|
||||
|
||||
```json
|
||||
{
|
||||
"step": "payment_intent_created",
|
||||
"recipient": "buyer",
|
||||
"expected": true,
|
||||
"observed": false,
|
||||
"gap": "No notification emitted for pending_payment status",
|
||||
"linkedDoc": "Notification Flow#Purchase request status coverage gap"
|
||||
}
|
||||
```
|
||||
|
||||
## Pass/Fail Rules
|
||||
|
||||
| Condition | Result |
|
||||
|---|---|
|
||||
| Expected notification appears within SLA | pass |
|
||||
| Expected notification appears after SLA but before hard timeout | warning |
|
||||
| Expected notification never appears and no approved gap exists | fail |
|
||||
| Notification appears for wrong user | fail |
|
||||
| Notification exists but has wrong request/payment/offer id | fail |
|
||||
| Socket missing but persistence exists | warning unless socket coverage is the target |
|
||||
| Unread count inconsistent with persisted notification | fail |
|
||||
|
||||
## Performance Metrics
|
||||
|
||||
Notification metrics must be included in concurrency profiles:
|
||||
|
||||
- notification persistence latency p50/p95/p99;
|
||||
- socket delivery latency p50/p95/p99;
|
||||
- notification failures by action and recipient;
|
||||
- unread count mismatch rate;
|
||||
- duplicate notification rate;
|
||||
- Mongo notification insert error rate.
|
||||
|
||||
@@ -38,6 +38,8 @@ have an automation owner, a smoke command, and a UAT checklist.
|
||||
| DISPUTE-003 | Admin resolves dispute for buyer and refund path is unblocked | P0 | Not complete | [[Testing Expansion Backlog]] |
|
||||
| CI-001 | Code push builds and deploys backend/frontend/scanner via Woodpecker | P0 | Live used | [[Smoke and Regression Procedure]] |
|
||||
| UI-CHAIN-001 | Checkout shows BSC Testnet, tUSDT contract, and testnet explorer links | P1 | Implemented in frontend `2.8.118` | [[Scanner BSC Testnet Payment Procedure]] |
|
||||
| NOTIF-E2E-001 | Every E2E state-changing step issues expected notifications | P0 | Designed, needs automation | [[Notification Assertion Procedure]] |
|
||||
| PERF-CONC-001 | Ramp simultaneous full escrow E2E workers: 1, 2, 4, 8, 16, 32+ | P0 | Designed, needs automation and baseline report | [[Concurrency and Performance Profile]] |
|
||||
| AUTH-001 | Admin-created generated users can log in and execute role actions | P0 | Live used | [[Escrow Marketplace E2E Procedure]] |
|
||||
| CLEANUP-001 | Test users/requests can be identified and excluded from reports | P2 | Designed | [[Test Environment and Data]] |
|
||||
|
||||
@@ -52,9 +54,24 @@ flowchart LR
|
||||
E --> F["Scanner confirms payment"]
|
||||
F --> G["Seller delivers"]
|
||||
G --> H["Buyer confirms delivery"]
|
||||
H --> I["Policy boundary: grace, release, or dispute"]
|
||||
H --> I["Assert notifications after every step"]
|
||||
I --> J["Policy boundary: grace, release, or dispute"]
|
||||
```
|
||||
|
||||
## Concurrency Scenario Family
|
||||
|
||||
The concurrency profile runs the same full escrow worker in parallel and doubles
|
||||
the worker count until a stop condition is reached:
|
||||
|
||||
```text
|
||||
1 -> 2 -> 4 -> 8 -> 16 -> 32 -> 64 ...
|
||||
```
|
||||
|
||||
Each worker executes one isolated buyer/seller/payment/delivery flow with unique
|
||||
test users, request ids, destination addresses, and payment ids. Notification
|
||||
assertions remain mandatory inside each worker. See
|
||||
[[Concurrency and Performance Profile]].
|
||||
|
||||
## Payment Negative Scenario Families
|
||||
|
||||
| Family | What to mutate | Expected result |
|
||||
@@ -88,4 +105,3 @@ A scenario is complete when all are true:
|
||||
4. It has at least one automated test or smoke script where practical.
|
||||
5. It has a documented live-dev verification result.
|
||||
6. It names residual risk or product gaps.
|
||||
|
||||
|
||||
@@ -102,6 +102,50 @@ Add Playwright tests for:
|
||||
- payment submitted screen waits for scanner confirmation;
|
||||
- paid socket/poll fallback transitions to success.
|
||||
|
||||
## P1 - Notification Assertions
|
||||
|
||||
Turn [[Notification Assertion Procedure]] into reusable E2E helpers:
|
||||
|
||||
- capture per-user notification baseline;
|
||||
- poll `GET /api/notifications` and `GET /api/notifications/unread-count`;
|
||||
- optionally attach Socket.IO clients for buyer/seller/admin actors;
|
||||
- fail on wrong-recipient notifications;
|
||||
- record known notification gaps with route/action and expected recipient;
|
||||
- include notification latency in every E2E report.
|
||||
|
||||
Acceptance tests:
|
||||
|
||||
| ID | Expected |
|
||||
|---|---|
|
||||
| NOTIF-E2E-001 | New seller offer creates a buyer notification and unread-count increment. |
|
||||
| NOTIF-E2E-002 | Buyer offer acceptance creates selected seller notification. |
|
||||
| NOTIF-E2E-003 | Seller delivery creates buyer notification. |
|
||||
| NOTIF-E2E-004 | Buyer delivery confirmation creates seller notification. |
|
||||
| NOTIF-E2E-005 | Missing `pending_payment` notification is reported as a known gap, not a silent pass. |
|
||||
|
||||
## P1 - Concurrency and Performance Profile
|
||||
|
||||
Implement [[Concurrency and Performance Profile]] as an executable runner and
|
||||
report generator:
|
||||
|
||||
- worker abstraction for one isolated buyer/sellers/payment/delivery flow;
|
||||
- barrier start for simultaneous workers;
|
||||
- ramp stages `1, 2, 4, 8, 16, 32`;
|
||||
- JSON result per worker;
|
||||
- markdown performance profile report per run;
|
||||
- infrastructure snapshots before/during/after each stage;
|
||||
- stop-condition enforcement.
|
||||
|
||||
Acceptance tests:
|
||||
|
||||
| ID | Expected |
|
||||
|---|---|
|
||||
| PERF-CONC-001 | C1 completes with all notification assertions. |
|
||||
| PERF-CONC-002 | C2/C4 detect no cross-worker state leakage. |
|
||||
| PERF-CONC-003 | Report includes p50/p95/p99 API, scanner, notification, and total timings. |
|
||||
| PERF-CONC-004 | Runner stops the ramp on P0 notification/payment/ledger correctness failure. |
|
||||
| PERF-CONC-005 | Runner separates live-chain mode from scanner-fixture/API-only modes. |
|
||||
|
||||
## P1 - CI and Deployment
|
||||
|
||||
Add a deployment smoke checklist artifact after every pipeline:
|
||||
@@ -142,4 +186,3 @@ Add dashboards/log queries for:
|
||||
5. Add release-policy tests.
|
||||
6. Resolve dispute route/policy gaps.
|
||||
7. Add dispute-to-release/refund E2E tests.
|
||||
|
||||
|
||||
@@ -21,6 +21,8 @@ seller, payment scanner, delivery, payout, dispute, and deployment behavior.
|
||||
| [[Test Scenario Catalog]] | Canonical scenarios we have designed or need to extend. |
|
||||
| [[Escrow Marketplace E2E Procedure]] | Buyer/seller/request/bid/delivery procedure, including the current two-round flow. |
|
||||
| [[Scanner BSC Testnet Payment Procedure]] | BSC Testnet tUSDT scanner payment procedure and failure modes. |
|
||||
| [[Notification Assertion Procedure]] | Required notification checks after every E2E business step. |
|
||||
| [[Concurrency and Performance Profile]] | Ramp test design, profiling targets, metrics, and report template. |
|
||||
| [[Smoke and Regression Procedure]] | CLI, CI, and post-deploy smoke checks. |
|
||||
| [[Testing Expansion Backlog]] | Gaps to cover before broader release confidence. |
|
||||
|
||||
@@ -33,6 +35,7 @@ seller, payment scanner, delivery, payout, dispute, and deployment behavior.
|
||||
| Smoke tests | Fast checks for one deployment target, usually via `BASE_URL`. | Backend scripts |
|
||||
| Browser E2E | Validate user-visible web flows. | Frontend Playwright |
|
||||
| Live dev E2E | Validate real dev deployment, CI image, scanner service, BSC Testnet, and test tokens. | QA/operator |
|
||||
| Concurrency profile | Ramp simultaneous full-flow workers and measure API, DB, scanner, notification, and chain behavior. | QA/operator + backend |
|
||||
| UAT | Validate product scenarios and acceptance criteria. | Product + QA |
|
||||
|
||||
## Golden Path Coverage
|
||||
@@ -47,7 +50,8 @@ The minimum live-dev confidence path is:
|
||||
6. Scanner confirms the token transfer.
|
||||
7. Seller delivers.
|
||||
8. Buyer confirms delivery.
|
||||
9. Flow pauses at the current product-policy boundary: release/grace/dispute automation is not complete.
|
||||
9. After every state-changing step, assert notifications for every expected recipient.
|
||||
10. Flow pauses at the current product-policy boundary: release/grace/dispute automation is not complete.
|
||||
|
||||
The current live tested version of this path is documented in
|
||||
[[Escrow Marketplace E2E Procedure#Reference execution - 2026-06-06]].
|
||||
@@ -70,4 +74,3 @@ The current live tested version of this path is documented in
|
||||
- Record enough evidence to reproduce a failure: request id, offer id, payment id, tx hash, chain id, token address, HTTP status, and CI build number.
|
||||
- Treat a passing local test as necessary but not sufficient for scanner/payment work. Payment changes must also be verified against dev after deploy.
|
||||
- Every live-dev test should state what remains untested or blocked.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user