207 lines
7.3 KiB
Markdown
207 lines
7.3 KiB
Markdown
---
|
|
title: Testing Expansion Backlog
|
|
tags: [testing, backlog, qa, release]
|
|
created: 2026-06-06
|
|
---
|
|
|
|
# Testing Expansion Backlog
|
|
|
|
This is the testing backlog to complete before expanding the escrow test suite
|
|
or claiming release confidence.
|
|
|
|
## P0 - Product Policy Gaps
|
|
|
|
### Physical product grace period
|
|
|
|
Need implementation and tests for:
|
|
|
|
- seller marks delivery;
|
|
- buyer receives delivery window;
|
|
- grace period starts;
|
|
- buyer can confirm delivery before expiry;
|
|
- buyer can dispute before expiry;
|
|
- funds release only after buyer confirmation, no dispute, or grace expiry.
|
|
|
|
Acceptance tests:
|
|
|
|
| ID | Expected |
|
|
|---|---|
|
|
| RELEASE-PHYS-001 | Physical delivery starts hold/grace timer. |
|
|
| RELEASE-PHYS-002 | Buyer confirmation before expiry makes funds releasable. |
|
|
| RELEASE-PHYS-003 | Dispute before expiry blocks release. |
|
|
| RELEASE-PHYS-004 | Grace expiry without dispute makes funds releasable. |
|
|
|
|
### Immediate release for gift cards/digital goods
|
|
|
|
Need implementation and tests for:
|
|
|
|
- product type is gift card or digital;
|
|
- seller proof is required immediately;
|
|
- buyer acceptance or product policy triggers immediate/near-immediate release;
|
|
- dispute path can still stop release if policy includes a short challenge window.
|
|
|
|
Acceptance tests:
|
|
|
|
| ID | Expected |
|
|
|---|---|
|
|
| RELEASE-DIGI-001 | Gift card delivery does not use physical-product grace period. |
|
|
| RELEASE-DIGI-002 | Gift card proof advances to releasable immediately after buyer acceptance. |
|
|
| RELEASE-DIGI-003 | Wrong/missing proof blocks release. |
|
|
|
|
## P0 - Dispute Coverage
|
|
|
|
The dispute mechanism is not complete enough for end-to-end fund movement.
|
|
|
|
Tests needed:
|
|
|
|
| ID | Scenario | Expected |
|
|
|---|---|---|
|
|
| DISPUTE-001 | Buyer opens dispute after seller delivery, before release | payment/request enters dispute hold |
|
|
| DISPUTE-002 | Seller cannot force release while dispute is open | release attempt rejected |
|
|
| DISPUTE-003 | Admin resolves for seller | release becomes available |
|
|
| DISPUTE-004 | Admin resolves for buyer | refund becomes available |
|
|
| DISPUTE-005 | Duplicate dispute resolution | idempotent or rejected safely |
|
|
| DISPUTE-006 | Dashboard dispute route vs release-hold route | no route shadowing ambiguity |
|
|
|
|
## P0 - Payment Negative Tests
|
|
|
|
Automate:
|
|
|
|
- wrong token;
|
|
- wrong chain;
|
|
- wrong destination;
|
|
- underpayment;
|
|
- duplicate payment;
|
|
- late payment after cancelled/expired intent;
|
|
- payment with no gas;
|
|
- scanner unavailable during payment;
|
|
- scanner webhook signature invalid;
|
|
- balance check baseline missing or stale.
|
|
|
|
## P0 - Authorization and ID Boundaries
|
|
|
|
Add regression tests for cross-store id comparisons:
|
|
|
|
| Route/action | Required test |
|
|
|---|---|
|
|
| buyer delivery confirmation | buyer can confirm even when session id and request buyer id use different stores |
|
|
| buyer delivery confirmation | non-buyer cannot confirm |
|
|
| seller delivery update | only selected seller/admin can deliver |
|
|
| offer acceptance | only buyer/admin can accept |
|
|
| payment check | only buyer/admin/system can advance payment where applicable |
|
|
|
|
## P1 - UI/Browser E2E
|
|
|
|
Add Playwright tests for:
|
|
|
|
- checkout renders BSC Testnet chain label;
|
|
- token contract row shows canonical tUSDT;
|
|
- testnet explorer links are generated correctly;
|
|
- wrong wallet chain shows switch button;
|
|
- insufficient balance warning appears;
|
|
- payment submitted screen waits for scanner confirmation;
|
|
- paid socket/poll fallback transitions to success.
|
|
|
|
## P1 - Notification Assertions
|
|
|
|
Initial reusable E2E helpers are implemented in
|
|
`backend/scripts/smoke/marketplace-e2e-notifications.mjs`. Remaining work:
|
|
|
|
- capture per-user notification baseline;
|
|
- poll `GET /api/notifications` and `GET /api/notifications/unread-count`;
|
|
- optionally attach Socket.IO clients for buyer/seller/admin actors;
|
|
- fail on wrong-recipient notifications;
|
|
- record known notification gaps with route/action and expected recipient;
|
|
- include notification latency in every E2E report.
|
|
|
|
Current dev findings from the first two-round runner report:
|
|
|
|
- `NOTIF-E2E-001` passes for buyer new-offer notifications.
|
|
- Seller new-request fanout passes for privately targeted sellers.
|
|
- Rejected-seller notifications pass by rejected offer `relatedId`.
|
|
- `NOTIF-E2E-002` fails by selected offer `relatedId`; accepted-offer notification is not discoverable with the selected offer id.
|
|
|
|
Acceptance tests:
|
|
|
|
| ID | Expected |
|
|
|---|---|
|
|
| NOTIF-E2E-001 | New seller offer creates a buyer notification and unread-count increment. |
|
|
| NOTIF-E2E-002 | Buyer offer acceptance creates selected seller notification. |
|
|
| NOTIF-E2E-003 | Seller delivery creates buyer notification. |
|
|
| NOTIF-E2E-004 | Buyer delivery confirmation creates seller notification. |
|
|
| NOTIF-E2E-005 | Missing `pending_payment` notification is reported as a known gap, not a silent pass. |
|
|
|
|
## P1 - Concurrency and Performance Profile
|
|
|
|
Initial executable runner and report generator are implemented in
|
|
`backend/scripts/smoke/marketplace-e2e-notifications.mjs`. Remaining work:
|
|
|
|
- worker abstraction for one isolated buyer/sellers/payment/delivery flow;
|
|
- barrier start for simultaneous workers;
|
|
- ramp stages `1, 2, 4, 8, 16, 32`;
|
|
- JSON result per worker;
|
|
- markdown performance profile report per run;
|
|
- infrastructure snapshots before/during/after each stage;
|
|
- stop-condition enforcement.
|
|
|
|
Current dev blocker:
|
|
|
|
- Dev API rate limiting is `100` requests per `900s`, so C2+ ramp runs need a
|
|
fresh window, a lower-polling profile, or a non-rate-limited test environment.
|
|
- `PAYMENT_MODE=record` currently fails because `POST /api/marketplace/payments`
|
|
returns HTTP 500 after offer selection with generated test actors.
|
|
|
|
Acceptance tests:
|
|
|
|
| ID | Expected |
|
|
|---|---|
|
|
| PERF-CONC-001 | C1 completes with all notification assertions. |
|
|
| PERF-CONC-002 | C2/C4 detect no cross-worker state leakage. |
|
|
| PERF-CONC-003 | Report includes p50/p95/p99 API, scanner, notification, and total timings. |
|
|
| PERF-CONC-004 | Runner stops the ramp on P0 notification/payment/ledger correctness failure. |
|
|
| PERF-CONC-005 | Runner separates live-chain mode from scanner-fixture/API-only modes. |
|
|
|
|
## P1 - CI and Deployment
|
|
|
|
Add a deployment smoke checklist artifact after every pipeline:
|
|
|
|
- backend version endpoint;
|
|
- frontend root 200;
|
|
- scanner `/health`;
|
|
- chain 97 token balance check;
|
|
- current Docker image digests;
|
|
- last commit SHA per service.
|
|
|
|
## P1 - Test Data Lifecycle
|
|
|
|
Need cleanup strategy:
|
|
|
|
- flag generated users with run id;
|
|
- archive or tag generated purchase requests;
|
|
- prevent test data from polluting analytics;
|
|
- define whether test tx/payment rows are permanent audit evidence or can be purged.
|
|
|
|
## P2 - Observability
|
|
|
|
Add dashboards/log queries for:
|
|
|
|
- scanner intent lifecycle;
|
|
- balance-check deltas;
|
|
- payment state transitions;
|
|
- delivery status changes;
|
|
- dispute holds;
|
|
- release/refund attempts.
|
|
|
|
## Suggested Next Automation Order
|
|
|
|
1. Fix selected-seller accepted notification related id.
|
|
2. Fix or retire the legacy `POST /api/marketplace/payments` record path.
|
|
3. Run `PAYMENT_MODE=status` through delivery after the dev rate-limit window resets.
|
|
4. Run `PAYMENT_MODE=live` once wallet gas/tUSDT and rate-limit headroom are available.
|
|
5. Add non-buyer delivery confirmation regression test.
|
|
6. Add scanner negative tests for wrong token and underpayment.
|
|
7. Implement physical vs digital release policy.
|
|
8. Add release-policy tests.
|
|
9. Resolve dispute route/policy gaps.
|
|
10. Add dispute-to-release/refund E2E tests.
|