docs: sync from backend ca2b1c4 — marketplace e2e smoke runner

This commit is contained in:
Siavash Sameni
2026-06-06 11:32:11 +04:00
parent bee91dd01f
commit 79dab07243
7 changed files with 271 additions and 12 deletions

View File

@@ -11,6 +11,16 @@ entries on top. Maintained by agents per the rule in `../AGENTS.md`.
--- ---
### 2026-06-06 — backend@ca2b1c4, frontend@aac3304 — Marketplace E2E notification smoke runner
**Commits:** `ca2b1c4` `aac3304`
**Touched:** backend `scripts/smoke/marketplace-e2e-notifications.mjs`, `scripts/smoke/marketplace-e2e-notifications.sh`, `package.json`, `package-lock.json`; frontend `package.json`, `Dockerfile`; docs `11 - Testing/Marketplace E2E Smoke Runner.md`, testing procedure pages.
**Why:** Implement the requested buyer/seller marketplace E2E runner with at least two sellers, notification assertions after every state-changing step, report generation, concurrency ramp controls, and BSC Testnet live-payment support. Rebased on newer Forgejo `main` and bumped backend/frontend to `2.9.2`.
**Verification:** backend `node --check scripts/smoke/marketplace-e2e-notifications.mjs`, `npm run typecheck`, `git diff --check`; frontend `git diff --check` and version/Dockerfile sanity; nick-doc `git diff --check`. Dev two-round `PAYMENT_MODE=record` report exposed HTTP 500 on legacy `POST /api/marketplace/payments` and missing selected-seller accepted notification. Direct-backend one-round `PAYMENT_MODE=status` reached final request status `delivered` with notification gaps recorded. Woodpecker build pending after push.
**Linked docs updated:** [[11 - Testing/Marketplace E2E Smoke Runner]], [[11 - Testing/Escrow Marketplace E2E Procedure]], [[11 - Testing/Smoke and Regression Procedure]], [[11 - Testing/Concurrency and Performance Profile]], [[11 - Testing/Testing Expansion Backlog]]
---
### 2026-06-06 — backend@31b285f, frontend@2a3e5c9 — BSC Testnet checkout UI support ### 2026-06-06 — backend@31b285f, frontend@2a3e5c9 — BSC Testnet checkout UI support
**Commits:** `31b285f` `2a3e5c9` **Commits:** `31b285f` `2a3e5c9`

View File

@@ -36,6 +36,21 @@ Each worker must use unique:
Notifications are mandatory inside every worker. See Notifications are mandatory inside every worker. See
[[Notification Assertion Procedure]]. [[Notification Assertion Procedure]].
Implemented runner:
```bash
cd ~/CascadeProjects/escrow/backend
BASE_URL=https://dev.amn.gg \
PAYMENT_MODE=status \
CONCURRENCY_LEVELS=1,2,4,8,16,32 \
ROUNDS=1 \
bash scripts/smoke/marketplace-e2e-notifications.sh
```
Use `PAYMENT_MODE=live` for low-concurrency BSC Testnet tUSDT confirmation.
Use `PAYMENT_MODE=status` for high-concurrency marketplace/notification
profiling without consuming gas.
## Ramp Plan ## Ramp Plan
Start with one simultaneous worker and double until a stop condition is reached: Start with one simultaneous worker and double until a stop condition is reached:
@@ -59,6 +74,7 @@ stage.
| Mode | Payment behavior | Use | | Mode | Payment behavior | Use |
|---|---|---| |---|---|---|
| Live-chain mode | Real BSC Testnet tUSDT transfers | Final confidence at low concurrency; expensive/slower; consumes gas. | | Live-chain mode | Real BSC Testnet tUSDT transfers | Final confidence at low concurrency; expensive/slower; consumes gas. |
| Status-only smoke mode | Moves accepted request to `payment` through the status route | Implemented high-concurrency marketplace/notification profiling without chain variables. |
| Scanner fixture mode | Deterministic scanner/balance fixture or controlled test endpoint | High concurrency without chain bottleneck. Must not be enabled in production. | | Scanner fixture mode | Deterministic scanner/balance fixture or controlled test endpoint | High concurrency without chain bottleneck. Must not be enabled in production. |
| API-only dry run | Runs request/offer/delivery and skips payment finalization | Marketplace/notification profiling without chain variables. | | API-only dry run | Runs request/offer/delivery and skips payment finalization | Marketplace/notification profiling without chain variables. |
@@ -267,4 +283,3 @@ These are the expectations to validate:
- Scanner live-chain checks are likely bounded by BSC Testnet RPC latency and should be separated from API-only profiling. - Scanner live-chain checks are likely bounded by BSC Testnet RPC latency and should be separated from API-only profiling.
- Payment intent creation may become slower if destination derivation, token registry lookup, and scanner registration are serial. - Payment intent creation may become slower if destination derivation, token registry lookup, and scanner registration are serial.
- Socket fanout should be watched at C16+ because each worker has multiple actors and multiple tabs/devices may multiply room membership. - Socket fanout should be watched at C16+ because each worker has multiple actors and multiple tabs/devices may multiply room membership.

View File

@@ -30,6 +30,20 @@ or scanner changes.
## High-Level Procedure ## High-Level Procedure
Use the implemented backend runner for repeatable execution:
```bash
BASE_URL=https://dev.amn.gg \
PAYMENT_MODE=status \
CONCURRENCY_LEVELS=1 \
ROUNDS=2 \
bash scripts/smoke/marketplace-e2e-notifications.sh
```
Use `PAYMENT_MODE=live` for the funded BSC Testnet tUSDT rail and
`PAYMENT_MODE=record` only when explicitly testing the legacy marketplace
payment-record route. See [[Marketplace E2E Smoke Runner]].
1. Generate a run id. 1. Generate a run id.
2. Admin creates one buyer and at least two sellers. 2. Admin creates one buyer and at least two sellers.
3. Assert initial notification baselines for buyer and sellers. 3. Assert initial notification baselines for buyer and sellers.

View File

@@ -0,0 +1,185 @@
---
title: Marketplace E2E Smoke Runner
tags: [testing, smoke, e2e, notifications, performance]
created: 2026-06-06
---
# Marketplace E2E Smoke Runner
The backend repo has a committed marketplace runner:
```bash
backend/scripts/smoke/marketplace-e2e-notifications.sh
```
It creates isolated test actors, runs buyer/seller marketplace flows, checks
persisted notifications after every business mutation, and writes JSON/Markdown
reports under ignored `backend/tmp/e2e-reports/`.
## Required Environment
Do not put secrets in command history or docs. Load these from an ignored local
env file or a secure shell session:
| Variable | Purpose |
|---|---|
| `BASE_URL` | Target API, for example `https://dev.amn.gg`. Defaults to local `http://127.0.0.1:5001`. |
| `ADMIN_EMAIL` | Admin account email used only to create test users. |
| `ADMIN_PASSWORD` | Admin account password, never committed or printed. |
| `PAYMENT_MODE` | `status`, `record`, or `live`. Defaults to `status`. |
| `CONCURRENCY_LEVELS` | Comma-separated worker ramp, for example `1,2,4,8`. |
| `ROUNDS` | Rounds per worker. Use `2` for the basic two-round requirement. |
| `SELLERS_PER_WORKER` | Sellers per flow. Minimum enforced by runner is `2`; default is `3`. |
| `SKIP_WRONG_RECIPIENT_CHECK` | Optional escape hatch for rate-limited environments. Defaults to `false`. |
## Payment Modes
| Mode | Use | Notes |
|---|---|---|
| `status` | Cheap full-flow smoke without wallet funds. | Moves the request to `payment` through the status route, then exercises seller delivery and buyer confirmation. |
| `record` | Tests legacy `POST /api/marketplace/payments`. | Current dev run shows this route returns HTTP 500 with generated buyer/seller ids. Keep this mode for regression coverage. |
| `live` | Tests BSC Testnet tUSDT direct-balance scanner rail. | Requires `E2E_PAYMENT_PRIVATE_KEY` or `E2E_PAYMENT_MNEMONIC`, gas, and tUSDT. Sends ERC-20 transfer and calls direct-balance check. |
Live mode also reads:
| Variable | Default |
|---|---|
| `E2E_NETWORK` | `bsc-testnet` |
| `E2E_CHAIN_ID` | `97` |
| `E2E_TOKEN_SYMBOL` | `USDT` |
| `RPC_URL_CHAIN_97` / `BSC_TESTNET_RPC_URL` | public BSC Testnet RPC fallback |
## Example Commands
Two-round full-flow smoke without wallet funds:
```bash
BASE_URL=https://dev.amn.gg \
PAYMENT_MODE=status \
CONCURRENCY_LEVELS=1 \
ROUNDS=2 \
bash scripts/smoke/marketplace-e2e-notifications.sh
```
Concurrency profile starting at one worker:
```bash
BASE_URL=https://dev.amn.gg \
PAYMENT_MODE=status \
CONCURRENCY_LEVELS=1,2,4,8,16,32 \
ROUNDS=1 \
bash scripts/smoke/marketplace-e2e-notifications.sh
```
Live BSC Testnet tUSDT smoke:
```bash
BASE_URL=https://dev.amn.gg \
PAYMENT_MODE=live \
CONCURRENCY_LEVELS=1 \
ROUNDS=1 \
bash scripts/smoke/marketplace-e2e-notifications.sh
```
## Notification Behavior
The runner treats each state-changing business step as a notification checkpoint.
Each checkpoint records:
- actor and role;
- expected related id;
- whether a persisted notification was found;
- whether the missing notification is a known gap;
- unread count from the notification list response.
By default, known gaps do not abort the flow. Set `STRICT_NOTIFICATIONS=true`
to make known gaps fail the run. Set `ABORT_ON_NOTIFICATION_FAILURE=true` to
stop at the first missing notification.
## Output
Reports are ignored by Git:
```text
backend/tmp/e2e-reports/marketplace-e2e-<runId>.json
backend/tmp/e2e-reports/marketplace-e2e-<runId>.md
```
The report includes:
- flow pass/fail counts;
- notification found/missing/known-gap counts;
- wrong-recipient notification count;
- API p50/p95/p99/max timings;
- request, offer, payment, token, and transaction ids where applicable.
Actor tokens and admin credentials are never written to the report.
## Reference Dev Run - 2026-06-06
Command shape:
```bash
BASE_URL=https://dev.amn.gg PAYMENT_MODE=record CONCURRENCY_LEVELS=1 ROUNDS=2 ...
```
Result:
| Metric | Value |
|---|---:|
| flows passed | `0/2` |
| request-created notifications found | `8/8` |
| buyer new-offer notifications found | `6/6` |
| rejected-seller notifications found | `4/4` |
| selected-seller accepted notifications by related id | `0/2` |
| wrong-recipient notifications | `0` |
| legacy marketplace payment records | `0/2`, HTTP 500 |
Observed defects:
1. Selected-seller accepted notification is not discoverable by the selected
offer `relatedId`. The service path appears to create the accepted-offer
notification with an incomplete related id.
2. `POST /api/marketplace/payments` returned HTTP 500 after offer selection
with generated test actors. This blocks `PAYMENT_MODE=record`; use
`PAYMENT_MODE=status` for flow-only smoke or `PAYMENT_MODE=live` for scanner
verification.
3. Dev API rate limiting is `100` requests per `900s`; concurrency runs must
use a fresh window or a non-rate-limited test environment.
## Reference Direct-Backend Run - 2026-06-06
Command shape:
```bash
BASE_URL=http://127.0.0.1:15001 PAYMENT_MODE=status CONCURRENCY_LEVELS=1 ROUNDS=1 ...
```
This used a temporary SSH tunnel to the dev backend container to avoid the
public edge rate-limit window.
Result:
| Metric | Value |
|---|---:|
| flows passed | `1/1` |
| final request status | `delivered` |
| wrong-recipient notifications | `0` |
| required notification missing | `1` |
| known notification gaps | `2` |
Key ids:
| Field | Value |
|---|---|
| purchaseRequestId | `dc6ec076-9c15-46a1-ab6b-1d9d17604614` |
| selectedOfferId | `9f56dd00-8b08-4c9f-b91c-5875e1949113` |
Observed notification result:
- request-created notifications passed for buyer and three targeted sellers;
- buyer new-offer notifications passed for all three seller bids;
- rejected-seller notifications passed;
- selected-seller accepted notification still failed by selected offer `relatedId`;
- payment-confirmed notification was found in `PAYMENT_MODE=status`;
- seller delivery and buyer delivery confirmation notifications are known gaps.

View File

@@ -39,6 +39,7 @@ Current scripts:
| `bsc-testnet-payment-registry.sh` | Chain 97 token/RPC registry sanity. | | `bsc-testnet-payment-registry.sh` | Chain 97 token/RPC registry sanity. |
| `confirmation-thresholds.sh` | Confirmation threshold APIs/settings. | | `confirmation-thresholds.sh` | Confirmation threshold APIs/settings. |
| `funds-ledger-repo.sh` | Funds ledger repository behavior. | | `funds-ledger-repo.sh` | Funds ledger repository behavior. |
| `marketplace-e2e-notifications.sh` | Buyer/seller E2E smoke with notification assertions, concurrency levels, and reports. |
| `marketplace-request-budget.sh` | Purchase request budget validation. | | `marketplace-request-budget.sh` | Purchase request budget validation. |
| `rn-intent.sh` | Request Network / in-house intent creation smoke. | | `rn-intent.sh` | Request Network / in-house intent creation smoke. |
| `rn-webhook.sh` | Webhook shape/signature flow. | | `rn-webhook.sh` | Webhook shape/signature flow. |
@@ -52,6 +53,21 @@ cd ~/CascadeProjects/escrow/backend
BASE_URL=https://dev.amn.gg bash scripts/smoke/bsc-testnet-payment-registry.sh BASE_URL=https://dev.amn.gg bash scripts/smoke/bsc-testnet-payment-registry.sh
``` ```
Two-round marketplace smoke:
```bash
cd ~/CascadeProjects/escrow/backend
BASE_URL=https://dev.amn.gg \
PAYMENT_MODE=status \
CONCURRENCY_LEVELS=1 \
ROUNDS=2 \
bash scripts/smoke/marketplace-e2e-notifications.sh
```
Use `PAYMENT_MODE=live` when the BSC Testnet wallet has gas and canonical
tUSDT. The runner writes reports to ignored `backend/tmp/e2e-reports/`; see
[[Marketplace E2E Smoke Runner]].
Run against local: Run against local:
```bash ```bash
@@ -165,4 +181,3 @@ Stop or roll back if any P0 condition fails:
- backend/frontend/scanner registry mismatch exists; - backend/frontend/scanner registry mismatch exists;
- CI deploy reports success but live containers are not healthy; - CI deploy reports success but live containers are not healthy;
- secrets appear in git status, logs, docs, or CI output. - secrets appear in git status, logs, docs, or CI output.

View File

@@ -104,7 +104,8 @@ Add Playwright tests for:
## P1 - Notification Assertions ## P1 - Notification Assertions
Turn [[Notification Assertion Procedure]] into reusable E2E helpers: Initial reusable E2E helpers are implemented in
`backend/scripts/smoke/marketplace-e2e-notifications.mjs`. Remaining work:
- capture per-user notification baseline; - capture per-user notification baseline;
- poll `GET /api/notifications` and `GET /api/notifications/unread-count`; - poll `GET /api/notifications` and `GET /api/notifications/unread-count`;
@@ -113,6 +114,13 @@ Turn [[Notification Assertion Procedure]] into reusable E2E helpers:
- record known notification gaps with route/action and expected recipient; - record known notification gaps with route/action and expected recipient;
- include notification latency in every E2E report. - include notification latency in every E2E report.
Current dev findings from the first two-round runner report:
- `NOTIF-E2E-001` passes for buyer new-offer notifications.
- Seller new-request fanout passes for privately targeted sellers.
- Rejected-seller notifications pass by rejected offer `relatedId`.
- `NOTIF-E2E-002` fails by selected offer `relatedId`; accepted-offer notification is not discoverable with the selected offer id.
Acceptance tests: Acceptance tests:
| ID | Expected | | ID | Expected |
@@ -125,8 +133,8 @@ Acceptance tests:
## P1 - Concurrency and Performance Profile ## P1 - Concurrency and Performance Profile
Implement [[Concurrency and Performance Profile]] as an executable runner and Initial executable runner and report generator are implemented in
report generator: `backend/scripts/smoke/marketplace-e2e-notifications.mjs`. Remaining work:
- worker abstraction for one isolated buyer/sellers/payment/delivery flow; - worker abstraction for one isolated buyer/sellers/payment/delivery flow;
- barrier start for simultaneous workers; - barrier start for simultaneous workers;
@@ -136,6 +144,13 @@ report generator:
- infrastructure snapshots before/during/after each stage; - infrastructure snapshots before/during/after each stage;
- stop-condition enforcement. - stop-condition enforcement.
Current dev blocker:
- Dev API rate limiting is `100` requests per `900s`, so C2+ ramp runs need a
fresh window, a lower-polling profile, or a non-rate-limited test environment.
- `PAYMENT_MODE=record` currently fails because `POST /api/marketplace/payments`
returns HTTP 500 after offer selection with generated test actors.
Acceptance tests: Acceptance tests:
| ID | Expected | | ID | Expected |
@@ -179,10 +194,13 @@ Add dashboards/log queries for:
## Suggested Next Automation Order ## Suggested Next Automation Order
1. Convert the two-round BSC Testnet E2E procedure into a committed smoke script. 1. Fix selected-seller accepted notification related id.
2. Add non-buyer delivery confirmation regression test. 2. Fix or retire the legacy `POST /api/marketplace/payments` record path.
3. Add scanner negative tests for wrong token and underpayment. 3. Run `PAYMENT_MODE=status` through delivery after the dev rate-limit window resets.
4. Implement physical vs digital release policy. 4. Run `PAYMENT_MODE=live` once wallet gas/tUSDT and rate-limit headroom are available.
5. Add release-policy tests. 5. Add non-buyer delivery confirmation regression test.
6. Resolve dispute route/policy gaps. 6. Add scanner negative tests for wrong token and underpayment.
7. Add dispute-to-release/refund E2E tests. 7. Implement physical vs digital release policy.
8. Add release-policy tests.
9. Resolve dispute route/policy gaps.
10. Add dispute-to-release/refund E2E tests.

View File

@@ -23,6 +23,7 @@ seller, payment scanner, delivery, payout, dispute, and deployment behavior.
| [[Scanner BSC Testnet Payment Procedure]] | BSC Testnet tUSDT scanner payment procedure and failure modes. | | [[Scanner BSC Testnet Payment Procedure]] | BSC Testnet tUSDT scanner payment procedure and failure modes. |
| [[Notification Assertion Procedure]] | Required notification checks after every E2E business step. | | [[Notification Assertion Procedure]] | Required notification checks after every E2E business step. |
| [[Concurrency and Performance Profile]] | Ramp test design, profiling targets, metrics, and report template. | | [[Concurrency and Performance Profile]] | Ramp test design, profiling targets, metrics, and report template. |
| [[Marketplace E2E Smoke Runner]] | Implemented backend smoke runner, modes, commands, reports, and current dev findings. |
| [[Smoke and Regression Procedure]] | CLI, CI, and post-deploy smoke checks. | | [[Smoke and Regression Procedure]] | CLI, CI, and post-deploy smoke checks. |
| [[Testing Expansion Backlog]] | Gaps to cover before broader release confidence. | | [[Testing Expansion Backlog]] | Gaps to cover before broader release confidence. |
@@ -74,3 +75,4 @@ The current live tested version of this path is documented in
- Record enough evidence to reproduce a failure: request id, offer id, payment id, tx hash, chain id, token address, HTTP status, and CI build number. - Record enough evidence to reproduce a failure: request id, offer id, payment id, tx hash, chain id, token address, HTTP status, and CI build number.
- Treat a passing local test as necessary but not sufficient for scanner/payment work. Payment changes must also be verified against dev after deploy. - Treat a passing local test as necessary but not sufficient for scanner/payment work. Payment changes must also be verified against dev after deploy.
- Every live-dev test should state what remains untested or blocked. - Every live-dev test should state what remains untested or blocked.
- Use `backend/scripts/smoke/marketplace-e2e-notifications.sh` for implemented buyer/seller smoke runs before extending scenarios manually.