nick-doc/11 - Testing/Marketplace E2E Smoke Runner.md

title, tags, created

title	tags	created
Marketplace E2E Smoke Runner	testing smoke e2e notifications performance	2026-06-06

Marketplace E2E Smoke Runner

The backend repo has a committed marketplace runner:

backend/scripts/smoke/marketplace-e2e-notifications.sh

It creates isolated test actors, runs buyer/seller marketplace flows, checks persisted notifications after every business mutation, and writes JSON/Markdown reports under ignored backend/tmp/e2e-reports/.

Required Environment

Do not put secrets in command history or docs. Load these from an ignored local env file or a secure shell session:

Variable	Purpose
BASE_URL	Target API, for example https://dev.amn.gg. Defaults to local http://127.0.0.1:5001.
ADMIN_EMAIL	Admin account email used only to create test users.
ADMIN_PASSWORD	Admin account password, never committed or printed.
PAYMENT_MODE	status, record, or live. Defaults to status.
CONCURRENCY_LEVELS	Comma-separated worker ramp, for example 1,2,4,8.
ROUNDS	Rounds per worker. Use 2 for the basic two-round requirement.
SELLERS_PER_WORKER	Sellers per flow. Minimum enforced by runner is 2; default is 3.
SKIP_WRONG_RECIPIENT_CHECK	Optional escape hatch for rate-limited environments. Defaults to false.

Payment Modes

Mode	Use	Notes
status	Cheap full-flow smoke without wallet funds.	Moves the request to payment through the status route, then exercises seller delivery and buyer confirmation.
record	Tests legacy POST /api/marketplace/payments.	Current dev run shows this route returns HTTP 500 with generated buyer/seller ids. Keep this mode for regression coverage.
live	Tests BSC Testnet tUSDT direct-balance scanner rail.	Requires E2E_PAYMENT_PRIVATE_KEY or E2E_PAYMENT_MNEMONIC, gas, and tUSDT. Sends ERC-20 transfer and calls direct-balance check.

Live mode also reads:

Variable	Default
E2E_NETWORK	bsc-testnet
E2E_CHAIN_ID	97
E2E_TOKEN_SYMBOL	USDT
RPC_URL_CHAIN_97 / BSC_TESTNET_RPC_URL	public BSC Testnet RPC fallback

Example Commands

Two-round full-flow smoke without wallet funds:

BASE_URL=https://dev.amn.gg \
PAYMENT_MODE=status \
CONCURRENCY_LEVELS=1 \
ROUNDS=2 \
bash scripts/smoke/marketplace-e2e-notifications.sh

Concurrency profile starting at one worker:

BASE_URL=https://dev.amn.gg \
PAYMENT_MODE=status \
CONCURRENCY_LEVELS=1,2,4,8,16,32 \
ROUNDS=1 \
bash scripts/smoke/marketplace-e2e-notifications.sh

Live BSC Testnet tUSDT smoke:

BASE_URL=https://dev.amn.gg \
PAYMENT_MODE=live \
CONCURRENCY_LEVELS=1 \
ROUNDS=1 \
bash scripts/smoke/marketplace-e2e-notifications.sh

Notification Behavior

The runner treats each state-changing business step as a notification checkpoint. Each checkpoint records:

• actor and role;
• expected related id;
• whether a persisted notification was found;
• whether the missing notification is a known gap;
• unread count from the notification list response.

By default, known gaps do not abort the flow. Set STRICT_NOTIFICATIONS=true to make known gaps fail the run. Set ABORT_ON_NOTIFICATION_FAILURE=true to stop at the first missing notification.

Output

Reports are ignored by Git:

backend/tmp/e2e-reports/marketplace-e2e-<runId>.json
backend/tmp/e2e-reports/marketplace-e2e-<runId>.md

The report includes:

• flow pass/fail counts;
• notification found/missing/known-gap counts;
• wrong-recipient notification count;
• API p50/p95/p99/max timings;
• request, offer, payment, token, and transaction ids where applicable.

Actor tokens and admin credentials are never written to the report.

Reference Dev Run - 2026-06-06

Command shape:

BASE_URL=https://dev.amn.gg PAYMENT_MODE=record CONCURRENCY_LEVELS=1 ROUNDS=2 ...

Result:

Metric	Value
flows passed	0/2
request-created notifications found	8/8
buyer new-offer notifications found	6/6
rejected-seller notifications found	4/4
selected-seller accepted notifications by related id	0/2
wrong-recipient notifications	0
legacy marketplace payment records	0/2, HTTP 500

Observed defects:

1. Selected-seller accepted notification is not discoverable by the selected offer relatedId. The service path appears to create the accepted-offer notification with an incomplete related id.
2. POST /api/marketplace/payments returned HTTP 500 after offer selection with generated test actors. This blocks PAYMENT_MODE=record; use PAYMENT_MODE=status for flow-only smoke or PAYMENT_MODE=live for scanner verification.
3. Dev API rate limiting is 100 requests per 900s; concurrency runs must use a fresh window or a non-rate-limited test environment.

Reference Direct-Backend Run - 2026-06-06

Command shape:

BASE_URL=http://127.0.0.1:15001 PAYMENT_MODE=status CONCURRENCY_LEVELS=1 ROUNDS=1 ...

This used a temporary SSH tunnel to the dev backend container to avoid the public edge rate-limit window.

Result:

Metric	Value
flows passed	1/1
final request status	delivered
wrong-recipient notifications	0
required notification missing	1
known notification gaps	2

Key ids:

Field	Value
purchaseRequestId	dc6ec076-9c15-46a1-ab6b-1d9d17604614
selectedOfferId	9f56dd00-8b08-4c9f-b91c-5875e1949113

Observed notification result:

• request-created notifications passed for buyer and three targeted sellers;
• buyer new-offer notifications passed for all three seller bids;
• rejected-seller notifications passed;
• selected-seller accepted notification still failed by selected offer relatedId;
• payment-confirmed notification was found in PAYMENT_MODE=status;
• seller delivery and buyer delivery confirmation notifications are known gaps.