diff --git a/09 - Audits/Activity Log.md b/09 - Audits/Activity Log.md index 3bf55bd..fc282a7 100644 --- a/09 - Audits/Activity Log.md +++ b/09 - Audits/Activity Log.md @@ -11,6 +11,16 @@ entries on top. Maintained by agents per the rule in `../AGENTS.md`. --- +### 2026-06-06 — backend@ca2b1c4, frontend@aac3304 — Marketplace E2E notification smoke runner + +**Commits:** `ca2b1c4` `aac3304` +**Touched:** backend `scripts/smoke/marketplace-e2e-notifications.mjs`, `scripts/smoke/marketplace-e2e-notifications.sh`, `package.json`, `package-lock.json`; frontend `package.json`, `Dockerfile`; docs `11 - Testing/Marketplace E2E Smoke Runner.md`, testing procedure pages. +**Why:** Implement the requested buyer/seller marketplace E2E runner with at least two sellers, notification assertions after every state-changing step, report generation, concurrency ramp controls, and BSC Testnet live-payment support. Rebased on newer Forgejo `main` and bumped backend/frontend to `2.9.2`. +**Verification:** backend `node --check scripts/smoke/marketplace-e2e-notifications.mjs`, `npm run typecheck`, `git diff --check`; frontend `git diff --check` and version/Dockerfile sanity; nick-doc `git diff --check`. Dev two-round `PAYMENT_MODE=record` report exposed HTTP 500 on legacy `POST /api/marketplace/payments` and missing selected-seller accepted notification. Direct-backend one-round `PAYMENT_MODE=status` reached final request status `delivered` with notification gaps recorded. Woodpecker build pending after push. +**Linked docs updated:** [[11 - Testing/Marketplace E2E Smoke Runner]], [[11 - Testing/Escrow Marketplace E2E Procedure]], [[11 - Testing/Smoke and Regression Procedure]], [[11 - Testing/Concurrency and Performance Profile]], [[11 - Testing/Testing Expansion Backlog]] + +--- + ### 2026-06-06 — backend@31b285f, frontend@2a3e5c9 — BSC Testnet checkout UI support **Commits:** `31b285f` `2a3e5c9` diff --git a/11 - Testing/Concurrency and Performance Profile.md b/11 - Testing/Concurrency and Performance Profile.md index b09f2e6..74b3339 100644 --- a/11 - Testing/Concurrency and Performance Profile.md +++ b/11 - Testing/Concurrency and Performance Profile.md @@ -36,6 +36,21 @@ Each worker must use unique: Notifications are mandatory inside every worker. See [[Notification Assertion Procedure]]. +Implemented runner: + +```bash +cd ~/CascadeProjects/escrow/backend +BASE_URL=https://dev.amn.gg \ +PAYMENT_MODE=status \ +CONCURRENCY_LEVELS=1,2,4,8,16,32 \ +ROUNDS=1 \ +bash scripts/smoke/marketplace-e2e-notifications.sh +``` + +Use `PAYMENT_MODE=live` for low-concurrency BSC Testnet tUSDT confirmation. +Use `PAYMENT_MODE=status` for high-concurrency marketplace/notification +profiling without consuming gas. + ## Ramp Plan Start with one simultaneous worker and double until a stop condition is reached: @@ -59,6 +74,7 @@ stage. | Mode | Payment behavior | Use | |---|---|---| | Live-chain mode | Real BSC Testnet tUSDT transfers | Final confidence at low concurrency; expensive/slower; consumes gas. | +| Status-only smoke mode | Moves accepted request to `payment` through the status route | Implemented high-concurrency marketplace/notification profiling without chain variables. | | Scanner fixture mode | Deterministic scanner/balance fixture or controlled test endpoint | High concurrency without chain bottleneck. Must not be enabled in production. | | API-only dry run | Runs request/offer/delivery and skips payment finalization | Marketplace/notification profiling without chain variables. | @@ -267,4 +283,3 @@ These are the expectations to validate: - Scanner live-chain checks are likely bounded by BSC Testnet RPC latency and should be separated from API-only profiling. - Payment intent creation may become slower if destination derivation, token registry lookup, and scanner registration are serial. - Socket fanout should be watched at C16+ because each worker has multiple actors and multiple tabs/devices may multiply room membership. - diff --git a/11 - Testing/Escrow Marketplace E2E Procedure.md b/11 - Testing/Escrow Marketplace E2E Procedure.md index 375c4b2..4275309 100644 --- a/11 - Testing/Escrow Marketplace E2E Procedure.md +++ b/11 - Testing/Escrow Marketplace E2E Procedure.md @@ -30,6 +30,20 @@ or scanner changes. ## High-Level Procedure +Use the implemented backend runner for repeatable execution: + +```bash +BASE_URL=https://dev.amn.gg \ +PAYMENT_MODE=status \ +CONCURRENCY_LEVELS=1 \ +ROUNDS=2 \ +bash scripts/smoke/marketplace-e2e-notifications.sh +``` + +Use `PAYMENT_MODE=live` for the funded BSC Testnet tUSDT rail and +`PAYMENT_MODE=record` only when explicitly testing the legacy marketplace +payment-record route. See [[Marketplace E2E Smoke Runner]]. + 1. Generate a run id. 2. Admin creates one buyer and at least two sellers. 3. Assert initial notification baselines for buyer and sellers. diff --git a/11 - Testing/Marketplace E2E Smoke Runner.md b/11 - Testing/Marketplace E2E Smoke Runner.md new file mode 100644 index 0000000..5501c71 --- /dev/null +++ b/11 - Testing/Marketplace E2E Smoke Runner.md @@ -0,0 +1,185 @@ +--- +title: Marketplace E2E Smoke Runner +tags: [testing, smoke, e2e, notifications, performance] +created: 2026-06-06 +--- + +# Marketplace E2E Smoke Runner + +The backend repo has a committed marketplace runner: + +```bash +backend/scripts/smoke/marketplace-e2e-notifications.sh +``` + +It creates isolated test actors, runs buyer/seller marketplace flows, checks +persisted notifications after every business mutation, and writes JSON/Markdown +reports under ignored `backend/tmp/e2e-reports/`. + +## Required Environment + +Do not put secrets in command history or docs. Load these from an ignored local +env file or a secure shell session: + +| Variable | Purpose | +|---|---| +| `BASE_URL` | Target API, for example `https://dev.amn.gg`. Defaults to local `http://127.0.0.1:5001`. | +| `ADMIN_EMAIL` | Admin account email used only to create test users. | +| `ADMIN_PASSWORD` | Admin account password, never committed or printed. | +| `PAYMENT_MODE` | `status`, `record`, or `live`. Defaults to `status`. | +| `CONCURRENCY_LEVELS` | Comma-separated worker ramp, for example `1,2,4,8`. | +| `ROUNDS` | Rounds per worker. Use `2` for the basic two-round requirement. | +| `SELLERS_PER_WORKER` | Sellers per flow. Minimum enforced by runner is `2`; default is `3`. | +| `SKIP_WRONG_RECIPIENT_CHECK` | Optional escape hatch for rate-limited environments. Defaults to `false`. | + +## Payment Modes + +| Mode | Use | Notes | +|---|---|---| +| `status` | Cheap full-flow smoke without wallet funds. | Moves the request to `payment` through the status route, then exercises seller delivery and buyer confirmation. | +| `record` | Tests legacy `POST /api/marketplace/payments`. | Current dev run shows this route returns HTTP 500 with generated buyer/seller ids. Keep this mode for regression coverage. | +| `live` | Tests BSC Testnet tUSDT direct-balance scanner rail. | Requires `E2E_PAYMENT_PRIVATE_KEY` or `E2E_PAYMENT_MNEMONIC`, gas, and tUSDT. Sends ERC-20 transfer and calls direct-balance check. | + +Live mode also reads: + +| Variable | Default | +|---|---| +| `E2E_NETWORK` | `bsc-testnet` | +| `E2E_CHAIN_ID` | `97` | +| `E2E_TOKEN_SYMBOL` | `USDT` | +| `RPC_URL_CHAIN_97` / `BSC_TESTNET_RPC_URL` | public BSC Testnet RPC fallback | + +## Example Commands + +Two-round full-flow smoke without wallet funds: + +```bash +BASE_URL=https://dev.amn.gg \ +PAYMENT_MODE=status \ +CONCURRENCY_LEVELS=1 \ +ROUNDS=2 \ +bash scripts/smoke/marketplace-e2e-notifications.sh +``` + +Concurrency profile starting at one worker: + +```bash +BASE_URL=https://dev.amn.gg \ +PAYMENT_MODE=status \ +CONCURRENCY_LEVELS=1,2,4,8,16,32 \ +ROUNDS=1 \ +bash scripts/smoke/marketplace-e2e-notifications.sh +``` + +Live BSC Testnet tUSDT smoke: + +```bash +BASE_URL=https://dev.amn.gg \ +PAYMENT_MODE=live \ +CONCURRENCY_LEVELS=1 \ +ROUNDS=1 \ +bash scripts/smoke/marketplace-e2e-notifications.sh +``` + +## Notification Behavior + +The runner treats each state-changing business step as a notification checkpoint. +Each checkpoint records: + +- actor and role; +- expected related id; +- whether a persisted notification was found; +- whether the missing notification is a known gap; +- unread count from the notification list response. + +By default, known gaps do not abort the flow. Set `STRICT_NOTIFICATIONS=true` +to make known gaps fail the run. Set `ABORT_ON_NOTIFICATION_FAILURE=true` to +stop at the first missing notification. + +## Output + +Reports are ignored by Git: + +```text +backend/tmp/e2e-reports/marketplace-e2e-.json +backend/tmp/e2e-reports/marketplace-e2e-.md +``` + +The report includes: + +- flow pass/fail counts; +- notification found/missing/known-gap counts; +- wrong-recipient notification count; +- API p50/p95/p99/max timings; +- request, offer, payment, token, and transaction ids where applicable. + +Actor tokens and admin credentials are never written to the report. + +## Reference Dev Run - 2026-06-06 + +Command shape: + +```bash +BASE_URL=https://dev.amn.gg PAYMENT_MODE=record CONCURRENCY_LEVELS=1 ROUNDS=2 ... +``` + +Result: + +| Metric | Value | +|---|---:| +| flows passed | `0/2` | +| request-created notifications found | `8/8` | +| buyer new-offer notifications found | `6/6` | +| rejected-seller notifications found | `4/4` | +| selected-seller accepted notifications by related id | `0/2` | +| wrong-recipient notifications | `0` | +| legacy marketplace payment records | `0/2`, HTTP 500 | + +Observed defects: + +1. Selected-seller accepted notification is not discoverable by the selected + offer `relatedId`. The service path appears to create the accepted-offer + notification with an incomplete related id. +2. `POST /api/marketplace/payments` returned HTTP 500 after offer selection + with generated test actors. This blocks `PAYMENT_MODE=record`; use + `PAYMENT_MODE=status` for flow-only smoke or `PAYMENT_MODE=live` for scanner + verification. +3. Dev API rate limiting is `100` requests per `900s`; concurrency runs must + use a fresh window or a non-rate-limited test environment. + +## Reference Direct-Backend Run - 2026-06-06 + +Command shape: + +```bash +BASE_URL=http://127.0.0.1:15001 PAYMENT_MODE=status CONCURRENCY_LEVELS=1 ROUNDS=1 ... +``` + +This used a temporary SSH tunnel to the dev backend container to avoid the +public edge rate-limit window. + +Result: + +| Metric | Value | +|---|---:| +| flows passed | `1/1` | +| final request status | `delivered` | +| wrong-recipient notifications | `0` | +| required notification missing | `1` | +| known notification gaps | `2` | + +Key ids: + +| Field | Value | +|---|---| +| purchaseRequestId | `dc6ec076-9c15-46a1-ab6b-1d9d17604614` | +| selectedOfferId | `9f56dd00-8b08-4c9f-b91c-5875e1949113` | + +Observed notification result: + +- request-created notifications passed for buyer and three targeted sellers; +- buyer new-offer notifications passed for all three seller bids; +- rejected-seller notifications passed; +- selected-seller accepted notification still failed by selected offer `relatedId`; +- payment-confirmed notification was found in `PAYMENT_MODE=status`; +- seller delivery and buyer delivery confirmation notifications are known gaps. diff --git a/11 - Testing/Smoke and Regression Procedure.md b/11 - Testing/Smoke and Regression Procedure.md index 327161a..4cfcda1 100644 --- a/11 - Testing/Smoke and Regression Procedure.md +++ b/11 - Testing/Smoke and Regression Procedure.md @@ -39,6 +39,7 @@ Current scripts: | `bsc-testnet-payment-registry.sh` | Chain 97 token/RPC registry sanity. | | `confirmation-thresholds.sh` | Confirmation threshold APIs/settings. | | `funds-ledger-repo.sh` | Funds ledger repository behavior. | +| `marketplace-e2e-notifications.sh` | Buyer/seller E2E smoke with notification assertions, concurrency levels, and reports. | | `marketplace-request-budget.sh` | Purchase request budget validation. | | `rn-intent.sh` | Request Network / in-house intent creation smoke. | | `rn-webhook.sh` | Webhook shape/signature flow. | @@ -52,6 +53,21 @@ cd ~/CascadeProjects/escrow/backend BASE_URL=https://dev.amn.gg bash scripts/smoke/bsc-testnet-payment-registry.sh ``` +Two-round marketplace smoke: + +```bash +cd ~/CascadeProjects/escrow/backend +BASE_URL=https://dev.amn.gg \ +PAYMENT_MODE=status \ +CONCURRENCY_LEVELS=1 \ +ROUNDS=2 \ +bash scripts/smoke/marketplace-e2e-notifications.sh +``` + +Use `PAYMENT_MODE=live` when the BSC Testnet wallet has gas and canonical +tUSDT. The runner writes reports to ignored `backend/tmp/e2e-reports/`; see +[[Marketplace E2E Smoke Runner]]. + Run against local: ```bash @@ -165,4 +181,3 @@ Stop or roll back if any P0 condition fails: - backend/frontend/scanner registry mismatch exists; - CI deploy reports success but live containers are not healthy; - secrets appear in git status, logs, docs, or CI output. - diff --git a/11 - Testing/Testing Expansion Backlog.md b/11 - Testing/Testing Expansion Backlog.md index 6dd4f2e..44ec6b8 100644 --- a/11 - Testing/Testing Expansion Backlog.md +++ b/11 - Testing/Testing Expansion Backlog.md @@ -104,7 +104,8 @@ Add Playwright tests for: ## P1 - Notification Assertions -Turn [[Notification Assertion Procedure]] into reusable E2E helpers: +Initial reusable E2E helpers are implemented in +`backend/scripts/smoke/marketplace-e2e-notifications.mjs`. Remaining work: - capture per-user notification baseline; - poll `GET /api/notifications` and `GET /api/notifications/unread-count`; @@ -113,6 +114,13 @@ Turn [[Notification Assertion Procedure]] into reusable E2E helpers: - record known notification gaps with route/action and expected recipient; - include notification latency in every E2E report. +Current dev findings from the first two-round runner report: + +- `NOTIF-E2E-001` passes for buyer new-offer notifications. +- Seller new-request fanout passes for privately targeted sellers. +- Rejected-seller notifications pass by rejected offer `relatedId`. +- `NOTIF-E2E-002` fails by selected offer `relatedId`; accepted-offer notification is not discoverable with the selected offer id. + Acceptance tests: | ID | Expected | @@ -125,8 +133,8 @@ Acceptance tests: ## P1 - Concurrency and Performance Profile -Implement [[Concurrency and Performance Profile]] as an executable runner and -report generator: +Initial executable runner and report generator are implemented in +`backend/scripts/smoke/marketplace-e2e-notifications.mjs`. Remaining work: - worker abstraction for one isolated buyer/sellers/payment/delivery flow; - barrier start for simultaneous workers; @@ -136,6 +144,13 @@ report generator: - infrastructure snapshots before/during/after each stage; - stop-condition enforcement. +Current dev blocker: + +- Dev API rate limiting is `100` requests per `900s`, so C2+ ramp runs need a + fresh window, a lower-polling profile, or a non-rate-limited test environment. +- `PAYMENT_MODE=record` currently fails because `POST /api/marketplace/payments` + returns HTTP 500 after offer selection with generated test actors. + Acceptance tests: | ID | Expected | @@ -179,10 +194,13 @@ Add dashboards/log queries for: ## Suggested Next Automation Order -1. Convert the two-round BSC Testnet E2E procedure into a committed smoke script. -2. Add non-buyer delivery confirmation regression test. -3. Add scanner negative tests for wrong token and underpayment. -4. Implement physical vs digital release policy. -5. Add release-policy tests. -6. Resolve dispute route/policy gaps. -7. Add dispute-to-release/refund E2E tests. +1. Fix selected-seller accepted notification related id. +2. Fix or retire the legacy `POST /api/marketplace/payments` record path. +3. Run `PAYMENT_MODE=status` through delivery after the dev rate-limit window resets. +4. Run `PAYMENT_MODE=live` once wallet gas/tUSDT and rate-limit headroom are available. +5. Add non-buyer delivery confirmation regression test. +6. Add scanner negative tests for wrong token and underpayment. +7. Implement physical vs digital release policy. +8. Add release-policy tests. +9. Resolve dispute route/policy gaps. +10. Add dispute-to-release/refund E2E tests. diff --git a/11 - Testing/Testing Overview.md b/11 - Testing/Testing Overview.md index b96ae55..6ece65a 100644 --- a/11 - Testing/Testing Overview.md +++ b/11 - Testing/Testing Overview.md @@ -23,6 +23,7 @@ seller, payment scanner, delivery, payout, dispute, and deployment behavior. | [[Scanner BSC Testnet Payment Procedure]] | BSC Testnet tUSDT scanner payment procedure and failure modes. | | [[Notification Assertion Procedure]] | Required notification checks after every E2E business step. | | [[Concurrency and Performance Profile]] | Ramp test design, profiling targets, metrics, and report template. | +| [[Marketplace E2E Smoke Runner]] | Implemented backend smoke runner, modes, commands, reports, and current dev findings. | | [[Smoke and Regression Procedure]] | CLI, CI, and post-deploy smoke checks. | | [[Testing Expansion Backlog]] | Gaps to cover before broader release confidence. | @@ -74,3 +75,4 @@ The current live tested version of this path is documented in - Record enough evidence to reproduce a failure: request id, offer id, payment id, tx hash, chain id, token address, HTTP status, and CI build number. - Treat a passing local test as necessary but not sufficient for scanner/payment work. Payment changes must also be verified against dev after deploy. - Every live-dev test should state what remains untested or blocked. +- Use `backend/scripts/smoke/marketplace-e2e-notifications.sh` for implemented buyer/seller smoke runs before extending scenarios manually.