docs: add testing procedures and scenario catalog
This commit is contained in:
Siavash Sameni
168
11 - Testing/Smoke and Regression Procedure.md
Normal file
168
11 - Testing/Smoke and Regression Procedure.md
Normal file
@@ -0,0 +1,168 @@
|
||||
---
|
||||
title: Smoke and Regression Procedure
|
||||
tags: [testing, smoke, regression, ci, deploy]
|
||||
created: 2026-06-06
|
||||
---
|
||||
|
||||
# Smoke and Regression Procedure
|
||||
|
||||
This page defines the standard checks before and after backend/frontend/scanner
|
||||
changes.
|
||||
|
||||
## Local/Pre-Push Rules
|
||||
|
||||
For frontend or backend code changes:
|
||||
|
||||
1. Run focused tests for the touched area.
|
||||
2. Run typecheck.
|
||||
3. Run `git diff --check`.
|
||||
4. Bump backend and frontend patch versions together if a build/deploy will be pushed.
|
||||
5. For payment/scanner changes, run the relevant smoke script before pushing.
|
||||
|
||||
Docs-only changes do not require version bumps.
|
||||
|
||||
## Backend Smoke Scripts
|
||||
|
||||
Backend smoke scripts live in:
|
||||
|
||||
```text
|
||||
backend/scripts/smoke/
|
||||
```
|
||||
|
||||
Current scripts:
|
||||
|
||||
| Script | Purpose |
|
||||
|---|---|
|
||||
| `backend-health.sh` | Basic API health/version target check. |
|
||||
| `auth-basic.sh` | Basic auth behavior. |
|
||||
| `addresses-basic.sh` | Address APIs. |
|
||||
| `bsc-testnet-payment-registry.sh` | Chain 97 token/RPC registry sanity. |
|
||||
| `confirmation-thresholds.sh` | Confirmation threshold APIs/settings. |
|
||||
| `funds-ledger-repo.sh` | Funds ledger repository behavior. |
|
||||
| `marketplace-request-budget.sh` | Purchase request budget validation. |
|
||||
| `rn-intent.sh` | Request Network / in-house intent creation smoke. |
|
||||
| `rn-webhook.sh` | Webhook shape/signature flow. |
|
||||
| `user-admin-postgres.sh` | Admin user persistence. |
|
||||
| `user-dependencies.sh` | User dependency smoke. |
|
||||
|
||||
Run against dev:
|
||||
|
||||
```bash
|
||||
cd ~/CascadeProjects/escrow/backend
|
||||
BASE_URL=https://dev.amn.gg bash scripts/smoke/bsc-testnet-payment-registry.sh
|
||||
```
|
||||
|
||||
Run against local:
|
||||
|
||||
```bash
|
||||
BASE_URL=http://127.0.0.1:5001 bash scripts/smoke/backend-health.sh
|
||||
```
|
||||
|
||||
## Focused Automated Tests
|
||||
|
||||
Payment/scanner backend focus:
|
||||
|
||||
```bash
|
||||
cd ~/CascadeProjects/escrow/backend
|
||||
npm test -- --runTestsByPath \
|
||||
__tests__/rn-in-house-checkout.test.ts \
|
||||
__tests__/decentralized-payment-verifier.test.ts \
|
||||
__tests__/amn-pay-adapter-intent.test.ts \
|
||||
--runInBand
|
||||
```
|
||||
|
||||
Marketplace/delivery focus:
|
||||
|
||||
```bash
|
||||
npm test -- --runTestsByPath \
|
||||
__tests__/simple-marketplace.test.ts \
|
||||
__tests__/marketplace-idor.test.ts \
|
||||
__tests__/escrow-state-machine.test.ts \
|
||||
--runInBand
|
||||
```
|
||||
|
||||
Frontend checkout focus:
|
||||
|
||||
```bash
|
||||
cd ~/CascadeProjects/escrow/frontend
|
||||
npx tsc --noEmit --ignoreDeprecations 6.0
|
||||
npx eslint src/web3/config.ts src/web3/types.ts src/sections/payment/checkout/rn-in-house-checkout-view.tsx
|
||||
npm run build
|
||||
```
|
||||
|
||||
Scanner focus:
|
||||
|
||||
```bash
|
||||
cd ~/CascadeProjects/escrow/scanner
|
||||
go test ./...
|
||||
```
|
||||
|
||||
## CI/Deploy Verification
|
||||
|
||||
Use Woodpecker with token from the local environment:
|
||||
|
||||
```bash
|
||||
export WOODPECKER_SERVER=https://ci.tbs.amn.gg
|
||||
export WOODPECKER_TOKEN=<local secret>
|
||||
```
|
||||
|
||||
Never commit or paste the token into docs.
|
||||
|
||||
Poll latest pipelines by repo id:
|
||||
|
||||
| Repo | Woodpecker repo id | Expected pipeline |
|
||||
|---|---:|---|
|
||||
| backend | `5` | typecheck, build-and-deploy, notify |
|
||||
| frontend | `7` | build-and-deploy, notify |
|
||||
| scanner | `8` | build/test/deploy |
|
||||
|
||||
Required post-deploy checks:
|
||||
|
||||
```bash
|
||||
curl -fsS https://dev.amn.gg/api/version
|
||||
curl -fsS -I https://dev.amn.gg/
|
||||
ssh -i <ssh-key> root@<dev-host> \
|
||||
"docker ps --format '{{.Names}}\t{{.Image}}\t{{.Status}}' | grep -E 'escrow-(backend|frontend|scanner)'"
|
||||
```
|
||||
|
||||
Expected:
|
||||
|
||||
- `/api/version` matches package version for backend/frontend build pair;
|
||||
- frontend root returns HTTP 200;
|
||||
- backend/frontend/scanner containers are healthy;
|
||||
- corresponding Woodpecker pipelines are `success`.
|
||||
|
||||
## Regression Evidence Format
|
||||
|
||||
For every test run, record:
|
||||
|
||||
```markdown
|
||||
### YYYY-MM-DD - <short summary>
|
||||
|
||||
- Target: `https://dev.amn.gg`
|
||||
- Versions: backend/frontend/scanner
|
||||
- CI: backend #, frontend #, scanner #
|
||||
- Commands:
|
||||
- `...`
|
||||
- Live scenarios:
|
||||
- run id
|
||||
- request ids
|
||||
- payment ids
|
||||
- tx hashes
|
||||
- Result: pass/fail/blocked
|
||||
- Residual risk:
|
||||
- ...
|
||||
```
|
||||
|
||||
## When To Stop a Release
|
||||
|
||||
Stop or roll back if any P0 condition fails:
|
||||
|
||||
- payment can be marked paid without correct chain/token/destination/amount evidence;
|
||||
- buyer cannot complete a valid payment;
|
||||
- non-buyer can confirm delivery;
|
||||
- frontend cannot connect/switch to the chain returned by backend checkout;
|
||||
- backend/frontend/scanner registry mismatch exists;
|
||||
- CI deploy reports success but live containers are not healthy;
|
||||
- secrets appear in git status, logs, docs, or CI output.
|
||||
|
||||
Reference in New Issue
Block a user