docs: sync from backend cdc8df1 + frontend a5dd48e + scanner 8fee27e — AMN Pay Scanner
- Activity Log: new entry for AMN Pay Scanner implementation - Environment Variables: document AMN_SCANNER_URL, AMN_SCANNER_WEBHOOK_SECRET, AMN_SCANNER_DEFAULT - PRD status table: mark all components implemented
This commit is contained in:
Siavash Sameni
@@ -326,6 +326,11 @@ SWEEP_MASTER_PRIVKEY=
|
||||
SWEEP_GAS_MIN_BNB=0.001
|
||||
SWEEP_GAS_TOP_UP_BNB=0.002
|
||||
|
||||
# AMN Pay Scanner (replaces Request Network for pay-in detection)
|
||||
AMN_SCANNER_URL=
|
||||
AMN_SCANNER_WEBHOOK_SECRET=
|
||||
AMN_SCANNER_DEFAULT=false
|
||||
|
||||
# OAuth
|
||||
GOOGLE_CLIENT_ID=
|
||||
```
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
# TODO: Secret Management Overhaul + Deploy Migration
|
||||
|
||||
> Status: **Deferred — created 2026-05-29**
|
||||
> Owner: infra / nick
|
||||
> Trigger: discovered while wiring Telegram startup notifications (2026-05-29)
|
||||
|
||||
A dedicated pass to (a) rotate every secret that has lived in git, (b) move secrets out of committed files into the right injection layer, and (c) collapse the two-stack deploy split. All dev data is disposable (no data-migration concern).
|
||||
|
||||
## 1. Rotate secrets (all have been committed in git → treat as compromised)
|
||||
|
||||
Rotate at the provider, then place per the build-vs-runtime split (decided 2026-05-29).
|
||||
|
||||
**→ Runtime (live-stack env injection):**
|
||||
- `JWT_SECRET` (rotation logs everyone out — fine)
|
||||
- `MONGODB_URI` password + `MONGO_INITDB_ROOT_PASSWORD` (same value)
|
||||
- `REDIS_URI` password
|
||||
- `ADMIN_PASSWORD`
|
||||
- `GOOGLE_CLIENT_SECRET` (Google Cloud Console)
|
||||
- `SMTP_PASS` (Resend dashboard — revoke + new API key)
|
||||
- `REQUEST_NETWORK_API_KEY` (RN dashboard)
|
||||
- `REQUEST_NETWORK_WEBHOOK_SECRET` (+ update RN webhook config)
|
||||
- `TELEGRAM_BOT_TOKEN` (Mini App bot — @BotFather /revoke)
|
||||
- `TELEGRAM_WEBHOOK_SECRET_TOKEN` (+ re-set on Telegram setWebhook)
|
||||
- `TG_NOTIFY_BOT_TOKEN` (amnGG_MonitorBot — already a dedicated bot; rotate if desired)
|
||||
|
||||
**→ Build-time (frontend Woodpecker secret, baked into image):**
|
||||
- `NEXT_PUBLIC_ALCHEMY_API_KEY_*` (one secret `alchemy_api_key` feeds all 3 args) — rotate in Alchemy **and domain-restrict to dev.amn.gg** (it ships in the browser bundle, so the origin allowlist is the real protection).
|
||||
|
||||
**Public by design — no rotation:** Google client IDs, `NEXT_PUBLIC_WALLETCONNECT_PROJECT_ID` (domain-restrict instead), `NEXT_PUBLIC_TELEGRAM_BOT_ID`, wallet addresses, `REQUEST_NETWORK_MERCHANT_REFERENCE`, URLs/flags.
|
||||
|
||||
## 2. Strip secrets from committed files
|
||||
- `backend/.env.example` — currently holds live values; reduce to reference-only (keys + non-secret defaults). Safe: it is **docs-only**, not read at runtime.
|
||||
- `frontend/.env.production` — same; not copied into the runtime image by the Dockerfile.
|
||||
- Remove the gitleaks token allowlists once tokens are gone.
|
||||
|
||||
## 3. Collapse the two-stack deploy split
|
||||
See [[deploy_architecture_two_stacks]] (memory). Make `escrow-deploy` (gitops from `nick/deploy`) the canonical live stack; retire `devEscrow`; point the Woodpecker redeploy step at `escrow-deploy` (`8cbe7b2a…`) so sync + redeploy target the same project. This also fixes the cosmetic redeploy **400**.
|
||||
- Pending local edits already staged for this in `~/CascadeProjects/escrow/deployment` (uncommitted): `TG_NOTIFY_BOT_TOKEN` + `TG_NOTIFY_CHATS` added to `.env` and wired into both service `environment:` blocks in `docker-compose.yml`.
|
||||
- Container names are identical across both stacks → cutover has a brief collision/downtime window (acceptable; test data).
|
||||
|
||||
## Injection model (decided)
|
||||
- Build-time `NEXT_PUBLIC_*` → Woodpecker secrets (frontend `build_args`).
|
||||
- Runtime secrets → live-stack env (deploy-repo `.env` once `escrow-deploy` is canonical).
|
||||
@@ -11,6 +11,19 @@ entries on top. Maintained by agents per the rule in `../AGENTS.md`.
|
||||
|
||||
---
|
||||
|
||||
### 2026-05-29 — backend@cdc8df1 — AMN Pay Scanner integration (retire Request Network)
|
||||
|
||||
**Commits:** backend `cdc8df1`, scanner `8fee27e`
|
||||
**Touched:**
|
||||
- Backend: `src/services/payment/adapters/amnPayAdapter.ts`, `src/routes/amnScannerWebhookRoutes.ts`, `src/services/payment/adapters/types.ts`, `src/services/payment/providerConfig.ts`, `src/app.ts`, `.env.example`, `docker-compose.dev.yml`, `docker-compose.production.yml`
|
||||
- Scanner (new repo): `scanner/*.go`, `Dockerfile`, `supported-chains.json`
|
||||
- Frontend: `src/actions/network-registry.ts`, `src/sections/admin/networks/networks-list-view.tsx`
|
||||
**Why:** Implement AMN Pay Scanner per `PRD - Retire Request Network — In-House Payment Scanner.md`. Standalone Go microservice scans `ERC20FeeProxy` `TransferWithReferenceAndFee` events directly, eliminating RN API dependency. Supports any destination address (derived HD wallets enabled). Parallel run: RN stays active for existing payments; new payments route to scanner when `AMN_SCANNER_URL` is configured.
|
||||
**Verification:** `tsc --noEmit` clean. Scanner binary builds (`go build`). Go tests pass (3/3). Frontend networks page renders scanner lag column.
|
||||
**Linked docs updated:** [[07 - Development/Environment Variables]], [[PRD - Retire Request Network — In-House Payment Scanner]]
|
||||
|
||||
---
|
||||
|
||||
### 2026-05-29 — backend@7688f57 — Sweep gas strategy: PermitPull + GasTopUp signers
|
||||
|
||||
**Commits:** backend `7688f57`
|
||||
|
||||
197
PRD - Auth CAPTCHA (Cloudflare Turnstile).md
Normal file
197
PRD - Auth CAPTCHA (Cloudflare Turnstile).md
Normal file
@@ -0,0 +1,197 @@
|
||||
# PRD — Auth CAPTCHA: Replace Hard Rate Limiter with Cloudflare Turnstile
|
||||
|
||||
> Status: **Ready for implementation**
|
||||
> Task: #12
|
||||
> Priority: Medium — blocking dev velocity (15-min lockouts during testing)
|
||||
> Effort estimate: ~3-4 hours
|
||||
|
||||
---
|
||||
|
||||
## Problem
|
||||
|
||||
The current auth limiter (`authLimiter`) blocks all login attempts from an IP after 10 requests in 15 minutes. This:
|
||||
- Locks out legitimate users (and developers) for 15 minutes on any burst of requests
|
||||
- Is bypassable by rotating IPs anyway — provides no real security
|
||||
- Does not distinguish between a brute-force bot and a test script or user with a password manager
|
||||
|
||||
**Stopgap already applied (2.6.54):** threshold raised to 100/15 min to unblock dev. This PRD replaces it with a real solution.
|
||||
|
||||
---
|
||||
|
||||
## Solution: Cloudflare Turnstile (invisible CAPTCHA)
|
||||
|
||||
Cloudflare Turnstile is the right choice:
|
||||
- **Invisible** — no "click the traffic lights" for users; challenge runs silently in the background
|
||||
- **Server-side verification** — a simple POST to `https://challenges.cloudflare.com/turnstile/v0/siteverify`
|
||||
- **Free tier** covers our scale
|
||||
- **No Google dependency** (unlike reCAPTCHA)
|
||||
|
||||
---
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
1. After **3 failed** login attempts from the same IP (within a 15-min window), subsequent attempts from that IP require a valid Turnstile token in the request body (`cf-turnstile-response`).
|
||||
2. Valid token → login attempt proceeds normally. Invalid/missing token → `429` with `{ captchaRequired: true }`.
|
||||
3. Successful login resets the failed-attempt counter for that IP.
|
||||
4. The hard 15-min IP block is **removed** (or raised to 1000+ as a last-resort DoS backstop only).
|
||||
5. A new env var `TURNSTILE_SECRET_KEY` (server-side) and `NEXT_PUBLIC_TURNSTILE_SITE_KEY` (frontend) configure the integration.
|
||||
6. If `TURNSTILE_SECRET_KEY` is not set, the middleware logs a warning and skips CAPTCHA enforcement (safe for local dev without creds).
|
||||
|
||||
---
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
### Backend (`backend/src/`)
|
||||
|
||||
**1. New middleware: `src/shared/middleware/captchaMiddleware.ts`**
|
||||
|
||||
```typescript
|
||||
// Pseudo-code — agent should implement this properly
|
||||
import { Request, Response, NextFunction } from 'express';
|
||||
|
||||
const TURNSTILE_VERIFY = 'https://challenges.cloudflare.com/turnstile/v0/siteverify';
|
||||
const failedAttempts = new Map<string, { count: number; resetAt: number }>();
|
||||
|
||||
export const captchaGate = async (req: Request, res: Response, next: NextFunction) => {
|
||||
const secret = process.env.TURNSTILE_SECRET_KEY;
|
||||
if (!secret) return next(); // local dev without creds: skip
|
||||
|
||||
const ip = req.ip || req.headers['x-forwarded-for'] as string || 'unknown';
|
||||
const now = Date.now();
|
||||
const WINDOW = 15 * 60 * 1000;
|
||||
const THRESHOLD = 3;
|
||||
|
||||
// Cleanup expired windows
|
||||
const state = failedAttempts.get(ip);
|
||||
if (state && now > state.resetAt) failedAttempts.delete(ip);
|
||||
|
||||
const current = failedAttempts.get(ip);
|
||||
if (!current || current.count < THRESHOLD) {
|
||||
// Under threshold — proceed, but mark as "pending" to track outcome
|
||||
(req as any)._captchaIp = ip;
|
||||
return next();
|
||||
}
|
||||
|
||||
// Over threshold — require CAPTCHA token
|
||||
const token = req.body?.['cf-turnstile-response'];
|
||||
if (!token) {
|
||||
return res.status(429).json({
|
||||
success: false,
|
||||
captchaRequired: true,
|
||||
message: 'Too many failed attempts. Please complete the CAPTCHA.',
|
||||
});
|
||||
}
|
||||
|
||||
// Verify with Cloudflare
|
||||
const formData = new FormData();
|
||||
formData.append('secret', secret);
|
||||
formData.append('response', token);
|
||||
formData.append('remoteip', ip);
|
||||
|
||||
const cfRes = await fetch(TURNSTILE_VERIFY, { method: 'POST', body: formData });
|
||||
const cfJson = await cfRes.json() as { success: boolean };
|
||||
|
||||
if (!cfJson.success) {
|
||||
return res.status(429).json({
|
||||
success: false,
|
||||
captchaRequired: true,
|
||||
message: 'CAPTCHA verification failed.',
|
||||
});
|
||||
}
|
||||
|
||||
(req as any)._captchaIp = ip;
|
||||
next();
|
||||
};
|
||||
|
||||
// Call this on failed login
|
||||
export const recordFailedLogin = (ip: string) => {
|
||||
const now = Date.now();
|
||||
const WINDOW = 15 * 60 * 1000;
|
||||
const state = failedAttempts.get(ip) || { count: 0, resetAt: now + WINDOW };
|
||||
state.count += 1;
|
||||
failedAttempts.set(ip, state);
|
||||
};
|
||||
|
||||
// Call this on successful login
|
||||
export const clearFailedLogin = (ip: string) => {
|
||||
failedAttempts.delete(ip);
|
||||
};
|
||||
```
|
||||
|
||||
**2. Wire into `app.ts`**
|
||||
|
||||
Replace `app.use("/api/auth", authLimiter)` with:
|
||||
```typescript
|
||||
import { captchaGate } from './shared/middleware/captchaMiddleware';
|
||||
app.use("/api/auth/login", captchaGate);
|
||||
app.use("/api/auth", authLimiter); // keep as DoS backstop at max: 1000
|
||||
```
|
||||
|
||||
**3. Wire `recordFailedLogin` / `clearFailedLogin` into the auth controller**
|
||||
|
||||
In the login handler (wherever `401 Invalid credentials` is returned), call `recordFailedLogin(req.ip)`.
|
||||
On successful login, call `clearFailedLogin(req.ip)`.
|
||||
|
||||
Find the login handler: `src/controllers/authController.ts` or similar — grep for `"Invalid credentials"` or `"incorrect password"`.
|
||||
|
||||
### Frontend (`frontend/src/`)
|
||||
|
||||
**4. Add Turnstile widget to the login form**
|
||||
|
||||
- Install: `@marsidev/react-turnstile` (lightweight React wrapper, ~3KB)
|
||||
- Render `<Turnstile siteKey={process.env.NEXT_PUBLIC_TURNSTILE_SITE_KEY} />` in the login form
|
||||
- The widget is **invisible by default** — it renders a hidden iframe, solves automatically, and calls `onSuccess(token)` with the token
|
||||
- Store the token in component state and include it as `cf-turnstile-response` in the login POST body
|
||||
- Only show the widget when the API returns `{ captchaRequired: true }` (progressive: invisible until needed)
|
||||
|
||||
Find login form: `src/app/(auth)/login/page.tsx` or `src/components/auth/LoginForm.tsx` — grep for the login form component.
|
||||
|
||||
**5. Handle `captchaRequired: true` in the API action**
|
||||
|
||||
In `src/actions/auth.ts` (or wherever login is dispatched):
|
||||
```typescript
|
||||
if (res.status === 429 && data.captchaRequired) {
|
||||
// Show the Turnstile widget — user needs to solve it
|
||||
setCaptchaRequired(true);
|
||||
return;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Environment Variables
|
||||
|
||||
| Variable | Where | Value |
|
||||
|---|---|---|
|
||||
| `TURNSTILE_SECRET_KEY` | Backend `.env` / deploy stack | From Cloudflare dashboard |
|
||||
| `NEXT_PUBLIC_TURNSTILE_SITE_KEY` | Frontend build arg (Woodpecker) | From Cloudflare dashboard |
|
||||
|
||||
### Getting Cloudflare Turnstile credentials
|
||||
1. Go to [dash.cloudflare.com](https://dash.cloudflare.com) → Turnstile → Add Site
|
||||
2. Domain: `dev.amn.gg` (and `amn.gg` when live)
|
||||
3. Widget type: **Managed** (invisible, automatic challenge)
|
||||
4. Copy **Site Key** → `NEXT_PUBLIC_TURNSTILE_SITE_KEY`
|
||||
5. Copy **Secret Key** → `TURNSTILE_SECRET_KEY`
|
||||
|
||||
---
|
||||
|
||||
## What NOT to do
|
||||
|
||||
- Do not use Google reCAPTCHA (data dependency, flaky v3 scores)
|
||||
- Do not block by IP permanently — Turnstile token clears the counter
|
||||
- Do not require CAPTCHA on first attempt — only after 3 failures
|
||||
- Do not add CAPTCHA to `/api/auth/refresh` or `/api/auth/logout` — only `/api/auth/login`
|
||||
- Do not store failed-attempt state in Redis for now — in-memory Map is fine for a single-instance deployment
|
||||
|
||||
---
|
||||
|
||||
## Files to touch (summary for agent)
|
||||
|
||||
| File | Change |
|
||||
|---|---|
|
||||
| `backend/src/shared/middleware/captchaMiddleware.ts` | **CREATE** — full middleware |
|
||||
| `backend/src/app.ts` | Wire `captchaGate` on `/api/auth/login`; raise authLimiter max to 1000 |
|
||||
| `backend/src/controllers/authController.ts` (or equivalent) | Call `recordFailedLogin` on bad credentials, `clearFailedLogin` on success |
|
||||
| `frontend/src/` (login form + auth action) | Add Turnstile widget, handle `captchaRequired` response |
|
||||
| `backend/.env.example` | Add `TURNSTILE_SECRET_KEY=` |
|
||||
| `frontend/.env.example` | Add `NEXT_PUBLIC_TURNSTILE_SITE_KEY=` |
|
||||
@@ -1,10 +1,23 @@
|
||||
# PRD — Retire Request Network: AMN Pay Scanner (Standalone Microservice)
|
||||
|
||||
> Status: **Ready for implementation**
|
||||
> Task: #13 (new)
|
||||
> Status: **Implemented — 2026-05-29**
|
||||
> Task: #13
|
||||
> Priority: High
|
||||
> Effort estimate: ~4–6 days (Rust/Go scanner service + Node.js adapter swap)
|
||||
> Depends on: Task #8 (done), Task #9 (confirmation thresholds), Task #11 (Trezor sweep — parallel, not blocking)
|
||||
>
|
||||
> | Component | Status |
|
||||
> |---|---|
|
||||
> | Go scanner service | ✅ Done — `scanner/*.go`, builds, tests pass |
|
||||
> | Backend adapter (`amnPayAdapter.ts`) | ✅ Done — implements `PaymentProviderAdapter` |
|
||||
> | Webhook receiver (`amnScannerWebhookRoutes.ts`) | ✅ Done — HMAC verify + PaymentCoordinator delegation |
|
||||
> | Provider config (`providerConfig.ts`) | ✅ Done — `amn.scanner` enabled when `AMN_SCANNER_URL` set |
|
||||
> | Admin scanner status proxy | ✅ Done — `GET /api/admin/scanner/status` |
|
||||
> | Frontend lag column | ✅ Done — color-coded chips in networks list |
|
||||
> | Docker compose (dev + prod) | ✅ Done — `amn-scanner` service with healthcheck |
|
||||
> | Env vars (backend + scanner) | ✅ Done — `AMN_SCANNER_URL`, `AMN_SCANNER_WEBHOOK_SECRET`, `AMN_SCANNER_DEFAULT` |
|
||||
> | Cross-language reference test | ✅ Done — Go `computePaymentReference` = TS `computeOnChainPaymentReference` |
|
||||
> | Live end-to-end probe | ⏳ Pending — requires deployed scanner + real on-chain payment |
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user