docs: sync from backend cab0719 - align request budget validation
This commit is contained in:
Siavash Sameni
@@ -6,10 +6,10 @@ created: 2026-05-23
|
||||
|
||||
# Backend Architecture
|
||||
|
||||
Module-level architecture of the Express 5 + TypeScript + Mongoose backend at `/Users/mojtabaheidari/code/backend` (development branch).
|
||||
Module-level architecture of the Express 5 + TypeScript backend. MongoDB/Mongoose is still the primary runtime persistence layer; the `integrate-main-into-development` backend also contains the Drizzle/Postgres migration layer.
|
||||
|
||||
> [!info]
|
||||
> Repo: `git@git.manko.yoga:222/nick/backend.git` · Branch: `development` · Version: 2.6.3-beta (`package.json:4`)
|
||||
> Repo: `git@git.manko.yoga:222/nick/backend.git` · Active integration branch: `integrate-main-into-development` · Current baseline: backend `2.6.79` at `3a50dc4`
|
||||
|
||||
---
|
||||
|
||||
@@ -24,6 +24,7 @@ backend/src/
|
||||
│ ├── database/ # Mongoose connection, retries, graceful shutdown
|
||||
│ └── socket/socketService.ts # Socket.IO server, rooms, emit helpers
|
||||
├── models/ # Mongoose models — see 02 - Data Models/
|
||||
├── db/ # Drizzle/Postgres migration layer: schemas, migrations, repos, backfill, verify
|
||||
├── routes/ # Express Router definitions (mounted in app.ts)
|
||||
├── scripts/ # CLI utilities (seed:users, seed:categories, ...)
|
||||
├── seeds/ # Seed data fixtures
|
||||
@@ -60,6 +61,9 @@ backend/src/
|
||||
└── utils/ # Pure utility fns (logger, currencyUtils, etc.)
|
||||
```
|
||||
|
||||
> [!warning] Postgres is not the default runtime store yet
|
||||
> `src/db/repositories/factory.ts` can select `mongo`, `dual`, or `pg` implementations for user, payment, points, and marketplace domains, but the broad service layer still imports Mongoose models directly. A code scan on 2026-05-31 found no runtime calls to `createRepositories()` / `getPaymentRepo()` / `getMarketplaceRepo()` outside the factory itself. See [[Postgres Runtime Cutover Status]] before assuming a `REPO_*` flag changes live behavior.
|
||||
|
||||
> [!tip]
|
||||
> Service folders are self-contained: each typically has `<feature>Service.ts`, `<feature>Controller.ts`, `<feature>Routes.ts`, `<feature>Validation.ts`. This makes each service movable to a microservice later with minimal coupling.
|
||||
|
||||
@@ -82,7 +86,7 @@ The bootstrap is intentionally linear and easy to audit. Execution order:
|
||||
11. **Error handler** — central `errorHandler` middleware formats responses via `response-handler.ts`.
|
||||
12. **HTTP server creation** — `const server = http.createServer(app)`.
|
||||
13. **Socket.IO attach** — `initSocket(server, corsOptions)` (see [[Real-time Layer]]).
|
||||
14. **DB connect** — `await connectDatabase()`.
|
||||
14. **DB connect** — `await connectDatabase()` for MongoDB/Mongoose. Postgres connects lazily only when PG modules are imported (for example oracle quote persistence with `ORACLE_QUOTING_ENABLED=true`) and requires `PG_URL`.
|
||||
15. **Redis connect** — `await connectRedis()`.
|
||||
16. **Listen** — `server.listen(config.port, ...)`.
|
||||
17. **Graceful shutdown** — SIGTERM/SIGINT handlers close server, drain sockets, close Mongoose, close Redis.
|
||||
|
||||
@@ -1,16 +1,18 @@
|
||||
# Database Strategy — Mongo vs Postgres Assessment
|
||||
|
||||
**Status:** Living assessment. Not a decision yet. Written 2026-05-28.
|
||||
**Status:** Superseded by active Postgres migration work, but still useful as the risk assessment. Written 2026-05-28; updated 2026-05-31 for backend `integrate-main-into-development@3a50dc4`.
|
||||
**Owner:** nick + claude
|
||||
**Decision deadline:** Open. Re-evaluate when one of the trigger conditions below fires.
|
||||
**Decision:** Proceed with a staged hybrid migration, not an immediate full cutover.
|
||||
|
||||
---
|
||||
|
||||
## TL;DR
|
||||
|
||||
Amanat runs on MongoDB (primary store) + Redis (cache/sessions/rate limits). For an escrow product that moves money, Postgres would be the structurally better fit — FK constraints, ACID across rows, mature audit/reporting tooling. But a full migration today is a **3–6 month, single-engineer-equivalent project with high schedule risk** and zero user-visible value during the cutover.
|
||||
Amanat still runs on MongoDB (primary store) + Redis (cache/sessions/rate limits). Backend `2.6.79` adds Postgres 18 support, Drizzle schemas/migrations, repository implementations, backfill/verify tooling, and conditional `payment_quotes` persistence, but this is **not** a full runtime cutover.
|
||||
|
||||
**Current recommendation:** Don't migrate. Pay down the specific weaknesses Mongo creates (cross-collection consistency, audit trails, FK-shaped bugs) with targeted in-place hardening. Revisit the decision when one of the trigger conditions below fires.
|
||||
**Current recommendation:** continue the staged hybrid migration. Keep Mongo authoritative for live traffic until each domain is wired through the repository layer, backfilled, dual-written, shadow-read, and explicitly flipped.
|
||||
|
||||
See [[Postgres Runtime Cutover Status]] for the current line between code that can use Postgres and code that still uses Mongo.
|
||||
|
||||
---
|
||||
|
||||
@@ -18,9 +20,19 @@ Amanat runs on MongoDB (primary store) + Redis (cache/sessions/rate limits). For
|
||||
|
||||
| Store | Use | Notes |
|
||||
|---|---|---|
|
||||
| MongoDB (Mongoose 8.x) | Primary store — all domain data | 22 models, ~454 query call sites across 171 backend TS files |
|
||||
| MongoDB (Mongoose 8.x) | Primary runtime store — normal domain traffic | 22 models, ~454 query call sites across 171 backend TS files |
|
||||
| PostgreSQL 18 + Drizzle | Migration target and conditional oracle quote store | Schemas/migrations through `0008`, repo implementations, backfill/verify tooling; broad service wiring still pending |
|
||||
| Redis | Sessions, cache, rate limits (paymentLimiter etc.) | Not in scope for any migration. Keep as-is either way. |
|
||||
|
||||
### Current Postgres implementation state (2026-05-31)
|
||||
|
||||
| Implemented | Not yet cut over |
|
||||
|---|---|
|
||||
| `src/db/client.ts` fail-fast PG client, Drizzle schema/index barrel, migrations through `0008`, `id_map`, `pg_dualwrite_gaps`, `payment_quotes` | Service layer still imports Mongoose models directly; no broad runtime use of `createRepositories()` / `get*Repo()` factory |
|
||||
| Drizzle/Mongo/Dual repository classes for user, payment, points, marketplace | Auth, marketplace, payment, wallet, points, chat, notification, dispute, and admin paths still use Mongoose directly |
|
||||
| Backfill and verification scripts guarded by `MIGRATION_PG_URL` | Backfills are not auto-run and no domain is verified as PG-authoritative |
|
||||
| Oracle quote persistence can write PG `payment_quotes` when `ORACLE_QUOTING_ENABLED=true` | Payment records themselves are still created/updated in Mongo; PG quote insert depends on a resolvable PG parent row |
|
||||
|
||||
### Mongoose models (22)
|
||||
|
||||
Ranked by how naturally they map to a relational schema:
|
||||
|
||||
@@ -6,10 +6,10 @@ created: 2026-05-23
|
||||
|
||||
# Frontend Architecture
|
||||
|
||||
Module-level architecture of the Next.js 16 (App Router) + TypeScript + MUI v7 frontend at `/Users/mojtabaheidari/code/frontend` (development branch).
|
||||
Module-level architecture of the Next.js 16 (App Router) + TypeScript + MUI v7 frontend. The current integration worktree observed locally is on `integrate-main-into-development`.
|
||||
|
||||
> [!info]
|
||||
> Repo: `git@git.manko.yoga:222/nick/frontend.git` · Branch: `development` · Version: 1.9.6 (`package.json:4`) · Dev port `3000`, Docker port `8083`.
|
||||
> Repo: `git@git.manko.yoga:222/nick/frontend.git` · Active integration branch observed locally: `integrate-main-into-development` · Version: 2.7.19 (`package.json`) · Dev port `3000`, Docker port `8083`.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -107,7 +107,7 @@ The Nginx proxy at `./nginx/nginx.conf` (mounted read-only) is responsible for:
|
||||
Both `nickapp-backend` and `nickapp-frontend` carry the `watchtower.enable=true` label. Watchtower polls the container registry on its configured interval and re-pulls when the `latest` tag moves.
|
||||
|
||||
Release cycle:
|
||||
1. Developer pushes commits to a feature branch → merged into `development`.
|
||||
1. Developer pushes commits to a feature branch → merged into the active integration branch (`integrate-main-into-development` for the current dev stack; historically `development`).
|
||||
2. Manual Gitea workflow `docker-build-simple.yml` builds & pushes `nickapp-backend:latest` (and a versioned tag) to `git.manko.yoga/manawenuz/escrow-backend`.
|
||||
3. Within the next poll interval (default 5 min) Watchtower restarts the affected service.
|
||||
|
||||
@@ -121,6 +121,7 @@ Release cycle:
|
||||
| Volume | What it stores | Backup priority |
|
||||
|---|---|---|
|
||||
| `mongodb_data` | All business data (users, requests, payments, chats, disputes...) | **Critical** — daily dump |
|
||||
| `postgres_data` | Postgres 18 migration/backfill store and `payment_quotes` when enabled | **Critical after cutover; medium before cutover** — dump before/after migrations |
|
||||
| `redis_data` | Cache, session, rate counters | Low — losing it logs everyone out but no data loss |
|
||||
| `./uploads` (host bind) | Avatars, product images, dispute evidence, documents | **High** — daily rsync |
|
||||
| `./nginx/logs` | Access / error logs | Medium |
|
||||
|
||||
@@ -4,7 +4,7 @@ status: implemented on backend integrate-main-into-development
|
||||
owner: backend
|
||||
created: 2026-05-31
|
||||
branch: backend integrate-main-into-development at 3a50dc4
|
||||
storage: Postgres `payment_quotes` plus Mongo `Payment.quote` mirror during dual-write
|
||||
storage: conditional Postgres `payment_quotes` plus Mongo `Payment.quote` mirror during dual-write
|
||||
---
|
||||
|
||||
# Oracle Pricing & Stablecoin Depeg Protection
|
||||
@@ -89,7 +89,7 @@ Providers are registered in a small registry (same pattern as the payment-provid
|
||||
2. Validate `(token, chain)` against the seller allowlist (`assertPaymentChoiceAllowed`).
|
||||
3. `PriceOracle` → `fxRate`, `tokenPriceUSD`; run guardrails.
|
||||
4. Compute `rawSettle` → `settle` (rounding) → `onChainUnits`.
|
||||
5. Persist a **locked quote** in Postgres and mirror it on the Mongo Payment, then use `settle` as the intent amount.
|
||||
5. When `ORACLE_QUOTING_ENABLED=true`, persist a **locked quote** in Postgres if the PG parent payment row exists, mirror it on the Mongo Payment, then use `settle` as the intent amount.
|
||||
|
||||
**Payment quote fields** (Mongo mirror plus Postgres `payment_quotes` row):
|
||||
```
|
||||
@@ -103,9 +103,9 @@ quote: {
|
||||
- **Validity window** `QUOTE_VALIDITY_S` (default 60–120 s). On expiry → re-quote before submit; never settle against a stale quote.
|
||||
- The quote is **immutable once a payment is detected** (audit trail of exactly what rate the buyer agreed to).
|
||||
|
||||
## 7. Data-model changes (Postgres-native)
|
||||
## 7. Data-model changes (Postgres-capable, not full cutover)
|
||||
|
||||
Because the feature was promoted through the money-core migration branch, the quote is stored **natively in Postgres** via the Drizzle schema/repos:
|
||||
Because the feature was promoted through the money-core migration branch, the quote can be stored **natively in Postgres** via the Drizzle schema/repos. The live payment record remains Mongo-backed until the payment service itself is wired through the PG repository path:
|
||||
|
||||
- **Drizzle schema**: `payment_quotes` child table keyed by `payment_id -> payments.id` — decimal columns (`numeric(38,18)`) for `offer_amount`, `invoice_usd`, `fx_rate`, `token_price_usd`, `raw_settle_amount`, `settle_amount`; text for currencies/sources; `rounding_bps`, `depeg_adjustment_bps`, `fetched_at`, `expires_at`. Additive migration `0008`, preserving every `0005`/`0006` money-safety object.
|
||||
- **Pricing-currency enum**: extend `budget_currency` / the offer currency enum to add `TRY` (and any others) — additive.
|
||||
|
||||
@@ -22,7 +22,8 @@ flowchart LR
|
||||
Nginx[Nginx Reverse Proxy<br/>:80/:443]
|
||||
FE[Next.js Frontend<br/>standalone server<br/>:8083]
|
||||
BE[Express Backend<br/>+ Socket.IO<br/>:5001]
|
||||
Mongo[(MongoDB 8)]
|
||||
Mongo[(MongoDB 8<br/>primary runtime store)]
|
||||
PG[(PostgreSQL 18<br/>migration target / quote table)]
|
||||
Redis[(Redis 8)]
|
||||
RN[Request Network<br/>Pay-in + webhooks]
|
||||
CFWorker[Durable webhook ingress<br/>roadmap]
|
||||
@@ -37,6 +38,7 @@ flowchart LR
|
||||
FE -->|REST /api/*| BE
|
||||
FE -.->|Socket.IO| BE
|
||||
BE --> Mongo
|
||||
BE -.->|PG_URL + migration/quote paths| PG
|
||||
BE --> Redis
|
||||
BE -->|Pay-in intent / status| RN
|
||||
RN -.->|Signed webhook| CFWorker
|
||||
@@ -79,6 +81,9 @@ sequenceDiagram
|
||||
FE-->>U: UI re-render
|
||||
```
|
||||
|
||||
> [!note] Postgres status on `integrate-main-into-development`
|
||||
> Backend `2.6.79` includes Drizzle schemas, migrations, repository implementations, backfill/verify tooling, and conditional oracle quote persistence to Postgres. It is not a full runtime cutover: ordinary services still call Mongoose models directly and MongoDB remains the primary store. See [[Postgres Runtime Cutover Status]] for the current boundary.
|
||||
|
||||
Concurrent realtime path:
|
||||
|
||||
```mermaid
|
||||
@@ -106,6 +111,7 @@ Production runs as a single Docker Compose stack (`backend/docker-compose.produc
|
||||
| App | Frontend | `nickapp-frontend:latest` | 8083 (internal) | Next.js standalone |
|
||||
| App | Backend | `nickapp-backend:latest` | 5001 (internal) | Express + Socket.IO |
|
||||
| Data | MongoDB | `mongo:8.0` | 27017 (internal) | Primary store |
|
||||
| Data | PostgreSQL | `postgres:18` / `postgres:18-alpine` | 5432 (internal) | Migration target; required for PG backfill/verify and oracle `payment_quotes` when enabled |
|
||||
| Data | Redis | `redis:8-alpine` | 6379 (internal) | Cache + sessions + rate-limit counters |
|
||||
|
||||
External SSL termination, DNS, and CDN are assumed to live in front of Nginx (CloudFlare / nginx-proxy / similar).
|
||||
@@ -176,6 +182,7 @@ See [[PRD - Request Network In-House Checkout]] and [[Request Network Integratio
|
||||
|---|---|---|
|
||||
| Backend stateless? | Yes — JWT-only auth, no in-memory session | Run N replicas behind LB; use Redis pub/sub adapter for Socket.IO |
|
||||
| MongoDB | Single-node | Replica set → sharding by `buyerId` |
|
||||
| PostgreSQL | Dev/staging service for migration work | Managed Postgres or hardened self-hosted PG with backups/PITR before cutover |
|
||||
| Redis | Single-node | Cluster mode; separate cache vs session DBs |
|
||||
| Socket.IO | Single process | `@socket.io/redis-adapter` for multi-node fan-out |
|
||||
| File uploads | Local `uploads/` mount | S3 / R2; multer-s3 adapter |
|
||||
|
||||
Reference in New Issue
Block a user