docs: sync from backend cab0719 - align request budget validation
This commit is contained in:
Siavash Sameni
@@ -78,4 +78,4 @@ A handful of design choices set Amn apart from generic marketplace software:
|
||||
|
||||
## Project status at a glance
|
||||
|
||||
Amn is at version **2.6.x** across both repositories, on the `development` branch, and tagged "production-ready with minor enhancements" by the project leads. The core escrow loop, real-time chat, multi-language UI, dispute system, points programme, and blog are all live. Active work focuses on Request Network hardening, durable webhook ingress, derived-destination custody, admin signing, and a more granular permissions matrix. The custody/smart-contract strategy lives in [[PRD - Decentralized Custody and Smart-Contract Escrow Roadmap]].
|
||||
Amn is at version **2.6.x/2.7.x** across the integration worktrees, with backend `integrate-main-into-development@3a50dc4` at `2.6.79`. The core escrow loop, real-time chat, multi-language UI, dispute system, points programme, and blog are all live. Active work focuses on Request Network/AMN scanner hardening, Postgres migration readiness, oracle/depeg quote protection, durable webhook ingress, derived-destination custody, admin signing, and a more granular permissions matrix. The Postgres status lives in [[Postgres Runtime Cutover Status]]; the custody/smart-contract strategy lives in [[PRD - Decentralized Custody and Smart-Contract Escrow Roadmap]].
|
||||
|
||||
@@ -14,7 +14,7 @@ created: 2026-05-23
|
||||
Amn is a **two-repo system**:
|
||||
|
||||
- **Frontend** (`/Users/mojtabaheidari/code/frontend`) — a Next.js 16 App Router application that serves the marketplace UI, the admin dashboard, the public blog, and the user-facing Web3 wallet flow.
|
||||
- **Backend** (`/Users/mojtabaheidari/code/backend`) — an Express 5 + TypeScript API server that owns all business logic, persists to MongoDB, caches in Redis, and brokers all external integrations.
|
||||
- **Backend** (`/Users/mojtabaheidari/code/backend`) — an Express 5 + TypeScript API server that owns all business logic, persists live app state primarily to MongoDB, caches in Redis, and brokers all external integrations. The active integration backend also contains the Postgres/Drizzle migration layer, but it is not yet the broad runtime store.
|
||||
|
||||
The two repos are deployable independently. They communicate over **HTTPS (REST)** for stateful actions and over **WebSocket (Socket.IO)** for live updates. The frontend never talks directly to MongoDB, Redis, Request Network API keys, OpenAI, or admin custody secrets -- every sensitive external interaction is mediated by the backend so that secrets stay on the server.
|
||||
|
||||
@@ -52,7 +52,8 @@ flowchart TB
|
||||
end
|
||||
|
||||
subgraph Data["Data tier"]
|
||||
Mongo[("MongoDB<br/>via Mongoose")]
|
||||
Mongo[("MongoDB<br/>via Mongoose<br/>primary runtime")]
|
||||
PG[("PostgreSQL 18<br/>Drizzle migration layer")]
|
||||
RedisDB[("Redis<br/>cache + locks")]
|
||||
Disk[("Local disk<br/>/uploads")]
|
||||
end
|
||||
@@ -84,6 +85,7 @@ flowchart TB
|
||||
SocketS --> ChatSvc & Notif & Market
|
||||
|
||||
Auth & Market & ChatSvc & PaySvc & Disp & Points & BlogSvc & TelegramSvc --> Mongo
|
||||
PaySvc -.->|oracle payment_quotes when enabled| PG
|
||||
Auth & PaySvc & Notif --> RedisDB
|
||||
Files --> Disk
|
||||
|
||||
@@ -135,6 +137,7 @@ Payments are where Amn is most distinctive. The live backend has converged on **
|
||||
- **Derived destination wallets** -- `/api/payment/derived-destinations` admin endpoints manage per-`(buyer, sellerOffer, chainId)` receiving addresses, sweep status, and config health.
|
||||
- **Funds ledger** -- `backend/src/services/payment/ledger/` tracks payment detection, holds, releases, refunds, fees, and adjustments independently of provider metadata.
|
||||
- **Release/refund orchestration** -- `/api/payment/:id/(release|refund)` builds instructions; `/confirm` records confirmed transaction hashes. Optional Trezor enforcement gates confirmation when `TREZOR_SAFEKEEPING_REQUIRED=true`.
|
||||
- **Postgres migration layer** -- backend `2.6.79` includes Drizzle migrations/repos and can persist oracle quote rows to `payment_quotes` when enabled. Payment records, ledger state, wallet destinations, and marketplace entities still flow through Mongo-backed services until the cutover work in [[Postgres Runtime Cutover Status]] is completed.
|
||||
|
||||
Historical SHKeeper and DePay docs remain in the vault for migration context, but the current backend tree no longer has `backend/src/services/payment/shkeeper/`. The current strategic path is in [[PRD - Decentralized Custody and Smart-Contract Escrow Roadmap]].
|
||||
|
||||
|
||||
@@ -7,11 +7,11 @@ created: 2026-05-23
|
||||
# Tech Stack
|
||||
|
||||
> [!info] Versions
|
||||
> Versions below are pulled directly from `frontend/package.json` and `backend/package.json` on the `development` branch. Where a `^` range is declared in package.json, the **declared minimum** is shown — the lockfile may have resolved a newer patch. When in doubt, check `yarn.lock` in each repo.
|
||||
> Versions below are pulled from the current integration worktrees. Backend baseline: `integrate-main-into-development@3a50dc4`, package version `2.6.79`. Frontend integration worktree observed at `2.7.19`. Where a `^` range is declared in package.json, the **declared minimum** is shown — the lockfile may have resolved a newer patch.
|
||||
|
||||
## Frontend stack
|
||||
|
||||
The frontend is a Next.js 16 App Router application written in TypeScript. The build is deliberately heavy on best-in-class libraries rather than home-grown solutions: MUI for components, Wagmi for Web3, React Query / SWR for data, Zod for validation, Sentry for errors. The package is `amn-frontend@2.6.5-beta` and requires Node `>=20`.
|
||||
The frontend is a Next.js 16 App Router application written in TypeScript. The build is deliberately heavy on best-in-class libraries rather than home-grown solutions: MUI for components, Wagmi for Web3, React Query / SWR for data, Zod for validation, Sentry for errors. The current integration package observed locally is `amn-frontend@2.7.19` and requires Node `>=20`.
|
||||
|
||||
### Core framework & language
|
||||
|
||||
@@ -117,7 +117,7 @@ The frontend is a Next.js 16 App Router application written in TypeScript. The b
|
||||
|
||||
## Backend stack
|
||||
|
||||
The backend is `amn-backend@2.6.3-beta`, an Express 5 server in TypeScript backed by MongoDB (Mongoose), Redis, and Socket.IO. It owns all integrations with Request Network, EVM chains, OpenAI, Google OAuth, Telegram, SMTP, and custody/signing controls.
|
||||
The backend is `amn-backend@2.6.79`, an Express 5 server in TypeScript backed by MongoDB (Mongoose), Redis, Socket.IO, and a Postgres/Drizzle migration layer. MongoDB remains the primary runtime store; Postgres is currently used for migrations/backfill tooling and conditional oracle quote persistence. It owns all integrations with Request Network, AMN scanner, EVM chains, OpenAI, Google OAuth, Telegram, SMTP, and custody/signing controls.
|
||||
|
||||
### Core runtime & framework
|
||||
|
||||
@@ -146,6 +146,10 @@ The backend is `amn-backend@2.6.3-beta`, an Express 5 server in TypeScript backe
|
||||
| Tool | Version | Purpose | Where used |
|
||||
|---|---|---|---|
|
||||
| mongoose | ^8.16.4 | MongoDB ODM | `backend/src/models/**` |
|
||||
| pg | ^8.16.0 | PostgreSQL driver | `backend/src/db/client.ts`, Drizzle runtime |
|
||||
| drizzle-orm | ^0.44.1 | Type-safe SQL ORM | `backend/src/db/schema/**`, repositories |
|
||||
| drizzle-kit | ^0.31.1 | Migration CLI | `backend/src/db/migrations/**`, `drizzle.config.ts` |
|
||||
| decimal.js | ^10.5.0 | Decimal-exact money/oracle math | payment quote engine |
|
||||
| redis | ^5.6.0 | Cache, locks, rate-limit store | `services/redis/`, `app.ts:362` |
|
||||
| mongodb-memory-server | ^10.2.0 (dev) | In-memory Mongo for tests | `__tests__/` |
|
||||
|
||||
@@ -200,7 +204,8 @@ The backend is `amn-backend@2.6.3-beta`, an Express 5 server in TypeScript backe
|
||||
|---|---|---|---|
|
||||
| Container engine | Docker + Docker Compose | Dev & prod deployment | `docker-compose.dev.yml`, `docker-compose.production.yml` in each repo |
|
||||
| Reverse proxy | Nginx (external) | TLS termination, routing | `TRUST_PROXY=true` recognised in `app.ts:64` |
|
||||
| Database | MongoDB | Primary store | Connection string via env |
|
||||
| Database | MongoDB | Primary runtime store | Connection string via env |
|
||||
| Database | PostgreSQL 18 + Drizzle | Migration target, backfill/verify store, conditional `payment_quotes` | `PG_URL` / `MIGRATION_PG_URL`; not a full cutover yet |
|
||||
| Cache | Redis | Sessions, locks, ephemeral data | Optional — backend boots without it |
|
||||
| Object storage | Local disk `/uploads` | User uploads | `UPLOAD_PATH` env override |
|
||||
| Process manager | Docker `restart: unless-stopped` (typical) | Crash recovery | Per compose file |
|
||||
|
||||
@@ -6,10 +6,10 @@ created: 2026-05-23
|
||||
|
||||
# Backend Architecture
|
||||
|
||||
Module-level architecture of the Express 5 + TypeScript + Mongoose backend at `/Users/mojtabaheidari/code/backend` (development branch).
|
||||
Module-level architecture of the Express 5 + TypeScript backend. MongoDB/Mongoose is still the primary runtime persistence layer; the `integrate-main-into-development` backend also contains the Drizzle/Postgres migration layer.
|
||||
|
||||
> [!info]
|
||||
> Repo: `git@git.manko.yoga:222/nick/backend.git` · Branch: `development` · Version: 2.6.3-beta (`package.json:4`)
|
||||
> Repo: `git@git.manko.yoga:222/nick/backend.git` · Active integration branch: `integrate-main-into-development` · Current baseline: backend `2.6.79` at `3a50dc4`
|
||||
|
||||
---
|
||||
|
||||
@@ -24,6 +24,7 @@ backend/src/
|
||||
│ ├── database/ # Mongoose connection, retries, graceful shutdown
|
||||
│ └── socket/socketService.ts # Socket.IO server, rooms, emit helpers
|
||||
├── models/ # Mongoose models — see 02 - Data Models/
|
||||
├── db/ # Drizzle/Postgres migration layer: schemas, migrations, repos, backfill, verify
|
||||
├── routes/ # Express Router definitions (mounted in app.ts)
|
||||
├── scripts/ # CLI utilities (seed:users, seed:categories, ...)
|
||||
├── seeds/ # Seed data fixtures
|
||||
@@ -60,6 +61,9 @@ backend/src/
|
||||
└── utils/ # Pure utility fns (logger, currencyUtils, etc.)
|
||||
```
|
||||
|
||||
> [!warning] Postgres is not the default runtime store yet
|
||||
> `src/db/repositories/factory.ts` can select `mongo`, `dual`, or `pg` implementations for user, payment, points, and marketplace domains, but the broad service layer still imports Mongoose models directly. A code scan on 2026-05-31 found no runtime calls to `createRepositories()` / `getPaymentRepo()` / `getMarketplaceRepo()` outside the factory itself. See [[Postgres Runtime Cutover Status]] before assuming a `REPO_*` flag changes live behavior.
|
||||
|
||||
> [!tip]
|
||||
> Service folders are self-contained: each typically has `<feature>Service.ts`, `<feature>Controller.ts`, `<feature>Routes.ts`, `<feature>Validation.ts`. This makes each service movable to a microservice later with minimal coupling.
|
||||
|
||||
@@ -82,7 +86,7 @@ The bootstrap is intentionally linear and easy to audit. Execution order:
|
||||
11. **Error handler** — central `errorHandler` middleware formats responses via `response-handler.ts`.
|
||||
12. **HTTP server creation** — `const server = http.createServer(app)`.
|
||||
13. **Socket.IO attach** — `initSocket(server, corsOptions)` (see [[Real-time Layer]]).
|
||||
14. **DB connect** — `await connectDatabase()`.
|
||||
14. **DB connect** — `await connectDatabase()` for MongoDB/Mongoose. Postgres connects lazily only when PG modules are imported (for example oracle quote persistence with `ORACLE_QUOTING_ENABLED=true`) and requires `PG_URL`.
|
||||
15. **Redis connect** — `await connectRedis()`.
|
||||
16. **Listen** — `server.listen(config.port, ...)`.
|
||||
17. **Graceful shutdown** — SIGTERM/SIGINT handlers close server, drain sockets, close Mongoose, close Redis.
|
||||
|
||||
@@ -1,16 +1,18 @@
|
||||
# Database Strategy — Mongo vs Postgres Assessment
|
||||
|
||||
**Status:** Living assessment. Not a decision yet. Written 2026-05-28.
|
||||
**Status:** Superseded by active Postgres migration work, but still useful as the risk assessment. Written 2026-05-28; updated 2026-05-31 for backend `integrate-main-into-development@3a50dc4`.
|
||||
**Owner:** nick + claude
|
||||
**Decision deadline:** Open. Re-evaluate when one of the trigger conditions below fires.
|
||||
**Decision:** Proceed with a staged hybrid migration, not an immediate full cutover.
|
||||
|
||||
---
|
||||
|
||||
## TL;DR
|
||||
|
||||
Amanat runs on MongoDB (primary store) + Redis (cache/sessions/rate limits). For an escrow product that moves money, Postgres would be the structurally better fit — FK constraints, ACID across rows, mature audit/reporting tooling. But a full migration today is a **3–6 month, single-engineer-equivalent project with high schedule risk** and zero user-visible value during the cutover.
|
||||
Amanat still runs on MongoDB (primary store) + Redis (cache/sessions/rate limits). Backend `2.6.79` adds Postgres 18 support, Drizzle schemas/migrations, repository implementations, backfill/verify tooling, and conditional `payment_quotes` persistence, but this is **not** a full runtime cutover.
|
||||
|
||||
**Current recommendation:** Don't migrate. Pay down the specific weaknesses Mongo creates (cross-collection consistency, audit trails, FK-shaped bugs) with targeted in-place hardening. Revisit the decision when one of the trigger conditions below fires.
|
||||
**Current recommendation:** continue the staged hybrid migration. Keep Mongo authoritative for live traffic until each domain is wired through the repository layer, backfilled, dual-written, shadow-read, and explicitly flipped.
|
||||
|
||||
See [[Postgres Runtime Cutover Status]] for the current line between code that can use Postgres and code that still uses Mongo.
|
||||
|
||||
---
|
||||
|
||||
@@ -18,9 +20,19 @@ Amanat runs on MongoDB (primary store) + Redis (cache/sessions/rate limits). For
|
||||
|
||||
| Store | Use | Notes |
|
||||
|---|---|---|
|
||||
| MongoDB (Mongoose 8.x) | Primary store — all domain data | 22 models, ~454 query call sites across 171 backend TS files |
|
||||
| MongoDB (Mongoose 8.x) | Primary runtime store — normal domain traffic | 22 models, ~454 query call sites across 171 backend TS files |
|
||||
| PostgreSQL 18 + Drizzle | Migration target and conditional oracle quote store | Schemas/migrations through `0008`, repo implementations, backfill/verify tooling; broad service wiring still pending |
|
||||
| Redis | Sessions, cache, rate limits (paymentLimiter etc.) | Not in scope for any migration. Keep as-is either way. |
|
||||
|
||||
### Current Postgres implementation state (2026-05-31)
|
||||
|
||||
| Implemented | Not yet cut over |
|
||||
|---|---|
|
||||
| `src/db/client.ts` fail-fast PG client, Drizzle schema/index barrel, migrations through `0008`, `id_map`, `pg_dualwrite_gaps`, `payment_quotes` | Service layer still imports Mongoose models directly; no broad runtime use of `createRepositories()` / `get*Repo()` factory |
|
||||
| Drizzle/Mongo/Dual repository classes for user, payment, points, marketplace | Auth, marketplace, payment, wallet, points, chat, notification, dispute, and admin paths still use Mongoose directly |
|
||||
| Backfill and verification scripts guarded by `MIGRATION_PG_URL` | Backfills are not auto-run and no domain is verified as PG-authoritative |
|
||||
| Oracle quote persistence can write PG `payment_quotes` when `ORACLE_QUOTING_ENABLED=true` | Payment records themselves are still created/updated in Mongo; PG quote insert depends on a resolvable PG parent row |
|
||||
|
||||
### Mongoose models (22)
|
||||
|
||||
Ranked by how naturally they map to a relational schema:
|
||||
|
||||
@@ -6,10 +6,10 @@ created: 2026-05-23
|
||||
|
||||
# Frontend Architecture
|
||||
|
||||
Module-level architecture of the Next.js 16 (App Router) + TypeScript + MUI v7 frontend at `/Users/mojtabaheidari/code/frontend` (development branch).
|
||||
Module-level architecture of the Next.js 16 (App Router) + TypeScript + MUI v7 frontend. The current integration worktree observed locally is on `integrate-main-into-development`.
|
||||
|
||||
> [!info]
|
||||
> Repo: `git@git.manko.yoga:222/nick/frontend.git` · Branch: `development` · Version: 1.9.6 (`package.json:4`) · Dev port `3000`, Docker port `8083`.
|
||||
> Repo: `git@git.manko.yoga:222/nick/frontend.git` · Active integration branch observed locally: `integrate-main-into-development` · Version: 2.7.19 (`package.json`) · Dev port `3000`, Docker port `8083`.
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -107,7 +107,7 @@ The Nginx proxy at `./nginx/nginx.conf` (mounted read-only) is responsible for:
|
||||
Both `nickapp-backend` and `nickapp-frontend` carry the `watchtower.enable=true` label. Watchtower polls the container registry on its configured interval and re-pulls when the `latest` tag moves.
|
||||
|
||||
Release cycle:
|
||||
1. Developer pushes commits to a feature branch → merged into `development`.
|
||||
1. Developer pushes commits to a feature branch → merged into the active integration branch (`integrate-main-into-development` for the current dev stack; historically `development`).
|
||||
2. Manual Gitea workflow `docker-build-simple.yml` builds & pushes `nickapp-backend:latest` (and a versioned tag) to `git.manko.yoga/manawenuz/escrow-backend`.
|
||||
3. Within the next poll interval (default 5 min) Watchtower restarts the affected service.
|
||||
|
||||
@@ -121,6 +121,7 @@ Release cycle:
|
||||
| Volume | What it stores | Backup priority |
|
||||
|---|---|---|
|
||||
| `mongodb_data` | All business data (users, requests, payments, chats, disputes...) | **Critical** — daily dump |
|
||||
| `postgres_data` | Postgres 18 migration/backfill store and `payment_quotes` when enabled | **Critical after cutover; medium before cutover** — dump before/after migrations |
|
||||
| `redis_data` | Cache, session, rate counters | Low — losing it logs everyone out but no data loss |
|
||||
| `./uploads` (host bind) | Avatars, product images, dispute evidence, documents | **High** — daily rsync |
|
||||
| `./nginx/logs` | Access / error logs | Medium |
|
||||
|
||||
@@ -4,7 +4,7 @@ status: implemented on backend integrate-main-into-development
|
||||
owner: backend
|
||||
created: 2026-05-31
|
||||
branch: backend integrate-main-into-development at 3a50dc4
|
||||
storage: Postgres `payment_quotes` plus Mongo `Payment.quote` mirror during dual-write
|
||||
storage: conditional Postgres `payment_quotes` plus Mongo `Payment.quote` mirror during dual-write
|
||||
---
|
||||
|
||||
# Oracle Pricing & Stablecoin Depeg Protection
|
||||
@@ -89,7 +89,7 @@ Providers are registered in a small registry (same pattern as the payment-provid
|
||||
2. Validate `(token, chain)` against the seller allowlist (`assertPaymentChoiceAllowed`).
|
||||
3. `PriceOracle` → `fxRate`, `tokenPriceUSD`; run guardrails.
|
||||
4. Compute `rawSettle` → `settle` (rounding) → `onChainUnits`.
|
||||
5. Persist a **locked quote** in Postgres and mirror it on the Mongo Payment, then use `settle` as the intent amount.
|
||||
5. When `ORACLE_QUOTING_ENABLED=true`, persist a **locked quote** in Postgres if the PG parent payment row exists, mirror it on the Mongo Payment, then use `settle` as the intent amount.
|
||||
|
||||
**Payment quote fields** (Mongo mirror plus Postgres `payment_quotes` row):
|
||||
```
|
||||
@@ -103,9 +103,9 @@ quote: {
|
||||
- **Validity window** `QUOTE_VALIDITY_S` (default 60–120 s). On expiry → re-quote before submit; never settle against a stale quote.
|
||||
- The quote is **immutable once a payment is detected** (audit trail of exactly what rate the buyer agreed to).
|
||||
|
||||
## 7. Data-model changes (Postgres-native)
|
||||
## 7. Data-model changes (Postgres-capable, not full cutover)
|
||||
|
||||
Because the feature was promoted through the money-core migration branch, the quote is stored **natively in Postgres** via the Drizzle schema/repos:
|
||||
Because the feature was promoted through the money-core migration branch, the quote can be stored **natively in Postgres** via the Drizzle schema/repos. The live payment record remains Mongo-backed until the payment service itself is wired through the PG repository path:
|
||||
|
||||
- **Drizzle schema**: `payment_quotes` child table keyed by `payment_id -> payments.id` — decimal columns (`numeric(38,18)`) for `offer_amount`, `invoice_usd`, `fx_rate`, `token_price_usd`, `raw_settle_amount`, `settle_amount`; text for currencies/sources; `rounding_bps`, `depeg_adjustment_bps`, `fetched_at`, `expires_at`. Additive migration `0008`, preserving every `0005`/`0006` money-safety object.
|
||||
- **Pricing-currency enum**: extend `budget_currency` / the offer currency enum to add `TRY` (and any others) — additive.
|
||||
|
||||
@@ -22,7 +22,8 @@ flowchart LR
|
||||
Nginx[Nginx Reverse Proxy<br/>:80/:443]
|
||||
FE[Next.js Frontend<br/>standalone server<br/>:8083]
|
||||
BE[Express Backend<br/>+ Socket.IO<br/>:5001]
|
||||
Mongo[(MongoDB 8)]
|
||||
Mongo[(MongoDB 8<br/>primary runtime store)]
|
||||
PG[(PostgreSQL 18<br/>migration target / quote table)]
|
||||
Redis[(Redis 8)]
|
||||
RN[Request Network<br/>Pay-in + webhooks]
|
||||
CFWorker[Durable webhook ingress<br/>roadmap]
|
||||
@@ -37,6 +38,7 @@ flowchart LR
|
||||
FE -->|REST /api/*| BE
|
||||
FE -.->|Socket.IO| BE
|
||||
BE --> Mongo
|
||||
BE -.->|PG_URL + migration/quote paths| PG
|
||||
BE --> Redis
|
||||
BE -->|Pay-in intent / status| RN
|
||||
RN -.->|Signed webhook| CFWorker
|
||||
@@ -79,6 +81,9 @@ sequenceDiagram
|
||||
FE-->>U: UI re-render
|
||||
```
|
||||
|
||||
> [!note] Postgres status on `integrate-main-into-development`
|
||||
> Backend `2.6.79` includes Drizzle schemas, migrations, repository implementations, backfill/verify tooling, and conditional oracle quote persistence to Postgres. It is not a full runtime cutover: ordinary services still call Mongoose models directly and MongoDB remains the primary store. See [[Postgres Runtime Cutover Status]] for the current boundary.
|
||||
|
||||
Concurrent realtime path:
|
||||
|
||||
```mermaid
|
||||
@@ -106,6 +111,7 @@ Production runs as a single Docker Compose stack (`backend/docker-compose.produc
|
||||
| App | Frontend | `nickapp-frontend:latest` | 8083 (internal) | Next.js standalone |
|
||||
| App | Backend | `nickapp-backend:latest` | 5001 (internal) | Express + Socket.IO |
|
||||
| Data | MongoDB | `mongo:8.0` | 27017 (internal) | Primary store |
|
||||
| Data | PostgreSQL | `postgres:18` / `postgres:18-alpine` | 5432 (internal) | Migration target; required for PG backfill/verify and oracle `payment_quotes` when enabled |
|
||||
| Data | Redis | `redis:8-alpine` | 6379 (internal) | Cache + sessions + rate-limit counters |
|
||||
|
||||
External SSL termination, DNS, and CDN are assumed to live in front of Nginx (CloudFlare / nginx-proxy / similar).
|
||||
@@ -176,6 +182,7 @@ See [[PRD - Request Network In-House Checkout]] and [[Request Network Integratio
|
||||
|---|---|---|
|
||||
| Backend stateless? | Yes — JWT-only auth, no in-memory session | Run N replicas behind LB; use Redis pub/sub adapter for Socket.IO |
|
||||
| MongoDB | Single-node | Replica set → sharding by `buyerId` |
|
||||
| PostgreSQL | Dev/staging service for migration work | Managed Postgres or hardened self-hosted PG with backups/PITR before cutover |
|
||||
| Redis | Single-node | Cluster mode; separate cache vs session DBs |
|
||||
| Socket.IO | Single process | `@socket.io/redis-adapter` for multi-node fan-out |
|
||||
| File uploads | Local `uploads/` mount | S3 / R2; multer-s3 adapter |
|
||||
|
||||
@@ -6,7 +6,7 @@ aliases: [Models Index, Schema Overview]
|
||||
|
||||
# Data Model Overview
|
||||
|
||||
This section documents every Mongoose model that backs the marketplace. The persistence layer lives in `backend/src/models/` and is exported through a single barrel file at `backend/src/models/index.ts`. All models target MongoDB via Mongoose, lean on `timestamps: true` for `createdAt` / `updatedAt`, and follow a consistent pattern: one schema per file, an exported `I<Name>` TypeScript interface, and named exports for the compiled model.
|
||||
This section documents every Mongoose model that backs the marketplace. On backend `integrate-main-into-development@cab0719`, these Mongoose models are still the live application persistence layer. The repo also contains a Drizzle/Postgres migration layer, but most services still call `backend/src/models/*` directly.
|
||||
|
||||
> [!note] Scope
|
||||
> Twenty-two models are present in `backend/src/models/`. The "File" concept exists only at the service layer (`backend/src/services/file/`) and is not persisted as its own Mongoose collection, so it is not listed below.
|
||||
@@ -14,6 +14,9 @@ This section documents every Mongoose model that backs the marketplace. The pers
|
||||
> [!note] Documentation freshness
|
||||
> The 2026-05-24 audit note that marked `Dispute`, `BlogPost`, `Review`, `PointTransaction`, `LevelConfig`, and `ShopSettings` as missing is now stale: schema files exist for those models. Newer operational models such as [[ConfigSetting]], [[DerivedDestination]], [[FundsLedgerEntry]], and [[TrezorAccount]] should be expanded into dedicated model pages when the docs are next deepened.
|
||||
|
||||
> [!warning] Mongo vs Postgres runtime status
|
||||
> Postgres schemas and repositories exist for the money/relational core, but normal app traffic is not fully cut over. Payment quote rows are the only current conditional PG write in checkout, and even that requires `ORACLE_QUOTING_ENABLED=true` plus a resolvable PG payment row. See [[Postgres Runtime Cutover Status]].
|
||||
|
||||
## Index of Models
|
||||
|
||||
- [[User]] — Core identity. Stores credentials, profile, preferences, referral data, points, and WebAuthn passkeys. Every other model that records "who did what" points back at a `User._id`. Buyers, sellers, and admins all live in this collection, differentiated by a `role` enum.
|
||||
|
||||
120
02 - Data Models/Money-Core Migration Workflow — Audit Report.md
Normal file
120
02 - Data Models/Money-Core Migration Workflow — Audit Report.md
Normal file
@@ -0,0 +1,120 @@
|
||||
---
|
||||
title: Money-Core Migration Workflow — Audit Report
|
||||
tags: [migration, audit, workflow, postgres, drizzle]
|
||||
created: 2026-05-31
|
||||
target: .claude/workflows/pg-money-core-migration.js
|
||||
verdict: go-with-changes (0 blockers)
|
||||
companion: "[[MongoDB to PostgreSQL Migration Plan (Drizzle)]]"
|
||||
updated: 2026-05-31 after backend integrate-main-into-development@3a50dc4
|
||||
---
|
||||
|
||||
> [!info] Provenance
|
||||
> Generated 2026-05-31 by a 29-agent read-only audit workflow (4 review dimensions → adversarial verification → synthesis) over the `pg-money-core-migration` workflow design + the real money-core backend code. All recommended changes below have since been **applied** to the workflow script.
|
||||
|
||||
# Audit Report — `pg-money-core-migration.js` Workflow
|
||||
|
||||
**Target:** `/Users/manwe/CascadeProjects/escrow/.claude/workflows/pg-money-core-migration.js`
|
||||
**Status:** Not yet run. 8 phases (Preflight → Infra → Schema → Repos → Backfill → Tests → VerifyGate → SelfAudit).
|
||||
**Adversarial verification:** Every `high`/`blocker` finding was challenged against the actual script. **All of them were refuted.** The remaining open items are `medium`/`low` enforcement-gap concerns that were not independently re-verified (`n/a`).
|
||||
|
||||
> [!warning] Historical scope
|
||||
> This is a workflow audit, not the current runtime status. The backend branch now contains the generated Postgres layer, but Mongo remains authoritative for normal traffic and service-layer wiring is still incomplete. Use [[Postgres Runtime Cutover Status]] for the current code/runtime boundary.
|
||||
|
||||
---
|
||||
|
||||
## 1. Verdict: **GO (with recommended changes)**
|
||||
|
||||
The workflow **cannot itself cause fraud or fund/order dataloss**, and that is the load-bearing question. Three independent, code-level facts make it safe to *run*:
|
||||
|
||||
1. **It never touches production data.** SAFETY rule 1 (lines 34–36, 67) binds every agent to code/SQL/test generation only; any DB the harness spins up is an ephemeral local throwaway. The backfill scripts are generated as *artifacts for humans*, explicitly never auto-run (line 160), and are gated on a non-prod `MIGRATION_PG_URL` allowlist.
|
||||
2. **It cannot deploy.** It works on an isolated branch `feat/pg-money-core-migration` cut from `origin/main` (lines 54–55, 91), is additive-only under `backend/src/db/**` (rule 2), never bumps version (rule 3), and never pushes / opens a PR / flips a flag (rule 4). Per the CI audit (BRANCH_001, DEPLOY_001), creating or committing to this branch fires no CI; only a human push to `main`/`master` or a `v*` tag deploys — outside the workflow's reach.
|
||||
3. **The worst realistic failure is recoverable.** VerifyGate commits only on green typecheck+tests (lines 216–217), and to a *local* branch. Even the post-commit SelfAudit ordering (SAFETY_001, refuted) is benign because the commit is local and reversible; the verdict still surfaces to the human before any push.
|
||||
|
||||
Every adversarially-tested `blocker`/`high` claim was **refuted** because the corresponding safety constraint *is* present in the prompts (SAFETY rules 1–7, plus per-phase reinforcement on schema indexes line 127, transactions line 145, dual-write line 146, decimals line 126, backfill guard line 160). The legitimate residual risk is that these are **prompt-level instructions, not code-enforced gates** — an agent could under-implement one and VerifyGate (which only runs typecheck+tests) would not catch it. That is a *quality/trust* gap, not a *safety* gap, and it is fully contained by the mandatory human review before cutover. Hence: go, but tighten the gates below first so the human reviewer is handed verifiable evidence instead of having to re-derive it.
|
||||
|
||||
**Confirmed blockers: 0.**
|
||||
|
||||
---
|
||||
|
||||
## 2. Confirmed Blockers (must fix before first run)
|
||||
|
||||
**None.** All `blocker`/`high`-severity findings were adversarially verified and **refuted** — the safety contract they claimed was missing is in fact specified in the workflow prompts. The workflow is safe to run as-is. The items in §3 are improvements that raise trust and catch agent under-implementation; they are not gating.
|
||||
|
||||
| ID | Why it is NOT a blocker |
|
||||
|---|---|
|
||||
| IDEMPOTENCY_INDEX_PRESERVATION / FK-010 | Line 127 explicitly mandates the partial `WHERE provider='request.network' AND direction='in' AND status='pending'` and the ledger sparse-unique. Not enforced in code, but specified + audited by SelfAudit (line 240). |
|
||||
| DECIMAL_PRECISION_TRUNCATION_RISK / FK-009 | Line 126 mandates `numeric(38,18)`, never float; SelfAudit hunts float money (line 239). Token-decimal scale flagged as a risk to surface (line 131). |
|
||||
| MULTI_DOC_WRITE / FK-011 | Line 145 requires `db.transaction(...)` for payment+ledger, referral reward, dispute hold/release; SAFETY rule 7 repeats it. |
|
||||
| LOCKFILE_001 / BRANCH_002 / SAFETY_002 | Workflow never pushes; lockfile/`npm audit` are CI-merge-time concerns the human owns. Refuted as in-workflow blockers. |
|
||||
| MIXED_ID / immutability / prod-guard / version-bump | All specified in prompts (lines 124, 48/line 70, 160, rule 3). |
|
||||
|
||||
---
|
||||
|
||||
## 3. Recommended Changes Before Running (high / medium), by phase
|
||||
|
||||
These convert prompt-level promises into **verifiable evidence** so the human reviewer and VerifyGate can confirm them, and they fix the two genuine structural risks (factory race, test-skip-as-pass).
|
||||
|
||||
| Phase | ID | Change | Severity |
|
||||
|---|---|---|---|
|
||||
| **Preflight** | PREFLIGHT_BRANCH_ALREADY_EXISTS_REUSE_RISK | After `git switch ${BRANCH}` (line 91, reuse path), run `git status --porcelain` on the target branch and STOP if dirty — re-running must not accumulate prior uncommitted work. | medium |
|
||||
| **Repos** | PARALLEL_AGENTS_RACE / CONFLICT_001 | **Structural fix.** Four parallel domain agents all "update `factory.ts`" (lines 138, 147); last writer wins, silently dropping 3 domains' wiring. Split Repos into: (a) parallel — each agent writes only its own repo files, no `factory.ts`; (b) sequential aggregator agent that builds one unified `factory.ts`. Then have VerifyGate assert `factory.ts` imports/exports all 4 domains. | medium |
|
||||
| **Repos** | REPOS_PHASE_MONGO_WRAPPER_NOT_VERIFIED | Require the MongoRepo to carry the original Mongoose call as an inline comment (or a method→source map) so the human can confirm "verbatim, no behavior change" (line 144). | medium |
|
||||
| **Repos** | FK-012 | For multi-doc money writes, the dual-write "log+alert, don't throw" (line 146) can leave Mongo funds released with no PG ledger row during the dual-write window. Add an explicit escalation/alert requirement and a reconcile sweep for these gaps (covered by reconcile.ts, line 170). | medium |
|
||||
| **Schema** | IDEMPOTENCY_INDEX_REPRODUCTION_NOT_TESTED / SCHEMA_AGENTS_INDEX_REPRODUCTION_NO_VERIFY / FK-001/FK-006/FK-008 | Have each schema agent emit the generated SQL (or index/CHECK list) in its return. Add a schema-audit step in VerifyGate that greps the generated migration for: (a) the partial `WHERE` on the RN idempotency index, (b) ledger `idempotency_key IS NOT NULL`, (c) each Mixed-id CHECK constraint, (d) presence of every Mongo index. Missing partial-`WHERE` or CHECK → blocker. | high (where) / medium |
|
||||
| **Backfill** | NO_PROD_BACKFILL_RUNAWAY_GUARD / FK-003/FK-004/FK-005/FK-007 | Make the non-prod guard a **whitelist** (`localhost`/`127.0.0.1`/named staging hosts), not a `!includes('prod')` blacklist, and require a guard unit-test that aborts on a mock prod DSN. Add per-FK parent-completion checks (TrezorAccount `addresses[].paymentId`, PointTransaction `order`/`referredUser`, Category `parentId` string audit) before child upsert. | high (where) / medium |
|
||||
| **Backfill (verify)** | VERIFICATION_CHECKSUM_PRECISION / LEDGER_IMMUTABILITY | In `checksums.ts` (line 168) compare fund sums as `::numeric` cast to **text**, not float. In `reconcile.ts` (line 170) add an assertion that ledger rows are never UPDATE/DELETE (replicating the Mongo pre-save immutability hook) — via a PG trigger or a repo guard plus a test. | medium / high (where) |
|
||||
| **Tests** | TEST_SKIP_WITHOUT_TEST_PG_SILENTLY_PASSES / TEST_001 | **Structural fix.** Tests "skip if no test PG" (line 190) but VerifyGate treats skip as pass. Return `{testsPassed, testsSkipped, skipReason}`; VerifyGate must **fail the commit if `testsSkipped` is true**, so money-safety tests are never silently bypassed. | medium |
|
||||
| **Tests** | PAYMENT_CONFIRM_LEDGER_ATOMICITY / MISSING_CONSTRAINT_SERIALIZATION / DUAL_WRITE idempotency | Expand the atomicity test (line 182) to a concrete template: inject a mid-transaction ledger failure → assert payment status rolled back → retry → assert success (idempotent). Add a 2-concurrent-confirm test and document the required isolation level / `FOR UPDATE` locking to prevent double-release. | high (where) / medium |
|
||||
| **VerifyGate** | VERSION_BUMP_DETECTION / VERSION_001 | Add a hard diff check: `git diff ${BASE}...HEAD -- backend/package.json | grep '"version"'` → if non-empty, BLOCKER and do not commit. Cheap insurance against an accidental deploy-triggering bump. | high (where) / low |
|
||||
| **SelfAudit** | SELF_AUDIT_VERDICT_NOT_GATED | After SelfAudit, gate the final return: if `verdict === 'unsafe'` or `mustFixBeforeBackfill` is non-empty, return an error/flag prominently (the commit is local and reversible, so this is advisory, but it must not be buried in the JSON). | medium |
|
||||
| **New phase** | NO_MANUAL_BACKFILL_RUNBOOK_GENERATED | Add a Documentation phase that emits `backend/src/db/BACKFILL_RUNBOOK.md`: dependency order, per-script invocation + env, checksum/row-count verification, dual-write enablement, soak/monitor, rollback. This is the cutover team's source of truth. | medium |
|
||||
|
||||
---
|
||||
|
||||
## 4. What the Workflow Does Well (trust these parts)
|
||||
|
||||
- **Airtight non-prod / non-deploy posture.** SAFETY rules 1–4 are stated globally and re-stated in every phase prompt. Branch off `origin/main`, additive-only, no version bump, no push, no auto-backfill. CI audit independently confirms the branch cannot trigger a deploy.
|
||||
- **Correct scope and FK ordering.** Tier A (money/orders) + Tier B parents (User, Category) migrated first; `COLLECTIONS` ordered parents-first (line 59); backfill runner enforces `User, Category → PurchaseRequest, SellerOffer → Payment, ...` (line 159). Out-of-scope set (Chat, Notification, Address, Review) is sensible.
|
||||
- **Money-safety modeling is explicitly specified.** `numeric(38,18)` not float (line 126); the exact RN partial-unique `WHERE`, ledger sparse-unique, and derived-destination uniqueness (line 127); Mixed-id discriminator+typed-FK+external-ref+CHECK pattern (line 124); ledger append-only (rule 6); multi-doc writes in real PG transactions (rule 7, line 145).
|
||||
- **Dual-write direction is correct.** Reads authoritative from Mongo, writes Mongo-first then PG idempotent upsert, PG failure does not block Mongo (line 146) — the safe ordering for a not-yet-cut-over store.
|
||||
- **Real verification harness + reconciliation.** Row counts, fund-sum checksums, shadow-read diffing, and a money-specific reconcile (released/refunded payments have matching ledger entries, no double-release, monotonic escrow state) — lines 167–170.
|
||||
- **Layered safety net.** A `BULK`(sonnet)/`BRAIN`(opus) model split that reserves Opus for the commit gate and the adversarial fund-safety SelfAudit (lines 16–20, 247), which hunts exactly the right failure modes (float money, dropped idempotency, non-transactional writes, collapsed Mixed ids, prod backfill).
|
||||
|
||||
---
|
||||
|
||||
## 5. Refuted / Non-Issues (do not re-raise)
|
||||
|
||||
All of the following were challenged and confirmed already handled by the workflow prompts or by the no-push/no-prod design. They are **not** action items:
|
||||
|
||||
- **Prod backfill guard, version-bump, client.ts throw-on-unset-PG_URL, ledger immutability, Mixed-id CHECK, idempotency index `WHERE`, decimal-as-numeric, multi-doc transactions** — all specified in SAFETY rules 1–7 and per-phase prompts (lines 105, 124, 126, 127, 145, 146, 160, rule 3/6/7). The "not code-enforced" critique is valid as a *trust* improvement (see §3) but does not make the workflow unsafe to run.
|
||||
- **SAFETY_001 (self-audit after commit):** refuted — commit is to a local, reversible, non-deploying branch; the verdict still reaches the human.
|
||||
- **LOCKFILE_001 / BRANCH_002 / SAFETY_002 (lockfile, npm audit):** refuted as in-workflow blockers — the workflow never installs, never pushes; these are CI-merge-time concerns owned by the human at integration. (Note: CI is Gitea Actions, not Woodpecker as the SAFETY comment says — a cosmetic doc fix only.)
|
||||
- **BRANCH_001 / BRANCH_003 / DEPLOY_001 / VERSION_001 / MERGE_001 / PARALLEL_001:** refuted — branch isolation, additive package.json edits, and "never push" are operationally sound; the residual is human discipline at push time, covered by §6.
|
||||
|
||||
---
|
||||
|
||||
## 6. Pre-Run Checklist & Human-Only Gates
|
||||
|
||||
### Before invoking the workflow
|
||||
1. **Apply the two structural §3 fixes first** — they are cheap and prevent silent breakage:
|
||||
- Repos `factory.ts` race: split into parallel-repo-files + sequential-aggregator.
|
||||
- Tests skip-as-pass: return `testsSkipped` and have VerifyGate fail on skip.
|
||||
2. **Confirm a clean working tree** on the repo (Preflight will STOP otherwise — line 90) and that `origin` is fetched.
|
||||
3. **Confirm branch state:** if `feat/pg-money-core-migration` already exists, verify it has no stray uncommitted work (PREFLIGHT_BRANCH_ALREADY_EXISTS).
|
||||
4. **Provide an ephemeral local test PG** (container/throwaway) so money-safety tests actually run instead of skipping. If you cannot, accept that VerifyGate (post-fix) will refuse to commit.
|
||||
5. **Verify parallel-agent scope is disjoint** — per repo memory, the moojttaba agent pushes to the same branches. Ensure it is NOT touching `user/payment/points/marketplace` domains or `backend/src/db/repositories/factory.ts`.
|
||||
6. **Set no production env vars** in the harness shell. Ensure `PG_URL` (if set) and `MIGRATION_PG_URL` point only at local/staging.
|
||||
|
||||
### Gates enforced during the run
|
||||
- VerifyGate commits **only** on green typecheck + tests; otherwise leaves work uncommitted with verbatim blockers (lines 216–217). Trust this — but read the `blockers` array.
|
||||
- (After §3) VerifyGate also fails on version-bump diff, skipped tests, and missing partial-`WHERE`/CHECK in generated SQL.
|
||||
|
||||
### After the run — human reviews REQUIRED before anything leaves the branch
|
||||
1. Read `selfAudit.verdict` and `selfAudit.mustFixBeforeBackfill` (lines 222–248). **Do not proceed if `unsafe` or non-empty must-fix.**
|
||||
2. Code-review the generated schema for: partial-unique `WHERE` clauses, ledger immutability enforcement, Mixed-id CHECK constraints, `numeric` (no float), and `db.transaction(...)` around payment+ledger / referral / dispute writes.
|
||||
3. Confirm `package.json` version is unchanged and the lockfile situation is understood (update lockfiles + run `npm/yarn audit` at integration time, not in this workflow).
|
||||
|
||||
### STAYS HUMAN-ONLY (workflow must never do these — and does not)
|
||||
- **Running any backfill against real/staging data.** Scripts are artifacts; a human runs them, in dependency order, against a whitelisted non-prod DSN, with `--dry-run` first and checksum verification after each step (use the §3 runbook).
|
||||
- **Cutover / flipping any `mongo|dual|pg` rollout flag.**
|
||||
- **Pushing the branch, opening/merging a PR, tagging `v*`** — any of which can trigger CI/deploy. Cherry-pick reviewed changes into a proper feature branch if needed; never push `feat/pg-money-core-migration` to `main`.
|
||||
4864
02 - Data Models/MongoDB to PostgreSQL Migration Guide.md
Normal file
4864
02 - Data Models/MongoDB to PostgreSQL Migration Guide.md
Normal file
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,437 @@
|
||||
---
|
||||
title: MongoDB → PostgreSQL Migration Plan (Drizzle)
|
||||
tags: [data-model, migration, postgres, drizzle, plan, runbook]
|
||||
aliases: [Drizzle Migration Plan, PG Migration Plan]
|
||||
created: 2026-05-31
|
||||
companion: "[[MongoDB to PostgreSQL Migration Guide]]"
|
||||
updated: 2026-05-31 for backend integrate-main-into-development@cab0719
|
||||
---
|
||||
|
||||
# MongoDB → PostgreSQL Migration Plan (Drizzle)
|
||||
|
||||
> [!abstract] What this is
|
||||
> The **execution plan** for the recommendation in [[MongoDB to PostgreSQL Migration Guide]]: a **hybrid target** (Postgres for the money/relational core, Mongo retained for Chat/Notification/TTL-session collections) reached via the **strangler pattern with dual-write**, using **Drizzle ORM** + **drizzle-kit** migrations.
|
||||
>
|
||||
> It is opinionated and concrete: a repository seam, an `id_map` bridge, Drizzle schema sketches for the hard cases (Mixed ids, embedded arrays, partial-unique idempotency, TTL), per-phase backfill/verify/cutover mechanics, and a rollback runbook. Where it references fields it uses the **real schema** from `backend/src/models/`.
|
||||
>
|
||||
> **Scope reminder:** partial migration (Phases 0–5) is the recommended stopping point — ≈16–28 engineer-weeks. Full migration of Chat/Notification/sessions is explicitly deferred.
|
||||
|
||||
> [!warning] Current implementation status
|
||||
> Backend `2.6.80` has completed the first implementation slice of this plan: Postgres/Drizzle infra, schemas/migrations through `0008`, `id_map`, `pg_dualwrite_gaps`, Drizzle/Mongo/Dual repo implementations, backfill/verify tooling, conditional oracle `payment_quotes` persistence, and the `PurchaseRequest`/`RequestTemplate` budget enum alignment with PG `budget_currency`. It has **not** completed service-layer wiring or runtime cutover. Mongo remains authoritative for normal traffic. See [[Postgres Runtime Cutover Status]].
|
||||
|
||||
---
|
||||
|
||||
## 0. Guiding principles
|
||||
|
||||
1. **Never cut over without a soak.** Every collection goes through backfill → dual-write → shadow-read verify → flip reads → soak → decommission. Rollback at any point = flip reads back to Mongo.
|
||||
2. **The repository layer is the only thing that knows where data lives.** Services must stop calling Mongoose directly. This seam is what makes the swap invisible and per-collection reversible.
|
||||
3. **Parents before children.** FK remapping flows through `id_map`; you cannot migrate `Payment` before `User` exists in PG with stable uuids.
|
||||
4. **Money correctness is the point.** The migration's payoff is real ACID transactions around payment + ledger + dispute flows that today lean on Mongo per-document atomicity. Treat every money write as transactional from day one in PG.
|
||||
5. **No feature work during migration.** No new fields, no behavior changes. A migration that also ships features cannot be verified by row-count + checksum equality.
|
||||
6. **Mongo stays authoritative until cutover.** Dual-write writes both; reads come from Mongo until a collection's shadow-read window is clean.
|
||||
|
||||
---
|
||||
|
||||
## 1. Target architecture
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ Service layer │
|
||||
│ (marketplace, payment, dispute, points, …) │
|
||||
└───────────────────────┬─────────────────────┘
|
||||
│ calls interfaces only
|
||||
┌───────────────────────▼─────────────────────┐
|
||||
│ Repository layer │
|
||||
│ IUserRepo, IPaymentRepo, IPurchaseRepo, … │
|
||||
│ ── feature-flagged per collection ── │
|
||||
└───────┬───────────────────────────┬─────────┘
|
||||
reads/writes reads/writes
|
||||
│ │
|
||||
┌───────────▼─────────┐ ┌───────────▼─────────┐
|
||||
│ MongoRepo (today) │ │ DrizzleRepo (new) │
|
||||
│ Mongoose models │ │ Postgres + Drizzle │
|
||||
└─────────────────────┘ └─────────────────────┘
|
||||
│ │
|
||||
┌─────▼─────┐ ┌─────▼─────┐
|
||||
│ MongoDB │◄── id_map ──────►│ Postgres │
|
||||
└───────────┘ (bridge) └───────────┘
|
||||
|
||||
Permanent on Mongo: Chat, Notification, TelegramSession,
|
||||
TempVerification, TelegramLink-state. Redis untouched.
|
||||
```
|
||||
|
||||
Each domain gets an interface (`IPaymentRepo`), a `MongoPaymentRepo` (wraps today's Mongoose calls verbatim), a `DrizzlePaymentRepo` (new), and a `DualWritePaymentRepo` (delegates reads to one, writes to both, behind a flag). A factory picks the implementation per collection from config:
|
||||
|
||||
```ts
|
||||
// repos/factory.ts
|
||||
type Mode = 'mongo' | 'dual' | 'pg';
|
||||
const MODE: Record<string, Mode> = {
|
||||
user: env.REPO_USER ?? 'mongo',
|
||||
payment: env.REPO_PAYMENT ?? 'mongo',
|
||||
// …per collection
|
||||
};
|
||||
export const paymentRepo: IPaymentRepo =
|
||||
MODE.payment === 'pg' ? new DrizzlePaymentRepo()
|
||||
: MODE.payment === 'dual' ? new DualWritePaymentRepo(new MongoPaymentRepo(), new DrizzlePaymentRepo())
|
||||
: new MongoPaymentRepo();
|
||||
```
|
||||
|
||||
A collection's migration is then just three flag flips: `mongo → dual → pg`.
|
||||
|
||||
---
|
||||
|
||||
## 2. Drizzle & infra setup (Phase 0)
|
||||
|
||||
### Packages
|
||||
```
|
||||
pnpm add drizzle-orm pg
|
||||
pnpm add -D drizzle-kit @types/pg
|
||||
```
|
||||
|
||||
### Layout
|
||||
```
|
||||
backend/src/db/
|
||||
schema/ # one file per table group
|
||||
users.ts
|
||||
payments.ts
|
||||
purchaseRequests.ts
|
||||
...
|
||||
idMap.ts
|
||||
index.ts # re-exports all tables + relations
|
||||
client.ts # drizzle(pg.Pool) singleton
|
||||
migrations/ # drizzle-kit generated SQL
|
||||
repositories/
|
||||
interfaces/ # IUserRepo, IPaymentRepo, …
|
||||
mongo/ # MongoUserRepo (wraps existing Mongoose)
|
||||
drizzle/ # DrizzleUserRepo
|
||||
dual/ # DualWriteUserRepo
|
||||
factory.ts
|
||||
backfill/ # per-collection batch copiers
|
||||
verify/ # row-count + checksum + shadow-read harness
|
||||
drizzle.config.ts
|
||||
```
|
||||
|
||||
### `drizzle.config.ts`
|
||||
```ts
|
||||
import { defineConfig } from 'drizzle-kit';
|
||||
export default defineConfig({
|
||||
schema: './src/db/schema/index.ts',
|
||||
out: './src/db/migrations',
|
||||
dialect: 'postgresql',
|
||||
dbCredentials: { url: process.env.PG_URL! },
|
||||
strict: true,
|
||||
verbose: true,
|
||||
});
|
||||
```
|
||||
|
||||
### Client
|
||||
```ts
|
||||
// src/db/client.ts
|
||||
import { drizzle } from 'drizzle-orm/node-postgres';
|
||||
import { Pool } from 'pg';
|
||||
import * as schema from './schema';
|
||||
export const pool = new Pool({ connectionString: process.env.PG_URL, max: 10 });
|
||||
export const db = drizzle(pool, { schema });
|
||||
```
|
||||
|
||||
> Mirror the current Mongo pool size (`maxPoolSize: 10` in `connection.ts`). Keep `mongoose.connect` alive in parallel — both drivers run for the whole migration.
|
||||
|
||||
### Migration workflow
|
||||
- Author tables in `schema/*.ts` → `pnpm drizzle-kit generate` → review the SQL in `migrations/` → `pnpm drizzle-kit migrate` in CI per environment.
|
||||
- **Migrations are versioned, reviewed, and reversible.** This is brand-new discipline — there is no migration framework today.
|
||||
|
||||
---
|
||||
|
||||
## 3. The `id_map` bridge
|
||||
|
||||
ObjectIds become uuids. Every legacy id is recorded so FKs can be remapped and dual-writes stay idempotent.
|
||||
|
||||
```ts
|
||||
// src/db/schema/idMap.ts
|
||||
import { pgTable, uuid, text, timestamp, uniqueIndex } from 'drizzle-orm/pg-core';
|
||||
export const idMap = pgTable('id_map', {
|
||||
collection: text('collection').notNull(), // 'users', 'payments', …
|
||||
legacyId: text('legacy_object_id').notNull(), // 24-char hex
|
||||
newId: uuid('new_id').notNull().defaultRandom(),
|
||||
createdAt: timestamp('created_at', { withTimezone: true }).defaultNow(),
|
||||
}, (t) => ({
|
||||
uq: uniqueIndex('id_map_collection_legacy_uq').on(t.collection, t.legacyId),
|
||||
}));
|
||||
```
|
||||
|
||||
Rules:
|
||||
- Backfill allocates `new_id` once per `(collection, legacyId)` and upserts here. Re-running backfill is safe.
|
||||
- Resolving a foreign reference = look up the parent's `legacyId` in `id_map` to get its `new_id`. **A child cannot backfill until its parents are mapped** (enforces parents-before-children).
|
||||
- Keep `legacy_object_id` as a real column on each migrated table too, for traceability and for the dual-write path to match Mongo docs.
|
||||
|
||||
---
|
||||
|
||||
## 4. Resolving the hard data-modeling cases in Drizzle
|
||||
|
||||
These are the patterns from §3 of the guide, made concrete. Get these right once; they recur.
|
||||
|
||||
### 4.1 Mixed / polymorphic ids — `Payment`, `FundsLedgerEntry`, `DerivedDestination`
|
||||
|
||||
Today `Payment.purchaseRequestId`, `sellerOfferId`, `sellerId` are `Schema.Types.Mixed` — an ObjectId for normal flows, a **string** for template checkout. **Never** store "uuid-or-string" in one PG column. Split into a typed FK + a nullable free-text ref + a discriminator.
|
||||
|
||||
```ts
|
||||
// src/db/schema/payments.ts
|
||||
import { pgTable, uuid, text, numeric, boolean, timestamp, jsonb, pgEnum, index, uniqueIndex } from 'drizzle-orm/pg-core';
|
||||
|
||||
export const paymentProvider = pgEnum('payment_provider', ['request.network','amn.scanner','shkeeper','other']);
|
||||
export const paymentDirection = pgEnum('payment_direction', ['in','out','refund']);
|
||||
export const paymentStatus = pgEnum('payment_status', ['pending','processing','completed','failed','cancelled','refunded']); // confirm full enum from model
|
||||
export const escrowState = pgEnum('escrow_state', ['funded','releasable','released','refunded','releasing','failed','cancelled','partial']);
|
||||
export const refKind = pgEnum('ref_kind', ['entity','template']); // discriminator
|
||||
|
||||
export const payments = pgTable('payments', {
|
||||
id: uuid('id').primaryKey().defaultRandom(),
|
||||
legacyObjectId: text('legacy_object_id'),
|
||||
|
||||
// purchaseRequestId (Mixed) → typed FK OR free string
|
||||
purchaseRequestRefKind: refKind('purchase_request_ref_kind').notNull(),
|
||||
purchaseRequestId: uuid('purchase_request_id').references(() => purchaseRequests.id), // null when template
|
||||
purchaseRequestExternalRef: text('purchase_request_external_ref'), // set when template
|
||||
|
||||
// sellerOfferId (Mixed) → same shape
|
||||
sellerOfferRefKind: refKind('seller_offer_ref_kind').notNull(),
|
||||
sellerOfferId: uuid('seller_offer_id').references(() => sellerOffers.id),
|
||||
sellerOfferExternalRef: text('seller_offer_external_ref'),
|
||||
|
||||
buyerId: uuid('buyer_id').notNull().references(() => users.id),
|
||||
|
||||
// sellerId (Mixed)
|
||||
sellerRefKind: refKind('seller_ref_kind').notNull(),
|
||||
sellerId: uuid('seller_id').references(() => users.id),
|
||||
sellerExternalRef: text('seller_external_ref'),
|
||||
|
||||
// amount subdoc → inline columns
|
||||
amount: numeric('amount', { precision: 38, scale: 18 }).notNull(),
|
||||
currency: text('currency').notNull().default('USDT'),
|
||||
|
||||
provider: paymentProvider('provider').notNull().default('request.network'),
|
||||
direction: paymentDirection('direction').notNull().default('in'),
|
||||
status: paymentStatus('status').notNull().default('pending'),
|
||||
escrowState: escrowState('escrow_state'),
|
||||
|
||||
providerPaymentId: text('provider_payment_id'),
|
||||
blockchain: jsonb('blockchain'), // transactionHash etc. — read-as-blob, GIN if filtered
|
||||
metadata: jsonb('metadata'), // provider-specific, schema-varying
|
||||
|
||||
isRefunded: boolean('is_refunded').notNull().default(false),
|
||||
completedAt: timestamp('completed_at', { withTimezone: true }),
|
||||
createdAt: timestamp('created_at', { withTimezone: true }).defaultNow(),
|
||||
updatedAt: timestamp('updated_at', { withTimezone: true }).defaultNow(),
|
||||
}, (t) => ({
|
||||
byStatusCreated: index('payments_status_created_idx').on(t.status, t.createdAt),
|
||||
byBuyerStatus: index('payments_buyer_status_idx').on(t.buyerId, t.status),
|
||||
bySellerStatus: index('payments_seller_status_idx').on(t.sellerId, t.status),
|
||||
txHash: index('payments_tx_hash_idx').on(t.providerPaymentId),
|
||||
// Partial-unique idempotency — the real Mongo index 'uniq_pending_request_network_by_buyer_session_offer'
|
||||
pendingRnUq: uniqueIndex('uniq_pending_rn_by_buyer_offer')
|
||||
.on(t.buyerId, t.purchaseRequestId, t.sellerOfferId, t.provider, t.direction)
|
||||
.where(sql`provider = 'request.network' AND direction = 'in' AND status = 'pending'`),
|
||||
}));
|
||||
```
|
||||
|
||||
Add a CHECK so a discriminator always agrees with which column is populated:
|
||||
```sql
|
||||
ALTER TABLE payments ADD CONSTRAINT payments_pr_ref_ck CHECK (
|
||||
(purchase_request_ref_kind = 'entity' AND purchase_request_id IS NOT NULL AND purchase_request_external_ref IS NULL) OR
|
||||
(purchase_request_ref_kind = 'template' AND purchase_request_id IS NULL AND purchase_request_external_ref IS NOT NULL)
|
||||
);
|
||||
```
|
||||
|
||||
`FundsLedgerEntry` has the same Mixed `purchaseRequestId`/`paymentId` plus a **`idempotencyKey` sparse-unique** → partial unique index `WHERE idempotency_key IS NOT NULL`.
|
||||
|
||||
### 4.2 Embedded arrays → child tables
|
||||
|
||||
| Source (embedded) | PG | Notes |
|
||||
|---|---|---|
|
||||
| `PurchaseRequest.offers[]` (array of SellerOffer ids) | junction `purchase_request_offers(pr_id, offer_id)` | FK integrity; also drop the denormalized array. |
|
||||
| `PurchaseRequest.preferredSellerIds[]` | junction `pr_preferred_sellers(pr_id, user_id)` | — |
|
||||
| `PurchaseRequest.deliveryInfo / serviceInfo` (nested subdocs) | child tables `pr_delivery_info`, `pr_service_info` (1:1) | queried logistics; not blobbed. |
|
||||
| `Dispute.evidence[]`, `Dispute.timeline[]` | `dispute_evidence`, `dispute_timeline` | timeline pre-save append → explicit INSERT. |
|
||||
| `User.passkeys[]`, `User.refreshTokens[]` | `user_passkeys`, `user_refresh_tokens` | append/revoke + lookup semantics. |
|
||||
| `DerivedDestination` sweep history, `TrezorAccount.addresses[]` | child tables | per-address rows referenced by payments. |
|
||||
| `Payment.blockchain`, `Payment.metadata`, `Notification.metadata`, `PointTransaction.metadata` | **JSONB** | read-as-blob, never filtered/joined. |
|
||||
|
||||
Rule: **child table when you query/index/FK/aggregate it; JSONB when you read it whole and never filter on it.**
|
||||
|
||||
### 4.3 Self-referential FK — `Category`
|
||||
```ts
|
||||
export const categories = pgTable('categories', {
|
||||
id: uuid('id').primaryKey().defaultRandom(),
|
||||
legacyObjectId: text('legacy_object_id'),
|
||||
name: text('name').notNull(),
|
||||
nameEn: text('name_en'),
|
||||
parentId: uuid('parent_id'), // self-FK, see relations
|
||||
isActive: boolean('is_active').notNull().default(true),
|
||||
}, (t) => ({
|
||||
parentIdx: index('categories_parent_idx').on(t.parentId),
|
||||
activeIdx: index('categories_active_idx').on(t.isActive),
|
||||
}));
|
||||
// relations(): parentId → categories.id, ON DELETE SET NULL
|
||||
```
|
||||
`Category.parentId` is itself Mixed (ObjectId | string) in the model — verify all rows are ObjectIds during the pre-migration audit; treat stray strings as data errors to clean.
|
||||
|
||||
### 4.4 Sparse-unique → partial unique index — `User.email`, `User.referralCode`
|
||||
The runtime code in `connection.ts` rebuilds `users.email` as unique+sparse. In PG:
|
||||
```ts
|
||||
emailUq: uniqueIndex('users_email_uq').on(t.email).where(sql`email IS NOT NULL`),
|
||||
referralUq: uniqueIndex('users_referral_uq').on(t.referralCode).where(sql`referral_code IS NOT NULL`),
|
||||
```
|
||||
Reimplement `toJSON()` password/token stripping in the repository's read mapper (it deletes `refreshTokens`, `emailVerification*` before returning).
|
||||
|
||||
### 4.5 Atomic counter — `DerivedDestination.derivationIndex`
|
||||
Today allocation relies on Mongo atomicity. In PG use a real transaction with `SELECT … FOR UPDATE` on a per-(buyer,chain) counter row, or a dedicated sequence per chain. The `uniq_destination_by_buyer_seller_chain` unique index ports directly. `status` enum `('active','swept','sweeping','quarantined')` → `pgEnum`.
|
||||
|
||||
### 4.6 TTL → `pg_cron`
|
||||
`TempVerification` and `TelegramSession` stay on Mongo (ephemeral, recommended). If `Notification` (90-day TTL) ever moves: monthly range-partition + drop, or
|
||||
```sql
|
||||
SELECT cron.schedule('notifications_ttl', '0 3 * * *',
|
||||
$$DELETE FROM notifications WHERE created_at < now() - interval '90 days'$$);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. The dual-write seam (the mechanic that makes it safe)
|
||||
|
||||
```ts
|
||||
// repositories/dual/DualWritePaymentRepo.ts
|
||||
export class DualWritePaymentRepo implements IPaymentRepo {
|
||||
constructor(private mongo: IPaymentRepo, private pg: IPaymentRepo) {}
|
||||
|
||||
// READS: source of truth = Mongo until cutover
|
||||
findById(id) { return this.mongo.findById(id); }
|
||||
|
||||
// WRITES: both, idempotently. Mongo first (authoritative); PG must not break the request.
|
||||
async create(input) {
|
||||
const m = await this.mongo.create(input); // returns doc incl. _id
|
||||
try {
|
||||
await this.pg.upsertFromMongo(m); // keyed by legacyObjectId / idempotencyKey
|
||||
} catch (e) {
|
||||
metrics.dualWriteError('payments', 'create', e); // alert, do NOT throw
|
||||
}
|
||||
return m;
|
||||
}
|
||||
|
||||
async update(id, patch) {
|
||||
const m = await this.mongo.update(id, patch);
|
||||
try { await this.pg.upsertFromMongo(m); } catch (e) { metrics.dualWriteError('payments','update',e); }
|
||||
return m;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- **Mongo write is authoritative and must succeed**; PG write failures are logged + alerted, never surfaced to the user, during `dual` mode. (Once in `pg` mode, PG is authoritative and wrapped in real transactions.)
|
||||
- All PG writes are **idempotent upserts** keyed on `legacyObjectId` (or natural idempotency keys: `Payment` partial-unique set, `FundsLedgerEntry.idempotencyKey`). This lets backfill and live dual-write overlap without double-insert.
|
||||
- `$inc`/`$push` translate inside the repo: `$inc points` → `UPDATE … SET points = points + $1` in a transaction; `$push offers` → `INSERT INTO purchase_request_offers …`.
|
||||
|
||||
---
|
||||
|
||||
## 6. Phased execution
|
||||
|
||||
Same phases as the guide §2, here with Drizzle-concrete entry/exit gates. Each phase ends with a collection in `pg` mode and dual-write removed only after the soak.
|
||||
|
||||
### Phase 0 — Foundations (2–5 wk) — *no data moves*
|
||||
- Stand up Postgres (per env), Drizzle, drizzle-kit, CI migrations. **Status 2026-05-31:** implemented in code and dev stack, but migrations must still be applied per target DB.
|
||||
- Build repository interfaces + `MongoRepo` wrappers for the relational-core domains (refactor services to call repos, not Mongoose directly). **Status 2026-05-31:** repo interfaces/implementations exist; service-layer wiring remains the bulk of the cutover risk.
|
||||
- Create `id_map`, the verification harness (§7), and the backfill batch runner skeleton.
|
||||
- **Exit:** all relational-core services call repositories; PG reachable everywhere; `id_map` + verify harness exist; CI runs migrations.
|
||||
|
||||
### Phase 1 — Address pilot (1–2 wk)
|
||||
- Smallest real domain; proves backfill → dual-write → verify → cutover end-to-end.
|
||||
- Reimplement the **one-primary-per-user** pre-save invariant as either a partial unique index `UNIQUE (user_id) WHERE primary = true` or a trigger.
|
||||
- **Exit:** `addresses` in `pg` mode in prod, invariant proven under concurrent writes, verify green, dual-write removed.
|
||||
|
||||
### Phase 2 — Reference/config (2–3 wk)
|
||||
- `Category` (self-FK, soft-delete), `LevelConfig`, `ConfigSetting`, `ConfigSettingHistory`, `ShopSettings`, `Review`.
|
||||
- Port seeds to run in dependency order. Enforce `ShopSettings.sellerId` unique, Category `parentId` ON DELETE SET NULL.
|
||||
- **Exit:** these read from PG; seeds run in PG.
|
||||
|
||||
### Phase 3 — User + auth core (3–5 wk)
|
||||
- `User` is the FK hub — **must precede the money core** so `id_map` for users is authoritative.
|
||||
- Normalize `profile`/`preferences`/`points`/`referralStats` into columns; extract `passkeys[]`, `refreshTokens[]` to child tables; partial-unique `email`/`referralCode`; reimplement `toJSON()` stripping; passkey `default: Date.now()` in app code.
|
||||
- Redis session/rate-limit + in-memory passkey challenge store stay as-is.
|
||||
- **Exit:** `users` in `pg` mode; referral self-FK intact; all auth flows pass; user uuids authoritative in `id_map`.
|
||||
|
||||
### Phase 4 — Money core (6–10 wk) — *the point of the project*
|
||||
- `PurchaseRequest`, `SellerOffer`, `Payment`, `FundsLedgerEntry`, `DerivedDestination`, `TrezorAccount`, `PointTransaction`.
|
||||
- Apply §4.1 (Mixed→discriminator+FK), §4.2 (offers/preferredSellers junctions, deliveryInfo/serviceInfo child tables), §4.5 (derivation counter).
|
||||
- **Wrap in real PG transactions the multi-doc writes that today have none:** `raiseDispute` (PurchaseRequest + Payment), payment confirm + `FundsLedgerEntry` AML-fee insert, referral reward (points + referralStats), PointsService flows (migrate its 2 `withTransaction` sites to PG `BEGIN/COMMIT`).
|
||||
- Preserve the `Payment` partial-unique idempotency index and `FundsLedgerEntry.idempotencyKey` uniqueness.
|
||||
- **Exit:** money core in `pg` mode; checksum equality on `funds_ledger_entries` sums & `payments` amounts across a full soak; idempotency + escrow-hold invariants pass concurrency tests.
|
||||
|
||||
### Phase 5 — Dispute + delivery (2–4 wk)
|
||||
- `Dispute.evidence[]`/`timeline[]` → child tables; pre-save timeline-append → explicit INSERT; delivery `$set/$push` nested updates → SQL.
|
||||
- `Dispute ↔ Chat` becomes a **cross-store call** (Chat stays on Mongo) — define the boundary API.
|
||||
- **Exit:** dispute lifecycle in `pg` mode; release-hold sync transactional.
|
||||
|
||||
### Phase 6 (deferred / optional) — `RequestTemplate`, `BlogPost`
|
||||
- Behind a search abstraction; `$regex` → PG trigram/FTS only if migrated. Otherwise leave on Mongo.
|
||||
|
||||
### Permanent on Mongo
|
||||
`Chat`, `Notification`, `TelegramSession`, `TempVerification`, `TelegramLink` link-state. Revisit only if dual-stack ops cost exceeds migration cost.
|
||||
|
||||
---
|
||||
|
||||
## 7. Verification (gate for every cutover)
|
||||
|
||||
Three layers, **all green before any read flip**:
|
||||
|
||||
1. **Row counts** — per collection and per FK relationship, Mongo vs PG. Catches dropped/dangling rows. Run continuously during dual-write.
|
||||
2. **Checksums** — column-level hashes; special attention to financial sums (`SUM(funds_ledger_entries.amount)`, `SUM(payments.amount)` grouped by status/provider) and the partial-unique idempotency set.
|
||||
3. **Shadow reads** — in prod, serve from Mongo, asynchronously read PG for the same key, diff, alert on mismatch. **A clean shadow-read window (e.g. 7 days, zero diffs on hot paths) is the exit criterion for cutover.**
|
||||
|
||||
```ts
|
||||
// verify/shadow.ts — wrap a repo read in dual mode
|
||||
async function shadowRead(key, mongoFn, pgFn) {
|
||||
const m = await mongoFn(key);
|
||||
pgFn(key).then(p => { if (!deepEqualNormalized(m, p)) metrics.shadowMismatch(key, diff(m, p)); })
|
||||
.catch(e => metrics.shadowError(key, e));
|
||||
return m; // user always gets Mongo result
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. Cutover & rollback runbook (per collection)
|
||||
|
||||
1. **Backfill** in batches with checkpointing; allocate uuids → `id_map`; remap FKs from already-migrated parents. Re-runnable (idempotent upserts).
|
||||
2. **Enable `dual`** (flag) — writes go to both; shadow-read diffing on. Backfill the delta accumulated during step 1.
|
||||
3. **Soak** until row-count + checksum + shadow-read are clean for the agreed window.
|
||||
4. **Flip reads to `pg`** (flag). Keep dual-write on.
|
||||
5. **Soak again** (shorter). Rollback = flip reads back to `mongo`; data still mirrored, so rollback is instant.
|
||||
6. **Decommission**: stop writing Mongo for that collection; archive the collection.
|
||||
|
||||
> Near-zero downtime: there is no global write freeze except, optionally, a brief one during final ledger reconciliation for the money core.
|
||||
|
||||
---
|
||||
|
||||
## 9. First two weeks — concrete starter checklist
|
||||
|
||||
- [ ] Add `drizzle-orm`, `pg`, `drizzle-kit`; create `src/db/{schema,client.ts,migrations}` + `drizzle.config.ts`.
|
||||
- [x] Provision Postgres in dev (compose) + define `PG_URL`; keep Mongo running alongside. Use Postgres 18 volume mount `/var/lib/postgresql`, not `/var/lib/postgresql/data`.
|
||||
- [ ] Write `id_map` schema; generate + run the first migration in CI.
|
||||
- [ ] Define `IAddressRepo`; implement `MongoAddressRepo` by moving the existing Mongoose calls behind it; refactor address service to use the repo. **No behavior change** — prove the seam is invisible (existing tests pass).
|
||||
- [ ] Build the verification harness (row count + checksum) against `addresses`.
|
||||
- [ ] Author `addresses` Drizzle schema (incl. one-primary partial unique index) + `DrizzleAddressRepo` + `DualWriteAddressRepo`.
|
||||
- [ ] Write the batch backfill for `addresses`; run dev backfill; confirm verify is green.
|
||||
- [ ] Flip dev to `dual`, then `pg`; document the flag flips. This is the template for all later phases.
|
||||
|
||||
---
|
||||
|
||||
## 10. Effort recap (from the guide)
|
||||
|
||||
| Scope | Eng-weeks | Notes |
|
||||
|---|---|---|
|
||||
| **Partial — money/relational core (Phases 0–5 + cross-cutting)** | **~16–28** | Recommended stopping point; captures ~90% of value (ACID money + relational integrity). |
|
||||
| Full — all 23 collections | ~23–40 | Extra 7–12+ wks mostly buys Chat/Notification normalization the access patterns don't reward. |
|
||||
|
||||
Add ~20% contingency for data-audit surprises in the Mixed-id fields. One focused engineer assumed; parallelize to compress wall-clock, not effort.
|
||||
|
||||
---
|
||||
|
||||
> [!warning] Before trusting the code sketches
|
||||
> Drizzle schemas above use the real field names from `backend/src/models/` but are **first-pass sketches** — confirm the full `Payment.status` enum, the exact `amount` precision/scale your tokens need (USDT/USDC decimals), and audit which `Mixed` rows are actually strings vs ObjectIds **before** writing the money-core migration. See [[MongoDB to PostgreSQL Migration Guide]] §3/§5 for the authoritative per-field detail.
|
||||
@@ -10,6 +10,9 @@ aliases: [Payment Record, Escrow, IPayment]
|
||||
|
||||
Records every monetary movement in the marketplace: buyer pay-ins, seller payouts, and refunds. The current model is centered on Request Network pay-in, in-house checkout metadata, on-chain transaction verification, escrow state, and provider request IDs. The `provider` and `direction` discriminators let one collection hold incoming buyer payments, outgoing seller releases, refunds, and legacy/other provider records.
|
||||
|
||||
> [!warning] Runtime store
|
||||
> The `Payment` document is still created, read, and updated through Mongoose on normal request paths. Backend `2.6.80` can persist oracle quotes to Postgres `payment_quotes`, but that is conditional on `ORACLE_QUOTING_ENABLED=true` and does not make the whole payment domain PG-authoritative. See [[Postgres Runtime Cutover Status]].
|
||||
|
||||
> [!note] Source
|
||||
> `backend/src/models/Payment.ts:3` — schema definition
|
||||
> `backend/src/models/Payment.ts:257` — model export (default export)
|
||||
@@ -121,7 +124,7 @@ Defined at `backend/src/models/Payment.ts:174-188`:
|
||||
|
||||
## Postgres Quote Table
|
||||
|
||||
The Postgres money-core branch stores oracle quotes in `payment_quotes`, a 1:1 child table keyed by `payment_id → payments.id`. Amount/rate columns use `numeric(38,18)` and the route resolves the PG parent through `payments.legacy_object_id` or `id_map` during the Mongo/PG dual-write window. If the PG payment row is missing, the quote is mirrored to this Mongo `quote` subdocument and a `pg_dualwrite_gaps` row is recorded for reconciliation.
|
||||
The Postgres money-core branch can store oracle quotes in `payment_quotes`, a 1:1 child table keyed by `payment_id → payments.id`. Amount/rate columns use `numeric(38,18)` and the route resolves the PG parent through `payments.legacy_object_id` or `id_map` during the Mongo/PG dual-write window. If the PG payment row is missing, the quote is mirrored to this Mongo `quote` subdocument and a `pg_dualwrite_gaps` row is recorded for reconciliation. This table is quote/audit storage only until the payment service itself is wired through the PG repository path.
|
||||
|
||||
## Pre/Post Hooks
|
||||
|
||||
|
||||
68
02 - Data Models/Postgres Runtime Cutover Status.md
Normal file
68
02 - Data Models/Postgres Runtime Cutover Status.md
Normal file
@@ -0,0 +1,68 @@
|
||||
---
|
||||
title: Postgres Runtime Cutover Status
|
||||
tags: [data-model, postgres, migration, runtime-status]
|
||||
aliases: [Postgres Status, PG Cutover Status, Mongo vs Postgres Runtime]
|
||||
created: 2026-05-31
|
||||
source: backend origin/integrate-main-into-development@cab0719
|
||||
---
|
||||
|
||||
# Postgres Runtime Cutover Status
|
||||
|
||||
> **Current branch:** backend `origin/integrate-main-into-development` at `cab0719`, version `2.6.80`.
|
||||
>
|
||||
> **Bottom line:** this branch is **Postgres-capable**, not fully Postgres-backed. MongoDB remains the live source of truth for normal app traffic until services are actually rewired through the repository factory and cutover flags are verified in a running environment.
|
||||
|
||||
## What Uses Postgres Now
|
||||
|
||||
| Area | Runtime status | Notes |
|
||||
|---|---|---|
|
||||
| Postgres connection | Available when `PG_URL` is set | `src/db/client.ts` fail-fast requires an explicit DSN before importing PG code. |
|
||||
| Schema/migrations | Implemented through migration `0008` | Drizzle schemas cover users, categories, purchase requests, seller offers, payments, funds ledger, derived destinations, trezor accounts, points, `id_map`, `pg_dualwrite_gaps`, and `payment_quotes`. |
|
||||
| Repository implementations | Implemented but not broadly called | `src/db/repositories/{mongo,drizzle,dual}` and `factory.ts` exist for user, payment, points, and marketplace domains. |
|
||||
| Oracle quote persistence | Conditional runtime PG write | `/api/payment/request-network/intents` lazily imports `quoteRepo` only when `ORACLE_QUOTING_ENABLED=true`; it writes `payment_quotes` if the PG parent payment row exists, mirrors to Mongo `Payment.quote`, and records `pg_dualwrite_gaps` if PG is behind. |
|
||||
| Backfill/verify scripts | Available as operator tooling | `MIGRATION_PG_URL` drives backfill scripts; guards restrict allowed target hosts. These scripts are not run automatically by app startup. |
|
||||
|
||||
## What Is Still Mongo-Backed
|
||||
|
||||
The service layer still imports Mongoose models directly. A source scan on 2026-05-31 found no runtime calls to `createRepositories()` / `getPaymentRepo()` / `getMarketplaceRepo()` outside `src/db/repositories/factory.ts`; the only normal service import of `src/db/client.ts` is `priceOracle/quoteRepo.ts`.
|
||||
|
||||
| Domain | Current live store | Why not Postgres yet |
|
||||
|---|---|---|
|
||||
| Auth, users, passkeys, refresh tokens | MongoDB | Auth/user controllers call `User` and related Mongoose models directly. |
|
||||
| Marketplace requests/offers/templates/categories/shop settings | MongoDB | Marketplace, checkout, and seller-offer services call `PurchaseRequest`, `SellerOffer`, `RequestTemplate`, `Category`, and `ShopSettings` directly. |
|
||||
| Payments and escrow state | MongoDB primary | Request Network, AMN scanner, webhook, admin, release/refund, adapter, reconciliation, and legacy payment paths still create/update `Payment` Mongoose documents directly. |
|
||||
| Funds ledger | MongoDB primary | `FundsLedgerEntry` remains the ledger used by current services; PG ledger tables exist but are not the live write target. |
|
||||
| Derived destinations and sweeps | MongoDB | Wallet destination allocation and sweep services call `DerivedDestination` directly. |
|
||||
| Points/referrals/levels | MongoDB | Points service uses `User`, `PointTransaction`, `LevelConfig`, and Mongo transactions. |
|
||||
| Chat/messages | MongoDB | Chat intentionally remains document-shaped and is not part of the current PG cutover. |
|
||||
| Notifications | MongoDB | Notification TTL/read-state paths remain Mongo-backed. |
|
||||
| Disputes/reviews/blog/content/admin cleanup | MongoDB | These services still call their Mongoose models directly. |
|
||||
| Runtime config | MongoDB | `ConfigSetting` and `ConfigSettingHistory` remain Mongo-backed for admin-editable configuration. |
|
||||
| Telegram link/session/temp verification | MongoDB | These session/link/TTL records are not cut over to PG. |
|
||||
|
||||
## Env Flag Reality
|
||||
|
||||
| Flag | Current meaning |
|
||||
|---|---|
|
||||
| `PG_URL` | Makes PG code importable/reachable; does not cut over app domains by itself. |
|
||||
| `MIGRATION_PG_URL` | Used by backfill scripts and migration runbooks; not part of normal request handling. |
|
||||
| `REPO_USER`, `REPO_PAYMENT`, `REPO_POINTS`, `REPO_MARKETPLACE`, `REPO_DEFAULT` | Repository factory flags exist, but broad services are not yet wired through the factory. Treat them as migration controls that need integration verification before relying on them. |
|
||||
| `ORACLE_QUOTING_ENABLED` | Enables server-side quote computation and the only current PG write path in normal checkout: `payment_quotes`, when a PG parent row can be resolved. |
|
||||
|
||||
## Next Cutover Work
|
||||
|
||||
1. Apply Drizzle migrations to the target Postgres database.
|
||||
2. Run non-prod backfills in dependency order and record row-count/checksum results.
|
||||
3. Wire services to repository interfaces one domain at a time.
|
||||
4. Enable `dual` mode per domain only after wiring is proven by tests and smoke checks.
|
||||
5. Run shadow-read/reconcile during a soak window.
|
||||
6. Flip reads to `pg` per domain only after zero-diff shadow reads and a rollback plan are in place.
|
||||
|
||||
## Related Docs
|
||||
|
||||
- [[Database Strategy - Mongo vs Postgres Assessment]]
|
||||
- [[MongoDB to PostgreSQL Migration Plan (Drizzle)]]
|
||||
- [[Payment]]
|
||||
- [[Payment API]]
|
||||
- [[Environment Variables]]
|
||||
- [[Database Operations]]
|
||||
@@ -6,7 +6,7 @@ aliases: [Purchase Request, Buy Request, IPurchaseRequest]
|
||||
|
||||
# PurchaseRequest
|
||||
|
||||
> **Last updated:** 2026-05-30 — `budget.currency` locked to USDT; `categoryId` added to `IRequestTableItem`
|
||||
> **Last updated:** 2026-05-31 — `budget.currency` aligned with template/Postgres enum (`USD`, `EUR`, `IRR`, `USDT`, `USDC`); `categoryId` added to `IRequestTableItem`
|
||||
|
||||
The central buyer-side document. A `PurchaseRequest` captures what a buyer wants to acquire (physical product, digital product, service, or consultation), the budget envelope, urgency, delivery details, and the entire lifecycle from creation through payment, delivery, and completion. Sellers respond by attaching [[SellerOffer]] documents; the buyer accepts one, a [[Payment]] is opened, and delivery is verified by a 6-digit code.
|
||||
|
||||
@@ -31,7 +31,7 @@ The central buyer-side document. A `PurchaseRequest` captures what a buyer wants
|
||||
| `quantity` | Number | no | `1` | min 1 | — | Unit count. |
|
||||
| `budget.min` | Number | no | — | min 0 | — | Lower bound. |
|
||||
| `budget.max` | Number | no | — | min 0 | — | Upper bound. |
|
||||
| `budget.currency` | String | no | `USDT` | enum: `USDT` (escrow MVP — USD/EUR/IRR removed in commit 3aaa2fe) | — | Budget currency. |
|
||||
| `budget.currency` | String | no | `USDT` | enum: `USD` / `EUR` / `IRR` / `USDT` / `USDC` | — | Budget currency. Runtime Mongoose validation, request-template validation, and the PG `budget_currency` enum now share these values. |
|
||||
| `urgency` | String | no | `medium` | enum: `low` / `medium` / `high` / `urgent` | yes | Buyer urgency. |
|
||||
| `status` | String | no | `pending` | enum (13 values — see State Transitions below) | yes | Lifecycle state. |
|
||||
| `isPublic` | Boolean | no | `true` | — | — | Public marketplace listing vs. private request. |
|
||||
|
||||
@@ -28,7 +28,7 @@ A reusable template authored by a seller. When a buyer visits the template's `sh
|
||||
| `quantity` | Number | no | `1` | min 1 | — | Default unit count. |
|
||||
| `budget.min` | Number | no | — | min 0 | — | Lower bound. |
|
||||
| `budget.max` | Number | no | — | min 0 | — | Upper bound. |
|
||||
| `budget.currency` | String | no | `USD` | enum: `USD` / `EUR` / `IRR` | — | Currency. |
|
||||
| `budget.currency` | String | no | `USDT` | enum: `USD` / `EUR` / `IRR` / `USDT` / `USDC` | — | Currency. Shared with [[PurchaseRequest]] so a template can be converted without enum drift. |
|
||||
| `urgency` | String | no | `medium` | enum: `low` / `medium` / `high` / `urgent` | — | Urgency. |
|
||||
| `tags[]` | String[] | no | — | trim | — | Tags. |
|
||||
| `specifications[].key` | String | yes | — | trim | — | Spec key. |
|
||||
|
||||
@@ -71,7 +71,7 @@ The buyer-facing CRUD plus seller-side workflow endpoints. Model: [[PurchaseRequ
|
||||
size?: string;
|
||||
color?: string;
|
||||
quantity?: number; // default 1
|
||||
budget?: { min?: number; max?: number; currency: "USDT" | "USDC" }; // restricted to escrow-compatible stablecoins (commit d52feb7)
|
||||
budget?: { min?: number; max?: number; currency: "USD" | "EUR" | "IRR" | "USDT" | "USDC" };
|
||||
urgency?: "low" | "medium" | "high" | "urgent";
|
||||
deliveryInfo?: {
|
||||
deliveryType: "physical" | "online";
|
||||
@@ -343,8 +343,8 @@ A [[RequestTemplate]] is a re-usable "shop product" a seller can publish. Buyers
|
||||
size?: string; // <=100
|
||||
color?: string; // <=100
|
||||
quantity?: number; // 1-10000
|
||||
budget?: { min?: number; max?: number; currency: "USD" | "EUR" | "IRR" };
|
||||
urgency?: "low" | "medium" | "high";
|
||||
budget?: { min?: number; max?: number; currency: "USD" | "EUR" | "IRR" | "USDT" | "USDC" };
|
||||
urgency?: "low" | "medium" | "high" | "urgent";
|
||||
deliveryInfo?: { deliveryType: "physical" | "online"; email?: string };
|
||||
maxUsage?: number | null; // 0/null = unlimited
|
||||
expiresAt?: string | null; // ISO date
|
||||
|
||||
@@ -22,6 +22,9 @@ The payment surface is split across provider-neutral payment routers, Request Ne
|
||||
|
||||
Core model: [[Payment]]. Coordination logic to avoid race conditions when multiple sources update the same payment is in `paymentCoordinator.ts`.
|
||||
|
||||
> [!warning] Persistence status
|
||||
> Payment APIs still create/read/update Mongo `Payment` documents on backend `2.6.79`. The Postgres branch adds schemas, repos, migrations, and optional quote persistence, but it is not a full payment-domain cutover. `/api/payment/request-network/intents` can write `payment_quotes` only when `ORACLE_QUOTING_ENABLED=true`; the payment record itself remains Mongo-backed unless future service wiring changes that boundary.
|
||||
|
||||
## Configuration / health
|
||||
|
||||
### POST /api/payment/configuration
|
||||
|
||||
@@ -0,0 +1,286 @@
|
||||
---
|
||||
title: Oracle Depeg Checkout — UI Implementation Guide
|
||||
status: implementation-ready with backend-contract caveats (reconciled with backend integrate-main-into-development@3a50dc4)
|
||||
audience: frontend
|
||||
stack: Next.js 16 (App Router) · React 19 · TypeScript · MUI 7 + Emotion · SWR · axios · Socket.io · i18next (fa default, RTL)
|
||||
related: ../01 - Architecture/Oracle Pricing & Stablecoin Depeg Protection.md
|
||||
---
|
||||
|
||||
# Oracle Depeg Checkout — UI Implementation Guide
|
||||
|
||||
> Goal: a frontend dev can implement the depeg-protected checkout step **straight from this doc**. It maps every piece to the real codebase (file:line), gives the API contract + TypeScript types, the component tree (MUI), the state machine, display/formatting/RTL rules, error UX with copy (en + fa), wireframes, and acceptance criteria.
|
||||
>
|
||||
> **Backend reality 2026-05-31:** there is no separate read-only quote preview endpoint yet. The committed backend computes and returns a quote from `POST /api/payment/request-network/intents` when `ORACLE_QUOTING_ENABLED=true`; it also writes `payment_quotes` only if a PG parent payment row can be resolved. Mongo remains the runtime source for the Payment itself. See [[Postgres Runtime Cutover Status]].
|
||||
|
||||
## 0. What the buyer experiences (plain English)
|
||||
|
||||
The seller priced the item in some currency (e.g. **IRR / TRY / USD**). The buyer picks a **settlement stablecoin + chain** (USDC/USDT on an allow-listed chain). The app fetches a **live quote**: it converts the invoice to the token at the oracle rate, **adds depeg protection** (if USDC trades at \$0.97 the buyer pays ~3% more so the seller is made whole), and **snaps to a human-readable amount** when one is within 3%. The buyer sees exactly **what they pay, in what token, ≈ the invoice currency**, the depeg adjustment, and a short **expiry countdown**. On confirm, the payment intent is created against that locked quote.
|
||||
|
||||
## 1. Where it slots into the existing checkout
|
||||
|
||||
Existing flow (`src/sections/request-template/`): **cart → billing → payment → complete** (`view/request-template-checkout-view.tsx:20-77`, provider `context/request-template-checkout-provider.tsx`).
|
||||
|
||||
**Recommended:** insert a **Quote** sub-step at the top of the existing **payment** step (Option B from the explore — least restructuring). The token/chain selector already lives in the payment step (`request-template-checkout-payment.tsx` → `ProviderPayment`); we add the quote panel directly above the pay button and **block payment until a valid (non-expired, non-blocked) quote exists**.
|
||||
|
||||
| Touch point | File | Change |
|
||||
|---|---|---|
|
||||
| Token/chain selection | `request-template-checkout-payment.tsx` (ProviderPayment child) | On (token, chain) change → fetch quote |
|
||||
| Order summary | `request-template-checkout-payment.tsx:985-1050` | Add quote card + depeg banner + expiry timer above pay button |
|
||||
| Checkout state | `context/types.ts` | Add `oracleQuote` + `quoteStatus` fields |
|
||||
| API endpoints | `src/lib/axios.ts` (`endpoints` object, ~188-511) | Add `endpoints.payments.quote` |
|
||||
| Types | `src/types/payment.ts` | Add `IPaymentQuote` (below) |
|
||||
| Currency format | `src/utils/currencyUtils.ts` | Add fiat formatting (IRR/TRY) + dual-amount helper |
|
||||
|
||||
## 2. API contract
|
||||
|
||||
> The amount is computed **server-side**; the UI never sends or trusts a money amount. Field names below include the target preview contract plus the currently committed `/intents` behavior.
|
||||
|
||||
### 2.1 Future `POST /payment/quote` — preview a quote (read-only, no Payment created)
|
||||
|
||||
> [!warning] Not implemented in backend `3a50dc4`
|
||||
> Build the first frontend integration against `POST /api/payment/request-network/intents` for now. Add this preview endpoint later if the UX needs live quotes before creating/updating a pending Payment.
|
||||
|
||||
Request:
|
||||
```jsonc
|
||||
{
|
||||
"purchaseRequestId": "…", // or sellerOfferId / template session id, matching current /intents inputs
|
||||
"sellerOfferId": "…",
|
||||
"token": "USDC", // buyer's chosen settlement token
|
||||
"network": "bsc" // chain key (must be in the seller allowlist)
|
||||
}
|
||||
```
|
||||
Success `200`:
|
||||
```jsonc
|
||||
{
|
||||
"quote": {
|
||||
"quoteId": "q_…",
|
||||
"pricingCurrency": "IRR", // the invoice/obligation currency
|
||||
"offerAmount": "1500000.00", // decimal STRING, in pricingCurrency
|
||||
"invoiceUsd": "35.50", // decimal string
|
||||
"token": "USDC",
|
||||
"chainId": 56,
|
||||
"tokenPriceUsd": "0.971", // depeg oracle price (decimal string)
|
||||
"fxRate": "0.0000236", // pricingCurrency → USD (decimal string)
|
||||
"rawSettleAmount": "36.56", // exact depeg-protected token amount (decimal string)
|
||||
"settleAmount": "37.00", // amount the buyer pays (after snap-up rounding) — decimal string
|
||||
"settleAmountOnChain": "37000000000000000000", // base units (per-chain decimals) — string
|
||||
"depegAdjustmentBps": 299, // +2.99% vs par (number)
|
||||
"roundingBps": 120, // rounding delta vs rawSettle (number, >=0)
|
||||
"fxSource": "offchain_fx",
|
||||
"depegSource": "chainlink",
|
||||
"fetchedAt": "2026-05-31T12:00:00.000Z",
|
||||
"expiresAt": "2026-05-31T12:01:30.000Z" // QUOTE_VALIDITY_S after fetchedAt
|
||||
}
|
||||
}
|
||||
```
|
||||
Error `4xx` (typed `code`):
|
||||
```jsonc
|
||||
{ "error": { "code": "DEPEG_LIMIT_EXCEEDED", "message": "…", "details": { "tokenPriceUsd": "0.93", "capBps": 500 } } }
|
||||
```
|
||||
|
||||
### 2.2 Intent creation (committed route, now quote-capable)
|
||||
`POST /api/payment/request-network/intents` and the amn.scanner path (`src/lib/axios.ts` → `endpoints.payments.requestNetwork.intents`). Current committed behavior:
|
||||
- The server **recomputes/validates the amount** from the offer + a fresh quote; any client `amount` is ignored.
|
||||
- The response can include `quote` fields when `ORACLE_QUOTING_ENABLED=true`.
|
||||
- Quote persistence is best-effort PG + Mongo mirror. If the PG payment row is not present yet, the quote is mirrored to Mongo and a `pg_dualwrite_gaps` row is recorded.
|
||||
- Binding a separately previewed `quoteId` is future work, because the read-only preview endpoint is not committed yet.
|
||||
|
||||
### 2.3 Error codes the UI must handle
|
||||
| `code` | Meaning | UI behavior |
|
||||
|---|---|---|
|
||||
| `DEPEG_LIMIT_EXCEEDED` | Stablecoin off peg beyond hard cap | Block pay; show warning banner; offer "try another token/chain" or "retry later" |
|
||||
| `ORACLE_UNAVAILABLE` | No provider could price the pair | Block pay; "pricing temporarily unavailable, retry"; auto-retry w/ backoff |
|
||||
| `ORACLE_STALE` | Rate too old | Same as unavailable; auto-refetch |
|
||||
| `QUOTE_EXPIRED` / `QUOTE_MOVED` | Locked quote no longer valid at submit | Re-quote; if amount moved > threshold, require explicit re-confirm |
|
||||
| `PAYMENT_CHOICE_NOT_ALLOWED` | token/chain not in seller allowlist | Disable that option in the selector |
|
||||
|
||||
## 3. TypeScript types (add to `src/types/payment.ts`)
|
||||
|
||||
```ts
|
||||
export type QuoteErrorCode =
|
||||
| 'DEPEG_LIMIT_EXCEEDED' | 'ORACLE_UNAVAILABLE' | 'ORACLE_STALE'
|
||||
| 'QUOTE_EXPIRED' | 'QUOTE_MOVED' | 'PAYMENT_CHOICE_NOT_ALLOWED';
|
||||
|
||||
export interface IPaymentQuote {
|
||||
quoteId: string;
|
||||
pricingCurrency: string; // 'IRR' | 'TRY' | 'USD' | 'EUR' | 'USDT' | 'USDC'
|
||||
offerAmount: string; // decimal string, in pricingCurrency
|
||||
invoiceUsd: string;
|
||||
token: 'USDC' | 'USDT';
|
||||
chainId: number;
|
||||
tokenPriceUsd: string;
|
||||
fxRate: string;
|
||||
rawSettleAmount: string;
|
||||
settleAmount: string; // what the buyer pays (display this)
|
||||
settleAmountOnChain: string; // base units
|
||||
depegAdjustmentBps: number; // + = buyer pays more (depeg), - = buyer pays less (premium)
|
||||
roundingBps: number;
|
||||
fxSource: string;
|
||||
depegSource: string;
|
||||
fetchedAt: string; // ISO
|
||||
expiresAt: string; // ISO
|
||||
}
|
||||
|
||||
export type QuoteStatus =
|
||||
| 'idle' | 'loading' | 'quoted' | 'expired' | 'requoting' | 'blocked' | 'unavailable';
|
||||
```
|
||||
|
||||
> **All money/rate fields are decimal strings.** Never `parseFloat` them for math — only for display formatting. If you must do arithmetic (you shouldn't on the client), use a decimal lib; the authoritative amount is always `settleAmount` from the server.
|
||||
|
||||
## 4. Data layer (SWR + axios)
|
||||
|
||||
Add to the `endpoints` object in `src/lib/axios.ts`:
|
||||
```ts
|
||||
payments: {
|
||||
// …existing…
|
||||
quote: '/payment/quote',
|
||||
}
|
||||
```
|
||||
|
||||
Hook (`src/actions/payment-quote.ts`), mirroring the existing SWR convention:
|
||||
```ts
|
||||
import useSWR from 'swr';
|
||||
import { axiosInstance, endpoints, fetcher } from 'src/lib/axios';
|
||||
import type { IPaymentQuote } from 'src/types/payment';
|
||||
|
||||
export function usePaymentQuote(args: { purchaseRequestId?: string; sellerOfferId?: string; token?: string; network?: string } | null) {
|
||||
// POST-based quote: use a tuple key + a custom fetcher (SWR mutate on (token,network) change)
|
||||
const key = args && args.token && args.network ? ['payment-quote', args] as const : null;
|
||||
const { data, error, isLoading, mutate } = useSWR(key, async ([, body]) => {
|
||||
const res = await axiosInstance.post(endpoints.payments.quote, body);
|
||||
return res.data.quote as IPaymentQuote;
|
||||
}, {
|
||||
refreshInterval: 0, // we drive refresh off the expiry timer, not polling
|
||||
revalidateOnFocus: false,
|
||||
shouldRetryOnError: false, // typed errors are handled by the caller, not retried blindly
|
||||
});
|
||||
return { quote: data, error, isLoading, refetch: mutate };
|
||||
}
|
||||
```
|
||||
- **Refetch on:** (token, chain) change, manual "refresh rate", and on expiry (timer hits 0 → `refetch()` → `requoting`).
|
||||
- **Map axios errors** to `QuoteErrorCode` via `err.response?.data?.error?.code`.
|
||||
|
||||
## 5. Component tree (MUI)
|
||||
|
||||
```
|
||||
<OracleQuotePanel> // new — src/sections/request-template/checkout-oracle-quote.tsx
|
||||
├─ <TokenChainSelector/> // reuse/extend ProviderPayment's Select; disable disallowed (allowlist)
|
||||
├─ <QuoteSummaryCard quote=… status=…>// the headline: "You pay 37.00 USDC ≈ ﷼1,500,000"
|
||||
│ ├─ dual amount (token primary, pricingCurrency secondary)
|
||||
│ ├─ <DepegBadge bps=…/> // "+2.99% depeg protection" (Chip)
|
||||
│ ├─ rounding note ("rounded up 0.44 to 37.00")
|
||||
│ └─ <QuoteExpiryTimer expiresAt=… onExpire=refetch/>
|
||||
├─ <DepegWarningBanner code=…/> // Alert when blocked/unavailable
|
||||
└─ used by the pay button: disabled unless status==='quoted'
|
||||
```
|
||||
|
||||
### 5.1 `QuoteSummaryCard` (MUI `Card`)
|
||||
Props: `{ quote: IPaymentQuote; status: QuoteStatus }`.
|
||||
- Primary line (`Typography variant="h5"`, `dir="ltr"`): **`{settleAmount} {token}`**.
|
||||
- Secondary (`body2`, muted): **`≈ {offerAmount} {pricingCurrency}`** (formatted, RTL-aware — see §6).
|
||||
- Row of `Chip`s: depeg badge, network, "rate locked · {countdown}".
|
||||
- If `status==='loading'|'requoting'` → MUI `Skeleton` rows.
|
||||
- If `roundingBps>0` → small caption: "Rounded up to a round number (within 3%)".
|
||||
|
||||
### 5.2 `DepegBadge` (MUI `Chip`)
|
||||
- `depegAdjustmentBps > 0` → color `warning`, label "Depeg protection +{bps/100}%", tooltip "Your stablecoin trades below \$1; you pay slightly more so the seller receives the full {pricingCurrency} value."
|
||||
- `depegAdjustmentBps < 0` → color `success`, label "Premium −{|bps|/100}%", tooltip "Your stablecoin trades above \$1; you pay slightly less."
|
||||
- `=== 0` → hide or neutral "At peg".
|
||||
|
||||
### 5.3 `QuoteExpiryTimer`
|
||||
- Counts down to `expiresAt`. At `T-15s` turn amber; at `0` call `onExpire()` (sets `requoting`, refetches). Show "Refresh rate" button always.
|
||||
|
||||
### 5.4 `DepegWarningBanner` (MUI `Alert`)
|
||||
- `DEPEG_LIMIT_EXCEEDED` → severity `error`, "We can't price {token} safely right now (it's {x}% off peg). Try another token/chain or retry shortly." + retry button.
|
||||
- `ORACLE_UNAVAILABLE`/`ORACLE_STALE` → severity `warning` + auto-retry spinner.
|
||||
|
||||
## 6. Formatting, RTL & i18n
|
||||
|
||||
- **Amounts are LTR even in RTL layouts** — wrap every number/token/hash in `dir="ltr"` (project convention, see explore §4 + CLAUDE.md). The card layout flips with the theme (`stylis-plugin-rtl`), but the numerals don't.
|
||||
- Extend `src/utils/currencyUtils.ts`:
|
||||
```ts
|
||||
// fiat display: IRR/TRY have no decimals typically; group thousands per locale
|
||||
export function formatFiat(amount: string, currency: string, locale?: string): string;
|
||||
// dual display helper used by the card
|
||||
export function formatPayLine(quote: IPaymentQuote): { primary: string; secondary: string };
|
||||
```
|
||||
- IRR: symbol ﷼, 0 decimals, fa-IR grouping. TRY: ₺, 2 decimals. USDT/USDC: 2 decimals.
|
||||
- **i18n keys** (add to `src/locales/langs/{en,fa}/messages.json`), e.g.:
|
||||
```
|
||||
checkout.quote.youPay = "You pay {{amount}} {{token}}"
|
||||
checkout.quote.approx = "≈ {{amount}} {{currency}}"
|
||||
checkout.quote.depegProtection = "Depeg protection +{{pct}}%"
|
||||
checkout.quote.premium = "Premium −{{pct}}%"
|
||||
checkout.quote.roundedUp = "Rounded up to {{amount}} {{token}}"
|
||||
checkout.quote.expiresIn = "Rate locked · {{seconds}}s"
|
||||
checkout.quote.refresh = "Refresh rate"
|
||||
checkout.quote.err.depegCap = "Can't price {{token}} safely ({{pct}}% off peg). Try another token or retry."
|
||||
checkout.quote.err.unavailable = "Pricing temporarily unavailable. Retrying…"
|
||||
checkout.quote.err.expired = "Rate updated — please review the new amount."
|
||||
```
|
||||
Persian (`fa`) is the default locale — provide fa strings too.
|
||||
|
||||
## 7. State machine
|
||||
|
||||
```
|
||||
idle ──(token+chain chosen)──► loading ──ok──► quoted ──(timer 0)──► requoting ──ok──► quoted
|
||||
│ │ │
|
||||
└─err─► unavailable/blocked └─err─► unavailable/blocked
|
||||
quoted ──(buyer confirms)──► [POST intents] ──QUOTE_EXPIRED/MOVED──► requoting (then re-confirm if moved > 50 bps)
|
||||
└─ok──► existing payment-pending flow (Socket.io)
|
||||
```
|
||||
- **Pay button** enabled **only** in `quoted`. In `blocked`/`unavailable` it's disabled with the banner explaining why.
|
||||
- After intent creation succeeds, hand off to the **existing** Socket.io payment-status flow (`request-template-checkout-payment.tsx:457-736`) — unchanged.
|
||||
|
||||
## 8. Wireframe (quoted, depeg case)
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ Pay with: [ USDC ▾ ] on [ BSC ▾ ] │
|
||||
├─────────────────────────────────────────────┤
|
||||
│ You pay │
|
||||
│ 37.00 USDC ← h5, dir=ltr │
|
||||
│ ≈ ﷼1,500,000 ← muted, RTL-aware │
|
||||
│ │
|
||||
│ [ +2.99% depeg protection ] [ BSC ] │
|
||||
│ Rounded up to a round number (within 3%) │
|
||||
│ Rate locked · 01:18 [ Refresh rate ] │
|
||||
├─────────────────────────────────────────────┤
|
||||
│ [ Pay 37.00 USDC ] │
|
||||
└─────────────────────────────────────────────┘
|
||||
|
||||
Blocked (DEPEG_LIMIT_EXCEEDED):
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ ⚠ Can't price USDC safely (7% off peg). │
|
||||
│ Try another token/chain or retry shortly. │
|
||||
│ [ Retry ] [ Change token ] │
|
||||
└─────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
## 9. Edge cases
|
||||
|
||||
- **Token/chain not allowed** → disable the option (`PAYMENT_CHOICE_NOT_ALLOWED`), don't even quote.
|
||||
- **Quote expires while buyer idles** → auto-`requoting`; if `settleAmount` moves > 50 bps, surface "Rate updated — review new amount" before re-enabling pay.
|
||||
- **Network flip mid-quote** → cancel in-flight quote (SWR key change handles it), show skeleton.
|
||||
- **Premium (token > \$1)** → show green "Premium −x%", buyer pays less; never below the obligation.
|
||||
- **Decimal precision** → display rounds for humans, but submit/track uses the server `settleAmount`/`settleAmountOnChain` strings verbatim.
|
||||
- **Slow oracle** → skeleton + "fetching live rate"; don't show a stale amount.
|
||||
|
||||
## 10. Acceptance criteria
|
||||
|
||||
1. Changing token or chain refetches a quote; pay is disabled until `quoted`.
|
||||
2. Displayed amount always equals server `settleAmount`; the client never computes or sends an amount.
|
||||
3. Depeg up shows a warning-colored badge and a higher token amount; premium shows a success badge and a lower amount; seller is never shorted.
|
||||
4. Beyond the hard cap, pay is blocked with a clear banner (no silent overcharge).
|
||||
5. Expiry countdown works; on expiry it re-quotes; a > 50 bps move forces re-confirm.
|
||||
6. Amounts render `dir="ltr"` inside the RTL (fa) layout; fa + en strings present.
|
||||
7. Successful confirm transitions into the existing Socket.io payment-status UI unchanged.
|
||||
|
||||
## 11. Backend dependencies to confirm (with backend `integrate-main-into-development@3a50dc4`)
|
||||
|
||||
- [ ] `POST /payment/quote` preview endpoint exists and returns the shape in §2.1 (add it if the build only wired the `/intents` seam).
|
||||
- [ ] Intent route accepts/validates `quoteId` and returns `QUOTE_EXPIRED`/`QUOTE_MOVED`.
|
||||
- [ ] Typed error `code`s in §2.3 are emitted.
|
||||
- [ ] `TRY` (and any other) pricing currencies enabled.
|
||||
- [ ] Final field names match §3 (this doc will be reconciled to the committed code).
|
||||
```
|
||||
@@ -34,9 +34,14 @@ Next.js auto-loads `.env`, `.env.local`, `.env.development`, `.env.production` i
|
||||
|------|------|----------|---------|---------|---------|
|
||||
| `MONGODB_URI` | backend | ✅ | — | `mongodb://mongodb:27017` | Mongo connection string (no auth in dev) |
|
||||
| `DB_NAME` | backend | ✅ | — | `marketplace` | Database name appended to the URI |
|
||||
| `PG_URL` | backend | conditional | — | `postgres://amanat:...@postgres:5432/amanat_dev` | Drizzle runtime DSN. Required before importing PG-backed code such as `quoteRepo`; does not cut over app domains by itself. |
|
||||
| `MIGRATION_PG_URL` | backend | migration only | — | `postgres://amanat:...@postgres:5432/amanat_dev` | DSN used by backfill/migration scripts. Guarded by non-prod host allowlist. |
|
||||
|
||||
In `docker-compose.production.yml` the Mongo service is `mongodb` and is reachable as `mongodb://mongodb:27017` from the backend container.
|
||||
|
||||
> [!warning] Postgres cutover flags
|
||||
> `REPO_*` flags exist in the backend repository factory, but broad services still call Mongoose models directly on `integrate-main-into-development@3a50dc4`. Do not assume setting `REPO_DEFAULT=pg` or `REPO_PAYMENT=pg` fully moves live traffic to Postgres without service wiring and verification. See [[Postgres Runtime Cutover Status]].
|
||||
|
||||
---
|
||||
|
||||
## Cache / Redis
|
||||
@@ -128,6 +133,18 @@ Request Network is the current primary payment provider. See [[PRD - Request Net
|
||||
|
||||
---
|
||||
|
||||
## Repository Mode Flags (Migration Layer)
|
||||
|
||||
| Name | Repo | Required | Default | Example | Purpose |
|
||||
|------|------|----------|---------|---------|---------|
|
||||
| `REPO_DEFAULT` | backend | optional | `mongo` | `dual` | Fallback repository mode for domains that do not set their own flag. Current broad runtime services are not yet wired through the factory. |
|
||||
| `REPO_USER` | backend | optional | `REPO_DEFAULT` or `mongo` | `dual` | Intended user/auth repository mode. Requires service wiring before it affects normal requests. |
|
||||
| `REPO_PAYMENT` | backend | optional | `REPO_DEFAULT` or `mongo` | `dual` | Intended payment/ledger repository mode. Current payment APIs still call Mongoose directly. |
|
||||
| `REPO_POINTS` | backend | optional | `REPO_DEFAULT` or `mongo` | `dual` | Intended points/referral repository mode. Current points service still calls Mongoose directly. |
|
||||
| `REPO_MARKETPLACE` | backend | optional | `REPO_DEFAULT` or `mongo` | `dual` | Intended purchase request / seller offer repository mode. Current marketplace services still call Mongoose directly. |
|
||||
|
||||
---
|
||||
|
||||
## Payments — Oracle Quoting / Depeg Protection
|
||||
|
||||
| Name | Repo | Required | Default | Example | Purpose |
|
||||
|
||||
@@ -60,11 +60,11 @@ git clone ssh://git@git.manko.yoga:222/nick/backend.git
|
||||
git clone ssh://git@git.manko.yoga:222/nick/frontend.git
|
||||
```
|
||||
|
||||
Switch each repo to the `development` branch:
|
||||
Switch each repo to the active integration branch for the stack you are testing. As of 2026-05-31, the dev stack work is on `integrate-main-into-development`:
|
||||
|
||||
```bash
|
||||
cd ~/code/backend && git checkout development
|
||||
cd ~/code/frontend && git checkout development
|
||||
cd ~/code/backend && git checkout integrate-main-into-development
|
||||
cd ~/code/frontend && git checkout integrate-main-into-development
|
||||
```
|
||||
|
||||
> [!warning] `main`/`master` is the production branch and is consumed by the Watchtower auto-update flow. Never push WIP commits there. See [[Git Workflow]].
|
||||
|
||||
@@ -5,7 +5,7 @@ tags: [operations]
|
||||
|
||||
# Database Operations
|
||||
|
||||
Day-to-day operations for the two stateful services: **MongoDB 8.2** (primary data store) and **Redis 8** (cache, rate-limit counters, ephemeral session data).
|
||||
Day-to-day operations for stateful services: **MongoDB 8.x** (primary runtime data store), **PostgreSQL 18** (migration target and conditional oracle quote store), and **Redis 8** (cache, rate-limit counters, ephemeral session data).
|
||||
|
||||
For schema details see [[Data Models]]. For backup procedures and disaster recovery see [[Backup & Recovery]].
|
||||
|
||||
@@ -177,9 +177,79 @@ Optional auto-seed on startup: set `AUTO_SEED_ON_START=true` in `.env`. The boot
|
||||
|
||||
---
|
||||
|
||||
## 2. Redis
|
||||
## 2. PostgreSQL 18
|
||||
|
||||
### 2.1 Connection
|
||||
### 2.1 Runtime role
|
||||
|
||||
Postgres is present in the current dev/integration stack, but MongoDB remains the primary runtime store. Use Postgres for:
|
||||
|
||||
- Drizzle migrations and schema verification.
|
||||
- Mongo → Postgres backfill and reconciliation work.
|
||||
- `payment_quotes` when `ORACLE_QUOTING_ENABLED=true` and a PG parent payment row exists.
|
||||
|
||||
Do **not** treat Postgres as the authoritative app database until the relevant domain has been wired through repository interfaces, backfilled, shadow-read, and cut over. See [[Postgres Runtime Cutover Status]].
|
||||
|
||||
### 2.2 Docker volume layout for Postgres 18
|
||||
|
||||
Postgres 18 Docker images expect the mount at `/var/lib/postgresql`, not directly at `/var/lib/postgresql/data`, because the image stores data under a major-version-specific directory such as `/var/lib/postgresql/18/docker`.
|
||||
|
||||
```yaml
|
||||
postgres:
|
||||
image: postgres:18-alpine
|
||||
environment:
|
||||
POSTGRES_DB: amanat_dev
|
||||
POSTGRES_USER: amanat
|
||||
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
|
||||
volumes:
|
||||
- /var/data/escrowDev/postgres_data:/var/lib/postgresql
|
||||
```
|
||||
|
||||
For a disposable dev reset:
|
||||
|
||||
```bash
|
||||
docker rm -f amanat-postgres 2>/dev/null || true
|
||||
rm -rf /var/data/escrowDev/postgres_data
|
||||
mkdir -p /var/data/escrowDev/postgres_data
|
||||
```
|
||||
|
||||
### 2.3 Apply migrations
|
||||
|
||||
Run migrations only after the database is healthy and the DSN points at the intended non-production target:
|
||||
|
||||
```bash
|
||||
PG_URL=postgres://amanat:...@postgres:5432/amanat_dev npx drizzle-kit migrate
|
||||
```
|
||||
|
||||
The backend image contains migrations through `0008`. Application startup does not apply them automatically.
|
||||
|
||||
### 2.4 Backfill and verification
|
||||
|
||||
Backfills use `MIGRATION_PG_URL`, not `PG_URL`, and the scripts enforce a host allowlist. Run dry-run and verification before any dual-write/PG read flip:
|
||||
|
||||
```bash
|
||||
MIGRATION_MONGO_URL=mongodb://mongodb:27017/marketplace \
|
||||
MIGRATION_PG_URL=postgres://amanat:...@postgres:5432/amanat_dev \
|
||||
node dist/db/backfill/run-backfill.js --dry-run
|
||||
```
|
||||
|
||||
Verify row counts/checksums and inspect `pg_dualwrite_gaps` before enabling any cutover flag.
|
||||
|
||||
### 2.5 Backup
|
||||
|
||||
For dev/staging:
|
||||
|
||||
```bash
|
||||
docker exec amanat-postgres pg_dump -U amanat -d amanat_dev --format=custom \
|
||||
> backups/amanat_dev_pg_$(date +%F).dump
|
||||
```
|
||||
|
||||
Before production cutover, use managed backups or self-hosted WAL archiving/PITR. A plain dev bind mount is not a production backup strategy.
|
||||
|
||||
---
|
||||
|
||||
## 3. Redis
|
||||
|
||||
### 3.1 Connection
|
||||
|
||||
Dev: `redis://redis:6379` (no password).
|
||||
Prod: `redis://:<REDIS_PASSWORD>@redis:6379`. The compose command line is `redis-server --requirepass "$REDIS_PASSWORD"`.
|
||||
@@ -193,7 +263,7 @@ docker exec -it nickapp-redis redis-cli -a "$REDIS_PASSWORD"
|
||||
> KEYS * # prod-unsafe on large datasets, use SCAN
|
||||
```
|
||||
|
||||
### 2.2 What we store
|
||||
### 3.2 What we store
|
||||
|
||||
- **Rate-limit counters** for `express-rate-limit`
|
||||
- **Session data** for refresh-token tracking and revocation lists
|
||||
@@ -203,7 +273,7 @@ docker exec -it nickapp-redis redis-cli -a "$REDIS_PASSWORD"
|
||||
|
||||
Key prefixes follow `<service>:<entity>:<id>`. E.g. `payment:idem:<requestId>`, `auth:refresh:<userId>`.
|
||||
|
||||
### 2.3 Persistence
|
||||
### 3.3 Persistence
|
||||
|
||||
Redis 8 defaults to **RDB snapshots** + optional **AOF**. Our compose uses the default config:
|
||||
|
||||
@@ -220,7 +290,7 @@ redis:
|
||||
|
||||
`appendfsync everysec` is the common compromise: at most 1 second of writes lost on crash, with negligible perf impact.
|
||||
|
||||
### 2.4 Eviction policy
|
||||
### 3.4 Eviction policy
|
||||
|
||||
Default is `noeviction` — Redis refuses writes when memory is full. For our use (caches that can be regenerated), set:
|
||||
|
||||
@@ -233,7 +303,7 @@ docker exec nickapp-redis redis-cli -a "$REDIS_PASSWORD" \
|
||||
|
||||
Persist by adding to a custom `redis.conf` mounted at `/usr/local/etc/redis/redis.conf` (then change the compose `command:` to `["redis-server","/usr/local/etc/redis/redis.conf","--requirepass",...]`).
|
||||
|
||||
### 2.5 Backup
|
||||
### 3.5 Backup
|
||||
|
||||
Redis backups are usually unnecessary (the data is regeneratable) but still cheap:
|
||||
|
||||
@@ -245,7 +315,7 @@ docker cp nickapp-redis:/data/dump.rdb ./backups/redis-$(date +%F).rdb
|
||||
|
||||
`BGSAVE` is non-blocking (forks). For AOF, copy `/data/appendonly.aof` too.
|
||||
|
||||
### 2.6 Cache flush
|
||||
### 3.6 Cache flush
|
||||
|
||||
When deploying breaking changes to cached schemas:
|
||||
|
||||
@@ -261,7 +331,7 @@ docker exec nickapp-redis redis-cli -a "$REDIS_PASSWORD" \
|
||||
|
||||
> [!warning] `FLUSHALL` will sign out every user with an active refresh token and reset every rate-limit counter. Avoid in production unless that is what you want.
|
||||
|
||||
### 2.7 Monitoring
|
||||
### 3.7 Monitoring
|
||||
|
||||
```bash
|
||||
docker exec nickapp-redis redis-cli -a "$REDIS_PASSWORD" INFO stats
|
||||
@@ -273,7 +343,7 @@ Watch `evicted_keys`, `keyspace_misses`, `rejected_connections` — see [[Monito
|
||||
|
||||
---
|
||||
|
||||
## 3. Maintenance windows
|
||||
## 4. Maintenance windows
|
||||
|
||||
For both DBs, schedule a window when:
|
||||
|
||||
@@ -293,7 +363,7 @@ Suggested checklist:
|
||||
|
||||
---
|
||||
|
||||
## 4. Cross-links
|
||||
## 5. Cross-links
|
||||
|
||||
- [[Backup & Recovery]] — formal backup/restore procedures, RTO/RPO targets, offsite storage.
|
||||
- [[Monitoring]] — what metrics to watch (slow queries, evictions, replication lag).
|
||||
|
||||
@@ -11,6 +11,30 @@ entries on top. Maintained by agents per the rule in `../AGENTS.md`.
|
||||
|
||||
---
|
||||
|
||||
### 2026-05-31 — backend@cab0719, frontend@ec2f765 — align request budget validation after Postgres migration
|
||||
|
||||
**Commits:** backend `cab0719`, frontend `ec2f765` (backend `2.6.80`, frontend `2.7.20`)
|
||||
**Touched:**
|
||||
- Backend: `src/shared/constants/marketplace.ts`, `src/models/PurchaseRequest.ts`, `src/models/RequestTemplate.ts`, `src/services/marketplace/requestTemplateRoutes.ts`, `src/services/marketplace/PurchaseRequestService.ts`, `src/db/schema/purchaseRequest.ts`, `src/db/repositories/drizzle/DrizzleMarketplaceRepo.ts`, `__tests__/marketplace-request-budget-validation.test.ts`, `scripts/smoke/marketplace-request-budget.sh`
|
||||
- Frontend: `package.json`, `package-lock.json` version bump only.
|
||||
**Why:** Product/template creation could return `400` when the UI sent `urgency: "urgent"`, and template-to-purchase conversion could later fail when a template budget used `USD` / `EUR` / `IRR` while `PurchaseRequest` only accepted `USDT` / `USDC`. Runtime Mongoose validation, request-template route validation, and the PG `budget_currency` enum now share `USD`, `EUR`, `IRR`, `USDT`, `USDC`; urgency validation includes `urgent`.
|
||||
**Verification:** Backend `npm test -- --runTestsByPath __tests__/marketplace-request-budget-validation.test.ts`; backend `npm run typecheck`; backend `git diff --check`; frontend `npx tsc --noEmit --ignoreDeprecations 6.0`; smoke helper added at `scripts/smoke/marketplace-request-budget.sh` but not run against dev because no `ACCESS_TOKEN` / `CATEGORY_ID` were available in this session.
|
||||
**Linked docs updated:** [[PurchaseRequest]], [[RequestTemplate]], [[Marketplace API]], [[MongoDB to PostgreSQL Migration Guide]], [[MongoDB to PostgreSQL Migration Plan (Drizzle)]], [[Postgres Runtime Cutover Status]], [[Data Model Overview]], [[Payment]]
|
||||
|
||||
---
|
||||
|
||||
### 2026-05-31 — nick-doc@local — clarify Postgres runtime cutover status
|
||||
|
||||
**Commits:** docs-only sync after backend `3a50dc4` (`integrate-main-into-development`, backend `2.6.79`)
|
||||
**Touched:**
|
||||
- Added [[Postgres Runtime Cutover Status]].
|
||||
- Updated overview, architecture, data-model, payment API, env, database-ops, migration-plan, migration-guide, and oracle checkout docs.
|
||||
**Why:** Correct the overbroad assumption that the promoted Postgres branch means normal runtime traffic is already PG-backed. The code has Postgres infrastructure, migrations, repository classes, and conditional `payment_quotes`, but live services still call Mongoose directly and Mongo remains authoritative until repository wiring/backfill/shadow-read cutover is completed.
|
||||
**Verification:** Code audit via `rg` on backend `origin/integrate-main-into-development@3a50dc4`: no runtime `createRepositories()` / `get*Repo()` use outside `src/db/repositories/factory.ts`; only normal service import of `src/db/client.ts` is `services/payment/priceOracle/quoteRepo.ts`.
|
||||
**Linked docs updated:** [[Postgres Runtime Cutover Status]], [[System Overview]], [[Tech Stack]], [[System Architecture]], [[Backend Architecture]], [[Database Strategy - Mongo vs Postgres Assessment]], [[Data Model Overview]], [[Payment]], [[Payment API]], [[Environment Variables]], [[Database Operations]], [[MongoDB to PostgreSQL Migration Guide]], [[MongoDB to PostgreSQL Migration Plan (Drizzle)]], [[Oracle Pricing & Stablecoin Depeg Protection]], [[Oracle Depeg Checkout — UI Implementation Guide]]
|
||||
|
||||
---
|
||||
|
||||
### 2026-05-31 — backend@3a50dc4 — promote Postgres integration branch with oracle/depeg + gasless backports
|
||||
|
||||
**Commits:** backend `11bfd02` `74d73c5` `1730c4d` `148c803` `8aa4473` `a5e4da2` `3a50dc4` (backend `2.6.76` → `2.6.79`)
|
||||
|
||||
@@ -10,7 +10,7 @@ created: 2026-05-23
|
||||
Complete technical & operational documentation for the **Amn** (a.k.a. "nick app") crypto-escrow marketplace platform. This vault is exhaustive enough to **re-implement the system from scratch** with no access to the source code.
|
||||
|
||||
> [!info]
|
||||
> **Repos:** `git@git.manko.yoga:222/nick/{backend,frontend}.git` · **Branch:** `development` · **Vault generated:** 2026-05-23
|
||||
> **Repos:** `git@git.manko.yoga:222/nick/{backend,frontend}.git` · **Active backend integration branch:** `integrate-main-into-development` · **Current backend baseline:** `2.6.79` at `3a50dc4` · **Vault generated:** 2026-05-23
|
||||
|
||||
---
|
||||
|
||||
@@ -44,7 +44,8 @@ Project context, the cast of characters, and shared vocabulary.
|
||||
How the system is composed at every layer.
|
||||
|
||||
- [[System Architecture]] — end-to-end topology + request lifecycle
|
||||
- [[Backend Architecture]] — Express 5 + Mongoose + Socket.IO module map
|
||||
- [[Backend Architecture]] — Express 5 + Mongoose + Socket.IO module map, plus current Postgres migration layer status
|
||||
- [[Database Strategy - Mongo vs Postgres Assessment]] — current Mongo primary posture and Postgres cutover assessment
|
||||
- [[Frontend Architecture]] — Next.js 16 App Router + provider tree
|
||||
- [[Request Network Integration Constraints]] — current RN integration constraints and rollout gates
|
||||
- [[PRD - Decentralized Custody and Smart-Contract Escrow Roadmap]] — custody decentralization and smart-contract decision roadmap
|
||||
@@ -57,6 +58,7 @@ How the system is composed at every layer.
|
||||
Per-entity Mongoose schemas — fields, relationships, state machines.
|
||||
|
||||
- [[Data Model Overview]] — ER-style map + reading order
|
||||
- [[Postgres Runtime Cutover Status]] — what is actually using Postgres vs still Mongo-backed on `integrate-main-into-development`
|
||||
- Core entities: [[User]] · [[PurchaseRequest]] · [[SellerOffer]] · [[Payment]] · [[Chat]] · [[Notification]] · [[Dispute]]
|
||||
- Marketplace extras: [[RequestTemplate]] · [[ShopSettings]] · [[Category]] · [[Review]]
|
||||
- User extras: [[Address]] · [[TempVerification]]
|
||||
@@ -236,4 +238,3 @@ The vault is the project's internal documentation. Treat all credentials, addres
|
||||
Welcome to the codebase. If anything here is unclear, the source is in the [[Backend Architecture]] / [[Frontend Architecture]] cited files — fix the docs as you go.
|
||||
|
||||
<!-- git sync test: 2026-05-23 -->
|
||||
|
||||
|
||||
Reference in New Issue
Block a user