Compare commits
3 Commits
a283f0ef21
...
4b1d8ea36d
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
4b1d8ea36d | ||
|
|
d072238fe8 | ||
|
|
6f13903644 |
@@ -2,14 +2,15 @@
|
||||
title: Backend Architecture
|
||||
tags: [architecture, backend]
|
||||
created: 2026-05-23
|
||||
updated: 2026-06-03
|
||||
---
|
||||
|
||||
# Backend Architecture
|
||||
|
||||
Module-level architecture of the Express 5 + TypeScript backend. MongoDB/Mongoose is still the primary runtime persistence layer; the `integrate-main-into-development` backend also contains the Drizzle/Postgres migration layer.
|
||||
Module-level architecture of the Express 5 + TypeScript backend. The system is mid-migration: MongoDB/Mongoose remains the authoritative read store for most domains, with PostgreSQL (Drizzle ORM) running in dual-write mode across 17 landed migrations. The repository factory pattern (`src/db/repositories/factory.ts`) controls which backend each domain reads and writes through env flags.
|
||||
|
||||
> [!info]
|
||||
> Repo: `git@git.manko.yoga:222/nick/backend.git` · Active integration branch: `integrate-main-into-development` · Current baseline: backend `2.6.79` at `3a50dc4`
|
||||
> Repo: `git@git.manko.yoga:222/nick/backend.git` · Current version: `2.8.56` · 17 Drizzle migrations landed · Dual-write active across all major domains
|
||||
|
||||
---
|
||||
|
||||
@@ -24,10 +25,13 @@ backend/src/
|
||||
│ ├── database/ # Mongoose connection, retries, graceful shutdown
|
||||
│ └── socket/socketService.ts # Socket.IO server, rooms, emit helpers
|
||||
├── models/ # Mongoose models — see 02 - Data Models/
|
||||
├── db/ # Drizzle/Postgres migration layer: schemas, migrations, repos, backfill, verify
|
||||
├── db/ # Drizzle/Postgres layer: schemas, migrations, repos, backfill, verify
|
||||
│ ├── schema/ # Per-table Drizzle schema files + index.ts barrel
|
||||
│ ├── migrations/ # 17 numbered SQL migration files
|
||||
│ └── repositories/ # Drizzle repos, dual-write wrappers, factory.ts
|
||||
├── routes/ # Express Router definitions (mounted in app.ts)
|
||||
├── scripts/ # CLI utilities (seed:users, seed:categories, ...)
|
||||
├── seeds/ # Seed data fixtures
|
||||
├── seeds/ # Seed data fixtures (Postgres-capable as of v2.8.47)
|
||||
├── services/
|
||||
│ ├── ai/ # OpenAI integration (descriptions, moderation)
|
||||
│ ├── auth/ # JWT, OAuth, Passkey, password reset
|
||||
@@ -61,8 +65,8 @@ backend/src/
|
||||
└── utils/ # Pure utility fns (logger, currencyUtils, etc.)
|
||||
```
|
||||
|
||||
> [!warning] Postgres is not the default runtime store yet
|
||||
> `src/db/repositories/factory.ts` can select `mongo`, `dual`, or `pg` implementations for user, payment, points, and marketplace domains, but the broad service layer still imports Mongoose models directly. A code scan on 2026-05-31 found no runtime calls to `createRepositories()` / `getPaymentRepo()` / `getMarketplaceRepo()` outside the factory itself. See [[Postgres Runtime Cutover Status]] before assuming a `REPO_*` flag changes live behavior.
|
||||
> [!warning] Reads still go to Mongo for all dual-write domains
|
||||
> Even when `REPO_*=dual`, Mongo is the authoritative read source. The dual-write seam exists and is exercised in production, but no domain has been cut over to PG reads yet. See [[Postgres Runtime Cutover Status]] before assuming a `REPO_*` flag changes live read behavior.
|
||||
|
||||
> [!tip]
|
||||
> Service folders are self-contained: each typically has `<feature>Service.ts`, `<feature>Controller.ts`, `<feature>Routes.ts`, `<feature>Validation.ts`. This makes each service movable to a microservice later with minimal coupling.
|
||||
@@ -75,22 +79,25 @@ The bootstrap is intentionally linear and easy to audit. Execution order:
|
||||
|
||||
1. **Imports & env load** — `dotenv` (if used), then `import { config } from './shared/config'`.
|
||||
2. **Express app construction** — `const app = express();`
|
||||
3. **Trust proxy** — `app.set('trust proxy', config.trustProxy)` so X-Forwarded-For works behind Nginx.
|
||||
3. **Trust proxy** — `app.set('trust proxy', config.trustProxy)` so X-Forwarded-For works behind Traefik.
|
||||
4. **Security headers** — `app.use(helmet({ ... }))`.
|
||||
5. **CORS** — `cors({ origin: config.frontendUrl, credentials: true, methods: [...] })`.
|
||||
6. **Body parsers** — `express.json({ limit: '10mb' })`, `express.urlencoded({ extended: true })`.
|
||||
7. **Static uploads** — `app.use('/uploads', express.static(uploadDir))`.
|
||||
8. **Health endpoint** — `GET /health` for Docker healthcheck and external monitors.
|
||||
8. **Health endpoint** — `GET /health` for Docker healthcheck and external monitors. Now surfaces active Postgres store modes.
|
||||
9. **Route mounting** — every `/api/*` route registered before the error handler.
|
||||
10. **404 handler** — catches unmatched `/api/*`.
|
||||
11. **Error handler** — central `errorHandler` middleware formats responses via `response-handler.ts`.
|
||||
12. **HTTP server creation** — `const server = http.createServer(app)`.
|
||||
13. **Socket.IO attach** — `initSocket(server, corsOptions)` (see [[Real-time Layer]]).
|
||||
14. **DB connect** — `await connectDatabase()` for MongoDB/Mongoose. Postgres connects lazily only when PG modules are imported (for example oracle quote persistence with `ORACLE_QUOTING_ENABLED=true`) and requires `PG_URL`.
|
||||
14. **DB connect** — controlled by `MONGO_CONNECT_MODE`:
|
||||
- `always` (default) — connects Mongoose (Mongo) and PostgreSQL (via `PG_URL`) on boot.
|
||||
- `never` — skips Mongo entirely; Postgres is the only persistence layer. Seeds are Postgres-capable in this mode.
|
||||
- `optional` — connects Postgres; Mongo is attempted but failures are non-fatal.
|
||||
15. **Redis connect** — `await connectRedis()`.
|
||||
16. **Listen** — `server.listen(config.port, ...)`.
|
||||
17. **Graceful shutdown** — SIGTERM/SIGINT handlers close server, drain sockets, close Mongoose, close Redis.
|
||||
18. **Optional dev seeding** — when `NODE_ENV === 'development'` and `SEED_USERS !== 'false'`, the bootstrap calls the seed scripts to provision default test users.
|
||||
18. **Optional dev seeding** — when `NODE_ENV === 'development'` and `SEED_USERS !== 'false'`, the bootstrap calls the seed scripts to provision default test users. Seeds are store-aware and run correctly against both Mongo and PG.
|
||||
|
||||
---
|
||||
|
||||
@@ -104,14 +111,14 @@ The bootstrap is intentionally linear and easy to audit. Execution order:
|
||||
| 4 | `morgan` (dev only) | global | HTTP request log to stdout. |
|
||||
| 5 | `requestId` | global | Adds `X-Request-Id` for log correlation. |
|
||||
| 6 | `authMiddleware` | per-route | Verifies JWT, attaches `req.user`. Mounted only on protected routes. |
|
||||
| 7 | `roleGuard('admin'|'seller'|...)` | per-route | RBAC check after auth. |
|
||||
| 7 | `roleGuard('admin'\|'seller'\|'guard'\|...)` | per-route | RBAC check after auth. Roles: `admin`, `buyer`, `seller`, `resolver`, `guard`. |
|
||||
| 8 | `validate(schema)` | per-route | express-validator + zod inputs. |
|
||||
| 9 | `controllerFn` | per-route | Delegates to service layer. |
|
||||
| 10 | `notFound` | tail | Returns 404 envelope for unmatched routes. |
|
||||
| 11 | `errorHandler` | tail | Catches thrown errors, formats response. |
|
||||
|
||||
> [!note]
|
||||
> Rate-limit middleware is **active** as of 2026-05-24: auth 10 req/15 min, payment 30/15 min, AI 20/15 min, global 100/15 min. Request Network and Telegram webhooks are exempt from the global limiter. Counters are in-memory — a Redis adapter is planned for distributed deployments.
|
||||
> Rate-limit middleware is **active**: auth 10 req/15 min, payment 30/15 min, AI 20/15 min, global 100/15 min. `GET /api/payment/:id` is exempt from the payment limiter (polling route). Request Network and Telegram webhooks are exempt from the global limiter. Counters are in-memory — a Redis adapter is planned for distributed deployments.
|
||||
|
||||
---
|
||||
|
||||
@@ -128,7 +135,7 @@ The full route table mounted by `app.ts`:
|
||||
| `/api/marketplace/offers` | `services/marketplace/controllerRoutes.ts` | JWT (seller) | SellerOffer CRUD |
|
||||
| `/api/marketplace/templates` | `services/marketplace/controllerRoutes.ts` | JWT (seller) | RequestTemplate CRUD |
|
||||
| `/api/marketplace/categories` | `services/marketplace/controllerRoutes.ts` | public read | Category list |
|
||||
| `/api/marketplace/shop-settings` | `services/marketplace/shopSettingsController.ts` | JWT (seller) | Shop profile |
|
||||
| `/api/marketplace/shop-settings` | `services/marketplace/shopSettingsController.ts` | JWT (seller) | Shop profile; lookup tolerant of uuid/legacy id formats |
|
||||
| `/api/payment` | `services/payment/paymentControllerRoutes.ts` + `paymentRoutes.ts` | JWT | Payment CRUD, health, export |
|
||||
| `/api/payment/decentralized` | `services/payment/decentralizedPaymentRoutes.ts` | mixed | Legacy/manual Web3 save, verify, receiver |
|
||||
| `/api/payment/request-network` | `services/payment/requestNetwork/requestNetworkRoutes.ts` | mixed + HMAC sig on webhook | Request Network pay-in creation, in-house checkout rehydrate, webhooks |
|
||||
@@ -136,12 +143,12 @@ The full route table mounted by `app.ts`:
|
||||
| `/api/admin/rn/networks` | `services/payment/requestNetwork/networkRegistryRoutes.ts` | JWT (admin) | Supported RN chain/token registry |
|
||||
| `/api/admin/settings/confirmation-thresholds` | `services/admin/confirmationThresholdRoutes.ts` | JWT (admin) | Runtime min-confirmation thresholds |
|
||||
| `/api/admin/payments/awaiting-confirmation` | `services/admin/awaitingConfirmationRoutes.ts` | JWT (admin) | Payments blocked on safety confirmations |
|
||||
| `/api/telegram` | `services/telegram/telegramRoutes.ts` | mixed (some JWT, webhook uses secret-token) | Mini App verify/session, identity link/unlink, bot webhook |
|
||||
| `/api/telegram` | `services/telegram/telegramRoutes.ts` | mixed (some JWT, webhook uses secret-token) | Mini App verify/session, identity link/unlink, bot webhook; notifications delivered via Telegram as of v2.8.56 |
|
||||
| `/api/chat` | `services/chat/chatRoutes.ts` | JWT | Conversations, messages |
|
||||
| `/api/notification` | `services/notification/notificationRoutes.ts` + `notificationControllerRouter` | JWT | List, mark read |
|
||||
| `/api/disputes` | `routes/disputeRoutes.ts` + `services/dispute/disputeRoutes.ts` | JWT | Dispute CRUD plus release-hold helpers |
|
||||
| `/api/blog` | `services/blog/blogRoutes.ts` | mixed | Public reads, admin writes |
|
||||
| `/api/admin/cleanup` | `services/admin/dataCleanupRoutes.ts` | JWT (admin) | Data cleanup operations |
|
||||
| `/api/admin/cleanup` | `services/admin/dataCleanupRoutes.ts` | JWT (admin) | Data cleanup; scoped by provider to avoid wiping RN/multi-seller records |
|
||||
| `/api/points` | `services/points/pointsRoutes.ts` | JWT | Points, levels, referrals |
|
||||
| `/api/ai` | `services/ai/aiRoutes.ts` | JWT | OpenAI-backed helpers |
|
||||
| `/api/files` | `services/file/fileRoutes.ts` | JWT | Multipart upload |
|
||||
@@ -209,10 +216,11 @@ flowchart TB
|
||||
points -.-> notify
|
||||
notify --> socket
|
||||
notify --> email
|
||||
notify --> telegram
|
||||
```
|
||||
|
||||
> [!note]
|
||||
> `socket` and `email` are leaf services — every notification path funnels through them. Mocking these two in tests covers most side-effect verification.
|
||||
> `socket`, `email`, and `telegram` are leaf notification sinks — every notification path funnels through them. Mocking these three in tests covers most side-effect verification. Telegram notification delivery was added in v2.8.56.
|
||||
|
||||
---
|
||||
|
||||
@@ -254,12 +262,26 @@ Full table in [[Environment Variables]]. Critical ones:
|
||||
| Key | Default | Notes |
|
||||
|---|---|---|
|
||||
| `PORT` | `5001` | Listen port |
|
||||
| `MONGODB_URI` | `mongodb://localhost:27017/nickapp` | Includes db name |
|
||||
| `MONGODB_URI` | `mongodb://localhost:27017/nickapp` | Includes db name; not required when `MONGO_CONNECT_MODE=never` |
|
||||
| `MONGO_CONNECT_MODE` | `always` | `always` \| `never` \| `optional` — controls whether Mongoose connects on boot |
|
||||
| `PG_URL` | required for PG | PostgreSQL connection string for Drizzle; required when any `REPO_*=pg\|dual` |
|
||||
| `REDIS_URI` | `redis://localhost:6379` | + `REDIS_PASSWORD` |
|
||||
| `JWT_SECRET` | required | ≥32 chars |
|
||||
| `JWT_EXPIRES_IN` | `7d` | |
|
||||
| `REFRESH_TOKEN_EXPIRES_IN` | `30d` | |
|
||||
| `FRONTEND_URL` | `http://localhost:3000` | CORS origin |
|
||||
| `REPO_DEFAULT` | `mongo` | Global fallback store mode for all domains (`mongo` \| `dual` \| `pg`) |
|
||||
| `REPO_USER` | inherits `REPO_DEFAULT` | Per-domain override for user store |
|
||||
| `REPO_PAYMENT` | inherits `REPO_DEFAULT` | Per-domain override for payment store |
|
||||
| `REPO_POINTS` | inherits `REPO_DEFAULT` | Per-domain override for points store |
|
||||
| `REPO_MARKETPLACE` | inherits `REPO_DEFAULT` | Per-domain override for marketplace store |
|
||||
| `REPO_TREZOR` | inherits `REPO_DEFAULT` | Per-domain override for trezor store |
|
||||
| `REPO_DERIVED_DESTINATION` | inherits `REPO_DEFAULT` | Per-domain override for derived destination store |
|
||||
| `REPO_BLOG` \| `BLOG_STORE` | inherits `REPO_DEFAULT` | Per-domain override for blog store |
|
||||
| `REPO_NOTIFICATION` \| `NOTIFICATION_STORE` | inherits `REPO_DEFAULT` | Per-domain override for notification store |
|
||||
| `REPO_DISPUTE` \| `DISPUTE_STORE` | inherits `REPO_DEFAULT` | Per-domain override for dispute store |
|
||||
| `REPO_CHAT` \| `CHAT_STORE` | inherits `REPO_DEFAULT` | Chat dual-write not implemented; `dual` silently uses Mongo |
|
||||
| `REPO_RELEASE_HOLD` \| `RELEASE_HOLD_STORE` | inherits `REPO_DEFAULT` | Release-hold dual-write not implemented; `dual` silently uses Mongo |
|
||||
| `REQUEST_NETWORK_API_BASE_URL` | `https://api.request.network` | Request Network API |
|
||||
| `REQUEST_NETWORK_API_KEY` | required | Request Network API credential |
|
||||
| `REQUEST_NETWORK_WEBHOOK_SECRET` | required | Webhook HMAC key |
|
||||
@@ -268,16 +290,77 @@ Full table in [[Environment Variables]]. Critical ones:
|
||||
| `DERIVED_DESTINATION_SWEEP_SIGNER` | `build-only` | Target hardware/Safe-backed signer |
|
||||
| `SMTP_*` | required | Nodemailer |
|
||||
| `OPENAI_API_KEY` | required | |
|
||||
| `ORACLE_QUOTING_ENABLED` | `false` | Enables oracle-based depeg-protected payment quotes; requires `PG_URL` |
|
||||
|
||||
---
|
||||
|
||||
## 9. Database & connection management
|
||||
|
||||
- **Mongoose** is the ODM. Connection in `src/infrastructure/database/`.
|
||||
The backend runs a **dual-database architecture** during the Mongo→Postgres migration. Both stores may be active simultaneously; which one serves each domain is controlled by `REPO_*` env flags.
|
||||
|
||||
### MongoDB / Mongoose
|
||||
|
||||
- ODM: Mongoose. Connection in `src/infrastructure/database/`.
|
||||
- Connection options enable retryable writes, exponential backoff on reconnect.
|
||||
- Indexes are defined on each model and auto-created on connect (Mongoose `autoIndex: true` in dev, recommend `false` in prod with explicit migration).
|
||||
- Indexes defined on each model and auto-created on connect (`autoIndex: true` in dev; recommend `false` in prod with explicit migration scripts).
|
||||
- Remains the **authoritative read store** for all dual-write domains until read cutover is explicitly executed per domain.
|
||||
- See [[Data Model Overview]] for the relational map and per-model docs.
|
||||
|
||||
### PostgreSQL / Drizzle
|
||||
|
||||
- ORM: Drizzle. Schemas in `src/db/schema/`, migrations in `src/db/migrations/` (17 migrations landed as of 2026-06-03).
|
||||
- Managed via `drizzle-kit migrate` — never edit migration files manually.
|
||||
- Connects lazily when any PG-capable store is imported, or eagerly on boot when `MONGO_CONNECT_MODE=never`.
|
||||
- Every migrated table carries a `legacy_object_id text` column with a partial-unique index for idempotent backfill upserts.
|
||||
- Money columns use `numeric(38,18)` (except `seller_offers`: `numeric(18,8)`). Blockchain balance columns use `numeric(78,0)` to hold uint256 without overflow.
|
||||
- See [[Drizzle Schema Reference]] for the full per-table breakdown.
|
||||
|
||||
### Repository factory — `src/db/repositories/factory.ts`
|
||||
|
||||
The factory is the single routing layer between service code and the underlying store. It exposes per-domain getters and resolves the mode (`mongo` | `dual` | `pg`) in this order:
|
||||
|
||||
1. Per-domain env flag (e.g. `REPO_PAYMENT`)
|
||||
2. `REPO_DEFAULT` (global staging-wide fallback)
|
||||
3. Hardcoded default: `mongo`
|
||||
|
||||
Unrecognized values silently fall back to `mongo` — intentional safety net against typos on money writes.
|
||||
|
||||
| Domain | Getter | Dual-write | PG-only |
|
||||
|---|---|---|---|
|
||||
| user | `getUserRepo` | Yes (full trio) | Yes |
|
||||
| payment | `getPaymentRepo` | Yes (full trio) | Yes |
|
||||
| points | `getPointsRepo` | Yes (full trio) | Yes |
|
||||
| marketplace | `getMarketplaceRepo` | Yes (full trio) | Yes |
|
||||
| trezor | `getTrezorRepo` | Yes (full trio) | Yes |
|
||||
| derivedDestination | `getDerivedDestinationRepo` | Yes (full trio) | Yes |
|
||||
| blog | `getBlogRepo` | Yes (full trio) | Yes |
|
||||
| notification | `getNotificationRepo` | Yes (full trio) | Yes |
|
||||
| dispute | `getDisputeRepo` | Yes (full trio) | Yes |
|
||||
| releaseHold | `getReleaseHoldRepo` | No — `dual` silently uses Mongo | Yes |
|
||||
| chat | `getChatRepo` | No — `dual` silently uses Mongo | Yes |
|
||||
|
||||
> [!warning] `MONGO_CONNECT_MODE` is not handled by the factory
|
||||
> `MONGO_CONNECT_MODE` is consumed by the Mongoose connection module, not by `factory.ts`. The factory only reads `REPO_*` flags. These two controls are orthogonal: `MONGO_CONNECT_MODE=never` prevents Mongoose from connecting, while `REPO_*=pg` prevents the factory from routing to Mongo. For a full PG-only boot, set **both**.
|
||||
|
||||
### Migration phase status (as of 2026-06-03)
|
||||
|
||||
| Phase | Status |
|
||||
|---|---|
|
||||
| Schema / migrations | Done — 17 migrations landed, all domain tables exist in PG |
|
||||
| Dual-write seam | Done — active for all major domains via factory |
|
||||
| Backfill tooling | Done — backfill + verification harness in `src/db/` |
|
||||
| Reads cutover | Not started — all reads still served from Mongo |
|
||||
| Chat normalization | Blocked — Chat stored as JSONB blobs; normalization required before PG read cutover |
|
||||
| Mongo retirement | Future — blocked on per-domain read cutover completion |
|
||||
|
||||
### Infrastructure / bridge tables (PG-only)
|
||||
|
||||
- **`id_map`** — ObjectId → UUID bridge; every migrated entity upserts here during backfill/dual-write.
|
||||
- **`pg_dualwrite_gaps`** — Append-only reconciliation log for failed PG dual-writes; includes severity, resolver notes, and error stack.
|
||||
- **`payment_quotes`** — Oracle-based depeg-protected quote snapshots (1:1 with payments); PG-only, no Mongo equivalent. Only active when `ORACLE_QUOTING_ENABLED=true`.
|
||||
|
||||
### Redis
|
||||
|
||||
Redis client (in `src/services/redis/`) provides:
|
||||
- Session caching (login attempts, lockout counters)
|
||||
- Rate-limit counters (when middleware is enabled)
|
||||
@@ -323,6 +406,8 @@ Run with `npm run test:all`. CI runs the same. Reach for `npm run test:models`,
|
||||
| `src/shared/utils/response-handler.ts` | Standard response shape |
|
||||
| `src/shared/middleware/auth.ts` | JWT verify + RBAC |
|
||||
| `src/infrastructure/socket/socketService.ts` | All socket plumbing |
|
||||
| `src/db/repositories/factory.ts` | Store routing — which backend each domain uses |
|
||||
| `src/db/schema/index.ts` | Drizzle schema barrel — all 25+ PG tables |
|
||||
| `src/services/payment/requestNetwork/requestNetworkRoutes.ts` | Request Network checkout and webhook route |
|
||||
| `src/services/payment/ledger/fundsLedgerService.ts` | Immutable payment ledger writes |
|
||||
| `src/services/marketplace/PurchaseRequestService.ts` | Core marketplace state machine |
|
||||
@@ -338,5 +423,7 @@ Run with `npm run test:all`. CI runs the same. Reach for `npm run test:models`,
|
||||
- [[Frontend Architecture]] — how the FE talks to this BE
|
||||
- [[Real-time Layer]] — Socket.IO room model
|
||||
- [[Security Architecture]] — JWT, passkeys, webhook HMAC
|
||||
- [[Data Model Overview]] — entity-relationship map
|
||||
- [[Data Model Overview]] — entity-relationship map (Mongoose)
|
||||
- [[Drizzle Schema Reference]] — PostgreSQL table definitions, enums, migration status
|
||||
- [[Postgres Runtime Cutover Status]] — per-domain read cutover tracker
|
||||
- [[Authentication Flow]] · [[Escrow Flow]] · [[Dispute Flow]]
|
||||
|
||||
@@ -2,14 +2,15 @@
|
||||
title: Frontend Architecture
|
||||
tags: [architecture, frontend, nextjs]
|
||||
created: 2026-05-23
|
||||
updated: 2026-06-03
|
||||
---
|
||||
|
||||
# Frontend Architecture
|
||||
|
||||
Module-level architecture of the Next.js 16 (App Router) + TypeScript + MUI v7 frontend. The current integration worktree observed locally is on `integrate-main-into-development`.
|
||||
Module-level architecture of the Next.js 16 (App Router) + TypeScript + MUI v9 frontend. The current integration worktree observed locally is on `integrate-main-into-development`.
|
||||
|
||||
> [!info]
|
||||
> Repo: `git@git.manko.yoga:222/nick/frontend.git` · Active integration branch observed locally: `integrate-main-into-development` · Version: 2.7.19 (`package.json`) · Dev port `3000`, Docker port `8083`.
|
||||
> Repo: `git@git.manko.yoga:222/nick/frontend.git` · Active integration branch observed locally: `integrate-main-into-development` · Version: 2.8.59 (`package.json`) · Dev port `3000`, Docker port `8083`.
|
||||
|
||||
---
|
||||
|
||||
@@ -36,12 +37,17 @@ frontend/src/
|
||||
│ │ ├── post/ # Admin blog editor
|
||||
│ │ ├── shop-settings/ # Seller shop config
|
||||
│ │ └── shops/ # Browse / checkout (dashboard scope)
|
||||
│ ├── telegram/ # Telegram Mini App shell (see §19)
|
||||
│ │ ├── layout.tsx # TMA root — TonConnectUIProvider + minimal providers
|
||||
│ │ ├── shop/ # Seller list + product browsing
|
||||
│ │ ├── cart/ # In-shell cart + checkout handoff
|
||||
│ │ └── account/ # Account tab (dashboard parity)
|
||||
│ ├── error/ # Global error page
|
||||
│ └── not-found.tsx # 404
|
||||
├── sections/ # Page-specific composition modules (one folder per feature)
|
||||
│ └── (chat|payment|request|request-template|dispute|user|points|...)
|
||||
│ └── (chat|payment|request|request-template|dispute|user|points|telegram|...)
|
||||
├── components/ # Reusable UI primitives (hook-form, table, upload, editor, ...)
|
||||
├── layouts/ # Page-template wrappers (auth-centered, auth-split, dashboard, main)
|
||||
├── layouts/ # Page-template wrappers (auth-centered, auth-split, dashboard, main, telegram)
|
||||
├── theme/ # MUI theme creation, palette, typography, overrides
|
||||
├── settings/ # Settings drawer (mode, layout, direction, color, font)
|
||||
├── contexts/ # React Context providers (socket-context)
|
||||
@@ -80,6 +86,8 @@ flowchart TB
|
||||
|
||||
Order matters: theme must wrap query (because mutations show snackbars styled by theme); socket wraps snackbar (so socket-driven notifications can fire snackbars).
|
||||
|
||||
The Telegram Mini App shell (`app/telegram/`) uses its own slimmer layout that replaces the dashboard shell with `TonConnectUIProvider` and skips the settings drawer (see §19).
|
||||
|
||||
---
|
||||
|
||||
## 4. Route layout & guards
|
||||
@@ -92,8 +100,9 @@ Order matters: theme must wrap query (because mutations show snackbars styled by
|
||||
| `dashboard/user/*` | dashboard | + `role: admin` |
|
||||
| `dashboard/post/*` (editor) | dashboard | + `role: admin` |
|
||||
| `dashboard/shop-settings/*` | dashboard | + `role: seller` |
|
||||
| `telegram/*` | `layouts/telegram` (bottom-tab shell) | Telegram `initData` token guard + role check |
|
||||
|
||||
Guards live in `frontend/src/auth/` (HOC + hook). They consult the JWT-derived user context and redirect unauthenticated to `/auth/jwt/sign-in?returnTo=...`.
|
||||
Guards live in `frontend/src/auth/` (HOC + hook). They consult the JWT-derived user context and redirect unauthenticated to `/auth/jwt/sign-in?returnTo=...`. The Telegram guard additionally validates `window.Telegram.WebApp.initData` before issuing a session.
|
||||
|
||||
---
|
||||
|
||||
@@ -189,6 +198,8 @@ Higher-level hooks build on this:
|
||||
| `use-marketplace-socket` | broad market events |
|
||||
| `use-unified-real-time` | multi-event aggregator |
|
||||
|
||||
The Telegram Mini App shell reuses the same `SocketProvider` — live socket updates are available in the TMA shop, cart, and account tabs.
|
||||
|
||||
See [[Real-time Layer]] for the full event catalog.
|
||||
|
||||
---
|
||||
@@ -213,6 +224,8 @@ const config = createConfig({
|
||||
|
||||
Wallet UI: connect / disconnect / show address / show balance via `use-web3-wagmi`, `use-web3-context`. The current checkout target is the Request Network in-house flow; the DePay widget package remains legacy/frontier context and should not be treated as the primary path.
|
||||
|
||||
TON wallet support is handled separately via `@ton/core` + `@tonconnect/ui-react` in the Telegram Mini App layer (see §19).
|
||||
|
||||
---
|
||||
|
||||
## 10. Internationalization
|
||||
@@ -288,6 +301,9 @@ See [[Theme Configuration]] and [[Design System Overview]].
|
||||
|
||||
State persists in `localStorage` under `settings-key`.
|
||||
|
||||
> [!note]
|
||||
> The Telegram Mini App shell does not render the settings drawer; theme and direction are inherited from the parent app's stored settings at launch.
|
||||
|
||||
---
|
||||
|
||||
## 14. Editor (TipTap)
|
||||
@@ -350,6 +366,7 @@ See [[Docker Setup]], [[CI-CD Pipeline]], and [[Deployment]].
|
||||
| File | Why it matters |
|
||||
|---|---|
|
||||
| `src/app/layout.tsx` | Provider tree |
|
||||
| `src/app/telegram/layout.tsx` | TMA shell — TonConnectUIProvider + slim provider tree |
|
||||
| `src/lib/axios.ts` | Every HTTP call goes through this |
|
||||
| `src/contexts/socket-context.tsx` | Realtime plumbing |
|
||||
| `src/theme/index.ts` | Theme creation entry |
|
||||
@@ -359,6 +376,67 @@ See [[Docker Setup]], [[CI-CD Pipeline]], and [[Deployment]].
|
||||
|
||||
---
|
||||
|
||||
## 19. Telegram Mini App (TMA) layer
|
||||
|
||||
### Overview
|
||||
|
||||
The app ships a dedicated Telegram Mini App shell at `app/telegram/`. It is served from the same Next.js process and Docker image as the main web app; no separate deployment is required. The Telegram bot registers the Mini App URL pointing at `/telegram`.
|
||||
|
||||
### Provider tree (TMA layout)
|
||||
|
||||
The TMA layout replaces the full dashboard shell with a minimal provider stack:
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
A[TelegramLayout]
|
||||
A --> B[AppRouterCacheProvider]
|
||||
B --> C[ThemeProvider]
|
||||
C --> D[QueryClientProvider]
|
||||
D --> E[SocketProvider]
|
||||
E --> F[TonConnectUIProvider<br/>manifestUrl: /tonconnect-manifest.json]
|
||||
F --> G[SnackbarProvider]
|
||||
G --> H[Children — telegram routes]
|
||||
```
|
||||
|
||||
`TonConnectUIProvider` is the only addition relative to the web tree. Settings drawer, i18n provider, and auth guards are replaced by a Telegram `initData` token guard.
|
||||
|
||||
### Routes and features
|
||||
|
||||
| Route | Description |
|
||||
|---|---|
|
||||
| `telegram/shop` | Seller list with product browsing; infinite scroll |
|
||||
| `telegram/shop/[seller]` | Single seller's catalogue |
|
||||
| `telegram/cart` | In-shell shopping cart; checkout hands off to full web checkout URL |
|
||||
| `telegram/account` | Account tab with dashboard parity: profile, wallet, order history |
|
||||
|
||||
### Authentication flow
|
||||
|
||||
1. Telegram injects `window.Telegram.WebApp.initData` on launch.
|
||||
2. The TMA guard sends `initData` to `/api/auth/telegram` for HMAC verification.
|
||||
3. On success the backend issues a short-lived JWT that the axios instance attaches as `Bearer`.
|
||||
4. Role-based access (seller vs buyer views) is honoured via the same guard mechanism used in the dashboard.
|
||||
|
||||
### Real-time
|
||||
|
||||
`SocketProvider` is reused unchanged. The TMA shop, cart, and account tabs receive live socket updates (new messages, payment status, cart changes) on the same room infrastructure as the web dashboard.
|
||||
|
||||
### TON Connect (Telegram Wallet)
|
||||
|
||||
**Dependencies added**: `@ton/core`, `@tonconnect/ui-react`.
|
||||
|
||||
`TonConnectUIProvider` wraps the TMA routes and exposes a `useTonConnectUI()` hook. The manifest at `public/tonconnect-manifest.json` declares the app identity to the TON Connect protocol.
|
||||
|
||||
Current status: the wallet connection UI is in place (connect / disconnect / show address). **Actual TON payment processing is not yet wired to the backend** — the provider is pre-positioned for a future TON payment rail on the escrow platform. When that rail is built, the checkout handoff in `telegram/cart` will be extended to emit a TON transaction instead of redirecting to the web checkout.
|
||||
|
||||
### Constraints and differences from web
|
||||
|
||||
- No settings drawer (theme follows web localStorage, defaults to light/ltr).
|
||||
- No TipTap editor or file-upload dropzone in TMA routes.
|
||||
- `@mui/x-date-pickers` and DataGrid are not loaded in the TMA bundle.
|
||||
- COOP/COEP headers required for WalletConnect popups are relaxed for TMA routes because Telegram's WebView does not support `SharedArrayBuffer`.
|
||||
|
||||
---
|
||||
|
||||
## Related
|
||||
|
||||
- [[System Architecture]] — bird's-eye topology
|
||||
|
||||
@@ -1,11 +1,11 @@
|
||||
---
|
||||
title: Chat
|
||||
tags: [data-model, mongoose]
|
||||
tags: [data-model, mongoose, postgres]
|
||||
aliases: [Conversation, IChat, IMessage]
|
||||
---
|
||||
|
||||
# Chat
|
||||
> **Last updated:** 2026-05-29 — aligned with code (see [Doc vs Code Audit Report](../09%20-%20Audits/Doc%20vs%20Code%20Audit%20Report%20-%202026-05-29.md))
|
||||
> **Last updated:** 2026-06-03 — added Postgres/Drizzle schema section; migration status clarified.
|
||||
|
||||
Conversation container with embedded messages. Used for buyer-seller direct chats, group chats, and support tickets. Each chat carries a list of participants, an embedded `messages[]` array (with reactions, replies, edit history), a denormalised `lastMessage` snapshot for list views, and per-user `unreadCounts`. A chat can be linked to any other entity through the `relatedTo` discriminator (currently `PurchaseRequest`, `SellerOffer`, or `Transaction`).
|
||||
|
||||
@@ -13,6 +13,7 @@ Conversation container with embedded messages. Used for buyer-seller direct chat
|
||||
> `backend/src/models/Chat.ts:130` — chat schema definition
|
||||
> `backend/src/models/Chat.ts:69` — message subdocument schema
|
||||
> `backend/src/models/Chat.ts:348` — model export
|
||||
> `backend/src/db/schema/chat.ts` — Drizzle/Postgres schema
|
||||
|
||||
> [!warning] Embedded messages
|
||||
> Messages live inside the chat document. Very long-running chats can grow past the 16 MB document limit. Treat this as a known constraint of the current schema.
|
||||
@@ -20,7 +21,10 @@ Conversation container with embedded messages. Used for buyer-seller direct chat
|
||||
> [!warning] `relatedTo` is NOT set via `POST /api/chat`
|
||||
> Although `relatedTo` exists in the schema, it is **not accepted** by the `POST /api/chat` create endpoint. Purchase-request linkage is established server-side through the dedicated `POST /api/chat/purchase-request`, not by passing `relatedTo` to the generic create endpoint.
|
||||
|
||||
## Schema — Chat
|
||||
> [!danger] Migration status — DUAL-WRITE, reads still on Mongo
|
||||
> Chat writes go to **both** MongoDB and Postgres (via `DrizzleChatRepo`). However, **all reads still come from MongoDB**. The Postgres `chats` table is a conservative shim: `participants` and `messages` are stored as JSONB blobs, not normalised child tables. Full normalisation (splitting messages into a separate table with proper threading) is a **known open blocker** for the Mongo → PG read cutover. Do not assume PG data is queryable relationally until that work is complete.
|
||||
|
||||
## Schema — Chat (MongoDB / Mongoose)
|
||||
|
||||
| Field | Type | Required | Default | Validation | Index | Description |
|
||||
| --- | --- | --- | --- | --- | --- | --- |
|
||||
@@ -57,7 +61,7 @@ Conversation container with embedded messages. Used for buyer-seller direct chat
|
||||
> [!note] Soft removal of participants
|
||||
> Removing a participant (via `DELETE /api/chat/:id/participants/:participantId`) does **not** delete the subdocument. It is a soft removal: `isActive` is set to `false` and `leftAt` is timestamped, preserving message attribution and history.
|
||||
|
||||
## Schema — Message (embedded)
|
||||
## Schema — Message (embedded, MongoDB)
|
||||
|
||||
| Field | Type | Required | Default | Validation | Description |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
@@ -80,13 +84,65 @@ Conversation container with embedded messages. Used for buyer-seller direct chat
|
||||
> [!note] Messages are soft-deleted
|
||||
> Deleting a message sets `deletedAt` and clears `content` (the body becomes empty). The message subdocument is **not** physically removed from `messages[]`, and a `message-deleted` socket event is emitted.
|
||||
|
||||
## Schema — `chats` table (Postgres / Drizzle)
|
||||
|
||||
> Source: `backend/src/db/schema/chat.ts`
|
||||
|
||||
> [!warning] Conservative JSONB shim — not normalised
|
||||
> Unlike most other migrated tables, participants and messages are stored as **JSONB blobs** (`ChatParticipant[]` and `ChatMessage[]`), not as separate relational child tables. This was a deliberate trade-off to unblock dual-write without committing to a normalisation design. The normalised schema (separate `chat_messages` and `chat_participants` tables with proper FKs and threading support) is the **primary blocker** for cutting reads over to Postgres.
|
||||
|
||||
### Enums (declared in `_enums.ts`)
|
||||
|
||||
| Enum | Values |
|
||||
| --- | --- |
|
||||
| `chat_type` | `direct`, `group`, `support` |
|
||||
| `chat_participant_role` | `member`, `admin`, `owner` |
|
||||
| `chat_message_type` | `text`, `image`, `file`, `system` |
|
||||
| `chat_related_to_type` | `PurchaseRequest`, `SellerOffer`, `Transaction` |
|
||||
|
||||
### Table: `chats`
|
||||
|
||||
| Column | PG type | Nullable | Default | Notes |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| `id` | `uuid` | NOT NULL | `gen_random_uuid()` | Primary key |
|
||||
| `legacy_object_id` | `text` | nullable | — | Mongo ObjectId bridge; partial-unique index WHERE NOT NULL |
|
||||
| `type` | `chat_type` enum | NOT NULL | `'direct'` | |
|
||||
| `name` | `text` | nullable | — | Group chat display name |
|
||||
| `description` | `text` | nullable | — | |
|
||||
| `participants` | `jsonb` | nullable | — | `ChatParticipant[]` blob — **not normalised** |
|
||||
| `messages` | `jsonb` | nullable | — | `ChatMessage[]` blob — **not normalised** |
|
||||
| `related_to` | `jsonb` | nullable | — | `{ type: chat_related_to_type, id: string }` blob |
|
||||
| `last_message` | `jsonb` | nullable | — | Denormalised snapshot |
|
||||
| `unread_counts` | `jsonb` | nullable | — | `{ userId, count }[]` blob |
|
||||
| `settings_is_archived` | `boolean` | nullable | `false` | |
|
||||
| `settings_is_muted` | `boolean` | nullable | `false` | |
|
||||
| `settings_muted_until` | `timestamp with time zone` | nullable | — | |
|
||||
| `settings_notifications` | `boolean` | nullable | `true` | |
|
||||
| `created_by` | `text` | nullable | — | Mongo ObjectId or UUID string of creator |
|
||||
| `created_at` | `timestamp with time zone` | NOT NULL | `now()` | |
|
||||
| `updated_at` | `timestamp with time zone` | NOT NULL | `now()` | |
|
||||
| `last_activity` | `timestamp with time zone` | nullable | `now()` | Sort key for chat lists |
|
||||
|
||||
### Indexes on `chats`
|
||||
|
||||
| Index | Definition | Notes |
|
||||
| --- | --- | --- |
|
||||
| PK | `id` | |
|
||||
| partial-unique | `legacy_object_id` WHERE NOT NULL | Idempotent backfill upsert |
|
||||
| regular | `type` | |
|
||||
| regular | `created_by` | |
|
||||
| regular | `last_activity` | |
|
||||
|
||||
> [!note] No FK to `users`
|
||||
> `created_by` is stored as `text` (not `uuid` FK) to accommodate both Mongo ObjectIds and PG UUIDs during the transition period.
|
||||
|
||||
## Virtuals
|
||||
|
||||
| Virtual | Returns | Definition |
|
||||
| --- | --- | --- |
|
||||
| `participantsCount` | Count of active participants | `backend/src/models/Chat.ts:259` |
|
||||
|
||||
## Indexes
|
||||
## Indexes (MongoDB)
|
||||
|
||||
Defined at `backend/src/models/Chat.ts:243-247`:
|
||||
|
||||
@@ -119,6 +175,24 @@ None defined.
|
||||
- **References**: [[User]] (`participants[].userId`, `messages[].senderId`, `messages[].reactions[].userId`, `lastMessage.senderId`, `unreadCounts[].userId`, `metadata.createdBy`).
|
||||
- **Referenced by**: [[Dispute]] (`chatId`), and any [[PurchaseRequest]] / [[SellerOffer]] indirectly through `relatedTo`.
|
||||
|
||||
## Migration Status
|
||||
|
||||
| Dimension | Status |
|
||||
| --- | --- |
|
||||
| Dual-write repo | `DrizzleChatRepo` — active |
|
||||
| Writes | Both MongoDB and Postgres receive writes |
|
||||
| Reads | **MongoDB only** — not yet cut over |
|
||||
| Postgres schema style | JSONB shim (participants + messages as blobs) |
|
||||
| Normalisation blocker | Chat message threading design not finalised — blocks PG read cutover |
|
||||
|
||||
The normalisation work required before reads can be cut to PG:
|
||||
1. Design a `chat_messages` table with proper threading/reply support (currently `replyTo` is an ObjectId embedded in a JSONB blob)
|
||||
2. Design a `chat_participants` table (currently a JSONB blob with soft-removal semantics)
|
||||
3. Migrate reactions, edit history, and read tracking to relational rows
|
||||
4. Align unread counts with the new structure
|
||||
|
||||
Until that work is complete, the Postgres `chats` table is treated as a write-ahead log / backup, not the source of truth for reads.
|
||||
|
||||
## State Transitions
|
||||
|
||||
No top-level status. Chat-level archival is a boolean flag (`settings.isArchived`):
|
||||
|
||||
@@ -1,47 +1,57 @@
|
||||
---
|
||||
title: Data Model Overview
|
||||
tags: [data-model, mongoose, overview]
|
||||
tags: [data-model, mongoose, postgres, drizzle, overview]
|
||||
aliases: [Models Index, Schema Overview]
|
||||
---
|
||||
|
||||
# Data Model Overview
|
||||
|
||||
This section documents every Mongoose model that backs the marketplace. On backend `integrate-main-into-development@cab0719`, these Mongoose models are still the live application persistence layer. The repo also contains a Drizzle/Postgres migration layer, but most services still call `backend/src/models/*` directly.
|
||||
This section documents every Mongoose model that backs the marketplace and the parallel Drizzle/Postgres schema that is progressively replacing it. On backend `integrate-main-into-development@cab0719`, Mongoose models are still the live read path for most domains. The Drizzle layer has 17 applied migrations (0000–0017) and active dual-write repos for the majority of tables.
|
||||
|
||||
> [!note] Scope
|
||||
> Twenty-two models are present in `backend/src/models/`. The "File" concept exists only at the service layer (`backend/src/services/file/`) and is not persisted as its own Mongoose collection, so it is not listed below.
|
||||
>
|
||||
> [!note] Documentation freshness
|
||||
> The 2026-05-24 audit note that marked `Dispute`, `BlogPost`, `Review`, `PointTransaction`, `LevelConfig`, and `ShopSettings` as missing is now stale: schema files exist for those models. Newer operational models such as [[ConfigSetting]], [[DerivedDestination]], [[FundsLedgerEntry]], and [[TrezorAccount]] should be expanded into dedicated model pages when the docs are next deepened.
|
||||
> As of 2026-06-03 the Postgres migration inventory reflects migrations 0000–0017. The dual-write summary table at the bottom of this page is the authoritative migration-status reference. Individual model pages should be updated to note their PG table name and dual-write repo when they are deepened.
|
||||
|
||||
> [!warning] Mongo vs Postgres runtime status
|
||||
> Postgres schemas and repositories exist for the money/relational core, but normal app traffic is not fully cut over. Payment quote rows are the only current conditional PG write in checkout, and even that requires `ORACLE_QUOTING_ENABLED=true` plus a resolvable PG payment row. See [[Postgres Runtime Cutover Status]].
|
||||
> Dual-write repos exist for the majority of domain tables, but **reads are still served from Mongo** for all dual-write tables. Postgres is the sole store only for infra/bridge tables (`id_map`, `pg_dualwrite_gaps`), oracle quote rows (`payment_quotes`), and `config_setting_history`. Full read cutover is human-gated. See [[Postgres Runtime Cutover Status]].
|
||||
|
||||
## Index of Models
|
||||
|
||||
- [[User]] — Core identity. Stores credentials, profile, preferences, referral data, points, and WebAuthn passkeys. Every other model that records "who did what" points back at a `User._id`. Buyers, sellers, and admins all live in this collection, differentiated by a `role` enum.
|
||||
- [[PurchaseRequest]] — The buyer-side document at the heart of the marketplace. Captures what a buyer wants, the budget, urgency, delivery preferences, and the full lifecycle status (`pending_payment` → `seller_paid`). Aggregates [[SellerOffer]] references and tracks delivery codes.
|
||||
- [[SellerOffer]] — A seller's bid against a [[PurchaseRequest]]. Holds price, delivery ETA, attachments, and a small status machine (`pending` / `accepted` / `rejected` / `withdrawn`).
|
||||
- [[Payment]] — Records monetary movement intent and state: buyer pay-in, seller release, and refund. The current primary provider path is Request Network plus in-house checkout, derived destinations, funds ledger entries, and Transaction Safety Provider metadata.
|
||||
- [[Chat]] — Conversation container with embedded messages, participants, unread counters, and reactions. Used for direct buyer-seller chats, group chats, and support tickets. Can be linked to a [[PurchaseRequest]] or [[SellerOffer]].
|
||||
- [[Notification]] — Per-user notification with category, type, and 90-day TTL for automatic cleanup. References any related entity by stringified id.
|
||||
- [[RequestTemplate]] — A seller-authored, sharable template that pre-fills a [[PurchaseRequest]]. Carries a public shareable link, usage counter, and an optional default proposal.
|
||||
- [[Dispute]] — Buyer-raised complaint tied to a [[PurchaseRequest]]. Captures evidence uploads, a timeline of admin actions, deadlines, and a structured resolution.
|
||||
- [[BlogPost]] — Editorial content: title, slug, rich content, media, SEO metadata, view/like counters, and a draft/published/archived workflow.
|
||||
- [[Address]] — User shipping address book entry. Enforces a single primary address per user via a pre-save hook.
|
||||
- [[Category]] — Hierarchical product/service taxonomy referenced by [[PurchaseRequest]] and [[RequestTemplate]]. Supports parent/child via `parentId` and bilingual `name` / `nameEn`.
|
||||
- [[Review]] — Polymorphic 1-5 star review against either a seller or a [[RequestTemplate]] (`subjectType` discriminator). One review per reviewer per subject (compound unique index).
|
||||
- [[PointTransaction]] — Ledger of point grants and spends per user. Sources include purchase, referral, bonus, admin grant, and redemption.
|
||||
- [[LevelConfig]] — Static configuration of loyalty tiers (level number, point thresholds, benefits, icon, color). Driven by admins; consumed by the [[User]].points.level field.
|
||||
- [[ShopSettings]] — One-to-one storefront branding for a seller: name, description, avatar, cover image, review toggles, and social links.
|
||||
- [[TempVerification]] — Short-lived signup record that holds candidate user data and a verification code. Auto-purges via TTL when `emailVerificationCodeExpires` passes.
|
||||
- [[TelegramLink]] — Permanent auditable association between a Telegram user ID and an Amanat [[User]]. Stores Telegram profile metadata, link source (`miniapp` / `bot` / `login_widget`), status (`active` / `blocked`), and last-seen timestamp. One per Telegram user (unique on both `userId` and `telegramUserId`).
|
||||
- [[TelegramSession]] — Short-lived Telegram Mini App session token issued when `initData` is verified. Carries the `initDataFingerprint` for replay protection and auto-expires via a MongoDB TTL index on `expiresAt`.
|
||||
- [[ConfigSetting]] — Runtime configuration persisted in MongoDB for operational knobs that need an admin surface rather than a deploy.
|
||||
- [[DerivedDestination]] — Per-payment derived wallet destination records used to reduce address reuse and reconcile on-chain pay-ins.
|
||||
- [[FundsLedgerEntry]] — Immutable accounting ledger rows for pay-in, hold, release, refund, fee, adjustment, and reversal events.
|
||||
- [[TrezorAccount]] — Hardware-wallet/safekeeping account metadata for custody operations and staged signer hardening.
|
||||
- [[ConfigSettingHistory]] — Immutable audit trail of numeric runtime-config changes. Currently used for per-chain confirmation threshold change events, keyed as `confirmation_threshold:<chainId>`. Added in commit `27fb15a`.
|
||||
### Mongo Models (still live read path)
|
||||
|
||||
- [[User]] — Core identity. Stores credentials, profile, preferences, referral data, points, and WebAuthn passkeys. Every other model that records "who did what" points back at a `User._id`. Buyers, sellers, admins, resolvers, and guards all live in this collection, differentiated by a `role` enum. PG table: `users` (dual-write active).
|
||||
- [[PurchaseRequest]] — The buyer-side document at the heart of the marketplace. Captures what a buyer wants, the budget, urgency, delivery preferences, and the full lifecycle status (`pending_payment` → `seller_paid`). Aggregates [[SellerOffer]] references and tracks delivery codes. PG table: `purchase_requests` + 6 child tables (dual-write active).
|
||||
- [[SellerOffer]] — A seller's bid against a [[PurchaseRequest]]. Holds price, delivery ETA, attachments, and a small status machine (`pending` / `accepted` / `rejected` / `withdrawn`). PG table: `seller_offers` (dual-write active).
|
||||
- [[Payment]] — Records monetary movement intent and state: buyer pay-in, seller release, and refund. The current primary provider path is Request Network plus in-house checkout, derived destinations, funds ledger entries, and Transaction Safety Provider metadata. PG table: `payments` (dual-write active).
|
||||
- [[Chat]] — Conversation container with embedded messages, participants, unread counters, and reactions. Used for direct buyer-seller chats, group chats, and support tickets. Can be linked to a [[PurchaseRequest]] or [[SellerOffer]]. PG table: `chats` (conservative JSONB shim; Chat normalization is an open blocker).
|
||||
- [[Notification]] — Per-user notification with category, type, and 90-day TTL for automatic cleanup. References any related entity by stringified id. PG table: `notifications` (dual-write active; `user_id` stored as `text`, no hard FK).
|
||||
- [[RequestTemplate]] — A seller-authored, sharable template that pre-fills a [[PurchaseRequest]]. Carries a public shareable link, usage counter, and an optional default proposal. PG table: `request_templates` (dual-write active).
|
||||
- [[Dispute]] — Buyer-raised complaint tied to a [[PurchaseRequest]]. Captures evidence uploads, a timeline of admin actions, deadlines, and a structured resolution. PG table: `disputes` (dual-write active; all IDs as `text` for ObjectId/UUID coexistence).
|
||||
- [[BlogPost]] — Editorial content: title, slug, rich content, media, SEO metadata, view/like counters, and a draft/published/archived workflow. PG table: `blog_posts` (dual-write active).
|
||||
- [[Address]] — User shipping address book entry. Enforces a single primary address per user via a pre-save hook. PG table: `addresses` (schema scaffolded, migration 0016; `addressStore.ts` reads PG directly).
|
||||
- [[Category]] — Hierarchical product/service taxonomy referenced by [[PurchaseRequest]] and [[RequestTemplate]]. Supports parent/child via `parentId` and bilingual `name` / `nameEn`. PG table: `categories` (dual-write active).
|
||||
- [[Review]] — Polymorphic 1-5 star review against either a seller or a [[RequestTemplate]] (`subjectType` discriminator). One review per reviewer per subject (compound unique index). PG table: `reviews` (schema scaffolded, no dual-write repo yet).
|
||||
- [[PointTransaction]] — Ledger of point grants and spends per user. Sources include purchase, referral, bonus, admin grant, and redemption. PG table: `point_transactions` (dual-write active).
|
||||
- [[LevelConfig]] — Static configuration of loyalty tiers (level number, point thresholds, benefits, icon, color). Driven by admins; consumed by the [[User]].points.level field. No PG table (read-only config; not yet migrated).
|
||||
- [[ShopSettings]] — One-to-one storefront branding for a seller: name, description, avatar, cover image, review toggles, and social links. PG table: `shop_settings` (schema scaffolded, no dual-write repo yet).
|
||||
- [[TempVerification]] — Short-lived signup record that holds candidate user data and a verification code. Auto-purges via TTL when `emailVerificationCodeExpires` passes. No PG table (TTL-only; not yet migrated).
|
||||
- [[TelegramLink]] — Permanent auditable association between a Telegram user ID and an Amanat [[User]]. Stores Telegram profile metadata, link source (`miniapp` / `bot` / `login_widget`), status (`active` / `blocked`), and last-seen timestamp. One per Telegram user (unique on both `userId` and `telegramUserId`). PG table: `telegram_links` (schema scaffolded, no dual-write repo yet).
|
||||
- [[TelegramSession]] — Short-lived Telegram Mini App session token issued when `initData` is verified. Carries the `initDataFingerprint` for replay protection and auto-expires via a MongoDB TTL index on `expiresAt`. PG table: `telegram_sessions` (schema scaffolded, no dual-write repo yet).
|
||||
- [[ConfigSetting]] — Runtime configuration persisted in MongoDB for operational knobs that need an admin surface rather than a deploy. PG table: `config_settings` (schema scaffolded, no dual-write repo yet).
|
||||
- [[DerivedDestination]] — Per-payment derived wallet destination records used to reduce address reuse and reconcile on-chain pay-ins. PG table: `derived_destinations` + `derived_destination_sweeps` (dual-write active).
|
||||
- [[FundsLedgerEntry]] — Immutable accounting ledger rows for pay-in, hold, release, refund, fee, adjustment, and reversal events. PG table: `funds_ledger_entries` (dual-write active; immutability enforced by DB trigger since migration 0015).
|
||||
- [[TrezorAccount]] — Hardware-wallet/safekeeping account metadata for custody operations and staged signer hardening. PG table: `trezor_accounts` + `trezor_derived_addresses` (dual-write active).
|
||||
- [[ConfigSettingHistory]] — Immutable audit trail of numeric runtime-config changes. Currently used for per-chain confirmation threshold change events, keyed as `confirmation_threshold:<chainId>`. Added in commit `27fb15a`. PG table: `config_setting_history` (PG-only; no Mongo equivalent).
|
||||
|
||||
### PG-Only Tables (no Mongo equivalent)
|
||||
|
||||
- `id_map` — ObjectId → UUID bridge. Every migrated table upserts here during backfill/dual-write. Composite PK on `(collection, legacy_object_id)`, unique on `new_id`.
|
||||
- `pg_dualwrite_gaps` — Append-only reconciliation gap log for failed PG dual-writes. Tracks collection, op, payload, severity, and resolution metadata.
|
||||
- `payment_quotes` — Oracle pricing quotes per payment (oracle depeg-protection feature). Stores `fx_rate`, `token_price_usd`, `depeg_adjustment_bps`, `settle_amount`, chain/token, and expiry. Requires `ORACLE_QUOTING_ENABLED=true`. 1:1 to `payments`.
|
||||
- `user_passkeys` — WebAuthn credential store (child of `users`). Columns: credential id (text PK), `user_id FK→users CASCADE`, `public_key`, `counter`, `device_type`, `device_name`.
|
||||
- `user_refresh_tokens` — Refresh token store (child of `users`). Columns: `token text PK`, `user_id FK→users CASCADE`.
|
||||
|
||||
## Relationship Diagram
|
||||
|
||||
@@ -59,6 +69,10 @@ erDiagram
|
||||
USER ||--o{ DISPUTE : "raises as buyer"
|
||||
USER ||--o{ USER : "referred by"
|
||||
USER ||--o{ TREZOR_ACCOUNT : "controls custody account"
|
||||
USER ||--o{ USER_PASSKEY : "authenticates with"
|
||||
USER ||--o{ USER_REFRESH_TOKEN : "sessions via"
|
||||
USER ||--o| TELEGRAM_LINK : "links identity"
|
||||
USER ||--o{ TELEGRAM_SESSION : "session for"
|
||||
|
||||
PURCHASE_REQUEST }o--|| CATEGORY : "belongs to"
|
||||
PURCHASE_REQUEST ||--o{ SELLER_OFFER : "receives"
|
||||
@@ -74,6 +88,7 @@ erDiagram
|
||||
PAYMENT }o--|| USER : "seller"
|
||||
PAYMENT ||--o{ FUNDS_LEDGER_ENTRY : "accounted by"
|
||||
PAYMENT ||--o| DERIVED_DESTINATION : "collects into"
|
||||
PAYMENT ||--o| PAYMENT_QUOTE : "oracle-priced by"
|
||||
|
||||
CHAT }o--o{ USER : "participants"
|
||||
CHAT ||--o{ DISPUTE : "support channel"
|
||||
@@ -91,10 +106,17 @@ erDiagram
|
||||
TELEGRAM_LINK }o--|| USER : "links identity"
|
||||
TELEGRAM_SESSION }o--o| USER : "session for"
|
||||
TELEGRAM_SESSION }o--|| TELEGRAM_LINK : "matches"
|
||||
|
||||
TREZOR_ACCOUNT ||--o{ TREZOR_DERIVED_ADDRESS : "issues"
|
||||
DERIVED_DESTINATION ||--o{ DERIVED_DESTINATION_SWEEP : "swept by"
|
||||
|
||||
ID_MAP ||..|| USER : "bridges ObjectId"
|
||||
```
|
||||
|
||||
## Conventions Across All Models
|
||||
|
||||
### Mongoose Conventions
|
||||
|
||||
> [!note] Shared schema patterns
|
||||
> - **Timestamps**: every model declares `{ timestamps: true }`, so `createdAt` and `updatedAt` are always present.
|
||||
> - **ObjectId references**: foreign keys use `Schema.Types.ObjectId` with an explicit `ref` (e.g. `ref: 'User'`). The two exceptions are [[Notification]] and [[Payment]] which use string-typed or `Mixed` identifiers in places to support template-flow payments.
|
||||
@@ -105,15 +127,126 @@ erDiagram
|
||||
> [!warning] Index discipline
|
||||
> Several schemas leave a comment noting that `unique: true` already creates an index — adding `schema.index({ field: 1 })` on top would produce a duplicate-index warning at startup. When introducing new indexes, search for `unique: true` first.
|
||||
|
||||
### Drizzle/Postgres Conventions
|
||||
|
||||
> [!note] PG schema patterns
|
||||
> - **Legacy bridge**: every migrated table carries `legacy_object_id text` with a partial-unique index `WHERE legacy_object_id IS NOT NULL` for idempotent backfill upserts. The `id_map` table records the ObjectId → UUID mapping centrally.
|
||||
> - **Money columns**: `numeric(38,18)` for fiat/crypto amounts throughout, except `seller_offers` which uses `numeric(18,8)` per the Migration Guide. Blockchain balance columns use `numeric(78,0)` to hold uint256 without overflow.
|
||||
> - **Polymorphic triples**: the `ref_kind` enum (`entity` | `template`) discriminator is expanded into three columns (`_ref_kind`, `_id`, `_external_ref`) with a CHECK constraint to enforce discriminator integrity. Used by `payments`, `funds_ledger_entries`, and `derived_destinations`.
|
||||
> - **Soft delete**: `addresses` uses `deleted_at timestamptz` (nullable) with partial-unique indexes scoped to `WHERE deleted_at IS NULL`. Most other tables retain the Mongo `status` flag approach.
|
||||
> - **Timestamps**: all timestamp columns declare `withTimezone: true`.
|
||||
> - **Immutability**: `funds_ledger_entries` has both an UPDATE-blocking and a DELETE-blocking trigger installed at the DB level (migrations 0004, 0015). A TRUNCATE trigger was added in migration 0013.
|
||||
> - **user_role enum**: values are `admin`, `buyer`, `seller`, `resolver`, `guard`. The `guard` value was added in migration 0017.
|
||||
|
||||
## Postgres Migration Inventory
|
||||
|
||||
Schema entry point: `backend/src/db/schema/index.ts`
|
||||
|
||||
| Migration | File | Summary |
|
||||
|---|---|---|
|
||||
| 0000 | `0000_slimy_veda.sql` | Initial: core enums + `id_map` + `categories` |
|
||||
| 0001 | `0001_wild_cargill.sql` | `trezor_accounts` + `trezor_derived_addresses` (later reset) |
|
||||
| 0002 | `0002_motionless_grey_gargoyle.sql` | Schema reset: drops 0000/0001 tables to be rebuilt in 0003; adds `categories.parent_id` self-FK |
|
||||
| 0003 | `0003_remarkable_retro_girl.sql` | Comprehensive rebuild: all enums + full core domain (`users`, `payments`, `funds_ledger_entries`, `derived_destinations`, `purchase_requests` + 6 children, `seller_offers`, `point_transactions`, `trezor_*`) |
|
||||
| 0004a | `0004_funds_ledger_entries.sql` | UPDATE-blocking immutability trigger on `funds_ledger_entries` |
|
||||
| 0004b | `0004_seller_offer.sql` | Physical FKs on `seller_offers` → `users` and `purchase_requests` (CASCADE) |
|
||||
| 0005 | `0005_simple_champions.sql` | `pg_dualwrite_gaps`; FKs on `payments`; `legacy_object_id` unique indexes; refined pending-RN payment unique index |
|
||||
| 0006 | `0006_normal_madame_hydra.sql` | CHECK: `purchase_requests.budget_currency` restricted to crypto (USDT, USDC) |
|
||||
| 0007 | `0007_woozy_shaman.sql` | Drops 0006 constraint; sets `budget_currency` default to `'USDT'` |
|
||||
| 0008 | `0008_giant_winter_soldier.sql` | Adds `'TRY'` to `offer_currency` enum; creates `payment_quotes` table |
|
||||
| 0009 | `0009_unique_active_categories.sql` | Category deduplication; partial unique index on normalized active category name |
|
||||
| 0010 | `0010_request_templates.sql` | Creates `request_templates`; deduplicates `purchase_request_specifications`; adds unique key constraint |
|
||||
| 0011 | `0011_chats.sql` | Creates `chats` with JSONB participant/message storage + chat-related enums |
|
||||
| 0012 | `0012_disputes.sql` | Creates `disputes` (text IDs, JSONB evidence/timeline/resolution) |
|
||||
| 0013 | `0013_money_constraints.sql` | Money-integrity CHECKs on `payments`, `payment_quotes`, `point_transactions`, `users`; TRUNCATE trigger on `funds_ledger_entries`; composite PK + unique on `id_map` |
|
||||
| 0014 | `0014_physical_fks.sql` | NOT VALID FKs across all major tables (validated immediately); composite indexes on `payments`, `purchase_requests`, `seller_offers` |
|
||||
| 0015 | `0015_funds_ledger_immutable_trigger.sql` | Replaces/extends ledger triggers: UPDATE-block + new DELETE-block on `funds_ledger_entries` |
|
||||
| 0016 | `0016_addresses_table.sql` | `address_type` enum + `addresses` table; partial-unique primary-address-per-user index |
|
||||
| 0017 | `0017_user_role_guard.sql` | Adds `'guard'` to `user_role` enum (idempotent `ADD VALUE IF NOT EXISTS`) |
|
||||
|
||||
## Drizzle Table Inventory and Migration Status
|
||||
|
||||
### Infrastructure / Bridge
|
||||
|
||||
| PG Table | Schema File | Status | Notes |
|
||||
|---|---|---|---|
|
||||
| `id_map` | `idMap.ts` | PG-only | ObjectId → UUID bridge; composite PK + unique on `new_id` |
|
||||
| `pg_dualwrite_gaps` | `pgDualwriteGaps.ts` | PG-only | Append-only reconciliation gap log for failed dual-writes |
|
||||
|
||||
### Core Domain
|
||||
|
||||
| PG Table | Schema File | Status | Dual-Write Repo |
|
||||
|---|---|---|---|
|
||||
| `users` | `users.ts` | Dual-write active | `DualWriteUserRepo` + `DrizzleUserRepo` + `MongoUserRepo` |
|
||||
| `user_passkeys` | `users.ts` | Dual-write active (child of users) | — |
|
||||
| `user_refresh_tokens` | `users.ts` | Dual-write active (child of users) | — |
|
||||
| `categories` | `category.ts` | Dual-write active | `DualWriteMarketplaceRepo` |
|
||||
| `purchase_requests` | `purchaseRequest.ts` | Dual-write active | `DualWriteMarketplaceRepo` |
|
||||
| `purchase_request_delivery_info` | `purchaseRequest.ts` | Dual-write active (1:1 child) | — |
|
||||
| `purchase_request_delivery_address` | `purchaseRequest.ts` | Dual-write active (1:1 child) | — |
|
||||
| `purchase_request_seller_delivery_info` | `purchaseRequest.ts` | Dual-write active (1:1 child) | — |
|
||||
| `delivery_attempts` | `purchaseRequest.ts` | Dual-write active (1:N child) | — |
|
||||
| `purchase_request_service_info` | `purchaseRequest.ts` | Dual-write active (1:1 child) | — |
|
||||
| `purchase_request_specifications` | `purchaseRequest.ts` | Dual-write active (1:N child) | — |
|
||||
| `purchase_request_preferred_sellers` | `purchaseRequest.ts` | Dual-write active (N:M junction) | — |
|
||||
| `seller_offers` | `sellerOffer.ts` | Dual-write active | `DualWriteMarketplaceRepo` |
|
||||
| `payments` | `payment.ts` | Dual-write active | `DualWritePaymentRepo` + `DrizzlePaymentRepo` + `MongoPaymentRepo` |
|
||||
| `payment_quotes` | `paymentQuote.ts` | PG-only | No Mongo equivalent; oracle depeg-protection feature |
|
||||
| `funds_ledger_entries` | `fundsLedgerEntry.ts` | Dual-write active | `DrizzlePaymentRepo` / `DualWritePaymentRepo` |
|
||||
| `derived_destinations` | `derivedDestination.ts` | Dual-write active | `DualWriteDerivedDestinationRepo` + `DrizzleDerivedDestinationRepo` |
|
||||
| `derived_destination_sweeps` | `derivedDestination.ts` | Dual-write active (append-only child) | — |
|
||||
| `trezor_accounts` | `trezorAccount.ts` | Dual-write active | `DualWriteTrezorAccountRepo` + `DrizzleTrezorAccountRepo` |
|
||||
| `trezor_derived_addresses` | `trezorAccount.ts` | Dual-write active (child of trezor_accounts) | — |
|
||||
| `point_transactions` | `pointTransaction.ts` | Dual-write active | `DualWritePointsRepo` + `DrizzlePointsRepo` |
|
||||
| `request_templates` | `requestTemplate.ts` | Dual-write active | `DualWriteMarketplaceRepo` |
|
||||
| `chats` | `chat.ts` | Dual-write active | `DrizzleChatRepo` |
|
||||
| `blog_posts` | `blogPost.ts` | Dual-write active | `DualWriteBlogRepo` + `DrizzleBlogRepo` |
|
||||
| `notifications` | `notification.ts` | Dual-write active | `DualWriteNotificationRepo` + `DrizzleNotificationRepo` |
|
||||
| `disputes` | `dispute.ts` | Dual-write active | `DualWriteDisputeRepo` + `DrizzleDisputeRepo` |
|
||||
| `addresses` | `address.ts` | Schema scaffolded | No dual-write repo; `addressStore.ts` reads PG directly (migration 0016) |
|
||||
| `shop_settings` | `shopSettings.ts` | Schema scaffolded | No dual-write repo |
|
||||
| `config_settings` | `configSetting.ts` | Schema scaffolded | No dual-write repo |
|
||||
| `config_setting_history` | `configSetting.ts` | PG-only | No Mongo equivalent; child of `config_settings` |
|
||||
| `telegram_links` | `telegramLink.ts` | Schema scaffolded | No dual-write repo |
|
||||
| `telegram_sessions` | `telegramSession.ts` | Schema scaffolded | No dual-write repo |
|
||||
| `reviews` | `review.ts` | Schema scaffolded | No dual-write repo |
|
||||
|
||||
> [!note] Read cutover status
|
||||
> **Dual-write active** means writes go to both Mongo and PG; reads still come from Mongo (per MEMORY.md as of 2026-06-03). **Schema scaffolded** means the Drizzle table exists but no DualWriteRepo plumbs it. **PG-only** means there is no Mongo model for that data.
|
||||
|
||||
## Shared Enum Reference
|
||||
|
||||
Enums live in `backend/src/db/schema/_enums.ts` (shared) and individual schema files. Key enums:
|
||||
|
||||
| Enum | Values |
|
||||
|---|---|
|
||||
| `user_role` | admin, buyer, seller, resolver, guard |
|
||||
| `auth_provider` | email, google, telegram |
|
||||
| `user_status` | active, suspended, deleted |
|
||||
| `purchase_request_status` | pending_payment, pending, received_offers, in_negotiation, payment_pending, payment_confirmed, in_progress, delivery, delivered, completed, disputed, refunded, seller_paid |
|
||||
| `offer_status` | pending, accepted, rejected, withdrawn, active |
|
||||
| `offer_currency` | USD, EUR, IRR, USDT, USDC, TRY |
|
||||
| `payment_provider` | request.network, amn.scanner, shkeeper, other |
|
||||
| `payment_status` | pending, processing, confirmed, completed, failed, cancelled, refunded |
|
||||
| `escrow_state` | funded, releasable, released, refunded, releasing, failed, cancelled, partial |
|
||||
| `funds_ledger_entry_type` | payment_detected, provider_fee, platform_fee, hold, release, refund, dispute_hold, adjustment |
|
||||
| `derived_destination_status` | active, swept, sweeping, quarantined |
|
||||
| `ref_kind` | entity, template |
|
||||
| `chat_type` | direct, group, support |
|
||||
| `review_subject_kind` | seller, template |
|
||||
| `address_type` | Home, Office, Other |
|
||||
| `telegram_link_source` | miniapp, bot, login_widget |
|
||||
| `telegram_link_status` | active, blocked |
|
||||
|
||||
## Lifecycle View
|
||||
|
||||
The dominant happy-path flow exercises five collections in order:
|
||||
|
||||
1. A buyer (`User`) creates a `PurchaseRequest` with `status: 'pending'`.
|
||||
2. Sellers (other `User`s) attach `SellerOffer` documents; the request transitions through `received_offers` → `in_negotiation` as the parties chat in a `Chat`.
|
||||
3. The buyer accepts an offer; a `Payment` is opened against the Request Network provider and, once verified by webhook/reconciliation and safety checks, advances to a funded escrow state.
|
||||
3. The buyer accepts an offer; a `Payment` is opened against the Request Network provider and, once verified by webhook/reconciliation and safety checks, advances to a funded escrow state. If `ORACLE_QUOTING_ENABLED=true`, a `payment_quote` row is written to PG at this point.
|
||||
4. The seller marks the request `delivery` → `delivered`; the buyer confirms with the 6-digit `deliveryCode` and the request becomes `completed`.
|
||||
5. The escrow `Payment` flips to `released` after a ledger-gated custody transfer instruction. Optionally the buyer writes a `Review` and earns a `PointTransaction`.
|
||||
5. The escrow `Payment` flips to `released` after a ledger-gated custody transfer instruction. Each ledger event appends an immutable `FundsLedgerEntry` row (Mongo + PG). Optionally the buyer writes a `Review` and earns a `PointTransaction`.
|
||||
|
||||
If anything goes sideways, the buyer can open a `Dispute`, which freezes release until an admin resolves it (refund, replacement, compensation, or no-action).
|
||||
|
||||
@@ -122,4 +255,4 @@ If anything goes sideways, the buyer can open a `Dispute`, which freezes release
|
||||
Each model has its own note in this folder. Cross-references use `[[wikilinks]]` so backlinks work in Obsidian's graph view. Schemas are documented at field-level granularity — every field is listed with its type, default, validation, and indexing decisions. Where a model carries a meaningful state machine, a Mermaid `stateDiagram-v2` accompanies the schema table.
|
||||
|
||||
> [!note] Source of truth
|
||||
> The information below is mirrored from the TypeScript schema definitions. If a field listed here disagrees with the code, the code wins — please update the note. All source citations use the form `backend/src/models/<File>.ts:<line>`.
|
||||
> The information below is mirrored from the TypeScript schema definitions. If a field listed here disagrees with the code, the code wins — please update the note. All source citations use the form `backend/src/models/<File>.ts:<line>` for Mongo and `backend/src/db/schema/<File>.ts:<line>` for Drizzle/PG.
|
||||
|
||||
@@ -1,23 +1,27 @@
|
||||
---
|
||||
title: Dispute
|
||||
tags: [data-model, mongoose]
|
||||
tags: [data-model, mongoose, postgres]
|
||||
aliases: [Complaint, IDispute]
|
||||
---
|
||||
|
||||
# Dispute
|
||||
|
||||
> **Last updated:** 2026-05-29 — aligned with code (see [Doc vs Code Audit Report](../09%20-%20Audits/Doc%20vs%20Code%20Audit%20Report%20-%202026-05-29.md))
|
||||
> **Last updated:** 2026-06-03 — added Postgres / Drizzle schema and migration status (see [Doc vs Code Audit Report](../09%20-%20Audits/Doc%20vs%20Code%20Audit%20Report%20-%202026-05-29.md))
|
||||
|
||||
Buyer-raised complaint tied to a [[PurchaseRequest]]. Captures the reason, priority, category, an array of evidence uploads, a chronological `timeline` of actions, an optional resolution, and SLA deadlines. An admin (`adminId`) is assigned during triage and resolves the dispute with a structured action (`refund`, `replacement`, `compensation`, `warning_seller`, `ban_seller`, or `no_action`).
|
||||
|
||||
> [!note] Implementation status
|
||||
> `backend/src/models/Dispute.ts`, `backend/src/services/dispute/DisputeService.ts`, `backend/src/routes/disputeRoutes.ts`, and release-hold helper routes now exist. The remaining gap is canonical state alignment between the full dispute document and the lighter `PurchaseRequest`/`Payment` hold flags used by release gating.
|
||||
>
|
||||
> Source: `backend/src/models/Dispute.ts` — schema definition and model export.
|
||||
> Sources: `backend/src/models/Dispute.ts` (Mongoose schema), `backend/src/db/schema/dispute.ts` (Drizzle/Postgres schema).
|
||||
|
||||
> ⚠️ **SECURITY** — The dispute `status` update endpoint and the `resolve` endpoint currently have **no role guards**. Any authenticated user (not just admins) can modify dispute status or submit a resolution. This is a known gap pending a role-guard audit.
|
||||
> WARNING — The dispute `status` update endpoint and the `resolve` endpoint currently have **no role guards**. Any authenticated user (not just admins) can modify dispute status or submit a resolution. This is a known gap pending a role-guard audit.
|
||||
|
||||
## Schema
|
||||
## Migration Status
|
||||
|
||||
**DUAL-WRITE** — `DualWriteDisputeRepo` + `DrizzleDisputeRepo` + `MongoDisputeRepo`. Writes go to both Mongo and Postgres. Reads still come from Mongo (cutover not yet executed).
|
||||
|
||||
## Mongo Schema
|
||||
|
||||
| Field | Type | Required | Default | Validation | Index | Description |
|
||||
| --- | --- | --- | --- | --- | --- | --- |
|
||||
@@ -57,26 +61,80 @@ Buyer-raised complaint tied to a [[PurchaseRequest]]. Captures the reason, prior
|
||||
|
||||
Valid values: `product_quality` · `delivery_delay` · `wrong_item` · `payment_issue` · `seller_behavior` · `other`
|
||||
|
||||
**Note:** `fraud` is **not** a valid category value. Use `seller_behavior` or `other` for fraud-related complaints.
|
||||
Note: `fraud` is NOT a valid category value. Use `seller_behavior` or `other` for fraud-related complaints.
|
||||
|
||||
### Status enum
|
||||
|
||||
Valid values: `pending` · `in_progress` · `waiting_response` · `resolved` · `rejected` · `closed`
|
||||
|
||||
**Note:** `under_review` does **not** exist in the schema. The equivalent lifecycle state is `in_progress`.
|
||||
Note: `under_review` does NOT exist in the schema. The equivalent lifecycle state is `in_progress`.
|
||||
|
||||
### Resolution action enum
|
||||
|
||||
Valid values: `refund` · `replacement` · `compensation` · `warning_seller` · `ban_seller` · `no_action`
|
||||
|
||||
> [!note] `messages` in the interface
|
||||
> The TypeScript interface mentions an optional embedded `messages[]` array, but the actual Mongoose schema does not declare it — messages live in [[Chat]] via `chatId`.
|
||||
Note: The TypeScript interface mentions an optional embedded `messages[]` array, but the actual Mongoose schema does not declare it — messages live in [[Chat]] via `chatId`.
|
||||
|
||||
## Postgres / Drizzle Schema
|
||||
|
||||
Source: `backend/src/db/schema/dispute.ts` — migration 0012.
|
||||
|
||||
### `disputes` table
|
||||
|
||||
| Column | PG Type | Notes |
|
||||
| --- | --- | --- |
|
||||
| `id` | `uuid` PK | Generated UUID primary key. |
|
||||
| `legacy_object_id` | `text` | Mongo ObjectId bridge; partial-unique WHERE NOT NULL. |
|
||||
| `purchase_request_id` | `text` | Stored as text (not uuid FK) to accommodate Mongo ObjectIds and PG UUIDs during cutover. No hard FK. |
|
||||
| `buyer_id` | `text` | Same cutover reason — text, no hard FK. |
|
||||
| `seller_id` | `text` | Optional; text, no hard FK. |
|
||||
| `admin_id` | `text` | Optional; text, no hard FK. |
|
||||
| `reason` | `text` | Short reason. |
|
||||
| `description` | `text` | Detailed description. |
|
||||
| `priority` | `text` | No DB-level enum; app-layer validated. |
|
||||
| `category` | `text` | No DB-level enum; app-layer validated. |
|
||||
| `status` | `text` | No DB-level enum; app-layer validated. |
|
||||
| `evidence` | `jsonb` | Array of evidence objects (serialized). |
|
||||
| `chat_id` | `text` | Optional; text reference to Chat. |
|
||||
| `messages` | `jsonb` | Embedded messages blob (conservative shim; normalization pending). |
|
||||
| `timeline` | `jsonb` | Array of timeline action objects. |
|
||||
| `resolution` | `jsonb` | Resolution object when resolved. |
|
||||
| `deadline` | `timestamptz` | Overall SLA deadline. |
|
||||
| `response_deadline` | `timestamptz` | Response SLA. |
|
||||
| `tags` | `jsonb` | Array of tag strings. |
|
||||
| `created_at` | `timestamptz` | Auto-managed. |
|
||||
| `updated_at` | `timestamptz` | Auto-managed. |
|
||||
| `closed_at` | `timestamptz` | Set when status reaches `closed`. |
|
||||
|
||||
> [!note] ID columns as `text`
|
||||
> `purchase_request_id`, `buyer_id`, `seller_id`, and `admin_id` are all stored as `text` (not `uuid` with a FK) to accommodate both legacy Mongo ObjectIds and PG UUIDs transparently during the cutover window. No referential integrity constraints exist at the DB layer for these columns.
|
||||
|
||||
> [!note] `messages` jsonb column
|
||||
> The Postgres schema includes a `messages jsonb` column that is absent from the Mongo schema (where messages live in Chat via `chatId`). This is a conservative shim added during migration scaffolding. Full normalization of chat/messages is flagged as an open blocker.
|
||||
|
||||
### Postgres Indexes
|
||||
|
||||
| Index | Type | Notes |
|
||||
| --- | --- | --- |
|
||||
| `(legacy_object_id)` WHERE NOT NULL | partial-unique | Idempotent backfill upserts. |
|
||||
| `(purchase_request_id)` | regular | Lookup by request. |
|
||||
| `(buyer_id)` | regular | Buyer's disputes. |
|
||||
| `(seller_id)` | regular | Seller's disputes. |
|
||||
| `(admin_id)` | regular | Admin workload. |
|
||||
| `(status)` | regular | Lifecycle filtering. |
|
||||
| `(priority)` | regular | Priority filtering. |
|
||||
| `(category)` | regular | Category filtering. |
|
||||
| `(created_at)` | regular | Time-ordered listing. |
|
||||
| `(status, priority)` | compound | Admin queue sort. |
|
||||
| `(admin_id, status)` | compound | Per-admin workload view. |
|
||||
|
||||
Mirrors the Mongo index set exactly.
|
||||
|
||||
## Virtuals
|
||||
|
||||
None defined.
|
||||
|
||||
## Indexes
|
||||
## Mongo Indexes
|
||||
|
||||
Defined at `backend/src/models/Dispute.ts`:
|
||||
|
||||
@@ -128,21 +186,35 @@ stateDiagram-v2
|
||||
## Common Queries
|
||||
|
||||
```ts
|
||||
// Admin queue
|
||||
// Admin queue (Mongo)
|
||||
Dispute.find({ status: { $in: ['pending', 'in_progress', 'waiting_response'] } })
|
||||
.sort({ priority: -1, createdAt: 1 });
|
||||
|
||||
// Buyer's disputes
|
||||
// Buyer's disputes (Mongo)
|
||||
Dispute.find({ buyerId }).sort({ createdAt: -1 });
|
||||
|
||||
// Seller's open disputes
|
||||
// Seller's open disputes (Mongo)
|
||||
Dispute.find({ sellerId, status: { $nin: ['resolved', 'rejected', 'closed'] } });
|
||||
|
||||
// Append timeline entry atomically
|
||||
// Append timeline entry atomically (Mongo)
|
||||
Dispute.updateOne(
|
||||
{ _id },
|
||||
{ $push: { timeline: { action, performedBy: adminId, performedAt: new Date(), details } } }
|
||||
);
|
||||
```
|
||||
|
||||
```sql
|
||||
-- Admin queue (Postgres)
|
||||
SELECT * FROM disputes
|
||||
WHERE status IN ('pending', 'in_progress', 'waiting_response')
|
||||
ORDER BY priority DESC, created_at ASC;
|
||||
|
||||
-- Buyer's disputes (Postgres)
|
||||
SELECT * FROM disputes WHERE buyer_id = $1 ORDER BY created_at DESC;
|
||||
|
||||
-- Seller's open disputes (Postgres)
|
||||
SELECT * FROM disputes
|
||||
WHERE seller_id = $1 AND status NOT IN ('resolved', 'rejected', 'closed');
|
||||
```
|
||||
|
||||
Related: [[PurchaseRequest]], [[User]], [[Chat]], [[Payment]].
|
||||
|
||||
@@ -3,56 +3,125 @@ title: Postgres Runtime Cutover Status
|
||||
tags: [data-model, postgres, migration, runtime-status]
|
||||
aliases: [Postgres Status, PG Cutover Status, Mongo vs Postgres Runtime]
|
||||
created: 2026-05-31
|
||||
source: backend integrate-main-into-development@cf59726 + frontend integrate-main-into-development@a2b972b + deployment main@8764fdf
|
||||
updated: 2026-06-03
|
||||
source: backend integrate-main-into-development@14d164c + deployment main@8764fdf
|
||||
---
|
||||
|
||||
# Postgres Runtime Cutover Status
|
||||
|
||||
> **Current branch:** backend `integrate-main-into-development` at `cf59726`, version `2.8.37`; frontend `integrate-main-into-development` at `a2b972b`, version `2.8.37`; dev deployment `main` at `8764fdf`.
|
||||
> **Current branch:** backend `integrate-main-into-development` at `14d164c`, version `2.8.56`; dev deployment `main` at `8764fdf`.
|
||||
>
|
||||
> **Bottom line:** this branch is **Postgres-capable**, not fully Postgres-backed. Dev deployment now defaults eight existing PG-capable runtime stores to Postgres: auth-owned users/Telegram auth, confirmation-threshold config/history, user addresses, categories, level config, shop settings, reviews, and notifications. Code-level defaults remain Mongo outside that deployment override, and Mongo remains the compatibility store for still-Mongo domains. The category PG path enforces one active visible category per normalized name. As of backend `2.8.37`, the active startup/health/admin/report import surface no longer has non-type top-level `mongoose` or `models/*` imports, repository factory flags accept `postgres` as an alias for `pg`, and the unmounted legacy marketplace router is no longer re-exported from the marketplace service index; legacy Mongo models are lazy-loaded only when fallback/backfill/maintenance actions run. All PG-backed stores require `PG_URL`.
|
||||
> **Bottom line:** the codebase is in **active dual-write phase**. All 11 repository domains in the factory now have Drizzle schemas, Drizzle repos, and dual-write wrappers (except Chat and ReleaseHold, which have Drizzle repos but no dual-write counterpart). 18 Drizzle migrations (0000–0017) have landed, covering every table in scope. Dev deployment defaults nine PG-capable stores to Postgres: auth-owned users/Telegram auth, confirmation-threshold config/history, user addresses, categories, level config, shop settings, reviews, notifications, and the oracle payment_quotes path. Code-level defaults remain `mongo` outside those deployment overrides. Repository factory normalizes `postgres` and `pg` as equivalent mode tokens. The unmounted legacy marketplace router is detached. As of `2.8.54–2.8.56`, the `guard` user role is in PG schema, chat routes are fixed, notifications deliver in real time, and PG response serialization/id resolution in marketplace is corrected. Reads are still Mongo-authoritative across all dual-write domains — read cutover is the remaining gate for each domain. Chat normalization (participants/messages stored as JSONB blobs, not relational child tables) remains an open blocker for full Chat cutover.
|
||||
|
||||
## Schema and Repository Coverage
|
||||
|
||||
### Tables with Full Drizzle Schema
|
||||
|
||||
All tables below have a `.ts` schema file in `src/db/schema/` and are covered by at least one migration:
|
||||
|
||||
**Infrastructure:** `id_map`, `pg_dualwrite_gaps`
|
||||
|
||||
**Auth/Users:** `users`, `user_passkeys`, `user_refresh_tokens`, `telegram_links`, `telegram_sessions`
|
||||
|
||||
**Marketplace:** `categories`, `purchase_requests`, `purchase_request_delivery_info`, `purchase_request_delivery_address`, `purchase_request_seller_delivery_info`, `purchase_request_service_info`, `purchase_request_specifications`, `purchase_request_preferred_sellers`, `delivery_attempts`, `seller_offers`, `request_templates`
|
||||
|
||||
**Payments:** `payments`, `payment_quotes`, `funds_ledger_entries`, `derived_destinations`, `derived_destination_sweeps`
|
||||
|
||||
**Points/Wallet:** `point_transactions`, `trezor_accounts`, `trezor_derived_addresses`
|
||||
|
||||
**Config/Ops:** `config_settings`, `config_setting_history`, `shop_settings`, `addresses`, `reviews`
|
||||
|
||||
**Content/Social:** `blog_posts`, `notifications`, `disputes`, `chats`
|
||||
|
||||
Total: **32 tables** across 18 migrations (0000–0017).
|
||||
|
||||
### Tables with a Drizzle Repository
|
||||
|
||||
| Drizzle Repo | Dual-Write Repo | Domain |
|
||||
|---|---|---|
|
||||
| `DrizzleUserRepo` | `DualWriteUserRepo` | Users, passkeys, refresh tokens |
|
||||
| `DrizzlePaymentRepo` | `DualWritePaymentRepo` | Payments, funds ledger |
|
||||
| `DrizzleMarketplaceRepo` | `DualWriteMarketplaceRepo` | Categories, purchase requests, seller offers, request templates |
|
||||
| `DrizzleDerivedDestinationRepo` | `DualWriteDerivedDestinationRepo` | Derived destinations, sweeps |
|
||||
| `DrizzleTrezorAccountRepo` | `DualWriteTrezorAccountRepo` | Trezor accounts, derived addresses |
|
||||
| `DrizzlePointsRepo` | `DualWritePointsRepo` | Point transactions |
|
||||
| `DrizzleNotificationRepo` | `DualWriteNotificationRepo` | Notifications |
|
||||
| `DrizzleDisputeRepo` | `DualWriteDisputeRepo` | Disputes |
|
||||
| `DrizzleBlogRepo` | `DualWriteBlogRepo` | Blog posts |
|
||||
| `DrizzleChatRepo` | _(none — no dual-write wrapper)_ | Chats (JSONB shim; Chat normalization is a blocker) |
|
||||
| `DrizzleReleaseHoldRepo` | _(none — no dual-write wrapper)_ | Release holds (bridges payments + purchase_requests) |
|
||||
|
||||
Tables with schema but no dedicated Drizzle repo yet: `addresses` (handled via addressStore facade), `shop_settings` (handled via shopSettings facade), `config_settings` / `config_setting_history` (handled via config-store facade), `telegram_links` / `telegram_sessions` (handled via auth-store facade), `reviews` (handled via review-store facade).
|
||||
|
||||
### Migration Count
|
||||
|
||||
18 migrations landed: **0000 through 0017**.
|
||||
|
||||
| Migration | Key change |
|
||||
|---|---|
|
||||
| 0000 | Core enums + `id_map` + `categories` |
|
||||
| 0001 | `trezor_accounts` + `trezor_derived_addresses` |
|
||||
| 0002 | Schema reset (drops 0000/0001 tables, adds category self-FK) |
|
||||
| 0003 | Full rebuild: all core domain tables (users, payments, marketplace, funds ledger, derived destinations, points, trezor) |
|
||||
| 0004 (×2) | Funds ledger immutability trigger; seller_offer physical FKs |
|
||||
| 0005 | `pg_dualwrite_gaps`; payment FKs; legacy_object_id uniques; pending payment index fix |
|
||||
| 0006 | budget_currency crypto-only CHECK on purchase_requests |
|
||||
| 0007 | Drops 0006 constraint; sets USDT default |
|
||||
| 0008 | `offer_currency` adds TRY; creates `payment_quotes` |
|
||||
| 0009 | Active category deduplication; `categories_active_name_norm_uq` |
|
||||
| 0010 | `request_templates`; purchase_request_specifications unique constraint |
|
||||
| 0011 | `chats` + chat enums |
|
||||
| 0012 | `disputes` |
|
||||
| 0013 | Money-integrity CHECK constraints; ledger TRUNCATE guard; id_map composite PK |
|
||||
| 0014 | Physical NOT VALID FKs across schema; validates all |
|
||||
| 0015 | Ledger immutability extended: UPDATE + DELETE triggers |
|
||||
| 0016 | `address_type` enum + `addresses` table |
|
||||
| 0017 | `guard` value added to `user_role` enum |
|
||||
|
||||
## What Uses Postgres Now
|
||||
|
||||
| Area | Runtime status | Notes |
|
||||
|---|---|---|
|
||||
| Postgres connection | Available when `PG_URL` is set | Current store facades use `src/infrastructure/postgres/client.ts`; the broader `src/db/` Drizzle layer and repository factory exist, but most live services are not wired through that factory yet. |
|
||||
| Runtime schema bootstrap | Implemented for auth, config, address, and reference stores | Auth tables are bootstrapped from `src/services/auth/postgresAuthSchema.ts`; store facades bootstrap their own tables at startup when their `*_STORE=postgres` flag is enabled. |
|
||||
| Health observability | Implemented in `/api/health` | `checks.postgres` reports `configured`, `required`, `storeModes`, `enabledStores`, and `enabledStoreCount`, so Gatus/operators can verify both PG reachability and which runtime stores are actively PG-backed. Dev Gatus now asserts all eight dev PG-backed store modes are `postgres`, including notifications. Backend `2.8.33` lazy-loads Mongoose for the legacy Mongo health check and skips it when Mongo is optional under `MONGO_CONNECT_MODE=auto/never`. |
|
||||
| Auth-owned user store | PG-backed in dev deployment; code opt-in with `AUTH_STORE=postgres` | Auth, passkey, Telegram auth/link/session/temp-verification, and `/api/user` profile paths use an auth-store facade. In PG mode, users are stored in Postgres and mirrored back to Mongo through `legacy_object_id` for compatibility with still-Mongo services. |
|
||||
| Confirmation-threshold runtime config | PG-backed in dev deployment; code opt-in with `CONFIG_STORE=postgres` | `ConfigSetting` / `ConfigSettingHistory` access for `/api/admin/settings/confirmation-thresholds` and transaction-safety confirmation thresholds uses a config-store facade. PG-mode writes mirror back to Mongo for rollback. Backend `2.8.32` removed top-level Mongo model imports from this facade; legacy models load only for Mongo fallback/backfill/mirror paths. |
|
||||
| User addresses | PG-backed in dev deployment; code opt-in with `ADDRESS_STORE=postgres` | `/api/addresses` CRUD uses an address-store facade. PG mode enforces one primary address per user with a partial unique index and mirrors writes/deletes back to Mongo for rollback. |
|
||||
| Marketplace categories | PG-backed in dev deployment; code opt-in with `CATEGORY_STORE=postgres` | `CategoryService` and the default `General` category path use a category-store facade. PG-mode writes mirror back to Mongo for rollback and still-Mongo request/template references. PG schema bootstrap/migration deactivates duplicate active category labels, repoints existing category references to the kept row, and enforces `categories_active_name_norm_uq` on `lower(btrim(name)) WHERE is_active = true`. List/cache reads also dedupe by normalized name. |
|
||||
| Level configuration | PG-backed in dev deployment; code opt-in with `LEVEL_CONFIG_STORE=postgres` | `PointsService` level reads use a level-config facade. `PointTransaction` and user points remain Mongo-backed. |
|
||||
| Shop settings | PG-backed in dev deployment; code opt-in with `SHOP_SETTINGS_STORE=postgres` | Shop settings controller, seller payment rail resolution, and review enable/disable checks use a shop-settings facade. PG-mode writes mirror back to Mongo. Backend `2.8.32` removed top-level Mongo model imports from this facade; legacy models load only for Mongo fallback/backfill/mirror paths. |
|
||||
| Marketplace reviews | PG-backed in dev deployment; code opt-in with `REVIEW_STORE=postgres` | Review list/summary/create routes use a review-store facade. PG-mode list responses still hydrate `reviewerId` from the user mirror to preserve frontend shape. Backend `2.8.32` removed top-level Mongo model imports from this facade; legacy models load only for Mongo fallback/backfill/mirror paths. |
|
||||
| Notifications | PG-backed in dev deployment; code opt-in with `NOTIFICATION_STORE=postgres` or `REPO_NOTIFICATION=pg` | `NotificationService` uses `getNotificationRepo()` for create/list/read/delete/count paths. Backend `2.8.34` adds Mongo→Postgres notification backfill tooling, ordered-runner support, a dry-run path, and `scripts/smoke/notifications-postgres.sh`. Deployment `8764fdf` defaults `NOTIFICATION_STORE=postgres` in dev and Gatus requires the notification store mode. Backend `2.8.37` fixes repository mode aliasing so this `postgres` store flag resolves to the Drizzle notification repo rather than Mongo. |
|
||||
| Repository implementations | Present with first payment-ledger runtime seam | `src/db/repositories/*` and Drizzle schemas exist for the target architecture. Backend `2.8.20` wires `fundsLedgerService` appends/balance reads through `getPaymentRepo()`, making that ledger slice controllable by `REPO_PAYMENT=mongo|dual|pg`. The broader payment, marketplace, and points services still need method-by-method service wiring before their repo flags are safe runtime cutovers. |
|
||||
| Postgres connection | Available when `PG_URL` is set | Store facades use `src/infrastructure/postgres/client.ts`; the broader `src/db/` Drizzle layer and repository factory are fully populated. |
|
||||
| Runtime schema bootstrap | Implemented for auth, config, address, and reference stores | Auth tables bootstrapped from `src/services/auth/postgresAuthSchema.ts`; store facades bootstrap their own tables at startup when their `*_STORE=postgres` flag is enabled. |
|
||||
| Health observability | Implemented in `/api/health` | `checks.postgres` reports `configured`, `required`, `storeModes`, `enabledStores`, and `enabledStoreCount`. Dev Gatus asserts all dev PG-backed store modes are `postgres`, including notifications. Mongoose health check is lazy-loaded and skipped when Mongo is optional under `MONGO_CONNECT_MODE=auto/never`. |
|
||||
| Auth-owned user store | PG-backed in dev deployment; code opt-in with `AUTH_STORE=postgres` | Auth, passkey, Telegram auth/link/session/temp-verification, and `/api/user` profile paths use an auth-store facade. PG-mode users are mirrored back to Mongo through `legacy_object_id` for compatibility with still-Mongo services. |
|
||||
| Confirmation-threshold runtime config | PG-backed in dev deployment; code opt-in with `CONFIG_STORE=postgres` | `ConfigSetting` / `ConfigSettingHistory` access for `/api/admin/settings/confirmation-thresholds` and transaction-safety confirmation thresholds uses a config-store facade. PG-mode writes mirror back to Mongo. Legacy models load only for Mongo fallback/backfill/mirror paths. |
|
||||
| User addresses | PG-backed in dev deployment; code opt-in with `ADDRESS_STORE=postgres` | `/api/addresses` CRUD uses an address-store facade. PG mode enforces one primary address per user with a partial unique index and mirrors writes/deletes back to Mongo. |
|
||||
| Marketplace categories | PG-backed in dev deployment; code opt-in with `CATEGORY_STORE=postgres` | `CategoryService` and the default `General` category path use a category-store facade. PG-mode writes mirror back to Mongo. Migration 0009 deactivated duplicate active category labels and enforces `categories_active_name_norm_uq` on `lower(btrim(name)) WHERE is_active = true`. |
|
||||
| Level configuration | PG-backed in dev deployment; code opt-in with `LEVEL_CONFIG_STORE=postgres` | `PointsService` level reads use a level-config facade. `PointTransaction` and user points remain Mongo-backed. `LEVEL_STORE=postgres` accepted as compatibility alias. |
|
||||
| Shop settings | PG-backed in dev deployment; code opt-in with `SHOP_SETTINGS_STORE=postgres` | Shop settings controller, seller payment rail resolution (`2.8.56` fixes seller shop lookup to handle both uuid and legacy id formats), and review enable/disable checks use a shop-settings facade. PG-mode writes mirror back to Mongo. |
|
||||
| Marketplace reviews | PG-backed in dev deployment; code opt-in with `REVIEW_STORE=postgres` | Review list/summary/create routes use a review-store facade. PG-mode list responses still hydrate `reviewerId` from the user mirror to preserve frontend shape. |
|
||||
| Notifications | PG-backed in dev deployment; code opt-in with `NOTIFICATION_STORE=postgres` or `REPO_NOTIFICATION=pg` | `NotificationService` uses `getNotificationRepo()` for create/list/read/delete/count paths. `2.8.55` fixes chat routes and delivers notifications in real time. `2.8.37` fixes repository mode aliasing so `postgres` resolves to the Drizzle notification repo. Backfill script (`npm run backfill:notification:postgres`) and smoke script (`scripts/smoke/notifications-postgres.sh`) are available. |
|
||||
| Oracle quote persistence | Conditional runtime PG write | `/api/payment/request-network/intents` lazily imports `quoteRepo` only when `ORACLE_QUOTING_ENABLED=true`; it writes `payment_quotes` if the PG parent payment row exists, mirrors to Mongo `Payment.quote`, and records `pg_dualwrite_gaps` if PG is behind. |
|
||||
| Backfill/verify scripts | Available as operator tooling | `MIGRATION_PG_URL` drives backfill scripts; guards restrict allowed target hosts. The marketplace-core runner group now backfills users/categories, request templates, purchase requests, seller offers, and the post-offer `selectedOfferId` remap in dependency order. These scripts are not run automatically by app startup. |
|
||||
| Funds ledger | Repository-backed, default Mongo | `appendFundsLedgerEntry` and `getFundsBalanceBy*` call `getPaymentRepo()`. Default is `MongoPaymentRepo`; `REPO_PAYMENT=dual`/`pg` exercises the Drizzle ledger after backfill/soak. |
|
||||
| Backfill/verify scripts | Available as operator tooling | `MIGRATION_PG_URL` drives all backfill scripts; guards restrict allowed target hosts. Marketplace-core runner backfills users/categories, request templates, purchase requests, seller offers, and `selectedOfferId` remap in dependency order. Not run automatically at startup. |
|
||||
| Guard user role | PG schema-ready | Migration 0017 adds `guard` to the `user_role` enum. `2.8.54` adds guard role support across auth and user management. |
|
||||
| PG response serialization | Fixed in `2.8.51–2.8.53` | PG response serialization and id resolution in marketplace purchase-request paths corrected; user creation and purchase request unblocked from a PG FK constraint error. |
|
||||
| Admin user management | PG-capable as of `2.8.50` | Admin user count queries route through postgres-capable stores; admin user management works end-to-end under PG. |
|
||||
| Seeds | Postgres-capable as of `2.8.47` | Seeds in `src/seeds/*` are store-aware and idempotent; can seed fresh PG under `MONGO_CONNECT_MODE=never`. |
|
||||
|
||||
## What Is Still Mongo-Backed
|
||||
|
||||
Active app startup, health, and the PG-capable store facades no longer top-load Mongoose models. Auth-owned paths now have an auth-store boundary; confirmation-threshold config, user addresses, categories, level config, shop settings, reviews, and notifications have store/repository boundaries, and their PG-capable facades lazy-load legacy Mongo fallbacks instead of top-loading them. Funds ledger appends/balance reads now use the payment repository seam, but default to Mongo unless `REPO_PAYMENT` is flipped. Broad marketplace requests/offers/templates, most payment paths, points transactions, chat, and admin maintenance actions remain Mongo-first when exercised.
|
||||
Writes across all dual-write domains go to both Mongo and Postgres. Reads remain Mongo-authoritative for every dual-write domain — read cutover has not been performed for any domain. Chat is repository-backed (Drizzle repo exists) but participants/messages are stored as JSONB blobs rather than normalized child tables; Chat normalization is the primary structural blocker for Chat read cutover.
|
||||
|
||||
| Domain | Current live store | Why not Postgres yet |
|
||||
|---|---|---|
|
||||
| Legacy/broad user consumers | MongoDB mirror | Auth-owned users can be PG-backed, but still-Mongo domains expect Mongo ObjectId user references. PG-mode writes therefore maintain a Mongo mirror until those domains are cut over. |
|
||||
| Admin cleanup / seed address tooling | MongoDB | User-facing address CRUD is PG-capable, but admin cleanup and seed scripts still operate on Mongo first. Backend `2.8.33` makes admin cleanup lazy-load its Mongo models only when cleanup/stat/user-data maintenance actions run. Seed scripts backfill addresses to PG when `ADDRESS_STORE=postgres`. |
|
||||
| Marketplace requests/offers/templates | Repository-backed, default Mongo | Controller/service paths route through `getMarketplaceRepo()`, and PurchaseRequest/SellerOffer/RequestTemplate backfill tooling is operator-ready. `REPO_MARKETPLACE` still defaults to Mongo, the old unmounted route file remains present for now, and full PG/dual marketplace runtime cutover still needs route/service smoke coverage before flipping. Backend `2.8.36` stops re-exporting the old Mongo-heavy `marketplaceRouter` from the marketplace service index. |
|
||||
| Payments and escrow state | MongoDB primary | Request Network, AMN scanner, webhook, admin, release/refund, adapter, reconciliation, and legacy payment paths still create/update `Payment` Mongoose documents directly. Payment repository methods exist but are not broadly wired into runtime services yet. The SHKeeper migration report lazy-loads `Payment` and `FundsLedgerEntry` as of backend `2.8.33`, but the report remains Mongo-backed. |
|
||||
| Funds ledger | Repository-backed, default Mongo | `appendFundsLedgerEntry` and `getFundsBalanceBy*` now call `getPaymentRepo()`. In default mode that is `MongoPaymentRepo`; `REPO_PAYMENT=dual`/`pg` can exercise the PG ledger implementation after backfill/soak. Drizzle balance reads support both UUID refs and external/string refs used by template checkout. |
|
||||
| Derived destinations and sweeps | Repository-backed, default Mongo | Wallet destination allocation and sweep paths use `getDerivedDestinationRepo()` / `getPaymentRepo()`, but `REPO_DERIVED_DESTINATION` still defaults to Mongo and has not been flipped in dev. |
|
||||
| Points/referrals/transactions | Repository-backed, default Mongo | `PointsService` uses `getPointsRepo()` and level configuration is PG-capable, but `REPO_POINTS` defaults to Mongo and point transaction/user-point flows have not been flipped in dev. |
|
||||
| Chat/messages | Repository-backed, default Mongo | Chat service uses `getChatRepo()`, but `REPO_CHAT` / `CHAT_STORE` defaults to Mongo and chat is still treated as a document-shaped domain until a deliberate PG cutover. |
|
||||
| Notifications | PG-backed in dev deployment; code default Mongo | `NotificationService` routes through `getNotificationRepo()`, so `NOTIFICATION_STORE=postgres` / `REPO_NOTIFICATION=pg` can exercise the Drizzle repo. Backend `2.8.34` adds backfill and smoke coverage; deployment `8764fdf` includes notification in the dev PG baseline; backend `2.8.37` ensures `postgres` is a valid repo-mode alias. |
|
||||
| Disputes/blog/content/admin cleanup | Mixed | Disputes and blog are repository-backed, but their code defaults still resolve to Mongo until `REPO_DISPUTE`/`BLOG_STORE` are flipped. Admin cleanup still lazy-loads and calls Mongoose models directly for maintenance cleanup/stat/user-data actions. |
|
||||
| Runtime config outside confirmation thresholds | MongoDB | `ConfigSetting` and `ConfigSettingHistory` are PG-capable for confirmation thresholds only; any future admin-editable settings need to route through the same config-store boundary before they count as cut over. |
|
||||
| Telegram link/session/temp verification | PG-backed in dev deployment; code default MongoDB | These records move with `AUTH_STORE=postgres`. Dev compose defaults that flag to `postgres`; environments without the override remain Mongo until the flag is flipped. |
|
||||
| User reads | MongoDB authoritative | Auth-owned users can be PG-backed for writes, but reads remain Mongo-authoritative. Still-Mongo domains expect Mongo ObjectId user references; PG-mode writes maintain a Mongo mirror until all consumers cut over. |
|
||||
| Admin cleanup / seed address tooling | MongoDB | User-facing address CRUD is PG-capable, but admin cleanup scripts still operate on Mongo first. Seed scripts backfill addresses to PG when `ADDRESS_STORE=postgres`. |
|
||||
| Marketplace requests/offers/templates reads | Repository-backed writes; Mongo reads | `getMarketplaceRepo()` wires all marketplace writes. `REPO_MARKETPLACE` defaults to Mongo; full PG/dual read cutover needs smoke coverage before flipping. The legacy `marketplaceRouter` is detached from the service index (`2.8.36`). |
|
||||
| Payments and escrow state reads | MongoDB primary | Request Network, AMN scanner, webhook, admin, release/refund, adapter, reconciliation, and legacy payment paths still create/update `Payment` Mongoose documents directly for reads. Payment Drizzle repo and dual-write repo exist; `REPO_PAYMENT=dual` enables dual-writes, but read paths remain Mongo. |
|
||||
| Derived destinations and sweeps | Repository-backed writes; Mongo reads | `getDerivedDestinationRepo()` wires writes; `REPO_DERIVED_DESTINATION` defaults to Mongo and has not been flipped in dev. |
|
||||
| Points/referrals/transactions | Repository-backed writes; Mongo reads | `getPointsRepo()` wires writes; `REPO_POINTS` defaults to Mongo. Level config is PG-capable for reads; point transaction and user-point flows are not flipped in dev. |
|
||||
| Chat/messages | Repository-backed writes (JSONB shim); Mongo reads | `getChatRepo()` wires writes. `REPO_CHAT` / `CHAT_STORE` defaults to Mongo. `DrizzleChatRepo` uses JSONB blobs for participants and messages — Chat normalization into relational child tables is required before safe read cutover. No dual-write wrapper exists. |
|
||||
| Disputes/blog | Repository-backed writes; Mongo reads | Both have Drizzle repos and dual-write wrappers. Code defaults resolve to Mongo until `REPO_DISPUTE`/`BLOG_STORE` are flipped. |
|
||||
| ReleaseHold | Drizzle repo only; default Mongo | `getReleaseHoldRepo()` returns Drizzle or Mongo; no dual-write wrapper — `dual` mode silently uses Mongo. Separate cutover needed. |
|
||||
| Runtime config outside confirmation thresholds | MongoDB | `ConfigSetting` and `ConfigSettingHistory` are PG-capable for confirmation thresholds only; other admin-editable settings need to route through the config-store boundary before counting as cut over. |
|
||||
| Telegram link/session/temp verification reads | PG-backed writes in dev; Mongo reads in code default | These records move with `AUTH_STORE=postgres`. Dev compose defaults that flag to `postgres`; read cutover follows the auth-store flag. |
|
||||
|
||||
## Env Flag Reality
|
||||
|
||||
The backend code defaults every store flag below to `mongo`. Dev deployment overrides eight PG-capable store flags to `postgres` in `deployment/docker-compose.yml` as of `deployment@8764fdf`.
|
||||
The backend code defaults every store flag below to `mongo`. Dev deployment overrides eight PG-capable store flags to `postgres` in `deployment/docker-compose.yml`. Repository factory normalizes `postgres` and `pg` as equivalent.
|
||||
|
||||
| Flag | Current meaning |
|
||||
|---|---|
|
||||
@@ -60,21 +129,50 @@ The backend code defaults every store flag below to `mongo`. Dev deployment over
|
||||
| `CONFIG_STORE` | Code default `mongo`; dev deployment default `postgres`. Routes confirmation-threshold settings/history through Postgres. |
|
||||
| `ADDRESS_STORE` | Code default `mongo`; dev deployment default `postgres`. Routes `/api/addresses` through Postgres. |
|
||||
| `CATEGORY_STORE` | Code default `mongo`; dev deployment default `postgres`. Routes marketplace category reads/writes through Postgres. Active PG categories are unique by normalized visible name. |
|
||||
| `LEVEL_CONFIG_STORE` | Code default `mongo`; dev deployment default `postgres`. Routes level configuration reads and seed replacement through Postgres. `LEVEL_STORE=postgres` is accepted as a compatibility alias. |
|
||||
| `SHOP_SETTINGS_STORE` | Code default `mongo`; dev deployment default `postgres`. Routes shop settings, review gates, and seller payment rails through Postgres. |
|
||||
| `LEVEL_CONFIG_STORE` | Code default `mongo`; dev deployment default `postgres`. Routes level configuration reads and seed replacement through Postgres. `LEVEL_STORE=postgres` accepted as compatibility alias. |
|
||||
| `SHOP_SETTINGS_STORE` | Code default `mongo`; dev deployment default `postgres`. Routes shop settings, review gates, and seller payment rails through Postgres. `2.8.56` fixes seller shop lookup tolerance for uuid vs legacy id formats. |
|
||||
| `REVIEW_STORE` | Code default `mongo`; dev deployment default `postgres`. Routes marketplace reviews through Postgres. |
|
||||
| `NOTIFICATION_STORE` / `REPO_NOTIFICATION` | Code default `mongo`; dev deployment default `postgres`. Routes notification inbox create/list/read/delete/count through the Drizzle notification repo. Backend `2.8.34` adds `npm run backfill:notification:postgres`, ordered-runner step `notifications`, and `scripts/smoke/notifications-postgres.sh`; backend `2.8.37` makes repository factory flags accept both `postgres` and `pg`. |
|
||||
| `NOTIFICATION_STORE` / `REPO_NOTIFICATION` | Code default `mongo`; dev deployment default `postgres`. Routes notification inbox create/list/read/delete/count through the Drizzle notification repo. `postgres` and `pg` both resolve correctly since `2.8.37`. |
|
||||
| `PG_URL` | Makes PG code importable/reachable. Required for any `*_STORE=postgres` flag; does not cut over unrelated app domains by itself. |
|
||||
| `MIGRATION_PG_URL` | Used by backfill scripts and migration runbooks; not part of normal request handling. Marketplace-core dry-run/non-dry backfills also require `MIGRATION_MONGO_URL`. |
|
||||
| `REPO_PAYMENT` | Code default `mongo`. As of backend `2.8.20`, funds ledger appends and balance reads use this flag through `getPaymentRepo()`. Do not flip broad payment runtime to `pg` yet; most payment services still call Mongoose directly. |
|
||||
| `REPO_USER`, `REPO_POINTS`, `REPO_MARKETPLACE`, `REPO_DEFAULT` | Repository factory flags exist, but broad services are not yet wired through the factory. Treat them as migration controls that need integration verification before relying on them. The factory lazy-loads PG/dual implementations so importing it in Mongo mode does not require `PG_URL`; as of backend `2.8.37`, `postgres` and `pg` both resolve to PG mode. |
|
||||
| `REPO_RELEASE_HOLD` / `RELEASE_HOLD_STORE` | Code default `mongo`. Release-hold mode must be flipped explicitly; backend `2.8.37` removed the previous fallback where `REPO_DISPUTE=pg` also made release holds look PG-backed. |
|
||||
| `ORACLE_QUOTING_ENABLED` | Enables server-side quote computation and the only current PG write path in normal checkout: `payment_quotes`, when a PG parent row can be resolved. |
|
||||
| `REPO_PAYMENT` | Code default `mongo`. Funds ledger appends and balance reads route through this flag. `dual` mode enables dual-write for the ledger seam. Do not flip broad payment runtime to `pg` yet; most payment services still call Mongoose directly for reads. |
|
||||
| `REPO_MARKETPLACE` | Code default `mongo`. All marketplace writes route through `getMarketplaceRepo()`. Full read cutover needs smoke coverage. |
|
||||
| `REPO_USER`, `REPO_POINTS`, `REPO_DERIVED_DESTINATION`, `REPO_TREZOR` | Factory flags with full trio (mongo/dual/pg). Writes are wired; reads remain Mongo-authoritative until each flag is flipped and verified. |
|
||||
| `REPO_DISPUTE` / `DISPUTE_STORE`, `REPO_BLOG` / `BLOG_STORE` | Code default `mongo`. Dual-write wrappers exist; flip per-domain after verification. |
|
||||
| `REPO_CHAT` / `CHAT_STORE` | Code default `mongo`. Chat normalization (JSONB→relational) is a structural blocker; no dual-write wrapper. |
|
||||
| `REPO_RELEASE_HOLD` / `RELEASE_HOLD_STORE` | Code default `mongo`. No dual-write wrapper; `dual` silently uses Mongo. Must be flipped explicitly. As of `2.8.37`, `REPO_DISPUTE=pg` no longer leaks into release hold mode. |
|
||||
| `ORACLE_QUOTING_ENABLED` | Enables server-side quote computation and the `payment_quotes` PG write in checkout when a PG parent payment row exists. |
|
||||
| `MONGO_CONNECT_MODE` | Handled in Mongoose connection setup (not in the repository factory). `auto`/`never` allow PG-only boot; lazy Mongoose health check skips when Mongo is optional. |
|
||||
|
||||
## Overall Migration Phase
|
||||
|
||||
| Phase | Status |
|
||||
|---|---|
|
||||
| Schema design | Complete — 32 tables, 18 migrations (0000–0017) |
|
||||
| Drizzle repos | Complete — all 11 factory domains have a Drizzle repo |
|
||||
| Dual-write wrappers | Mostly complete — 9 of 11 domains have a dual-write wrapper (Chat and ReleaseHold are exceptions) |
|
||||
| Write cutover (dual-write active) | Not yet enabled by default — `REPO_DEFAULT` is still `mongo`; must be flipped per-domain with care |
|
||||
| Read cutover | Not started for any domain — Mongo remains authoritative for all reads |
|
||||
| Prod backfill | Not run — backfill scripts are operator-ready but not executed against production |
|
||||
| Chat normalization | Blocked — participants/messages stored as JSONB; relational normalization required before Chat read cutover |
|
||||
|
||||
Estimated overall: **schema and infrastructure phase complete; write-seam phase substantially complete; read cutover and backfill execution remain for every domain.**
|
||||
|
||||
## Recent Progress Since Last Update (2.8.37 → 2.8.56)
|
||||
|
||||
- **2.8.38–2.8.46:** Complete dual-write repos for all remaining domains; Drizzle migrations pipeline finalized; TTL scheduler added; shop lookup bug-fixed.
|
||||
- **2.8.47:** Seeds made Postgres-capable and idempotent for PG-only boot (`MONGO_CONNECT_MODE=never`).
|
||||
- **2.8.48–2.8.49:** Fresh-DB PG migrate + seed path corrected; 0013/0014 migrations made valid for a fresh `drizzle-kit migrate` run.
|
||||
- **2.8.50:** Admin user counts routed through postgres-capable stores; admin user management works end-to-end under PG.
|
||||
- **2.8.51–2.8.53:** PG response serialization and id resolution corrected in marketplace; user creation and purchase request creation unblocked from PG FK constraint errors.
|
||||
- **2.8.54:** `guard` user role added to `user_role` enum (migration 0017); guard role support across auth and user management.
|
||||
- **2.8.55:** Chat routes fixed; notifications delivered in real time alongside chat.
|
||||
- **2.8.56:** Seller shop lookup made tolerant of both uuid and legacy id formats; `dataCleanupService` guarded against `MONGO_CONNECT_MODE=never`.
|
||||
|
||||
## Next Cutover Work
|
||||
|
||||
1. Apply Drizzle migrations to the target Postgres database.
|
||||
2. For dev/test data, either run the existing backfills below or reseed acceptable test data before relying on the PG-backed stores. The deployment default flip does not move historical Mongo rows by itself.
|
||||
1. Apply Drizzle migrations to the target Postgres database (0000–0017 must be in-order; 0002 is a reset migration — confirm idempotency on existing instances).
|
||||
2. For dev/test data, run the existing backfills or reseed acceptable test data before relying on PG-backed stores. The deployment default flip does not move historical Mongo rows.
|
||||
3. For auth cutover, run `PG_URL=... npm run backfill:auth:postgres`, verify counts, and confirm `AUTH_STORE=postgres` in the target runtime.
|
||||
4. For confirmation-threshold config cutover, run `PG_URL=... npm run backfill:config:postgres`, verify counts/history, and confirm `CONFIG_STORE=postgres`.
|
||||
5. For address cutover, run `PG_URL=... npm run backfill:address:postgres`, verify one-primary invariants, and confirm `ADDRESS_STORE=postgres`.
|
||||
@@ -83,16 +181,17 @@ The backend code defaults every store flag below to `mongo`. Dev deployment over
|
||||
- `PG_URL=... npm run backfill:level-config:postgres`
|
||||
- `PG_URL=... npm run backfill:shop-settings:postgres`
|
||||
- `PG_URL=... npm run backfill:review:postgres`
|
||||
7. Run `PG_URL=... scripts/smoke/categories-postgres-unique.sh` and `PG_URL=... MONGODB_URI=... scripts/smoke/reference-stores-postgres.sh`, then confirm `CATEGORY_STORE=postgres LEVEL_CONFIG_STORE=postgres SHOP_SETTINGS_STORE=postgres REVIEW_STORE=postgres` in non-prod.
|
||||
8. For marketplace-core data, run `MIGRATION_MONGO_URL=... MIGRATION_PG_URL=... npm run backfill:marketplace-core:postgres:dry-run`, then the non-dry `npm run backfill:marketplace-core:postgres` against non-prod. The group runs root dependencies, RequestTemplate rows, PurchaseRequest main rows, SellerOffer rows, then the selected-offer remap.
|
||||
7. Run `PG_URL=... scripts/smoke/categories-postgres-unique.sh` and `PG_URL=... MONGODB_URI=... scripts/smoke/reference-stores-postgres.sh`, then confirm reference store flags in non-prod.
|
||||
8. For marketplace-core data, run `MIGRATION_MONGO_URL=... MIGRATION_PG_URL=... npm run backfill:marketplace-core:postgres:dry-run`, then the non-dry run. The group runs root dependencies, RequestTemplate rows, PurchaseRequest main rows, SellerOffer rows, then the selected-offer remap.
|
||||
9. Run `scripts/smoke/marketplace-core-postgres-backfill.sh` with the same migration DSNs and record row-count/checksum results.
|
||||
10. For notifications, run `PG_URL=... npm run backfill:notification:postgres` and `PG_URL=... scripts/smoke/notifications-postgres.sh` against dev to validate the new default.
|
||||
11. Continue payment-domain wiring after the ledger seam: add the missing payment repo methods for provider lookups, transaction-hash/webhook lookups, metadata/blockchain patching, template duplicate cleanup, and quote updates before moving `paymentService`, `paymentCoordinator`, RN, or AMN scanner routes.
|
||||
12. Add a derived-destination/sweep repository seam before payment PG cutover; destination allocation is payment-address state and should not stay Mongo-only once payments become PG-backed.
|
||||
13. Wire remaining services to repository interfaces one domain at a time.
|
||||
14. Enable `dual` mode per large domain only after wiring is proven by tests and smoke checks.
|
||||
15. Run shadow-read/reconcile during a soak window.
|
||||
16. Flip reads to `pg` per domain only after zero-diff shadow reads and a rollback plan are in place.
|
||||
10. For notifications, run `PG_URL=... npm run backfill:notification:postgres` and `PG_URL=... scripts/smoke/notifications-postgres.sh` against dev to validate the current default.
|
||||
11. Enable `REPO_MARKETPLACE=dual`, `REPO_PAYMENT=dual`, `REPO_POINTS=dual`, `REPO_DERIVED_DESTINATION=dual`, `REPO_TREZOR=dual`, `REPO_DISPUTE=dual`, `REPO_BLOG=dual` one domain at a time after backfill verification, and run a soak window before flipping reads.
|
||||
12. Continue payment-domain wiring: add missing payment repo methods for provider lookups, transaction-hash/webhook lookups, metadata/blockchain patching, template duplicate cleanup, and quote updates before moving `paymentService`, `paymentCoordinator`, RN, or AMN scanner routes.
|
||||
13. Add a derived-destination/sweep repository seam before payment PG read cutover; destination allocation is payment-address state and should not remain Mongo-only once payments become PG-backed for reads.
|
||||
14. Resolve Chat normalization: design relational child tables for participants and messages; migrate `DrizzleChatRepo` away from JSONB blobs; add `DualWriteChatRepo`; flip `REPO_CHAT=dual` only after normalization.
|
||||
15. Add `DualWriteReleaseHoldRepo`; flip `REPO_RELEASE_HOLD=dual` explicitly after wiring is proven.
|
||||
16. Flip reads to `pg` per domain only after zero-diff shadow reads and a documented rollback plan are in place.
|
||||
17. Run prod backfill under a maintenance window with `MIGRATION_PG_URL` pointing at prod; validate row counts before cutting over reads.
|
||||
|
||||
## Related Docs
|
||||
|
||||
|
||||
@@ -1,20 +1,28 @@
|
||||
---
|
||||
title: PurchaseRequest
|
||||
tags: [data-model, mongoose]
|
||||
tags: [data-model, mongoose, postgres, drizzle]
|
||||
aliases: [Purchase Request, Buy Request, IPurchaseRequest]
|
||||
---
|
||||
|
||||
# PurchaseRequest
|
||||
|
||||
> **Last updated:** 2026-05-31 — `budget.currency` aligned with template/Postgres enum (`USD`, `EUR`, `IRR`, `USDT`, `USDC`); template checkout now preserves seller-owned delivery mode and overlays buyer address/email.
|
||||
> **Last updated:** 2026-06-03 — added Postgres / Drizzle schema section, child-table breakdowns, migration status, and dispute/escrow hold fields present in both Mongo and PG schemas.
|
||||
|
||||
The central buyer-side document. A `PurchaseRequest` captures what a buyer wants to acquire (physical product, digital product, service, or consultation), the budget envelope, urgency, delivery details, and the entire lifecycle from creation through payment, delivery, and completion. Sellers respond by attaching [[SellerOffer]] documents; the buyer accepts one, a [[Payment]] is opened, and delivery is verified by a 6-digit code.
|
||||
|
||||
> [!note] Source
|
||||
> `backend/src/models/PurchaseRequest.ts:95` — schema definition
|
||||
> `backend/src/models/PurchaseRequest.ts:387` — model export
|
||||
> [!note] Sources
|
||||
> Mongo model: `backend/src/models/PurchaseRequest.ts:95` — schema definition; `:387` — model export
|
||||
> Drizzle schema: `backend/src/db/schema/purchaseRequest.ts`
|
||||
|
||||
## Schema
|
||||
## Migration Status
|
||||
|
||||
**DUAL-WRITE active** — part of `DualWriteMarketplaceRepo`. Writes go to both Mongo and Postgres; reads still come from Mongo. Backfill and read-cutover are human-gated and not yet executed.
|
||||
|
||||
---
|
||||
|
||||
## Mongo Schema
|
||||
|
||||
### Fields
|
||||
|
||||
| Field | Type | Required | Default | Validation | Index | Description |
|
||||
| --- | --- | --- | --- | --- | --- | --- |
|
||||
@@ -74,12 +82,18 @@ The central buyer-side document. A `PurchaseRequest` captures what a buyer wants
|
||||
| `serviceInfo.location` | String | no | — | trim, maxlength 200 | — | Service location. |
|
||||
| `serviceInfo.requirements[]` | String[] | no | — | trim | — | Pre-requisites. |
|
||||
| `attachments[]` | String[] | no | — | — | — | Attached file URLs. |
|
||||
| `offers[]` | ObjectId → [[SellerOffer]] | no | — | — | — | Offers received. |
|
||||
| `offers[]` | ObjectId → [[SellerOffer]] | no | — | — | — | Offers received. **Dropped in PG** — query `SellerOffer WHERE purchase_request_id = ?` instead. |
|
||||
| `selectedOfferId` | ObjectId → [[SellerOffer]] | no | `null` | — | — | Accepted offer. |
|
||||
| `rating` | Number | no | `null` | min 1, max 5 | — | Buyer's post-delivery rating. |
|
||||
| `feedback` | String | no | `null` | maxlength 1000 | — | Buyer's feedback text. |
|
||||
| `deliveryConfirmed` | Boolean | no | `false` | — | — | Buyer confirmation flag. |
|
||||
| `deliveryConfirmedAt` | Date | no | `null` | — | — | Confirmation timestamp. |
|
||||
| `disputeRaised` | Boolean | no | `false` | — | — | Escrow: whether a dispute has been raised. |
|
||||
| `disputeRaisedAt` | Date | no | `null` | — | — | When the dispute was raised. |
|
||||
| `disputeResolved` | Boolean | no | `false` | — | — | Escrow: whether dispute is resolved. |
|
||||
| `disputeResolvedAt` | Date | no | `null` | — | — | When it was resolved. |
|
||||
| `disputeHoldReason` | String | no | `null` | — | — | Human-readable hold reason. |
|
||||
| `holdUntil` | Date | no | `null` | — | — | Escrow hold expiry; partial index in PG for expiry sweeps. |
|
||||
| `metadata.source` | String | no | `manual` | enum: `manual` / `template` / `api` | — | Where the request came from. |
|
||||
| `metadata.templateId` | String | no | — | trim | — | Originating [[RequestTemplate]] id. |
|
||||
| `metadata.version` | String | no | — | trim | — | Schema version. |
|
||||
@@ -92,13 +106,13 @@ The central buyer-side document. A `PurchaseRequest` captures what a buyer wants
|
||||
|
||||
**Note:** `finalized` and `archived` are **not** valid status values and do not appear in the `IPurchaseRequest` frontend type or the Mongoose schema enum. Using either would cause a validation error.
|
||||
|
||||
## Virtuals
|
||||
### Virtuals
|
||||
|
||||
None defined.
|
||||
|
||||
## Indexes
|
||||
### Mongo Indexes
|
||||
|
||||
Single-field — `backend/src/models/PurchaseRequest.ts:376-381`:
|
||||
Single-field — `backend/src/models/PurchaseRequest.ts:414-419`:
|
||||
|
||||
- `{ buyerId: 1 }`
|
||||
- `{ categoryId: 1 }`
|
||||
@@ -107,26 +121,262 @@ Single-field — `backend/src/models/PurchaseRequest.ts:376-381`:
|
||||
- `{ createdAt: -1 }`
|
||||
- `{ urgency: 1 }`
|
||||
|
||||
Compound — `backend/src/models/PurchaseRequest.ts:384-385`:
|
||||
Compound — `backend/src/models/PurchaseRequest.ts:422-423`:
|
||||
|
||||
- `{ productType: 1, status: 1 }`
|
||||
- `{ categoryId: 1, productType: 1 }`
|
||||
|
||||
## Pre/Post Hooks
|
||||
### Pre/Post Hooks
|
||||
|
||||
None declared at the schema level.
|
||||
|
||||
## Instance Methods
|
||||
### Instance Methods
|
||||
|
||||
None defined.
|
||||
|
||||
## Static Methods
|
||||
### Static Methods
|
||||
|
||||
None defined.
|
||||
|
||||
---
|
||||
|
||||
## Postgres / Drizzle Schema
|
||||
|
||||
Source: `backend/src/db/schema/purchaseRequest.ts`
|
||||
|
||||
The PG model normalises the embedded Mongo subdocuments into 7 tables. The `offers[]` array is dropped; [[SellerOffer]] holds `purchase_request_id` as a back-reference.
|
||||
|
||||
### Enums (PG-level)
|
||||
|
||||
| Enum name | Values |
|
||||
| --- | --- |
|
||||
| `purchase_request_status` | `pending_payment`, `pending`, `active`, `received_offers`, `in_negotiation`, `payment`, `processing`, `delivery`, `delivered`, `confirming`, `completed`, `cancelled`, `seller_paid` |
|
||||
| `product_type` | `physical_product`, `digital_product`, `service`, `consultation` |
|
||||
| `request_urgency` | `low`, `medium`, `high`, `urgent` |
|
||||
| `delivery_type` | `physical`, `online` |
|
||||
| `service_session_type` | `online`, `in_person`, `hybrid` |
|
||||
| `pr_metadata_source` | `manual`, `template`, `api` |
|
||||
| `budget_currency` | `USD`, `EUR`, `IRR`, `USDT`, `USDC` |
|
||||
|
||||
### Table: `purchase_requests` (main)
|
||||
|
||||
| Column | PG type | Nullable | Default | Notes |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| `id` | uuid PK | no | `gen_random_uuid()` | |
|
||||
| `legacy_object_id` | text | yes | — | 24-char Mongo ObjectId; partial-unique index |
|
||||
| `buyer_id` | uuid | no | — | FK → `users(id)` |
|
||||
| `category_id` | uuid | no | — | FK → `categories(id)` |
|
||||
| `title` | varchar(200) | no | — | |
|
||||
| `description` | text | no | — | |
|
||||
| `product_type` | enum | yes | `physical_product` | |
|
||||
| `product_link` | varchar(2000) | yes | — | CHECK: `^https?://.+` |
|
||||
| `size` | varchar(100) | yes | — | |
|
||||
| `color` | varchar(100) | yes | — | |
|
||||
| `brand` | varchar(100) | yes | — | |
|
||||
| `quantity` | integer | yes | `1` | CHECK ≥ 1 |
|
||||
| `budget_min` | numeric(38,18) | yes | — | CHECK ≥ 0 |
|
||||
| `budget_max` | numeric(38,18) | yes | — | CHECK ≥ 0 |
|
||||
| `budget_currency` | enum | yes | `USDT` | |
|
||||
| `urgency` | enum | no | `medium` | |
|
||||
| `status` | enum | no | `pending` | 13-value escrow-critical enum |
|
||||
| `is_public` | boolean | yes | `true` | |
|
||||
| `tags` | text[] | yes | `'{}'` | |
|
||||
| `attachments` | text[] | yes | `'{}'` | |
|
||||
| `selected_offer_id` | uuid | yes | — | FK → `seller_offers(id)` |
|
||||
| `rating` | smallint | yes | — | CHECK 1–5 or NULL |
|
||||
| `feedback` | text | yes | — | CHECK length ≤ 1000 or NULL |
|
||||
| `delivery_confirmed` | boolean | yes | `false` | |
|
||||
| `delivery_confirmed_at` | timestamptz | yes | — | |
|
||||
| `dispute_raised` | boolean | no | `false` | |
|
||||
| `dispute_raised_at` | timestamptz | yes | — | |
|
||||
| `dispute_resolved` | boolean | no | `false` | |
|
||||
| `dispute_resolved_at` | timestamptz | yes | — | |
|
||||
| `dispute_hold_reason` | text | yes | — | |
|
||||
| `hold_until` | timestamptz | yes | — | Partial index WHERE NOT NULL |
|
||||
| `metadata_source` | enum | yes | `manual` | |
|
||||
| `metadata_template_id` | varchar(100) | yes | — | |
|
||||
| `metadata_version` | varchar(50) | yes | — | |
|
||||
| `created_at` | timestamptz | no | `now()` | |
|
||||
| `updated_at` | timestamptz | no | `now()` | |
|
||||
|
||||
**Indexes on `purchase_requests`:**
|
||||
|
||||
| Index | Type | Columns / condition |
|
||||
| --- | --- | --- |
|
||||
| `idx_pr_buyer_id` | btree | `buyer_id` |
|
||||
| `idx_pr_category_id` | btree | `category_id` |
|
||||
| `idx_pr_product_type` | btree | `product_type` |
|
||||
| `idx_pr_status` | btree | `status` |
|
||||
| `idx_pr_created_at` | btree | `created_at` |
|
||||
| `idx_pr_urgency` | btree | `urgency` |
|
||||
| `purchase_requests_legacy_object_id_uq` | partial-unique | `legacy_object_id` WHERE NOT NULL |
|
||||
| `idx_pr_product_type_status` | btree | `(product_type, status)` |
|
||||
| `idx_pr_category_product_type` | btree | `(category_id, product_type)` |
|
||||
| `idx_pr_hold_until` | partial btree | `hold_until` WHERE NOT NULL |
|
||||
| `idx_pr_dispute_raised` | partial btree | `dispute_raised` WHERE `dispute_raised = true` |
|
||||
|
||||
**CHECK constraints on `purchase_requests`:**
|
||||
|
||||
| Name | Expression |
|
||||
| --- | --- |
|
||||
| `pr_rating_ck` | `rating IS NULL OR (rating >= 1 AND rating <= 5)` |
|
||||
| `pr_feedback_len_ck` | `feedback IS NULL OR length(feedback) <= 1000` |
|
||||
| `pr_quantity_min_ck` | `quantity IS NULL OR quantity >= 1` |
|
||||
| `pr_budget_min_ck` | `budget_min IS NULL OR budget_min >= 0` |
|
||||
| `pr_budget_max_ck` | `budget_max IS NULL OR budget_max >= 0` |
|
||||
| `pr_product_link_ck` | `product_link IS NULL OR product_link ~ '^https?://.+'` |
|
||||
|
||||
---
|
||||
|
||||
### Table: `purchase_request_delivery_info` (1:1)
|
||||
|
||||
Child of `purchase_requests`. Holds all delivery logistics.
|
||||
|
||||
| Column | PG type | Nullable | Default | Notes |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| `id` | uuid PK | no | random | |
|
||||
| `legacy_object_id` | text | yes | — | Parent PR's legacy ObjectId for traceability |
|
||||
| `purchase_request_id` | uuid UNIQUE | no | — | FK → `purchase_requests(id)` CASCADE |
|
||||
| `delivery_type` | enum | no | `physical` | |
|
||||
| `address` | varchar(500) | yes | — | |
|
||||
| `preferred_date` | timestamptz | yes | — | |
|
||||
| `notes` | text | yes | — | |
|
||||
| `email` | varchar(255) | yes | — | CHECK: email regex or NULL |
|
||||
| `delivery_date_time` | timestamptz | yes | — | |
|
||||
| `delivery_date` | date | yes | — | |
|
||||
| `shipped_at` | timestamptz | yes | — | |
|
||||
| `delivery_code` | varchar(6) | yes | — | CHECK: length = 6 or NULL |
|
||||
| `delivery_code_generated_at` | timestamptz | yes | — | |
|
||||
| `delivery_code_expires_at` | timestamptz | yes | — | |
|
||||
| `delivery_code_used` | boolean | yes | `false` | |
|
||||
| `delivery_code_used_at` | timestamptz | yes | — | |
|
||||
| `delivery_code_used_by` | uuid | yes | — | FK → `users(id)` |
|
||||
| `delivered_at` | timestamptz | yes | — | |
|
||||
| `created_at` | timestamptz | no | `now()` | |
|
||||
| `updated_at` | timestamptz | no | `now()` | |
|
||||
|
||||
**Indexes:** `idx_pr_delivery_info_pr_id` on `purchase_request_id`
|
||||
|
||||
**CHECK constraints:** `pr_di_delivery_code_len_ck` (`length = 6 or NULL`), `pr_di_email_fmt_ck` (email regex)
|
||||
|
||||
---
|
||||
|
||||
### Table: `purchase_request_delivery_address` (1:1 under delivery_info)
|
||||
|
||||
| Column | PG type | Nullable | Notes |
|
||||
| --- | --- | --- | --- |
|
||||
| `id` | uuid PK | no | |
|
||||
| `legacy_object_id` | text | yes | |
|
||||
| `delivery_info_id` | uuid UNIQUE | no | FK → `purchase_request_delivery_info(id)` CASCADE |
|
||||
| `recipient_name` | varchar(200) | yes | |
|
||||
| `phone_number` | varchar(20) | yes | |
|
||||
| `full_address` | text | yes | |
|
||||
| `address_type` | varchar(50) | yes | e.g. Home / Office |
|
||||
|
||||
**Index:** `idx_pr_delivery_addr_info_id` on `delivery_info_id`
|
||||
|
||||
---
|
||||
|
||||
### Table: `purchase_request_seller_delivery_info` (1:1 under delivery_info)
|
||||
|
||||
| Column | PG type | Nullable | Default | Notes |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| `id` | uuid PK | no | random | |
|
||||
| `legacy_object_id` | text | yes | — | |
|
||||
| `delivery_info_id` | uuid UNIQUE | no | — | FK → `purchase_request_delivery_info(id)` CASCADE |
|
||||
| `estimated_delivery_date` | timestamptz | yes | — | |
|
||||
| `estimated_delivery_time` | varchar(50) | yes | — | |
|
||||
| `tracking_number` | varchar(100) | yes | — | |
|
||||
| `delivery_notes` | text | yes | — | |
|
||||
| `shipping_method` | varchar(100) | yes | — | |
|
||||
| `download_link` | varchar(2000) | yes | — | |
|
||||
| `digital_files` | text[] | yes | `'{}'` | |
|
||||
| `created_at` | timestamptz | no | `now()` | |
|
||||
| `updated_at` | timestamptz | no | `now()` | |
|
||||
|
||||
**Index:** `idx_pr_seller_di_info_id` on `delivery_info_id`
|
||||
|
||||
---
|
||||
|
||||
### Table: `delivery_attempts` (1:N under delivery_info)
|
||||
|
||||
Append-only audit log of code-entry attempts.
|
||||
|
||||
| Column | PG type | Nullable | Default | Notes |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| `id` | uuid PK | no | random | |
|
||||
| `delivery_info_id` | uuid | no | — | FK → `purchase_request_delivery_info(id)` CASCADE |
|
||||
| `seller_id` | uuid | no | — | FK → `users(id)` |
|
||||
| `attempted_at` | timestamptz | no | `now()` | |
|
||||
| `success` | boolean | no | — | |
|
||||
| `code` | varchar(100) | yes | — | Only stored on successful attempts |
|
||||
|
||||
**Indexes:** `idx_delivery_attempts_info_id`, `idx_delivery_attempts_seller_id`, `idx_delivery_attempts_success`
|
||||
|
||||
---
|
||||
|
||||
### Table: `purchase_request_service_info` (1:1)
|
||||
|
||||
Only populated for `service` / `consultation` product types.
|
||||
|
||||
| Column | PG type | Nullable | Default | Notes |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| `id` | uuid PK | no | random | |
|
||||
| `legacy_object_id` | text | yes | — | |
|
||||
| `purchase_request_id` | uuid UNIQUE | no | — | FK → `purchase_requests(id)` CASCADE |
|
||||
| `duration` | numeric(5,2) | yes | — | CHECK ≥ 0.5 |
|
||||
| `session_type` | enum | yes | — | `online` / `in_person` / `hybrid` |
|
||||
| `location` | varchar(200) | yes | — | |
|
||||
| `requirements` | text[] | yes | `'{}'` | |
|
||||
| `created_at` | timestamptz | no | `now()` | |
|
||||
| `updated_at` | timestamptz | no | `now()` | |
|
||||
|
||||
**Index:** `idx_pr_service_info_pr_id`
|
||||
**CHECK:** `pr_si_duration_min_ck` (`duration IS NULL OR duration >= 0.5`)
|
||||
|
||||
---
|
||||
|
||||
### Table: `purchase_request_specifications` (1:N)
|
||||
|
||||
Queryable `{key, value, label}` specs extracted from the Mongo embedded array.
|
||||
|
||||
| Column | PG type | Nullable | Default | Notes |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| `id` | uuid PK | no | random | |
|
||||
| `purchase_request_id` | uuid | no | — | FK → `purchase_requests(id)` CASCADE |
|
||||
| `key` | varchar(255) | no | — | |
|
||||
| `value` | text | no | — | |
|
||||
| `label` | varchar(255) | yes | — | |
|
||||
| `position` | integer | no | `0` | Preserves array order for round-trip fidelity |
|
||||
|
||||
**Indexes:** `idx_pr_specs_pr_id`, `idx_pr_specs_key`, partial-unique `purchase_request_specifications_request_key_uq` on `(purchase_request_id, key)`
|
||||
|
||||
---
|
||||
|
||||
### Table: `purchase_request_preferred_sellers` (N:M junction)
|
||||
|
||||
| Column | PG type | Nullable | Notes |
|
||||
| --- | --- | --- | --- |
|
||||
| `purchase_request_id` | uuid | no | FK → `purchase_requests(id)` |
|
||||
| `seller_id` | uuid | no | FK → `users(id)` |
|
||||
|
||||
**Indexes:** composite unique `idx_pr_preferred_sellers_uq` on `(purchase_request_id, seller_id)`; `idx_pr_preferred_sellers_seller_id` on `seller_id`
|
||||
|
||||
---
|
||||
|
||||
### Design Notes
|
||||
|
||||
- **`offers[]` dropped in PG.** The Mongo `offers[]` array is not migrated. Query `SellerOffer WHERE purchase_request_id = ?` instead.
|
||||
- **Money scale.** `budget_min` / `budget_max` use `numeric(38,18)` (project-wide crypto convention) rather than the `numeric(15,8)` suggested in the migration guide, for consistency with `Payment` and `FundsLedgerEntry`.
|
||||
- **`tags` / `attachments`** stored as `text[]` (not JSONB) to enable `ANY()` array queries without a child table.
|
||||
- **`legacy_object_id`** on every table uses a partial-unique index (`WHERE NOT NULL`) for idempotent backfill upserts.
|
||||
- **Dispute / escrow hold fields** (`dispute_raised`, `dispute_raised_at`, `dispute_resolved`, `dispute_resolved_at`, `dispute_hold_reason`, `hold_until`) are present in both the Mongo interface (`IPurchaseRequest`) and the PG main table. They were added to the Mongo schema before the PG migration and are considered escrow-critical.
|
||||
|
||||
---
|
||||
|
||||
## Relationships
|
||||
|
||||
- **References**: [[User]] (`buyerId`, `preferredSellerIds[]`, `deliveryInfo.deliveryCodeUsedBy`, `deliveryInfo.deliveryAttempts[].sellerId`), [[Category]] (`categoryId`), [[SellerOffer]] (`offers[]`, `selectedOfferId`).
|
||||
- **References**: [[User]] (`buyerId`, `preferredSellerIds[]`, `deliveryInfo.deliveryCodeUsedBy`, `deliveryInfo.deliveryAttempts[].sellerId`), [[Category]] (`categoryId`), [[SellerOffer]] (`offers[]` Mongo only, `selectedOfferId`).
|
||||
- **Referenced by**: [[SellerOffer]] (`purchaseRequestId`), [[Payment]] (`purchaseRequestId`), [[Dispute]] (`purchaseRequestId`), [[Chat]] (`relatedTo.id` when `relatedTo.type === 'PurchaseRequest'`), [[Review]] (`purchaseRequestId`).
|
||||
|
||||
## Template Checkout Mapping
|
||||
@@ -175,7 +425,7 @@ PurchaseRequest.find({ isPublic: true, status: 'active' }).sort({ createdAt: -1
|
||||
// Sellers' eligible queue
|
||||
PurchaseRequest.find({ productType, status: 'active', categoryId });
|
||||
|
||||
// Populate offers
|
||||
// Populate offers (Mongo only — offers[] array is not in PG)
|
||||
PurchaseRequest.findById(id).populate('offers').populate('selectedOfferId');
|
||||
|
||||
// Redeem delivery code
|
||||
@@ -183,6 +433,12 @@ PurchaseRequest.findOneAndUpdate(
|
||||
{ _id: id, 'deliveryInfo.deliveryCode': code, 'deliveryInfo.deliveryCodeUsed': false },
|
||||
{ $set: { 'deliveryInfo.deliveryCodeUsed': true, 'deliveryInfo.deliveryCodeUsedAt': new Date() } }
|
||||
);
|
||||
|
||||
// PG: offers for a request
|
||||
// SELECT * FROM seller_offers WHERE purchase_request_id = $1;
|
||||
|
||||
// PG: find requests with live escrow hold
|
||||
// SELECT * FROM purchase_requests WHERE hold_until IS NOT NULL AND hold_until > now();
|
||||
```
|
||||
|
||||
Related: [[SellerOffer]], [[Payment]], [[Chat]], [[Dispute]], [[Review]], [[RequestTemplate]], [[Category]].
|
||||
|
||||
@@ -1,21 +1,28 @@
|
||||
---
|
||||
title: SellerOffer
|
||||
tags: [data-model, mongoose]
|
||||
tags: [data-model, mongoose, postgres]
|
||||
aliases: [Seller Offer, Bid, ISellerOffer]
|
||||
---
|
||||
|
||||
# SellerOffer
|
||||
|
||||
> **Last updated:** 2026-05-31 — added `TRY` pricing support for oracle/depeg quoting.
|
||||
> **Last updated:** 2026-06-03 — added Postgres/Drizzle table definition and migration status.
|
||||
|
||||
A seller's bid against a [[PurchaseRequest]]. Stores the proposed price, the delivery time commitment, optional notes/attachments, and a small status machine (`pending` / `accepted` / `rejected` / `withdrawn`). The parent `PurchaseRequest` keeps the array of offer ids in `offers[]` and the chosen one in `selectedOfferId`.
|
||||
|
||||
> [!note] Source
|
||||
> `backend/src/models/SellerOffer.ts:24` — schema definition
|
||||
> `backend/src/models/SellerOffer.ts:100` — model export
|
||||
> `backend/src/models/SellerOffer.ts:24` — Mongoose schema definition
|
||||
> `backend/src/models/SellerOffer.ts:100` — Mongoose model export
|
||||
> `backend/src/db/schema/sellerOffer.ts` — Drizzle/Postgres table definition
|
||||
|
||||
## Migration Status
|
||||
|
||||
**DUAL-WRITE** — part of `DualWriteMarketplaceRepo`. Writes go to both MongoDB and Postgres; reads still come from MongoDB.
|
||||
|
||||
## Schema
|
||||
|
||||
### Mongoose (MongoDB)
|
||||
|
||||
| Field | Type | Required | Default | Validation | Index | Description |
|
||||
| --- | --- | --- | --- | --- | --- | --- |
|
||||
| `sellerId` | ObjectId → [[User]] | yes | — | — | yes | Seller submitting the bid. |
|
||||
@@ -37,11 +44,62 @@ A seller's bid against a [[PurchaseRequest]]. Stores the proposed price, the del
|
||||
|
||||
> **Status enum note:** `active` is accepted by the current backend schema for marketplace/listing flows, in addition to the negotiation statuses `pending | accepted | rejected | withdrawn`.
|
||||
|
||||
### Postgres (Drizzle) — `seller_offers`
|
||||
|
||||
Table: `seller_offers` | Schema file: `backend/src/db/schema/sellerOffer.ts`
|
||||
|
||||
| PG Column | Drizzle Type | Nullable | Default | Notes |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| `id` | `uuid` PK | no | `gen_random_uuid()` | PG primary key |
|
||||
| `legacy_object_id` | `text` | yes | — | Mongo ObjectId bridge; partial-unique WHERE NOT NULL |
|
||||
| `seller_id` | `uuid` FK → `users` CASCADE | no | — | Maps from `sellerId` |
|
||||
| `purchase_request_id` | `uuid` FK → `purchase_requests` CASCADE | no | — | Maps from `purchaseRequestId` |
|
||||
| `title` | `varchar(200)` | no | — | |
|
||||
| `description` | `varchar(1000)` | no | — | |
|
||||
| `price_amount` | `numeric(18,8)` | no | — | CHECK `price_amount >= 0` |
|
||||
| `price_currency` | `offer_currency` enum | no | — | `USD \| EUR \| IRR \| USDT \| USDC \| TRY` |
|
||||
| `delivery_time_amount` | `int` | no | — | CHECK `delivery_time_amount >= 1` |
|
||||
| `delivery_time_unit` | `delivery_unit` enum | no | — | `hours \| days \| weeks` |
|
||||
| `status` | `offer_status` enum | no | `pending` | `pending \| accepted \| rejected \| withdrawn \| active` |
|
||||
| `attachments` | `text[]` | yes | — | |
|
||||
| `notes` | `text` | yes | — | |
|
||||
| `valid_until` | `timestamp with time zone` | yes | — | Maps from `validUntil` |
|
||||
| `require_aml_check` | `boolean` | yes | — | |
|
||||
| `aml_block_on_failure` | `boolean` | yes | — | CHECK: block requires check (AML coherence) |
|
||||
| `created_at` | `timestamp with time zone` | no | `now()` | |
|
||||
| `updated_at` | `timestamp with time zone` | no | `now()` | |
|
||||
|
||||
**Enums used:**
|
||||
|
||||
| Enum name | Values |
|
||||
| --- | --- |
|
||||
| `offer_status` | `pending`, `accepted`, `rejected`, `withdrawn`, `active` |
|
||||
| `offer_currency` | `USD`, `EUR`, `IRR`, `USDT`, `USDC`, `TRY` |
|
||||
| `delivery_unit` | `hours`, `days`, `weeks` |
|
||||
|
||||
**Constraints:**
|
||||
- `CHECK (price_amount >= 0)`
|
||||
- `CHECK (delivery_time_amount >= 1)`
|
||||
- AML coherence check: `aml_block_on_failure = true` requires `require_aml_check = true`
|
||||
|
||||
**Money precision note:** `price_amount` uses `numeric(18,8)` — differs from the `numeric(38,18)` used by `payments` and `funds_ledger_entries`. This matches the Migration Guide specification for offer amounts.
|
||||
|
||||
#### Postgres Indexes
|
||||
|
||||
| Index | Type | Notes |
|
||||
| --- | --- | --- |
|
||||
| `seller_id` | btree | |
|
||||
| `purchase_request_id` | btree | |
|
||||
| `status` | btree | |
|
||||
| `created_at DESC` | btree | |
|
||||
| `(purchase_request_id, seller_id)` | btree | composite |
|
||||
| `legacy_object_id` | partial-unique | WHERE NOT NULL; idempotent backfill upserts |
|
||||
|
||||
## Virtuals
|
||||
|
||||
None defined.
|
||||
|
||||
## Indexes
|
||||
## Mongoose Indexes
|
||||
|
||||
Defined at `backend/src/models/SellerOffer.ts:95-98`:
|
||||
|
||||
@@ -80,6 +138,8 @@ The frontend exposes this via the `withdrawOffer(offerId)` action in `src/action
|
||||
|
||||
- **References**: [[User]] (`sellerId`), [[PurchaseRequest]] (`purchaseRequestId`).
|
||||
- **Referenced by**: [[PurchaseRequest]] (`offers[]`, `selectedOfferId`), [[Payment]] (`sellerOfferId`), [[Chat]] (`relatedTo.id` when `relatedTo.type === 'SellerOffer'`).
|
||||
- **PG FKs**: `seller_offers.seller_id → users.id CASCADE`, `seller_offers.purchase_request_id → purchase_requests.id CASCADE`.
|
||||
- **Referenced by (PG)**: `payments.seller_offer_id` (polymorphic triple), `payment_quotes` (via payment join).
|
||||
|
||||
## State Transitions
|
||||
|
||||
@@ -96,6 +156,8 @@ stateDiagram-v2
|
||||
|
||||
## Common Queries
|
||||
|
||||
### MongoDB
|
||||
|
||||
```ts
|
||||
// Offers for a request
|
||||
SellerOffer.find({ purchaseRequestId }).sort({ createdAt: -1 });
|
||||
@@ -113,4 +175,36 @@ SellerOffer.updateMany(
|
||||
SellerOffer.find({ validUntil: { $lt: new Date() }, status: 'pending' });
|
||||
```
|
||||
|
||||
### Postgres (Drizzle)
|
||||
|
||||
```ts
|
||||
// Offers for a request
|
||||
db.select().from(sellerOffers)
|
||||
.where(eq(sellerOffers.purchaseRequestId, requestId))
|
||||
.orderBy(desc(sellerOffers.createdAt));
|
||||
|
||||
// Seller's pending offers
|
||||
db.select().from(sellerOffers)
|
||||
.where(and(
|
||||
eq(sellerOffers.sellerId, sellerId),
|
||||
eq(sellerOffers.status, 'pending')
|
||||
));
|
||||
|
||||
// Reject siblings on accept
|
||||
db.update(sellerOffers)
|
||||
.set({ status: 'rejected' })
|
||||
.where(and(
|
||||
eq(sellerOffers.purchaseRequestId, purchaseRequestId),
|
||||
ne(sellerOffers.id, acceptedId),
|
||||
eq(sellerOffers.status, 'pending')
|
||||
));
|
||||
|
||||
// Cleanup expired offers
|
||||
db.select().from(sellerOffers)
|
||||
.where(and(
|
||||
lt(sellerOffers.validUntil, new Date()),
|
||||
eq(sellerOffers.status, 'pending')
|
||||
));
|
||||
```
|
||||
|
||||
Related: [[PurchaseRequest]], [[Payment]], [[User]].
|
||||
|
||||
@@ -1,14 +1,108 @@
|
||||
---
|
||||
title: User
|
||||
tags: [data-model, mongoose]
|
||||
tags: [data-model, mongoose, postgres, dual-write]
|
||||
aliases: [User Model, IUser, Account]
|
||||
---
|
||||
|
||||
# User
|
||||
|
||||
> **Last updated:** 2026-05-29 — aligned with code (see [Doc vs Code Audit Report](../09%20-%20Audits/Doc%20vs%20Code%20Audit%20Report%20-%202026-05-29.md))
|
||||
> **Last updated:** 2026-06-03 — added Postgres/Drizzle schema, `guard` role (migration 0017), dual-write status. Previous update: 2026-05-29 (see [Doc vs Code Audit Report](../09%20-%20Audits/Doc%20vs%20Code%20Audit%20Report%20-%202026-05-29.md))
|
||||
|
||||
The core identity document for every actor in the marketplace: buyers, sellers, and admins. Stores credentials (password + WebAuthn passkeys), profile/preference data, referral bookkeeping, point balances, and a soft-delete status flag. Almost every other model carries an `ObjectId` reference back to `User`, so this collection is the relational hub of the system.
|
||||
The core identity document for every actor in the marketplace: buyers, sellers, and admins. Stores credentials (password + WebAuthn passkeys), profile/preference data, referral bookkeeping, point balances, and a soft-delete status flag. Almost every other model carries an `ObjectId` (Mongo) or `uuid` (Postgres) reference back to `User`, so this collection is the relational hub of the system.
|
||||
|
||||
> [!info] Migration status: DUAL-WRITE
|
||||
> Writes go to **both** MongoDB (`User` collection) and Postgres (`users` table) via `DualWriteUserRepo`.
|
||||
> Reads still come from **MongoDB** — PG reads are not yet enabled.
|
||||
> Repositories: `DrizzleUserRepo`, `MongoUserRepo`, `DualWriteUserRepo`
|
||||
> Postgres table: **`users`** — `backend/src/db/schema/users.ts`
|
||||
|
||||
---
|
||||
|
||||
## Postgres Table: `users`
|
||||
|
||||
> [!note] Source
|
||||
> `backend/src/db/schema/users.ts`
|
||||
|
||||
### Columns
|
||||
|
||||
| Column | PG Type | Nullable | Default | Notes |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| `id` | `uuid` | no | `gen_random_uuid()` | Primary key |
|
||||
| `legacy_object_id` | `text` | yes | — | Mongo ObjectId; partial-unique index WHERE NOT NULL; used for idempotent backfill upserts |
|
||||
| `email` | `varchar(255)` | yes | — | Partial-unique index WHERE NOT NULL |
|
||||
| `password` | `varchar(255)` | yes | — | Hashed |
|
||||
| `first_name` | `text` | yes | — | — |
|
||||
| `last_name` | `text` | yes | — | — |
|
||||
| `role` | `user_role` enum | no | `buyer` | Values: `admin`, `buyer`, `seller`, `resolver`, `guard` (added migration 0017) |
|
||||
| `is_email_verified` | `bool` | yes | `false` | — |
|
||||
| `auth_provider` | `auth_provider` enum | no | `email` | Values: `email`, `google`, `telegram` |
|
||||
| `telegram_verified` | `bool` | yes | `false` | — |
|
||||
| `email_verification_token` | `text` | yes | — | Legacy token flow |
|
||||
| `email_verification_code` | `text` | yes | — | OTP code |
|
||||
| `email_verification_code_expires` | `timestamptz` | yes | — | — |
|
||||
| `password_reset_token` | `text` | yes | — | — |
|
||||
| `password_reset_expires` | `timestamptz` | yes | — | — |
|
||||
| `password_reset_code` | `text` | yes | — | — |
|
||||
| `password_reset_code_expires` | `timestamptz` | yes | — | — |
|
||||
| `profile` | `jsonb` | yes | — | Stores avatar, photoURL, phone, address, bio, website, walletAddress, walletType, walletProvider, walletProofVerified, walletProofTimestamp, isPublic |
|
||||
| `preferences` | `jsonb` | yes | — | Stores language, currency, notifications.{email,sms,push} |
|
||||
| `status` | `user_status` enum | yes | `active` | Values: `active`, `suspended`, `deleted` |
|
||||
| `last_login_at` | `timestamptz` | yes | — | — |
|
||||
| `referral_code` | `varchar(255)` | yes | — | Partial-unique index |
|
||||
| `referred_by_id` | `uuid` | yes | — | Self-FK → `users(id)`; index |
|
||||
| `points_total` | `int` | yes | `0` | — |
|
||||
| `points_available` | `int` | yes | `0` | — |
|
||||
| `points_used` | `int` | yes | `0` | — |
|
||||
| `points_level` | `int` | yes | `1` | Indexed |
|
||||
| `referral_stats_total` | `int` | yes | `0` | — |
|
||||
| `referral_stats_active` | `int` | yes | `0` | — |
|
||||
| `referral_stats_total_earned` | `int` | yes | `0` | — |
|
||||
| `created_at` | `timestamptz` | no | `now()` | — |
|
||||
| `updated_at` | `timestamptz` | no | `now()` | — |
|
||||
|
||||
### Child Tables
|
||||
|
||||
**`user_passkeys`** — WebAuthn credentials extracted from the embedded array:
|
||||
|
||||
| Column | Type | Notes |
|
||||
| --- | --- | --- |
|
||||
| `id` | `text` (PK) | WebAuthn credential ID |
|
||||
| `user_id` | `uuid FK→users CASCADE` | Owner |
|
||||
| `public_key` | `text` | Stored public key |
|
||||
| `counter` | `int` | Signature counter |
|
||||
| `device_type` | `passkey_device_type` enum | `platform` / `cross-platform` |
|
||||
| `device_name` | `text` | Optional human label |
|
||||
| `created_at` | `timestamptz` | — |
|
||||
|
||||
**`user_refresh_tokens`** — Active JWT refresh tokens extracted from the Mongo array:
|
||||
|
||||
| Column | Type | Notes |
|
||||
| --- | --- | --- |
|
||||
| `token` | `text` (PK) | The refresh token string |
|
||||
| `user_id` | `uuid FK→users CASCADE` | Owner |
|
||||
|
||||
### Indexes (Postgres)
|
||||
|
||||
| Index | Type | Condition |
|
||||
| --- | --- | --- |
|
||||
| `users_email_unique` | partial-unique | WHERE `email IS NOT NULL` |
|
||||
| `users_referral_code_unique` | partial-unique | WHERE `referral_code IS NOT NULL` |
|
||||
| `users_legacy_object_id_unique` | partial-unique | WHERE `legacy_object_id IS NOT NULL` |
|
||||
| `users_role_idx` | btree | — |
|
||||
| `users_status_idx` | btree | — |
|
||||
| `users_auth_provider_idx` | btree | — |
|
||||
| `users_referral_code_idx` | btree | — |
|
||||
| `users_referred_by_id_idx` | btree | — |
|
||||
| `users_points_level_idx` | btree | — |
|
||||
|
||||
### Relations
|
||||
|
||||
- Self-referential: `referred_by_id → users.id` (parent/children for referral tree)
|
||||
- One-to-many: `user_passkeys.user_id`, `user_refresh_tokens.user_id`
|
||||
|
||||
---
|
||||
|
||||
## MongoDB Collection: `User` (legacy — reads still active)
|
||||
|
||||
> [!note] Source
|
||||
> `backend/src/models/User.ts:70` — schema definition
|
||||
@@ -20,7 +114,7 @@ The core identity document for every actor in the marketplace: buyers, sellers,
|
||||
> [!note] Wallet ownership proof
|
||||
> `PATCH /api/user/wallet-address` accepts both EVM and TON wallets. EVM addresses require an EIP-191 signature (`ethers.verifyMessage`); TON addresses are format-validated and may include an optional TonProof. A successful proof sets `profile.walletProofVerified = true` and `profile.walletProofTimestamp`.
|
||||
|
||||
## Schema
|
||||
### Schema
|
||||
|
||||
| Field | Type | Required | Default | Validation | Index | Description |
|
||||
| --- | --- | --- | --- | --- | --- | --- |
|
||||
@@ -28,8 +122,8 @@ The core identity document for every actor in the marketplace: buyers, sellers,
|
||||
| `password` | String | no | — | minlength 6 | — | Hashed password. Optional to support passkey-only, Google, and Telegram accounts. |
|
||||
| `firstName` | String | no | `"کاربر"` | trim | — | Persian default ("user"). |
|
||||
| `lastName` | String | no | `"جدید"` | trim | — | Persian default ("new"). |
|
||||
| `role` | String | yes | `"buyer"` | enum: `admin` / `buyer` / `seller` / `resolver` | yes | Authorisation tier. `resolver` was added in commit `fce8a19` — can view and resolve disputes, and bypass chat membership checks, but has no other admin privileges. |
|
||||
| `isEmailVerified` | Boolean | no | `false` | — | — | Set to true after the email verification code is consumed. ⚠️ Changing the email via `PUT /api/user/profile` **resets this to `false`** and dispatches a fresh **6-digit** verification code to the new address (see Email verification note below). |
|
||||
| `role` | String | yes | `"buyer"` | enum: `admin` / `buyer` / `seller` / `resolver` / `guard` | yes | Authorisation tier. `resolver` (commit `fce8a19`): can view/resolve disputes and bypass chat membership checks. `guard` (migration 0017): added in PG schema; purpose TBD. |
|
||||
| `isEmailVerified` | Boolean | no | `false` | — | — | Set to true after the email verification code is consumed. Warning: Changing the email via `PUT /api/user/profile` **resets this to `false`** and dispatches a fresh **6-digit** verification code to the new address. |
|
||||
| `authProvider` | String | yes | `"email"` | enum: `email` / `google` / `telegram` | yes | Provider used to create the account. Existing email/password accounts remain `email`; Telegram-only users are `telegram`. |
|
||||
| `telegramVerified` | Boolean | no | `false` | — | — | Set when Telegram identity has been signature-verified and linked through `TelegramLink`. |
|
||||
| `emailVerificationToken` | String | no | — | — | — | Legacy token-based email verification. |
|
||||
@@ -39,7 +133,7 @@ The core identity document for every actor in the marketplace: buyers, sellers,
|
||||
| `passwordResetExpires` | Date | no | — | — | — | Expiry of `passwordResetToken`. |
|
||||
| `passwordResetCode` | String | no | — | — | — | OTP reset code. |
|
||||
| `passwordResetCodeExpires` | Date | no | — | — | — | Expiry for OTP reset code. |
|
||||
| `passkeys[]` | Subdocument array | no | `[]` | — | — | WebAuthn credentials (see below). |
|
||||
| `passkeys[]` | Subdocument array | no | `[]` | — | — | WebAuthn credentials. Extracted to `user_passkeys` table in PG. |
|
||||
| `passkeys[].id` | String | yes | — | — | — | Credential ID. |
|
||||
| `passkeys[].publicKey` | String | yes | — | — | — | Stored public key. |
|
||||
| `passkeys[].counter` | Number | yes | `0` | — | — | Signature counter. |
|
||||
@@ -49,7 +143,7 @@ The core identity document for every actor in the marketplace: buyers, sellers,
|
||||
| `profile.avatar` | String | no | — | — | — | Avatar URL. |
|
||||
| `profile.photoURL` | String | no | — | — | — | Alternative photo URL. |
|
||||
| `profile.phone` | String | no | — | — | — | Contact phone. |
|
||||
| `profile.address.street` | String | no | — | — | — | Inline address (separate from [[Address]] book). |
|
||||
| `profile.address.street` | String | no | — | — | — | Inline address (separate from Address book). |
|
||||
| `profile.address.city` | String | no | — | — | — | — |
|
||||
| `profile.address.state` | String | no | — | — | — | — |
|
||||
| `profile.address.zipCode` | String | no | — | — | — | — |
|
||||
@@ -69,26 +163,26 @@ The core identity document for every actor in the marketplace: buyers, sellers,
|
||||
| `preferences.notifications.push` | Boolean | no | `true` | — | — | Opt-in for push notifications. |
|
||||
| `status` | String | no | `"active"` | enum: `active` / `suspended` / `deleted` | yes | Soft-delete and moderation flag. |
|
||||
| `lastLoginAt` | Date | no | — | — | — | Updated by auth middleware. |
|
||||
| `refreshTokens[]` | String[] | no | `[]` | — | — | Array of currently active JWT refresh tokens. ⚠️ Reset to `[]` on password change and on password reset, which invalidates every outstanding session and forces re-login everywhere. |
|
||||
| `referralCode` | String | no | — | — | unique, sparse | **Not yet implemented** in `User.ts` — planned for referral programme. |
|
||||
| `referredBy` | ObjectId → User | no | — | — | yes | **Not yet implemented** in `User.ts` — planned for referral programme. |
|
||||
| `points.total` | Number | no | `0` | — | — | **Not yet implemented** in `User.ts` — planned for loyalty system. |
|
||||
| `points.available` | Number | no | `0` | — | — | **Not yet implemented** in `User.ts`. |
|
||||
| `points.used` | Number | no | `0` | — | — | **Not yet implemented** in `User.ts`. |
|
||||
| `points.level` | Number | no | `1` | — | yes (`points.level`) | **Not yet implemented** in `User.ts` — planned for [[LevelConfig]] lookup. |
|
||||
| `referralStats.totalReferrals` | Number | no | `0` | — | — | **Not yet implemented** in `User.ts`. |
|
||||
| `referralStats.activeReferrals` | Number | no | `0` | — | — | **Not yet implemented** in `User.ts`. |
|
||||
| `referralStats.totalEarned` | Number | no | `0` | — | — | **Not yet implemented** in `User.ts`. |
|
||||
| `refreshTokens[]` | String[] | no | `[]` | — | — | Array of currently active JWT refresh tokens. Warning: Reset to `[]` on password change and on password reset, invalidating every outstanding session. Extracted to `user_refresh_tokens` table in PG. |
|
||||
| `referralCode` | String | no | — | — | unique, sparse | **Not yet implemented** — planned for referral programme. |
|
||||
| `referredBy` | ObjectId -> User | no | — | — | yes | **Not yet implemented** — planned for referral programme. |
|
||||
| `points.total` | Number | no | `0` | — | — | **Not yet implemented** — planned for loyalty system. |
|
||||
| `points.available` | Number | no | `0` | — | — | **Not yet implemented.** |
|
||||
| `points.used` | Number | no | `0` | — | — | **Not yet implemented.** |
|
||||
| `points.level` | Number | no | `1` | — | yes | **Not yet implemented** — planned for LevelConfig lookup. |
|
||||
| `referralStats.totalReferrals` | Number | no | `0` | — | — | **Not yet implemented.** |
|
||||
| `referralStats.activeReferrals` | Number | no | `0` | — | — | **Not yet implemented.** |
|
||||
| `referralStats.totalEarned` | Number | no | `0` | — | — | **Not yet implemented.** |
|
||||
| `createdAt` | Date | auto | — | — | — | Mongoose timestamp. |
|
||||
| `updatedAt` | Date | auto | — | — | — | Mongoose timestamp. |
|
||||
|
||||
## Virtuals
|
||||
### Virtuals
|
||||
|
||||
| Virtual | Returns | Definition |
|
||||
| --- | --- | --- |
|
||||
| `fullName` | `${firstName} ${lastName}` | `backend/src/models/User.ts:238` |
|
||||
|
||||
## Indexes
|
||||
### Indexes (MongoDB)
|
||||
|
||||
Defined explicitly:
|
||||
|
||||
@@ -97,27 +191,44 @@ Defined explicitly:
|
||||
- `{ status: 1 }` — `backend/src/models/User.ts:179`
|
||||
- `{ authProvider: 1 }` — supports provider-level account reporting and cleanup.
|
||||
|
||||
> [!warning] Missing indexes
|
||||
> The schema currently defines only `role` and `status` indexes. The `referralCode`, `referredBy`, and `points.level` indexes documented below are **not yet present** in `User.ts`:
|
||||
> [!warning] Missing indexes in Mongo schema
|
||||
> The schema currently defines only `role` and `status` indexes. The `referralCode`, `referredBy`, and `points.level` indexes documented below are **not yet present** in `User.ts`.
|
||||
|
||||
## Pre/Post Hooks
|
||||
### Pre/Post Hooks
|
||||
|
||||
None declared at the schema level.
|
||||
|
||||
## Instance Methods
|
||||
### Instance Methods
|
||||
|
||||
| Signature | Purpose |
|
||||
| --- | --- |
|
||||
| `toJSON(): object` | Strips `password`, `refreshTokens`, all `emailVerification*` and `passwordReset*` fields before serialisation. Defined at `backend/src/models/User.ts:243`. |
|
||||
|
||||
## Static Methods
|
||||
### Static Methods
|
||||
|
||||
None defined on the schema.
|
||||
|
||||
---
|
||||
|
||||
## Roles
|
||||
|
||||
| Role | Added | Capabilities |
|
||||
| --- | --- | --- |
|
||||
| `admin` | original | Full platform access |
|
||||
| `buyer` | original | Place purchase requests, confirm delivery |
|
||||
| `seller` | original | Submit offers, manage shop |
|
||||
| `resolver` | commit `fce8a19` | View/resolve disputes; bypass chat membership checks; no other admin privileges |
|
||||
| `guard` | migration 0017 (PG only) | Purpose TBD — defined in `user_role` PG enum, not yet in Mongo schema |
|
||||
|
||||
> [!warning] Role enum drift
|
||||
> The Postgres `user_role` enum includes `guard`; the Mongo schema enum does not. Until the Mongo schema is updated, any `guard`-role user created through PG will not be representable in Mongo and will break dual-write for that record.
|
||||
|
||||
---
|
||||
|
||||
## Relationships
|
||||
|
||||
- **References**: [[User]] (self, via `referredBy`).
|
||||
- **Referenced by**: [[PurchaseRequest]] (`buyerId`, `preferredSellerIds`, `deliveryInfo.deliveryCodeUsedBy`, `deliveryInfo.deliveryAttempts[].sellerId`), [[SellerOffer]] (`sellerId`), [[Payment]] (`buyerId`, `sellerId`), [[Chat]] (`participants[].userId`, `messages[].senderId`, `metadata.createdBy`), [[Notification]] (`userId` as string), [[RequestTemplate]] (`sellerId`), [[Dispute]] (`buyerId`, `sellerId`, `adminId`), [[BlogPost]] (`author.id`), [[Address]] (`userId`), [[Review]] (`sellerId`, `reviewerId`), [[PointTransaction]] (`user`, `referredUser`), [[ShopSettings]] (`sellerId`).
|
||||
- **References**: User (self, via `referredBy` / `referred_by_id`).
|
||||
- **Referenced by**: PurchaseRequest (`buyerId`, `preferredSellerIds`, `deliveryInfo.deliveryCodeUsedBy`, `deliveryInfo.deliveryAttempts[].sellerId`), SellerOffer (`sellerId`), Payment (`buyerId`, `sellerId`), Chat (`participants[].userId`, `messages[].senderId`, `metadata.createdBy`), Notification (`userId` as string), RequestTemplate (`sellerId`), Dispute (`buyerId`, `sellerId`, `adminId`), BlogPost (`author.id`), Address (`userId`), Review (`sellerId`, `reviewerId`), PointTransaction (`user`, `referredUser`), ShopSettings (`sellerId`).
|
||||
|
||||
## State Transitions
|
||||
|
||||
@@ -134,20 +245,31 @@ stateDiagram-v2
|
||||
## Common Queries
|
||||
|
||||
```ts
|
||||
// Find by email (login)
|
||||
// Mongo — Find by email (login)
|
||||
User.findOne({ email: email.toLowerCase() });
|
||||
|
||||
// Active sellers
|
||||
// Mongo — Active sellers
|
||||
User.find({ role: 'seller', status: 'active' });
|
||||
|
||||
// Validate referral
|
||||
// Mongo — Validate referral
|
||||
User.findOne({ referralCode: code });
|
||||
|
||||
// Leaderboard by points
|
||||
// Mongo — Leaderboard by points
|
||||
User.find({ status: 'active' }).sort({ 'points.total': -1 }).limit(10);
|
||||
|
||||
// Promote level
|
||||
// Mongo — Promote level
|
||||
User.updateOne({ _id: id }, { $set: { 'points.level': newLevel } });
|
||||
```
|
||||
|
||||
Related: [[TempVerification]], [[LevelConfig]], [[PointTransaction]], [[ShopSettings]].
|
||||
```sql
|
||||
-- PG — Find by email
|
||||
SELECT * FROM users WHERE email = lower($1) AND email IS NOT NULL;
|
||||
|
||||
-- PG — Active sellers
|
||||
SELECT * FROM users WHERE role = 'seller' AND status = 'active';
|
||||
|
||||
-- PG — Leaderboard by points
|
||||
SELECT * FROM users WHERE status = 'active' ORDER BY points_total DESC LIMIT 10;
|
||||
```
|
||||
|
||||
Related: TempVerification, LevelConfig, PointTransaction, ShopSettings.
|
||||
|
||||
594
04 - Flows/Telegram Mini App.md
Normal file
594
04 - Flows/Telegram Mini App.md
Normal file
@@ -0,0 +1,594 @@
|
||||
---
|
||||
title: Telegram Mini App Flow
|
||||
tags: [flow, telegram, mini-app, auth, bilingual, RTL, shop, cart]
|
||||
related_models: ["[[User]]"]
|
||||
related_apis: ["POST /api/auth/telegram", "[[Auth API]]"]
|
||||
task: "5.4"
|
||||
---
|
||||
|
||||
> **Last updated:** 2026-06-03
|
||||
> **Status:** IN PROGRESS — Task 5.4 (dependencies: 5.1 auth infra, 5.2 Telegram sign-in endpoint)
|
||||
> **Frontend branch:** `integrate-main-into-development` · v2.8.59
|
||||
> **Entry point:** `src/sections/telegram/` · route `/telegram`
|
||||
|
||||
# Telegram Mini App Flow
|
||||
|
||||
End-to-end specification for the **Amaneh Telegram Mini App** — a fully self-contained marketplace shell surfaced inside Telegram's in-app browser via the WebApp SDK. Buyers and sellers can browse requests, create new escrow requests, shop seller templates, manage a cart, review offer state, follow payments, and message each other without leaving Telegram.
|
||||
|
||||
---
|
||||
|
||||
## 1. Architecture Overview
|
||||
|
||||
```
|
||||
Telegram Client
|
||||
└─ Mini App iframe (https://amn.gg/telegram)
|
||||
└─ TelegramMiniAppView ← shell orchestrator
|
||||
├─ useTelegramLiveContext ← SDK probe + polling
|
||||
├─ useTelegramLanguage ← EN / FA detection
|
||||
├─ useTelegramAutoSignIn ← silent JWT exchange
|
||||
├─ useTelegramMainButton ← native chrome sync
|
||||
├─ useTelegramBackButton ← native chrome sync
|
||||
├─ useTelegramHaptic ← haptic wrapper
|
||||
├─ useTelegramCart ← shared localStorage cart
|
||||
│
|
||||
├─ [state: loading] → TelegramLoadingState
|
||||
├─ [state: unsupported] → TelegramUnsupportedState
|
||||
├─ [state: unlinked] → TelegramUnlinkedState
|
||||
└─ [state: linked]
|
||||
├─ TelegramHeader
|
||||
├─ TelegramTabBar (Home / Shop / Requests / Chat / Account)
|
||||
│
|
||||
├─ TelegramHomeView
|
||||
├─ TelegramShopView → TelegramSellerShopView
|
||||
├─ TelegramRequestsView → TelegramRequestDetailView
|
||||
├─ TelegramChatView → TelegramChatThreadView
|
||||
├─ TelegramAccountView
|
||||
└─ [overlay] TelegramNewRequestView
|
||||
└─ [overlay] TelegramNotificationsView
|
||||
└─ [overlay] TelegramCartView
|
||||
```
|
||||
|
||||
The shell is a **single-page, no-router** design: all navigation (tabs, overlays, detail drilldowns) is pure React state in `TelegramMiniAppView`. `window.location.assign` is only used as a final escape hatch to the full web dashboard.
|
||||
|
||||
---
|
||||
|
||||
## 2. Launch Points
|
||||
|
||||
| Entry | Mechanism | `startapp` context |
|
||||
|---|---|---|
|
||||
| Bot profile | User opens bot → taps "Open App" | none |
|
||||
| Menu button | Pinned button in any chat with the bot | none |
|
||||
| Inline button | Bot sends a card with an embedded button | `req_<requestId>` |
|
||||
| Direct deep link | `https://t.me/AmanehBot/app?startapp=req_<id>` | `req_<requestId>` |
|
||||
| Web fallback | Browser at `/telegram` | none (unsupported state) |
|
||||
|
||||
`startapp` / `tgWebAppStartParam` is read from either the WebApp SDK (`window.Telegram.WebApp`) or from URL query/hash params (for older Telegram clients that append them directly).
|
||||
|
||||
---
|
||||
|
||||
## 3. SDK Initialisation & Context Probe
|
||||
|
||||
**File:** `src/utils/telegram-webapp.ts` · `getTelegramContext()`
|
||||
|
||||
The function assembles a `TelegramContext` object from:
|
||||
|
||||
1. `window.Telegram.WebApp` — primary SDK surface (available when the app is opened inside Telegram).
|
||||
2. URL query/hash fallback — `tgWebAppStartParam`, `tgWebAppData`, `tgWebAppVersion`, `tgWebAppPlatform` — used by older clients or during dev testing.
|
||||
|
||||
**Fields extracted:**
|
||||
|
||||
| Field | Source | Notes |
|
||||
|---|---|---|
|
||||
| `isMiniApp` | Any Telegram signal present | Drives unsupported vs unlinked state |
|
||||
| `initData` | `webApp.initData` or `tgWebAppData` URL param | HMAC-signed payload sent to `/api/auth/telegram` |
|
||||
| `initDataUnsafe` | `webApp.initDataUnsafe` | Client-side user identity (not trusted) |
|
||||
| `safeArea` | `contentSafeAreaInset` or `safe_area_insets` | Parsed to `{top, right, bottom, left}` in px |
|
||||
| `theme` | `webApp.themeParams` | Both camelCase and snake_case normalised |
|
||||
| `platform` | `webApp.platform` or URL param | e.g. `ios`, `android`, `tdesktop` |
|
||||
| `startParam` | `startapp` / `tgWebAppStartParam` / `start_param` | Deep-link context |
|
||||
| `isUnsupported` | `!webApp && Boolean(startParam)` | Partial signal — no SDK but has URL param |
|
||||
|
||||
**Polling on mount** (`useTelegramLiveContext`): Telegram sometimes finishes injecting the WebApp object after the first React render. The hook re-probes at 0 ms, 100 ms, 500 ms, and 1000 ms after mount, and also re-probes on `hashchange` events (triggered by the native back-button on some platforms).
|
||||
|
||||
---
|
||||
|
||||
## 4. Shell State Machine
|
||||
|
||||
`getTelegramStatus(context, hasWebAccount)` returns one of three states:
|
||||
|
||||
```
|
||||
unsupported ─── !context.isMiniApp
|
||||
(opened in browser, not Telegram)
|
||||
|
||||
unlinked ─────── isMiniApp && (!user || !telegramUser.id)
|
||||
(inside Telegram but no JWT session linked)
|
||||
|
||||
linked ──────── isMiniApp && user && telegramUser.id
|
||||
(authenticated, full shell rendered)
|
||||
```
|
||||
|
||||
State transitions occur on:
|
||||
- Auth session check completing (`loading → false`)
|
||||
- Telegram auto sign-in completing (`tgAuthLoading → false`)
|
||||
- Manual sign-in button tap (unlinked → linked)
|
||||
|
||||
---
|
||||
|
||||
## 5. Authentication Flow
|
||||
|
||||
### 5.1 Silent Auto Sign-In
|
||||
|
||||
**Hook:** `useTelegramAutoSignIn` · **File:** `hooks/use-telegram-auto-sign-in.ts`
|
||||
|
||||
On mount, if `context.isMiniApp && context.initData && !user`:
|
||||
|
||||
1. Exchange `initData` for a JWT by calling `signInWithTelegram({ initData })` → `POST /api/auth/telegram`.
|
||||
2. On success, call `checkUserSession()` to refresh the auth context.
|
||||
3. If the backend returns `isNewUser: true`, show `TelegramOnboardingSheet`.
|
||||
4. A `useRef` deduplication guard (`attemptedInitDataRef`) prevents re-runs under React Strict Mode's double-effect behaviour.
|
||||
|
||||
### 5.2 Manual Sign-In (Unlinked State)
|
||||
|
||||
When `initData` is present but auto sign-in failed (or hasn't run yet), `TelegramUnlinkedState` renders:
|
||||
- **Continue with Telegram** — calls the same `signIn()` function from `useTelegramAutoSignIn`.
|
||||
- **Sign in with email** — `window.location.assign(paths.auth.jwt.signIn)`.
|
||||
- **Create an account** — `window.location.assign(paths.auth.jwt.register)`.
|
||||
|
||||
When `initData` is absent (accessed via a path that skips Telegram context), only the email/register buttons appear.
|
||||
|
||||
### 5.3 Backend Endpoint
|
||||
|
||||
`POST /api/auth/telegram` — expects `{ initData: string }`. Backend verifies the HMAC using the Telegram bot token, extracts `user` from the payload, upserts a `User` record (`telegramId`, `telegramVerified: true`), and issues a JWT + refresh token. Returns `{ token, refreshToken, isNewUser }`.
|
||||
|
||||
---
|
||||
|
||||
## 6. Navigation Model
|
||||
|
||||
All navigation is in-shell React state — no Next.js router is involved.
|
||||
|
||||
```
|
||||
activeTab : 'home' | 'shop' | 'requests' | 'chat' | 'account'
|
||||
overlayScreen : 'new-request' | 'notifications' | 'cart' | null
|
||||
openConversationId : string | null
|
||||
openRequestId : string | null
|
||||
openSellerId : string | null
|
||||
```
|
||||
|
||||
**Priority rendering** (first match wins):
|
||||
|
||||
1. `openConversationId` → `TelegramChatThreadView`
|
||||
2. `openRequestId` → `TelegramRequestDetailView`
|
||||
3. `openSellerId` → `TelegramSellerShopView`
|
||||
4. `overlayScreen === 'cart'` → `TelegramCartView`
|
||||
5. `overlayScreen === 'notifications'` → `TelegramNotificationsView`
|
||||
6. `overlayScreen === 'new-request'` → `TelegramNewRequestView`
|
||||
7. `activeTab` → appropriate tab view
|
||||
|
||||
**Back button** (Telegram native `BackButton`) dismisses in reverse priority order: chat thread → request detail → seller shop → overlay → returns to `home` tab.
|
||||
|
||||
`BackButton` visibility: shown whenever `state === 'linked'` and either an overlay/drilldown is active, or `activeTab !== 'home'`.
|
||||
|
||||
`MainButton` visibility: hidden while any overlay is open. When visible:
|
||||
- **Linked** → "New Request" (opens `overlayScreen = 'new-request'`)
|
||||
- **Unlinked** → "Sign In" (navigates to the JWT sign-in page)
|
||||
|
||||
Both chrome buttons are styled with the amaneh saffron palette (`color: #C2410C`, `text_color: #FFFFFF`) via `setParams` (WebApp SDK >= 6.1).
|
||||
|
||||
---
|
||||
|
||||
## 7. Tab Structure
|
||||
|
||||
The shell has **five bottom tabs** rendered by `TelegramTabBar`:
|
||||
|
||||
| Tab | Icon | View | Purpose |
|
||||
|---|---|---|---|
|
||||
| Home | house | `TelegramHomeView` | Welcome banner, quick-action cards, escrow-state chips |
|
||||
| Shop | storefront | `TelegramShopView` | Sellers list; drill into seller store; add templates to cart |
|
||||
| Requests | list | `TelegramRequestsView` | User's escrow requests with status stepper |
|
||||
| Chat | speech bubble | `TelegramChatView` | Conversation list + support entry |
|
||||
| Account | person | `TelegramAccountView` | Profile, preferences, links to web dashboard sections |
|
||||
|
||||
`handleTabSelect` clears all overlays and drill-down IDs before switching tab.
|
||||
|
||||
---
|
||||
|
||||
## 8. Supported Flows
|
||||
|
||||
### 8.1 Home Tab
|
||||
|
||||
`TelegramHomeView` is the landing screen shown on first open. It contains:
|
||||
- **Welcome banner** (`TelegramWelcomeBanner`): escrow account summary, primary CTA.
|
||||
- **Quick-action cards** (`TelegramQuickActions`): shortcuts to Requests, Payments, Chat.
|
||||
- **Escrow state chips** (`TelegramEscrowStateChips`): legend of status values visible in the platform.
|
||||
|
||||
### 8.2 Shop Tab — Sellers List
|
||||
|
||||
**`TelegramShopView`** (`telegram-shop-view.tsx`):
|
||||
- Fetches all sellers via `useTelegramShops()` → SWR wrapping `getTemplateSellers()` → `GET /api/request-templates/sellers`.
|
||||
- Renders `TelegramShopRow` per seller: avatar, name, rating, template count, sales count.
|
||||
- Shows a floating cart badge button in the header when `totalItems > 0`; tap opens `overlayScreen = 'cart'`.
|
||||
- Tap a seller row → sets `openSellerId` → navigates to `TelegramSellerShopView`.
|
||||
|
||||
### 8.3 Shop Tab — Seller Store
|
||||
|
||||
**`TelegramSellerShopView`** (`telegram-seller-shop-view.tsx`):
|
||||
- Fetches seller + active templates via `useTelegramSellerShop(sellerId)` → `GET /api/request-templates/sellers/:id`.
|
||||
- Dark header: seller avatar, name, rating, template count, description.
|
||||
- Each template card shows: image, title, 2-line description, budget range, usage count.
|
||||
- **Two actions per template:**
|
||||
- **Add to cart / Remove from cart** — toggles item in `useTelegramCart` (localStorage, no API). Button is filled blue when not in cart, outline when added.
|
||||
- **Order this template** — `<a href>` to `/dashboard/request/from-template?shareableLink=...`. Exits the Mini App to the web dashboard (single-template direct order, bypasses cart).
|
||||
- Floating "Cart · N templates" sticky button at bottom when `totalItems > 0`; tap calls `onOpenCart()`.
|
||||
|
||||
### 8.4 Shopping Cart Overlay
|
||||
|
||||
**`TelegramCartView`** (`telegram-cart-view.tsx`):
|
||||
- Rendered as `overlayScreen = 'cart'`; dismissed by Telegram BackButton.
|
||||
- Lists each cart item: image, name, seller name, USDT price × quantity, +/− quantity controls, remove button.
|
||||
- Subtotal/total in USDT, locale-formatted (`fa-IR` for Persian, `en-US` for English); amounts always `dir="ltr"`.
|
||||
- **"Continue to payment"** — plain `<a href={paths.shops.checkout}>` link; exits Mini App to web checkout.
|
||||
|
||||
**Cart storage (`useTelegramCart`):**
|
||||
|
||||
- Reads/writes `localStorage` key **`app-request-template-checkout`** — the same key the web `RequestTemplateCheckoutProvider` reads. This is the cart handoff mechanism: the cart built in Telegram IS the cart the web checkout page hydrates.
|
||||
- Dispatches a custom `tg-cart-changed` DOM event on every write; listens on both that event and the native `storage` event so all open tabs stay in sync.
|
||||
- Operations: `addTemplate(template, seller)`, `removeItem(itemId)`, `changeQuantity(itemId, qty)`, `isInCart(templateId)`.
|
||||
- No API calls — cart is purely client-side until checkout.
|
||||
- Cart item model: `id`, `templateId`, `name`, `description`, `price` (from `template.budget.min`), `quantity`, `image`, `sellerId`, `sellerName`, `category`, `shareableLink`, `deliveryInfo`, `maxUsage`, `usageCount`, `remainingCapacity`.
|
||||
|
||||
### 8.5 Web Checkout Handoff
|
||||
|
||||
Destination: `/dashboard/shops/checkout` — `RequestTemplateCheckoutView` wrapped by `RequestTemplateCheckoutProvider`.
|
||||
|
||||
The provider reads the shared `localStorage` key and hydrates the TMA cart. The checkout is a 3-step stepper:
|
||||
|
||||
| Step | Component | Description |
|
||||
|---|---|---|
|
||||
| 0 (Cart review) | `RequestTemplateCheckoutCart` | Item list, quantities, remove, totals, discount/shipping |
|
||||
| 1 (Address) | `RequestTemplateCheckoutBillingAddress` | Physical address or online delivery email |
|
||||
| 2 (Payment) | `RequestTemplateCheckoutPayment` | Wallet payment + socket confirmation |
|
||||
| Complete | `RequestTemplateCheckoutOrderComplete` | Confirmation dialog, cart reset |
|
||||
|
||||
Payment execution calls `convertTemplatesToRequests()` to create escrow records, then awaits a `template-checkout-payment-confirmed` socket event. A guard checks `createdRequestIds` is non-empty before advancing (prevents stray global socket events from triggering premature completion). Stock validation clamps or removes items exceeding `remainingCapacity` before payment.
|
||||
|
||||
### 8.6 Browse Requests (Requests Tab)
|
||||
|
||||
- `TelegramRequestsView` fetches the user's purchase requests via `useTelegramMyRequests` (GET `/api/requests`).
|
||||
- Displays a skeleton loader, then a scrollable list of `TelegramRequestRow` items.
|
||||
- Each row shows: title, status chip, budget, creation date.
|
||||
- Tap → sets `openRequestId` → renders `TelegramRequestDetailView`.
|
||||
|
||||
### 8.7 Request Detail with Stepper
|
||||
|
||||
- `TelegramRequestDetailView` fetches a single request via `useTelegramRequest`.
|
||||
- Renders `TelegramRequestStepper` — a visual timeline of the escrow status flow from `pending_payment` → `completed`.
|
||||
- `determineCurrentStepFromStatus` maps the current `status` to a step index.
|
||||
- Also renders: budget, description, creation date, category, urgency.
|
||||
- Dates formatted via `toLocaleDateString` with `fa-IR` locale for Persian.
|
||||
|
||||
### 8.8 Create New Request
|
||||
|
||||
- `TelegramNewRequestView` is a full-screen overlay (not a routed page).
|
||||
- Form fields: title, description, category (fetched from `/api/categories`), budget min/max, urgency.
|
||||
- On submit: calls `createPurchaseRequest()` → POST `/api/purchase-requests`.
|
||||
- On success: closes overlay, switches `activeTab` to `'requests'`.
|
||||
- `MainButton` is hidden while the overlay is open (submit lives in the form itself).
|
||||
|
||||
### 8.9 Chat Tab
|
||||
|
||||
- `TelegramChatView` shows the user's active conversations via `useTelegramConversations`.
|
||||
- Includes a Support row that calls `createSupportChat()` → `POST /api/chat/support`, then opens `TelegramChatThreadView` with the returned conversation ID.
|
||||
- Tap a conversation row → sets `openConversationId` → renders `TelegramChatThreadView`.
|
||||
- `TelegramChatThreadView` loads messages via `useTelegramChatThread`, renders `TelegramChatBubble` items, and includes `TelegramChatComposer` for sending.
|
||||
- Optimistic send: message appears immediately, confirmed/rolled back on API response.
|
||||
- Real-time updates via Socket.IO events; SWR is mutated on `new-notification` and `unread-count-update` events.
|
||||
|
||||
### 8.10 Account Tab
|
||||
|
||||
**`TelegramAccountView`** (`telegram-account-view.tsx`):
|
||||
|
||||
The account tab has four sections. All user data is passed as props from the shell (loaded via `useAuthContext()` — no fetch on mount).
|
||||
|
||||
**Profile header:**
|
||||
- Avatar (from `user.profile.avatar`, falls back to initials), full name, Telegram `@username`, role chip (buyer / seller / admin / resolver / guard).
|
||||
- Verification chips: "Telegram Verified" (if `user.telegramVerified`) and "Email Verified" (if `user.isEmailVerified`).
|
||||
|
||||
**Preferences section:**
|
||||
- Language toggle (FA / EN, in-shell via `TelegramLanguageToggle`).
|
||||
- General Settings → `/dashboard/account` (web, labeled "Opens in the web dashboard").
|
||||
- Wallet → truncated address (`0x1234…abcd`) or "not connected" → `/dashboard/account/wallet` (web).
|
||||
- Notifications → opens `TelegramNotificationsView` overlay in-shell.
|
||||
- Addresses → `/dashboard/account/address` (web).
|
||||
- Passkey → `/dashboard/account/passkey` (web).
|
||||
|
||||
**Help section:**
|
||||
- Support → `createSupportChat()` → opens `TelegramChatThreadView` in-shell.
|
||||
- Terms & Conditions → placeholder, "coming soon".
|
||||
|
||||
**Session section:**
|
||||
- Sign Out → `TelegramBottomSheet` confirmation dialog → `authSignOut()` + `window.location.assign(paths.auth.jwt.signIn)`.
|
||||
|
||||
### 8.11 Notifications Overlay
|
||||
|
||||
- `TelegramNotificationsView` is rendered as `overlayScreen = 'notifications'`.
|
||||
- Fetches via `useTelegramNotifications` → `getNotifications(userId, 1, 50)` → `GET /api/notifications?userId=...&page=1&limit=50`.
|
||||
- Real-time updates: Socket.IO events `new-notification`, `unread-count-update` trigger SWR mutate.
|
||||
- "Mark all read" calls `markAllNotificationsAsRead(userId)` → `PATCH /api/notifications/mark-all-read`.
|
||||
|
||||
---
|
||||
|
||||
## 9. API Calls
|
||||
|
||||
| Action | Hook / call | Backend endpoint |
|
||||
|---|---|---|
|
||||
| Auto sign-in | `useTelegramAutoSignIn` → `signInWithTelegram({initData})` | `POST /api/auth/telegram` |
|
||||
| Sellers list | `useTelegramShops` → `getTemplateSellers()` | `GET /api/request-templates/sellers` |
|
||||
| Seller + templates | `useTelegramSellerShop` → `getSellerWithTemplates(id)` | `GET /api/request-templates/sellers/:id` |
|
||||
| Marketplace sellers | `useTelegramSellers` → `getSellers()` | `GET /api/marketplace/sellers` |
|
||||
| My requests | `useTelegramMyRequests` | `GET /api/requests` |
|
||||
| Single request | `useTelegramRequest` | `GET /api/purchase-requests/:id` |
|
||||
| Create request | shell → `createPurchaseRequest()` | `POST /api/purchase-requests` |
|
||||
| Conversations | `useTelegramConversations` | `GET /api/chat/conversations` |
|
||||
| Chat thread | `useTelegramChatThread` | `GET /api/chat/:id` + Socket.IO real-time |
|
||||
| Support chat | `createSupportChat()` | `POST /api/chat/support` |
|
||||
| Notifications | `useTelegramNotifications` | `GET /api/notifications?userId=...&page=1&limit=50` |
|
||||
| Mark all read | `markAllNotificationsAsRead(userId)` | `PATCH /api/notifications/mark-all-read` |
|
||||
| Auth sign-out | `authSignOut()` | JWT sign-out endpoint |
|
||||
|
||||
Cart operations (add/remove/quantity) are **pure localStorage** — no API calls until web checkout.
|
||||
|
||||
---
|
||||
|
||||
## 10. Bilingual Support (EN / FA)
|
||||
|
||||
**Language detection priority** (`useTelegramLanguage`):
|
||||
|
||||
1. `?lang=` URL query param — dev preview override.
|
||||
2. `localStorage` key `amn_tg_lang` — user's persisted manual selection.
|
||||
3. `initDataUnsafe.user.language_code` — Telegram-reported language (`"fa"` or `"fa-IR"` → Persian).
|
||||
4. Fallback → English.
|
||||
|
||||
**Language toggle:** `TelegramLanguageToggle` in the header — two buttons `[ EN | فا ]`. On tap: haptic light + language switch + persist to `localStorage`.
|
||||
|
||||
**RTL layout:**
|
||||
|
||||
| Element | EN (LTR) | FA (RTL) |
|
||||
|---|---|---|
|
||||
| Root `dir` attribute | `ltr` | `rtl` |
|
||||
| Font family | IBM Plex Sans | Vazirmatn |
|
||||
| Arrow icons | `→` | `←` |
|
||||
| Text alignment | left | right (inherits from `dir`) |
|
||||
| Chip list wrap | left-to-right | right-to-left |
|
||||
| Amounts | always `dir="ltr"` | always `dir="ltr"` |
|
||||
|
||||
Font size bumps for Persian: body 13 px → 14 px, labels 10 px → 11 px (Vazirmatn renders optically smaller).
|
||||
|
||||
**Translation structure:**
|
||||
|
||||
```ts
|
||||
// src/sections/telegram/locales/en.ts + fa.ts
|
||||
const TR = {
|
||||
en: { loading, unsupported, unlinked, header, home, shop, requests,
|
||||
chat, account, newRequest, tabs, main, onboarding, errors, displayName, dir },
|
||||
fa: { /* same keys, Farsi strings, dir: 'rtl' */ },
|
||||
};
|
||||
```
|
||||
|
||||
All JSX uses `t.<section>.<key>` — no inline strings in components.
|
||||
|
||||
---
|
||||
|
||||
## 11. Design System
|
||||
|
||||
**File:** `src/sections/telegram/constants.ts` · `src/sections/telegram/telegram-shell-css.ts`
|
||||
|
||||
The Mini App has a distinct visual identity (cream/saffron Persian palette) that does not inherit from the main dashboard theme. All tokens are feature-scoped.
|
||||
|
||||
**Palette:** `TG_PALETTE`
|
||||
|
||||
| Token | Hex | Usage |
|
||||
|---|---|---|
|
||||
| `cream50` | `#FBF6EB` | Page background |
|
||||
| `ink900` | `#1C1410` | Primary text |
|
||||
| `ink600` | `#6B5D4E` | Secondary text / labels |
|
||||
| `saffron600` | `#C2410C` | Primary action, MainButton |
|
||||
| `saffron500` | `#D97757` | Hover states |
|
||||
| `pistachio700` | `#3D6B4F` | Success / released states |
|
||||
| `pomegranate700` | `#8E2424` | Error / disputed states |
|
||||
| `bgPage` | `#E7DFCB` | Shell outer background |
|
||||
|
||||
**Fonts:** `TG_FONTS` — Source Serif 4 (headings), IBM Plex Sans (body LTR), Vazirmatn (body RTL), IBM Plex Mono (amounts/addresses).
|
||||
|
||||
**CSS:** `buildTelegramShellCss()` injects a `<style>` tag at shell root with all class utilities (`.tg-chip`, `.tg-shell`, `.tg-tab-bar`, `.tg-header`, etc.). Theme CSS variables (`--cream-50`, `--ink-900`, etc.) are set on `.tg-shell` root.
|
||||
|
||||
**Safe area:** `getTelegramSafeAreaStyle(safeArea)` maps the Telegram-reported safe area insets to CSS padding using `max(${px}px, env(safe-area-inset-*))` to handle both Telegram-native and iOS/Android safe areas.
|
||||
|
||||
---
|
||||
|
||||
## 12. Telegram SDK Usage Patterns
|
||||
|
||||
### 12.1 Safe-Area Inset
|
||||
|
||||
```ts
|
||||
// TelegramContext.safeArea = { top, right, bottom, left } (px)
|
||||
// Source: webApp.contentSafeAreaInset || webApp.safe_area_insets
|
||||
// Normalised to number via parseNumber() — rejects non-finite strings
|
||||
const topInset = (context.safeArea?.top ?? 0) as number;
|
||||
```
|
||||
|
||||
All views receive `topInset` / `bottomInset` props and add them as explicit `paddingTop` / `paddingBottom` to avoid content being obscured by the Telegram chrome.
|
||||
|
||||
### 12.2 Haptic Feedback
|
||||
|
||||
```ts
|
||||
// useTelegramHaptic(webApp) → haptic('light' | 'medium')
|
||||
webApp?.HapticFeedback?.impactOccurred?.(type)
|
||||
```
|
||||
|
||||
Used on: tab switches (light), new-request CTA (medium), language toggle (light), back button (light). All calls are wrapped in try/catch — the API may be absent on older clients.
|
||||
|
||||
### 12.3 Back Button
|
||||
|
||||
```ts
|
||||
useTelegramBackButton({ webApp, isVisible, onClick })
|
||||
// Calls webApp.BackButton.show() / hide() and registers onClick handler
|
||||
// Cleanup: offClick() on unmount / visibility change
|
||||
```
|
||||
|
||||
### 12.4 Main Button
|
||||
|
||||
```ts
|
||||
useTelegramMainButton({ webApp, isReady, text, onClick })
|
||||
// Calls webApp.MainButton.show() / hide(), setText(), setParams()
|
||||
// Saffron palette: color: '#C2410C', text_color: '#FFFFFF'
|
||||
// setParams requires WebApp >= 6.1; silent fallback for older clients
|
||||
```
|
||||
|
||||
### 12.5 Theme Integration
|
||||
|
||||
Telegram's `themeParams` is normalised (both camelCase and snake_case accepted) and injected as CSS custom properties on the shell root (`--telegram-shell-bg`, `--telegram-shell-text`, etc.). The amaneh palette overrides these for the Mini App's own UI, but components can reference them for adaptive behaviours.
|
||||
|
||||
---
|
||||
|
||||
## 13. Edge Cases
|
||||
|
||||
| Scenario | Detection | Handling |
|
||||
|---|---|---|
|
||||
| Opened in browser (not Telegram) | `context.isMiniApp === false` | `TelegramUnsupportedState` — shows "Open in Telegram" badge, web dashboard link |
|
||||
| Partial Telegram signal (URL params but no SDK) | `!webApp && Boolean(startParam)` → `isUnsupported: true` | Same unsupported state |
|
||||
| Telegram SDK injected late | `useTelegramLiveContext` polls at 0/100/500/1000 ms | Re-probes until SDK is ready; seed context bypasses polling |
|
||||
| `initData` absent (no auth data) | `!context.initData` in unlinked state | Sign-in button triggers error string `t.errors.no_init_data`; email/create buttons remain available |
|
||||
| Auto sign-in replay (React Strict Mode) | `attemptedInitDataRef.current === context.initData` | Deduplication ref — second effect is a no-op |
|
||||
| Backend sign-in failure | Catch block in `useTelegramAutoSignIn` | Error string displayed in `TelegramUnlinkedState`; retry via "Continue with Telegram" |
|
||||
| New user first login | `result.isNewUser === true` | `TelegramOnboardingSheet` shown over the shell; dismissed to account settings or "Later" |
|
||||
| Expired session inside Mini App | Auth context `user === null` after session check | Shell falls back to `unlinked` state |
|
||||
| Old Telegram client (< 6.1) | `setParams` throws | Try/catch silences it; button shows without saffron colour |
|
||||
| RTL + keyboard overlap | Viewport shrinks on soft keyboard open | `flex: 1` + `overflowY: auto` on content area; bottom safe-area inset on tab bar |
|
||||
| Persian locale date formatting | `lang === 'fa'` | `toLocaleDateString('fa-IR', ...)` in `formatDate` helper |
|
||||
| Cart cross-tab sync | Multiple tabs / Mini App + web | `tg-cart-changed` DOM event + `storage` event both trigger re-render |
|
||||
| Template at capacity | `remainingCapacity === 0` at checkout | Stock validation clamps/removes over-capacity items before payment |
|
||||
| Stray global socket on checkout | `template-checkout-payment-confirmed` fires unexpectedly | Guard checks `createdRequestIds.length > 0` before advancing to completion step |
|
||||
|
||||
---
|
||||
|
||||
## 14. File Map
|
||||
|
||||
```
|
||||
src/
|
||||
app/telegram/page.tsx # Next.js route (thin shell, no auth guard)
|
||||
utils/telegram-webapp.ts # SDK probe, context types, shell style helpers
|
||||
sections/telegram/
|
||||
constants.ts # TG_PALETTE, TG_FONTS, TG_EASE, status maps
|
||||
telegram-shell-css.ts # buildTelegramShellCss() — inlined CSS blob
|
||||
index.ts # barrel
|
||||
locales/
|
||||
types.ts # TelegramDict, TelegramLang, TelegramTabId
|
||||
en.ts # English strings
|
||||
fa.ts # Persian strings
|
||||
index.ts # getTelegramDict(lang)
|
||||
hooks/
|
||||
use-telegram-live-context.ts # SDK polling
|
||||
use-telegram-language.ts # EN/FA detection + ?lang= + localStorage persist
|
||||
use-telegram-auto-sign-in.ts # initData → JWT exchange
|
||||
use-telegram-main-button.ts # MainButton lifecycle
|
||||
use-telegram-back-button.ts # BackButton lifecycle
|
||||
use-telegram-haptic.ts # HapticFeedback wrapper
|
||||
use-telegram-cart.ts # localStorage cart (shared with web checkout)
|
||||
use-telegram-shops.ts # GET /api/request-templates/sellers
|
||||
use-telegram-seller-shop.ts # GET /api/request-templates/sellers/:id
|
||||
use-telegram-sellers.ts # GET /api/marketplace/sellers
|
||||
use-telegram-my-requests.ts # GET /api/requests
|
||||
use-telegram-request.ts # GET /api/purchase-requests/:id
|
||||
use-telegram-conversations.ts # Chat conversation list
|
||||
use-telegram-chat-thread.ts # Chat thread + optimistic send
|
||||
use-telegram-notifications.ts # GET /api/notifications
|
||||
index.ts
|
||||
view/
|
||||
telegram-mini-app-view.tsx # Shell orchestrator (all state lives here)
|
||||
telegram-home-view.tsx # Home tab
|
||||
telegram-shop-view.tsx # Shop tab — sellers list
|
||||
telegram-seller-shop-view.tsx # Seller store drill-down + cart actions
|
||||
telegram-cart-view.tsx # Cart overlay
|
||||
telegram-requests-view.tsx # Requests list tab
|
||||
telegram-request-detail-view.tsx # Request drilldown + stepper
|
||||
telegram-new-request-view.tsx # New request overlay form
|
||||
telegram-chat-view.tsx # Chat conversation list tab
|
||||
telegram-chat-thread-view.tsx # Chat thread drilldown
|
||||
telegram-account-view.tsx # Account + preferences + sign-out tab
|
||||
telegram-notifications-view.tsx # Notifications overlay
|
||||
index.ts
|
||||
components/
|
||||
telegram-header.tsx # AMN logo + subtitle + language toggle
|
||||
telegram-tab-bar.tsx # Bottom tab bar (5 tabs)
|
||||
telegram-welcome-banner.tsx # Home: escrow account banner + CTA
|
||||
telegram-quick-actions.tsx # Home: action cards (Requests / Payments / Chat)
|
||||
telegram-escrow-state-chips.tsx # Home: status chip legend
|
||||
telegram-shop-row.tsx # Shop: seller list row
|
||||
telegram-request-row.tsx # Requests: list row
|
||||
telegram-request-stepper.tsx # Detail: visual escrow timeline
|
||||
telegram-list-row.tsx # Generic list row primitive
|
||||
telegram-list-skeleton.tsx # Skeleton loader for lists
|
||||
telegram-chat-row.tsx # Chat: conversation list row
|
||||
telegram-chat-bubble.tsx # Chat: message bubble
|
||||
telegram-chat-composer.tsx # Chat: message input
|
||||
telegram-loading-state.tsx # Loading spinner state
|
||||
telegram-unlinked-state.tsx # Unlinked / sign-in prompt state
|
||||
telegram-unsupported-state.tsx # Not-in-Telegram fallback state
|
||||
telegram-onboarding-sheet.tsx # New-user onboarding bottom sheet
|
||||
telegram-empty-state.tsx # Generic empty list state
|
||||
telegram-language-toggle.tsx # EN | FA header toggle
|
||||
telegram-bottom-sheet.tsx # Generic bottom sheet primitive
|
||||
telegram-form-field.tsx # Form field + input style helper
|
||||
telegram-seal-mark.tsx # SealMark logo component
|
||||
telegram-icons.tsx # Telegram-scoped icon set
|
||||
index.ts
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 15. Current Implementation Status (v2.8.59)
|
||||
|
||||
| Area | Status | Notes |
|
||||
|---|---|---|
|
||||
| Shell + state machine | Done | `TelegramMiniAppView` — all states wired |
|
||||
| SDK probe + live context | Done | Polling + hashchange listener |
|
||||
| Auto sign-in | Done | Deduped initData exchange |
|
||||
| Manual sign-in (unlinked) | Done | Email + create account fallbacks |
|
||||
| Bilingual EN/FA | Done | Full string inventory, RTL layout, Vazirmatn font |
|
||||
| Language toggle | Done | Header toggle + localStorage persist |
|
||||
| `?lang=` dev preview param | Done | URL param override added to `useTelegramLanguage` |
|
||||
| Home tab | Done | Banner + quick actions + state chips |
|
||||
| Shop tab — sellers list | Done | API-backed with skeleton + empty states, cart badge |
|
||||
| Shop tab — seller store | Done | Templates list, add/remove cart, direct order link |
|
||||
| Shopping cart (localStorage) | Done | Shared key with web checkout; cross-tab sync |
|
||||
| Cart overlay | Done | Quantity controls, remove, total, checkout link |
|
||||
| Web checkout handoff | Done | localStorage handoff; stock guard; socket guard |
|
||||
| Requests list | Done | API-backed with skeleton + empty states |
|
||||
| Request detail + stepper | Done | Status timeline, budget, dates with fa-IR locale |
|
||||
| New request form | Done | In-shell overlay, category fetch, validation |
|
||||
| Chat list | Done | API-backed conversation list + support row |
|
||||
| Chat thread | Done | Messages + optimistic send + Socket.IO real-time |
|
||||
| Account tab | Done | Profile, preferences, help, web-dashboard links, sign-out |
|
||||
| Notifications overlay | Done | API-backed; Socket.IO real-time; mark-all-read |
|
||||
| Telegram chrome (MainButton / BackButton) | Done | Saffron palette, lifecycle hooks |
|
||||
| Haptic feedback | Done | All tap interactions |
|
||||
| Safe area insets | Done | Normalised from SDK + CSS env() fallback |
|
||||
| Deep link `startapp` context | Partial | Parsed but not yet used to auto-navigate to a request |
|
||||
| Bilingual onboarding sheet | Done | Shown on `isNewUser` flag |
|
||||
| Unsupported / browser fallback | Done | Web dashboard link |
|
||||
|
||||
### Open Items
|
||||
|
||||
- `startapp` deep link routing: if `context.startParam` matches `req_<id>`, auto-open `TelegramRequestDetailView` on first render.
|
||||
- Backend room-scoped Socket.IO for real-time chat updates (global socket event broadcast was fixed client-side in v2.8.4; server-side scoping is a follow-up).
|
||||
|
||||
---
|
||||
|
||||
## 16. Related Documents
|
||||
|
||||
- [[PRD - Telegram Mini App Bilingual (EN + FA)]] — bilingual string inventory and RTL layout spec
|
||||
- [[PRD - Telegram Phone Number Authentication]] — phone-number auth as a future sign-in path
|
||||
- [[Authentication Flow]] — JWT lifecycle shared with the Mini App auth
|
||||
- [[Purchase Request Flow]] — escrow state machine surfaced in the stepper
|
||||
- [[Chat Flow]] — real-time messaging that the Mini App embeds
|
||||
- [[Request Template Checkout]] — web checkout flow that the Mini App cart hands off to
|
||||
@@ -11,6 +11,16 @@ entries on top. Maintained by agents per the rule in `../AGENTS.md`.
|
||||
|
||||
---
|
||||
|
||||
### 2026-06-03 — frontend@9bafbbb — Telegram Mini App: full in-shell shop, account tab parity, and shopping cart (v2.8.57–v2.8.59)
|
||||
|
||||
**Commits:** `a8ae1e3` (v2.8.57), `6dc3918` (v2.8.58), `9bafbbb` (v2.8.59) — frontend only; backend stays at v2.8.56
|
||||
**Touched:** `telegram-mini-app-view.tsx` (shell nav: `openSellerId`, `overlayScreen='cart'`, BackButton dismissal chain), `telegram-shop-view.tsx` (cart badge header button, `TelegramShopRow` converted from `<a href>` to in-shell `onOpen`), `telegram-seller-shop-view.tsx` (new — seller header, active templates with budget/usage, add/remove-to-cart buttons, floating cart CTA), `telegram-cart-view.tsx` (new — qty stepper, remove, USDT total, "Continue to payment" → web checkout), `telegram-account-view.tsx` (new — profile header, preferences section, help section, sign-out with bottom-sheet confirm), `hooks/use-telegram-seller-shop.ts` (new — SWR over `getSellerWithTemplates`), `hooks/use-telegram-cart.ts` (new — localStorage `app-request-template-checkout`, custom `tg-cart-changed` event), `hooks/use-telegram-shops.ts` (new — SWR over `getTemplateSellers`), telegram locales (fa/en/types)
|
||||
**Why:** Three sequential buyer-parity milestones shipped in one session. (1) v2.8.57: tapping a seller in فروشگاه previously opened the web dashboard inside the webview; the seller store now renders entirely in-shell with `TelegramSellerShopView` — seller header, active templates, budget/usage count — and template ordering hands off to the web from-template checkout where the wallet stack lives. (2) v2.8.58: the web account menu (تنظیمات عمومی، اعلانها، کیف پول، آدرسهای تحویل، Passkey) had no Mini App counterpart; all five are now accessible from the account tab — notifications open the existing in-shell overlay, the other four deep-link to web dashboard pages (labeled «در داشبورد وب باز میشود») because passkey and wallet require browser-context APIs unavailable in Telegram. (3) v2.8.59: phase 1 of full buyer parity — new `useTelegramCart` hook writes to the same localStorage key (`app-request-template-checkout`) that the web checkout provider reads, so the Mini App cart IS the web cart with no sync step; `TelegramCartView` adds qty controls, remove, and a "Continue to payment" link that hands off seamlessly. Remaining buyer-parity roadmap: offers view/accept on requests, delivery confirmation, payments list, points/referral, addresses CRUD, TON Connect payments (Telegram Wallet is TON-native; backend already has `tonProofService`).
|
||||
**Verification:** tsc + eslint clean across all three commits. Admin verifies after deploy: (shop) فروشگاه → tap seller → in-shell shop → سفارش این قالب → web checkout; (cart) افزودن به سبد → cart badge → سبد overlay → ادامه و پرداخت → web checkout shows same items; (account) account tab → preferences rows → notifications overlay in-shell, remaining rows open web dashboard.
|
||||
**Linked docs updated:** `04 - Flows/Telegram Mini App.md` (major update — navigation model, all view files, shop/cart/checkout flow, account tab, SDK surfaces, API call table), `01 - Architecture/Frontend Architecture.md` (updated — Telegram Mini App section)
|
||||
|
||||
---
|
||||
|
||||
### 2026-06-02 — backend@cf59726, frontend@a2b972b — normalize Postgres repository store modes
|
||||
|
||||
**Commits:** backend `cf59726` (version `2.8.37`), frontend `a2b972b` (version `2.8.37`)
|
||||
@@ -679,4 +689,14 @@ TON-only, no EVM; backend already has tonProofService).
|
||||
|
||||
---
|
||||
|
||||
### 2026-06-02 — backend@7c4dedf — complete dual-write repos, migrations pipeline, TTL scheduler, address reconciliation
|
||||
|
||||
**Commits:** `7c4dedf` (backend v2.8.44), frontend v2.8.44
|
||||
**Touched:** `src/db/repositories/dual/DualWriteDisputeRepo.ts` (new), `DualWriteTrezorAccountRepo.ts` (new), `DualWriteDerivedDestinationRepo.ts` (new), `src/db/repositories/factory.ts`, `src/db/schema/dispute.ts`, `src/db/schema/address.ts`, `src/db/schema/fundsLedgerEntry.ts`, `src/services/address/addressStore.ts`, `src/services/admin/dataCleanupService.ts`, `src/services/admin/ttlCleanupJob.ts` (new), `src/app.ts`, `src/shared/types/address.ts`, `drizzle.config.ts`, `migrations/` (new), `__tests__/dispute-dual-write.test.ts` (new)
|
||||
**Why:** Complete all remaining Phase 1 migration prerequisites: the three missing DualWrite repos (Dispute, TrezorAccount, DerivedDestination) unblock PG cutover for those domains; migrations pipeline enables drizzle-kit schema management; FundsLedgerEntry immutability trigger (DDL in migrations/) protects money records from mutation; Dispute composite indexes match Mongo query patterns; TTL scheduler replaces Mongo TTL indexes for Notification/TempVerification/TelegramSession; Address schema reconciled so Drizzle is authoritative.
|
||||
**Verification:** typecheck clean. 5 core suites (11 tests) pass. DualWriteDisputeRepo 21 tests pass.
|
||||
**Linked docs updated:** `MIGRATION_TODO.md` — all 9 tasks marked done.
|
||||
|
||||
---
|
||||
|
||||
<!-- Add new entries above this line. Newest at top. -->
|
||||
|
||||
449
mongo-to-pg-migration-prd.md
Normal file
449
mongo-to-pg-migration-prd.md
Normal file
@@ -0,0 +1,449 @@
|
||||
## Status Update — 2026-06-03
|
||||
|
||||
**Backend version:** v2.8.56
|
||||
**Updated:** 2026-06-03
|
||||
|
||||
### Infrastructure Milestones Reached
|
||||
|
||||
| Milestone | Status |
|
||||
|---|---|
|
||||
| Drizzle migrations landed | ✅ 17 migrations (0000–0017) |
|
||||
| Drizzle schema coverage | ✅ All 25+ entities have schemas |
|
||||
| Dual-write repositories | ✅ All entities covered |
|
||||
| PG-only boot (`MONGO_CONNECT_MODE=never`) | ✅ Confirmed working, seeds pass |
|
||||
| Fresh DB path (`drizzle-kit migrate` + seed) | ✅ Verified end-to-end |
|
||||
| Read cutover | ❌ Not yet — most domains still read from Mongo |
|
||||
| Chat normalization | ❌ Blocker for Chat read cutover (JSONB shim remains) |
|
||||
| Production backfill | ❌ Not yet executed — runbook at `src/db/BACKFILL_RUNBOOK.md` |
|
||||
|
||||
### Current State Summary
|
||||
|
||||
The migration has reached **write-parity**: every entity is dual-written to Postgres, the schema is complete, and the system can boot and seed against a fresh PG-only database. The remaining work is on the **read side**: flipping each domain's reads from Mongo to Postgres, gated by backfill execution (human-gated) and Chat normalization (engineering blocker).
|
||||
|
||||
**Critical path remains unchanged:** Chat normalization (4–6 wks) must precede any Chat read cutover. All other Phase 1/2 domains can proceed to read cutover independently once their backfill is confirmed correct.
|
||||
|
||||
### Next Steps (in order)
|
||||
|
||||
1. Execute production backfill per `src/db/BACKFILL_RUNBOOK.md` (human-gated, per-domain)
|
||||
2. Enable shadow-reads per domain, validate row-count and field parity
|
||||
3. Flip read flags domain by domain (User → Category/Shop/Level → Payment → Dispute → …)
|
||||
4. Begin Chat normalization (child tables replacing JSONB shim)
|
||||
|
||||
---
|
||||
|
||||
# PRD: Mongo→Postgres Migration — Escrow Backend
|
||||
|
||||
**Status:** In Progress — Foundation tasks complete, dual-write coverage 100%
|
||||
**Date:** 2026-06-02 (audit) / updated 2026-06-02 (9 tasks completed)
|
||||
**Backend version:** 2.8.44 on `integrate-main-into-development`
|
||||
**Source:** Automated audit via `mongo-to-pg-migration-audit` workflow (49 agents, 528 DB operations catalogued)
|
||||
**Target:** Full migration — 23 Mongoose models → Postgres + Drizzle ORM
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
The escrow backend currently runs a hybrid Mongo + Postgres architecture with an in-progress migration. The audit found **23 Mongoose models, 528 catalogued DB operations, 96 relationships, 110+ `.populate()` sites, 14 aggregation pipelines, and 6 transaction sites** that must be migrated. Migration infrastructure was **~80% scaffolded** at audit time.
|
||||
|
||||
**As of 2026-06-02, all 9 MIGRATION_TODO foundation tasks are complete:**
|
||||
- ✅ All **9 DualWrite repos** exist (3 missing repos implemented)
|
||||
- ✅ **Migration pipeline** established (migrations/ dir, drizzle-kit scripts)
|
||||
- ✅ **FundsLedgerEntry immutability trigger** SQL migration created
|
||||
- ✅ **Dispute composite indexes** added to Drizzle schema
|
||||
- ✅ **TTL scheduled cleanup** implemented and wired into app startup
|
||||
- ✅ **Address dual schema reconciled** (Drizzle authoritative)
|
||||
- ✅ **Seed script audit** complete
|
||||
- ✅ **21 new tests** for DualWriteDisputeRepo
|
||||
|
||||
**Remaining:** Backfill execution (human-gated, 14 scripts exist), Chat normalization decision, env var cutover.
|
||||
|
||||
**Recommendation:** Incremental strangler pattern, phased over 3 stages with dual-write + shadow-read verification at each cutover. Total realistic estimate: **28 engineer-weeks** for full migration (24–33 range), or **17 engineer-weeks** for money/relational core only.
|
||||
|
||||
**Critical path blocker:** The Chat domain's current JSONB document shim is non-scalable and must be normalized to child tables before production cutover (4–6 wks alone).
|
||||
|
||||
---
|
||||
|
||||
## 1. Scope & Inventory
|
||||
|
||||
### 1.1 Collections (23)
|
||||
|
||||
| Collection | Complexity | Migration Status | Phase |
|
||||
|---|---|---|---|
|
||||
| User | High (30+ fields, embedded arrays) | AuthStore dual-write behind flag | 1 |
|
||||
| PurchaseRequest | Critical (7 child tables, 40+ fields, 13-value enum) | Drizzle schema exists, no backfill | 1 |
|
||||
| SellerOffer | Medium (2 embedded subdocs) | Drizzle schema exists, no backfill | 1 |
|
||||
| Payment | Critical (3 polymorphic Mixed FKs) | Drizzle + DualWrite repos done | 1 |
|
||||
| FundsLedgerEntry | Critical (append-only, 3 polymorphic FKs) | In DrizzlePaymentRepo | 1 |
|
||||
| Dispute | High (4 embedded arrays, pre-save hook) | ✅ DualWrite done, composite indexes added, 21 tests | 1 |
|
||||
| Category | Low (self-referential FK) | Dual-wired in production | 1 |
|
||||
| ShopSettings | Low | Dual-wired in production | 1 |
|
||||
| LevelConfig | Low | Dual-wired in production | 1 |
|
||||
| ConfigSetting | Low (key-value) | Via configStore adapter | 1 |
|
||||
| Address | Low | ✅ Schema reconciled, `ensurePostgresAddressSchema` → stub, `IAddress` fixed | 2 |
|
||||
| Review | Low | Dual-write + backfill done | 2 |
|
||||
| BlogPost | Low | Full dual-write done | 2 |
|
||||
| PointTransaction | Low (append-only) | Dual-write done | 2 |
|
||||
| Notification | Medium (TTL index) | ✅ Dual-write done, TTL DataCleanupService implemented | 2 |
|
||||
| TrezorAccount | Medium (child table) | ✅ DualWrite repo done, factory wired | 2 |
|
||||
| RequestTemplate | Medium (6 embedded arrays) | Drizzle schema exists, no backfill | 2 |
|
||||
| DerivedDestination | Medium (polymorphic sellerId/sellerOfferId) | ✅ DualWrite repo done, factory wired | 2 |
|
||||
| Chat | **Highest** (4-level nesting, JSONB shim) | Drizzle shim exists, needs normalization | 3 |
|
||||
| TelegramSession | Low (TTL) | ✅ Dual-write in authStore, TTL DataCleanupService implemented | 3 |
|
||||
| TelegramLink | Low | Dual-write deployed in production | 3 |
|
||||
| TempVerification | Low (TTL) | ✅ PgTempVerification behind flag, TTL DataCleanupService implemented | 3 |
|
||||
| ConfigSettingHistory | Low (audit trail) | Raw SQL table exists, Drizzle schema exists | 1 |
|
||||
|
||||
### 1.2 Service Domains (19)
|
||||
|
||||
| Domain | Ops | Models Touched | Migration State |
|
||||
|---|---|---|---|
|
||||
| auth | 47+ | User, TempVerification, TelegramLink, TelegramSession | Behind AUTH_STORE flag |
|
||||
| marketplace | 60+ | PurchaseRequest, SellerOffer, Category, RequestTemplate, ShopSettings | Partial Drizzle ORM paths |
|
||||
| payment | 30 | Payment, FundsLedgerEntry | Full dual-write |
|
||||
| points | 48 | User, PointTransaction, LevelConfig | Triple parallel repos |
|
||||
| chat | 10 | Chat | JSONB shim only |
|
||||
| dispute | 22 | Dispute, Chat | DualWrite missing |
|
||||
| notification | 10 | Notification | Dual-write done |
|
||||
| delivery | 7 | PurchaseRequest (via marketplace repo) | Via repo seam |
|
||||
| blog | 12 | BlogPost | Dual-write done |
|
||||
| address | 11 | Address | Dual-write partially wired |
|
||||
| trezor | 3+ | TrezorAccount | DualWrite missing |
|
||||
| telegram | 19 | 6 collections | Mixed read paths |
|
||||
| user | 47+ | User | Via authStore |
|
||||
| admin | 5+ | User, various | Partial migration |
|
||||
| blockchain | 3 | Payment (via repo) | Via repo abstraction |
|
||||
| email | 0 | None | Pure side-effect layer |
|
||||
| file | 1 | User (avatar URL) | Filesystem + single write |
|
||||
| health | 0 | None | Connectivity probe only |
|
||||
| redis | 0 | None | Redis-only, no migration needed |
|
||||
|
||||
### 1.3 Cross-Cutting Inventory
|
||||
|
||||
| Concern | Count | Detail |
|
||||
|---|---|---|
|
||||
| `.populate()` call sites | **110+** | Across 7 files, User is dominant target (20+ FK paths) |
|
||||
| `.aggregate()` pipelines | **14** | 13 are simple $match+$group; 1 complex with 2× $lookup |
|
||||
| Transaction sites | **6** | 4 Mongoose + 2 PG SERIALIZABLE |
|
||||
| `$inc` atomic update sites | **6** | Points and referral flows |
|
||||
| Mixed/polymorphic ID fields | **11** | Across Payment, FundsLedgerEntry, DerivedDestination |
|
||||
| TTL indexes | **3** | Notification (90d), TempVerification, TelegramSession |
|
||||
| Mongoose pre/post hooks | **4 models** | Address, Dispute, BlogPost, PurchaseRequest |
|
||||
| Backfill scripts written | **8** | Per-store, variable maturity |
|
||||
| DualWrite repos implemented | **6** | Payment, Marketplace, Points, User, Blog, Notification |
|
||||
| Drizzle repos implemented | **11** | Covering all Phase 1 models + some Phase 2 |
|
||||
|
||||
---
|
||||
|
||||
## 2. Strategy Recommendation
|
||||
|
||||
### Option A — Big-Bang ❌
|
||||
|
||||
Cut all 23 collections in a single deployment. **Rejected** — unacceptable risk, no rollback plan, Chat shim would go live unscalable, TTL replacements not in place.
|
||||
|
||||
### Option B — Strangler (Incremental) ✅ **Recommended**
|
||||
|
||||
Per-domain dual-write → backfill → shadow-read → PG cutover → retire Mongo mirror. The codebase is already architected for this with 6 DualWrite repos, repository factory (425-line `factory.ts`), and AUTH_STORE=postgres proving the pattern works.
|
||||
|
||||
### Option C — Hybrid (Mongo for documents, PG for relational) ⚠️
|
||||
|
||||
Keep Chat/Notification/BlogPost/TelegramSession/TempVerification in Mongo permanently. **Pragmatic stepping stone** but strands the Drizzle investment already made for those collections.
|
||||
|
||||
### Decision
|
||||
|
||||
**Go with Option B (strangler), sequenced as Option C's subset first.** Phase 1 migrates the money/relational core where PG adds the most value (atomicity, FK constraints, money safety). Phase 2 migrates content/engagement. Phase 3 tackles document-shaped collections, with Chat normalization as the gating deliverable.
|
||||
|
||||
---
|
||||
|
||||
## 3. Phased Migration Plan
|
||||
|
||||
### Phase 0: Foundation — 2–3 eng-wks
|
||||
|
||||
| # | Deliverable | Detail |
|
||||
|---|---|---|
|
||||
| P0.1 | Migration pipeline | `npm run db:migrate` + `npm run db:generate` via drizzle-kit |
|
||||
| P0.2 | id_map utility | Cached (LRU) resolveLegacyId/resolveUuid, warm on startup |
|
||||
| P0.3 | Gap reconciliation | Script reads pg_dualwrite_gaps, retries PG writes, alerts on persistent failure |
|
||||
| P0.4 | Monitoring | Row-count diff metrics (Mongo vs PG per collection), shadow-read mismatch alerts |
|
||||
| P0.5 | CI verify step | `db:verify` connects both DBs, confirms dual-write mirrors alive, detects schema drift |
|
||||
| P0.6 | Rollback drill | Documented procedure: flip feature flag → Mongo, confirm reads/writes, reconcile gaps |
|
||||
|
||||
### Phase 1: Money + Marketplace Core — 14–16 eng-wks
|
||||
|
||||
**Order:** User → Category → ShopSettings + LevelConfig (parallel) → PurchaseRequest → SellerOffer → Payment + FundsLedgerEntry (atomic pair) → Dispute
|
||||
|
||||
| # | Collection | Schema | Repo | DualWrite | Backfill | Action Needed |
|
||||
|---|---|---|---|---|---|---|
|
||||
| 1.1 | **User** | ✅ Done | ✅ Done | ✅ Done | ✅ Done | Most mature — cutover first |
|
||||
| 1.2 | **Category** | ✅ Done | ✅ Done | ✅ Done | ✅ Done | Self-referential FK, already dual-wired |
|
||||
| 1.3 | **ShopSettings** | ✅ Done | ✅ Done | ✅ Done | ✅ Done | Clean model, already dual-wired |
|
||||
| 1.4 | **LevelConfig** | ✅ Done | ✅ Done | ✅ Done | ✅ Done | Already dual-wired |
|
||||
| 1.5 | **ConfigSetting** | ✅ Done | ✅ Via adapter | ✅ Via adapter | ✅ Done | Adapter pattern, no formal repo |
|
||||
| 1.6 | **PurchaseRequest** | ✅ Done | ✅ Done | ✅ Done | **NEEDED** | 7 child tables, 40+ fields, highest effort |
|
||||
| 1.7 | **SellerOffer** | ✅ Done | ✅ Done | ✅ Done | **NEEDED** | Simple model, many FKs |
|
||||
| 1.8 | **Dispute** | ✅ Done | ✅ Done | ✅ **DONE** | **NEEDED** | ✅ Composite indexes added, pre-save hook confirmed in Drizzle repo |
|
||||
| 1.9 | **Payment** | ✅ Done | ✅ Done | ✅ Done | **NEEDED** | Most money-critical, SERIALIZABLE transactions |
|
||||
| 1.10 | **FundsLedgerEntry** | ✅ Done | ✅ Done | ✅ Done | **NEEDED** | ✅ Immutability trigger SQL migration created |
|
||||
|
||||
### Phase 2: Content + Engagement — 4–5 eng-wks
|
||||
|
||||
| # | Collection | Effort | Key Action |
|
||||
|---|---|---|---|
|
||||
| 2.1 | **Address** | Low | ✅ Reconciled: Drizzle authoritative, `ensurePostgresAddressSchema` → stub, `IAddress` type fixed |
|
||||
| 2.2 | **Review** | Low | Dual-write + backfill done; polymorphic subjectId handled without FK |
|
||||
| 2.3 | **BlogPost** | Low | Full dual-write done; replicate slug generation hook |
|
||||
| 2.4 | **PointTransaction** | Low | Dual-write done; sparse index → partial index |
|
||||
| 2.5 | **Notification** | Medium | ✅ Dual-write done, TTL DataCleanupService implemented |
|
||||
| 2.6 | **TrezorAccount** | Medium | ✅ DualWrite repo done, factory wired; child table handled by Drizzle repo |
|
||||
| 2.7 | **RequestTemplate** | Medium | Decide child tables vs JSONB for 6 embedded arrays |
|
||||
| 2.8 | **DerivedDestination** | Medium | ✅ DualWrite repo done, factory wired; discriminator handled by Drizzle repo |
|
||||
|
||||
### Phase 3: Document-Shaped Collections — 5–8 eng-wks
|
||||
|
||||
| # | Collection | Effort | Key Action |
|
||||
|---|---|---|---|
|
||||
| 3.1 | **Chat** | **High (4–6 wks)** | Normalize JSONB shim → chat_messages + chat_participants child tables; rewrite all ChatService methods; migrate existing data; perf test with production volumes |
|
||||
| 3.2 | **TelegramSession** | Low | ✅ `integer`→`bigint` risk flagged, TTL DataCleanupService implemented |
|
||||
| 3.3 | **TelegramLink** | Low | Already dual-wired in production |
|
||||
| 3.4 | **TempVerification** | Low | ✅ Behind feature flag, TTL DataCleanupService implemented |
|
||||
|
||||
**End state:** `MONGO_CONNECT_MODE=auto` skips Mongo entirely. Mongoose driver removed from package.json.
|
||||
|
||||
---
|
||||
|
||||
## 4. Data Modeling Decisions
|
||||
|
||||
### 4.1 Polymorphic Mixed IDs
|
||||
|
||||
**Rule:** Three-column discriminator pattern for ALL `Schema.Types.Mixed` fields that hold ObjectId | string.
|
||||
|
||||
```
|
||||
field_ref_kind ref_kind ENUM('entity','template') NOT NULL
|
||||
field_id UUID NULL (FK when kind='entity')
|
||||
field_external_ref TEXT NULL (when kind='template')
|
||||
CHECK constraint: (kind='entity' AND id IS NOT NULL AND external_ref IS NULL)
|
||||
OR (kind='template' AND id IS NULL AND external_ref IS NOT NULL)
|
||||
```
|
||||
|
||||
**Applicable to:** Payment (purchaseRequestId, sellerOfferId, sellerId), FundsLedgerEntry (purchaseRequestId, paymentId, createdBy), DerivedDestination (sellerId, sellerOfferId), Dispute (chatId — schema gap, must add).
|
||||
|
||||
### 4.2 Interface-Level `ObjectId | string`
|
||||
|
||||
**Rule:** Treat as type-system bugs, NOT schema requirements. Resolve the string to UUID in the service layer before writing to PG. Found in ~10 interface fields (PurchaseRequest.categoryId, SellerOffer.purchaseRequestId/sellerId, RequestTemplate.sellerId/categoryId).
|
||||
|
||||
### 4.3 Embedded Arrays: Child Tables vs JSONB
|
||||
|
||||
**Child tables (preferred):**
|
||||
- PurchaseRequest.specifications[], deliveryAttempts[], preferredSellers[]
|
||||
- Chat.messages[] (**must** be child table), Chat.participants[]
|
||||
- TrezorAccount.addresses[] (already a child table)
|
||||
|
||||
**JSONB (acceptable):**
|
||||
- Dispute.evidence[], messages[], timeline[], resolution
|
||||
- Payment.blockchain, metadata
|
||||
- BlogPost.author, videos, seo
|
||||
- PurchaseRequest.deliveryInfo, serviceInfo (partial: extract flat fields to columns)
|
||||
|
||||
### 4.4 Money Numerics
|
||||
|
||||
**Rule:** `numeric(38,18)` for all amounts. `numeric(78,0)` for on-chain uint256 balances.
|
||||
|
||||
### 4.5 TTL Replacement
|
||||
|
||||
**Rule:** Application-level `DataCleanupService` with scheduled DELETE queries. Do NOT use pg_cron (extension dependency, may not be available on managed Postgres).
|
||||
|
||||
| Collection | Frequency | Query |
|
||||
|---|---|---|
|
||||
| Notification | Hourly | `DELETE WHERE created_at < NOW() - INTERVAL '90 days'` |
|
||||
| TempVerification | Every 5 min | `DELETE WHERE expires_at < NOW()` |
|
||||
| TelegramSession | Every 1 min | `DELETE WHERE expires_at < NOW()` |
|
||||
|
||||
---
|
||||
|
||||
## 5. Risk Register
|
||||
|
||||
| # | Risk | L | I | Mitigation |
|
||||
|---|---|---|---|---|
|
||||
| R1 | **Data loss during backfill** — concurrent writes cause gaps/doubles | M | H | ON CONFLICT DO NOTHING on legacy_object_id; backfill from snapshot; replay gaps |
|
||||
| R2 | **Polymorphic ID ambiguity** — string that looks like UUID but is external ref | L | M | Discriminator pattern makes ref_kind explicit; manual review of UUID-format strings |
|
||||
| R3 | **Atomicity gap in dual-write** — PG and Mongo diverge | H | H | Gap table reconciliation every 5–15min; money: PG first, Mongo best-effort |
|
||||
| R4 | **populate() → N+1** — 110+ sites convert to individual SELECTs | H | M | Use Drizzle relations with JOINs; monitor query count per request post-cutover |
|
||||
| R5 | **ObjectId→UUID remapping broken** — id_map misses rows | M | H | Startup verification: count Mongo vs PG per table; alert on difference |
|
||||
| R6 | **Downtime during cutover** — PG not ready, feature flag flips | L | M | Canary deploy; Mongo mirror keeps writing; cutover during low-traffic |
|
||||
| R7 | **Mongoose hooks not replicated** — unchecked data in PG | M | M | ✅ Address primary enforcement (reconciled to Drizzle schema + partial unique index), ✅ Dispute timeline push (confirmed in DrizzleDisputeRepo.create), BlogPost slug gen still needs PG equivalent |
|
||||
| R8 | **Aggregation SQL mismatch** — pipeline → SQL gives different results | M | H | Test all 14 pipelines against identical staging data; 12 are simple GROUP BYs |
|
||||
| R9 | **Chat JSONB scalability collapse** — loads entire messages[] into memory | H | H | **BLOCKS Phase 3** — normalize to child tables before cutover |
|
||||
| R10 | ~~FundsLedgerEntry updatable in PG~~ | L | C | ✅ MITIGATED — `0001_funds_ledger_immutable_trigger.sql` migration created with BEFORE UPDATE + BEFORE DELETE triggers rejecting all modifications |
|
||||
| R11 | ~~Dual PG schema definitions~~ | L | L | ✅ MITIGATED — Address schema reconciled (Drizzle authoritative), `ensurePostgresAddressSchema()` reduced to no-op stub. BlogPost, PointTransaction, ConfigSettingHistory still have dual definitions to check |
|
||||
| R12 | **Polymorphic id resolution at read time** — UUIDs must match id_map | M | L | LRU-cached id_map (10k entries), warmed on startup |
|
||||
| R13 | **Seed scripts bypass repo layer** — direct Mongoose calls | L | M | Update seed scripts per phase to go through PG path |
|
||||
|
||||
---
|
||||
|
||||
## 6. Verification & Cutover Requirements
|
||||
|
||||
### Per-Collection Cutover Checklist
|
||||
|
||||
1. ✅ Drizzle schema frozen and reviewed
|
||||
2. ✅ Repository implemented with all hooks replicated
|
||||
3. ✅ Backfill script run against staging snapshot, row counts verified
|
||||
4. ✅ Dual-write active, `pg_dualwrite_gaps` < 10 rows
|
||||
5. ✅ Shadow-read enabled (7 days money, 3 days content)
|
||||
6. ✅ Zero checksum mismatches in last 48 hours
|
||||
7. ✅ PG p95 latency < 1.5× Mongo p95 latency
|
||||
8. ✅ Rollback drill passed: flip to Mongo, no data loss, gaps reconciled
|
||||
9. ✅ PG-read cutover → monitor 48h → retire Mongo mirror
|
||||
|
||||
### Verification Layers
|
||||
|
||||
| Layer | Frequency | Method |
|
||||
|---|---|---|
|
||||
| Row count parity | Hourly | Mongo.countDocuments vs PG SELECT count per collection |
|
||||
| Checksum parity | Daily | Financial reconciliation by bucket (provider, status, date) |
|
||||
| Shadow-read parity | Continuous | MD5 hash of sorted result set; PagerDuty on mismatch |
|
||||
|
||||
---
|
||||
|
||||
## 7. Effort & Timeline
|
||||
|
||||
**Team:** 2 senior backend engineers + 1 staff engineer (50% architecture review)
|
||||
|
||||
| Scenario | Low | Realistic | High |
|
||||
|---|---|---|---|
|
||||
| **Money core only** (Phase 0+1) | 14 wks | **17 wks** | 19 wks |
|
||||
| **Full migration** (Phases 0–3) | 24 wks | **28 wks** | 33 wks |
|
||||
| Full with Chat JSONB kept | 19 wks | **22 wks** | 27 wks |
|
||||
|
||||
**Uncertainty buffer:** +25%. Chat workstream: +50% (unknown production data volume).
|
||||
|
||||
### Gantt Summary
|
||||
|
||||
```
|
||||
Phase 0: Foundation ██░░░░░░░░░░░░░░░░░░░░░░░░░░ 2-3 wks
|
||||
Phase 1a: User+Cat+Config ░░██████░░░░░░░░░░░░░░░░░░░░ 3-4 wks
|
||||
Phase 1b: PR+SellOffer ░░░░░░████████░░░░░░░░░░░░░░ 4-5 wks
|
||||
Phase 1c: Dispute+Pay+Ledger ░░░░░░░░░░░░████████░░░░░░░░ 4-5 wks
|
||||
Phase 1 QA + soak ░░░░░░░░░░░░░░░░░░░░██░░░░░░ 1-2 wks
|
||||
Phase 2: Content+Engagement ░░░░░░░░░░░░░░░░░░░░░░████░░ 4-5 wks
|
||||
Phase 3a: Chat normalization ░░░░░░░░░░░░░░░░░░░░░░░░░░██ 4-6 wks ← Critical path
|
||||
Phase 3b: Sessions+Temp ░░░░░░░░░░░░░░░░░░░░░░░░░░██ 1-2 wks (parallel with 3a)
|
||||
Phase 3c: Mongo decommission ░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 1 wk
|
||||
E2E migration test ░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 2-3 wks
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 8. Tooling & Approach
|
||||
|
||||
### ORM: Drizzle (confirmed)
|
||||
|
||||
Already v0.44.1 in package.json. 25 schema files, 11 repos, drizzle-kit v0.31.1. Team is familiar. No switch to Prisma/TypeORM/Kysely is justified.
|
||||
|
||||
### Migration Pipeline
|
||||
|
||||
```
|
||||
drizzle-kit generate → review + commit migration SQL → npm run db:migrate → CI verify no drift
|
||||
```
|
||||
|
||||
**Gap:** Create `backend/migrations/` directory, add `npm run db:migrate` script.
|
||||
|
||||
### Dual-Write Mechanics
|
||||
|
||||
**Money-core (Payment, FundsLedgerEntry):** PG authoritative, Mongo best-effort mirror. Gap table catches failures.
|
||||
|
||||
**Content (BlogPost, Review, Notification):** Mongo authoritative, PG best-effort mirror. Failed PG writes retry from gap log.
|
||||
|
||||
**Auth (User, TempVerification, TelegramLink, TelegramSession):** `AUTH_STORE=postgres` routes cleanly. No dual-write.
|
||||
|
||||
### Backfill Pattern
|
||||
|
||||
```
|
||||
Stream Mongo docs in batches (500) → transform (resolve ObjectIds, apply discriminator,
|
||||
flatten subdocs, serialize arrays) → INSERT ON CONFLICT DO NOTHING → replay gaps
|
||||
```
|
||||
|
||||
8 scripts exist as templates. 5 new backfill scripts needed (PurchaseRequest, SellerOffer, Payment, FundsLedgerEntry, Dispute).
|
||||
|
||||
---
|
||||
|
||||
## 9. Immediate Action Items — Status
|
||||
|
||||
### Completed 2026-06-02 (8 agents, 9 tasks)
|
||||
|
||||
| # | Action | Status |
|
||||
|---|---|---|
|
||||
| A1 | **Reconcile Address dual schema** — `ensurePostgresAddressSchema()` → stub, `IAddress` fixed | ✅ DONE |
|
||||
| A2 | **Implement DualWriteDisputeRepo** — PG-first pattern, **21 tests passing** | ✅ DONE |
|
||||
| A3 | **Add FundsLedgerEntry immutability trigger** — SQL migration created | ✅ DONE |
|
||||
| A4 | **Add missing Dispute indexes** — composite status+priority, adminId+status | ✅ DONE |
|
||||
| A5 | **Create `backend/migrations/` directory** — `db:generate`/`db:migrate`/`db:studio` scripts | ✅ DONE |
|
||||
| A6 | **Implement DataCleanupService TTL** — 3 purge methods + `ttlCleanupJob.ts` wired into app.ts | ✅ DONE |
|
||||
| A7 | **Decision: Chat normalization or JSONB shim** | ⏳ PENDING |
|
||||
| A8 | **Seed script audit** — 7 seed + 8 utility scripts audited, 4 npm paths broken | ✅ DONE |
|
||||
| A9 | **Implement DualWriteTrezorAccountRepo** — factory `dual` path wired | ✅ DONE |
|
||||
| A10 | **Implement DualWriteDerivedDestinationRepo** — factory `dual` path wired | ✅ DONE |
|
||||
|
||||
### Phase 1 Blockers Status
|
||||
|
||||
1. ~~Dispute DualWrite repo + pre-save hook~~ ✅ DONE
|
||||
2. **PurchaseRequest backfill script** with 7-child-table transform — still needed
|
||||
3. **Payment backfill script** with financial reconciliation — still needed
|
||||
4. ~~FundsLedgerEntry immutability trigger~~ ✅ DONE
|
||||
|
||||
### Quick Wins Status
|
||||
|
||||
- ✅ Reconcile `IAddress.addressType` to include `'Other'` — **DONE**
|
||||
- Fix `ObjectId | string` type bugs at the controller layer (~10 interface fields)
|
||||
- Add CI `db:verify` step that fails on uncommitted schema changes
|
||||
- Index audit: verify every Mongo index has a PG equivalent with matching EXPLAIN ANALYZE
|
||||
|
||||
### Seed Audit Key Findings (TASK 9)
|
||||
|
||||
- **All 7 seed scripts** in `src/seeds/` bypass the repo factory (use direct Mongoose)
|
||||
- **4 npm seed paths are BROKEN** — `seed:*` scripts point to `src/scripts/` but files live in `src/seeds/`
|
||||
- **`init-admin.ts`** runs on every startup, uses `AuthUser` with `new User()`/`user.save()` — first DB touch on boot
|
||||
- **`seedCategories.ts`** runs UNCONDITIONALLY on every startup, touches RequestTemplate + PurchaseRequest
|
||||
- **No DrizzleCategoryRepo or DrizzleAddressRepo exist** despite having Drizzle schemas — largest seed migration gaps
|
||||
- Money-critical tables (Payment, FundsLedgerEntry) are NOT touched by seed scripts — low risk
|
||||
|
||||
## 10. Appendices
|
||||
|
||||
### A. Cross-Cutting Detail: populate() Hotspots
|
||||
|
||||
| File | Populate Count | Dominant Join | Notes |
|
||||
|---|---|---|---|
|
||||
| MongoMarketplaceRepo.ts | 40+ | User, Category, SellerOffer | Heaviest file, all marketplace reads |
|
||||
| MongoChatRepo.ts | 20+ | User (participants, messages.author) | Nested inside embedded arrays |
|
||||
| MongoDisputeRepo.ts | 15+ | User, Chat | Polymorphic chatId populate |
|
||||
| MongoPaymentRepo.ts | 15+ | User, PurchaseRequest, SellerOffer | 3 Mixed fields populated without ref |
|
||||
| MongoReviewRepo.ts | 10+ | User | Polymorphic subjectId |
|
||||
| MongoNotificationRepo.ts | 5 | User | Straightforward 1:1 |
|
||||
| MongoPointsRepo.ts | 5 | User | One silently omitted in Drizzle migration |
|
||||
|
||||
### B. Cross-Cutting Detail: Aggregation Pipelines
|
||||
|
||||
| # | File | Model | Complexity | Stages |
|
||||
|---|---|---|---|---|
|
||||
| 1 | MongoMarketplaceRepo.ts:1158 | User, ShopSettings | **High** | $match + 2× $lookup + $project + $sort |
|
||||
| 2–4 | MongoMarketplaceRepo.ts | PurchaseRequest, SellerOffer | Low | $match + $group |
|
||||
| 5–6 | MongoPointsRepo.ts | PointTransaction | Low | $match + $group + $sort |
|
||||
| 7–8 | MongoPaymentRepo.ts | Payment | Low | $match + $group |
|
||||
| 9–10 | MongoDisputeRepo.ts | Dispute | Low | $match + $group |
|
||||
| 11–12 | MongoNotificationRepo.ts | Notification | Low | $match + $group + $count |
|
||||
| 13 | MongoChatRepo.ts | Chat | Low | $match + $unwind + $group |
|
||||
| 14 | MongoBlogRepo.ts | BlogPost | Low | $match + $group |
|
||||
|
||||
### C. Cross-Cutting Detail: Transaction & Atomicity Sites
|
||||
|
||||
| Site | Model | Pattern | Risk |
|
||||
|---|---|---|---|
|
||||
| MongoPointsRepo | User, PointTransaction | Mongoose session + transaction | Dual-write gap if PG insert fails |
|
||||
| MongoMarketplaceRepo (2 sites) | PurchaseRequest, SellerOffer | Mongoose session + transaction | Same dual-write gap |
|
||||
| DrizzleUserRepo | User | PG SERIALIZABLE | **Reference implementation** ✅ |
|
||||
| DrizzlePaymentRepo | Payment, FundsLedgerEntry | PG SERIALIZABLE + FOR UPDATE | **Best practice** ✅ |
|
||||
| PointsService.processReferralReward | User, PointTransaction | **NO TRANSACTION** | ⚠️ addPoints + updateReferralStats separate |
|
||||
| PaymentCoordinator AML fee | FundsLedgerEntry | **NO TRANSACTION** | ⚠️ Explicit TODO on line 298 |
|
||||
| MongoMarketplaceRepo.processReferralReward | User | **NO TRANSACTION** | ⚠️ referralStats outside session |
|
||||
|
||||
---
|
||||
|
||||
*Generated from automated audit: 49 agents, 23 model analyses, 19 service catalogues, 6 cross-cutting inventories. Full raw output: `~/.claude/projects/.../tasks/wbq6syb3q.output`*
|
||||
Reference in New Issue
Block a user