nick/nick-doc
3
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

docs: sync from backend 8fc2309 — M43/M44 missing FKs + H37 dispute enums

Browse Source
This commit is contained in:
Siavash Sameni
2026-06-07 07:16:02 +04:00
parent a2967ec594
commit 0bb60dbc98
24 changed files with 3428 additions and 906 deletions
Download Patch File Download Diff File Expand all files Collapse all files

207
02 - Data Models/Postgres Runtime Cutover Status.md
View File

@@ -3,15 +3,27 @@ title: Postgres Runtime Cutover Status
tags: [data-model, postgres, migration, runtime-status]
aliases: [Postgres Status, PG Cutover Status, Mongo vs Postgres Runtime]
created: 2026-05-31
updated: 2026-06-03
updated: 2026-06-06
source: backend integrate-main-into-development@41087c7 + deployment main@8764fdf
---
# Postgres Runtime Cutover Status
> **Current branch:** backend `integrate-main-into-development` at `41087c7`, version `2.8.79`; dev deployment `main` at `8764fdf`.
> **Current branch:** backend `integrate-main-into-development`, version `2.9.12`.
>
> **Bottom line:** the codebase is in **active dual-write phase**. All 11 repository domains in the factory now have Drizzle schemas, Drizzle repos, and dual-write wrappers (except Chat and ReleaseHold, which have Drizzle repos but no dual-write counterpart). 18 Drizzle migrations (0000–0017) have landed, covering every table in scope. Dev deployment defaults nine PG-capable stores to Postgres: auth-owned users/Telegram auth, confirmation-threshold config/history, user addresses, categories, level config, shop settings, reviews, notifications, and the oracle payment_quotes path. Code-level defaults remain `mongo` outside those deployment overrides. Repository factory normalizes `postgres` and `pg` as equivalent mode tokens. The unmounted legacy marketplace router is detached. As of `2.8.79`, the `guard` user role is in PG schema, chat routes are fixed, notifications deliver in real time, PG response serialization/id resolution in marketplace is corrected, and seller shop lookup is tolerant of uuid/legacy id formats. Reads are still Mongo-authoritative across all dual-write domains — read cutover is the remaining gate for each domain. Chat normalization (participants/messages stored as JSONB blobs, not relational child tables) remains an open blocker for full Chat cutover.
> **Bottom line: Migration complete as of 2026-06-06, backend v2.9.12.** MongoDB and Mongoose have been fully removed from the runtime. PostgreSQL (Drizzle ORM) is the sole database. All 11 repository domains use DrizzleXxxRepo exclusively. No dual-write wrappers are active. TypeScript compilation: 0 errors.
## Migration Status
| Phase | Status |
|---|---|
| Schema design | Complete — 32 tables, 19 migrations (0000–0019) |
| Drizzle repos | Complete — all 11 factory domains have a DrizzleXxxRepo |
| Dual-write wrappers | Decommissioned — removed from runtime as of v2.9.12 |
| Write cutover | Complete — all writes go to PostgreSQL only |
| Read cutover | Complete — all reads from PostgreSQL |
| Mongoose removal | Complete — no Mongoose imports in runtime src/ |
| TypeScript compilation | 0 errors |
## Schema and Repository Coverage
@@ -33,29 +45,29 @@ All tables below have a `.ts` schema file in `src/db/schema/` and are covered by
**Content/Social:** `blog_posts`, `notifications`, `disputes`, `chats`
Total: **32 tables** across 18 migrations (0000–0017).
Total: **32 tables** across 19 migrations (0000–0019).
### Tables with a Drizzle Repository
| Drizzle Repo | Dual-Write Repo | Domain |
|---|---|---|
| `DrizzleUserRepo` | `DualWriteUserRepo` | Users, passkeys, refresh tokens |
| `DrizzlePaymentRepo` | `DualWritePaymentRepo` | Payments, funds ledger |
| `DrizzleMarketplaceRepo` | `DualWriteMarketplaceRepo` | Categories, purchase requests, seller offers, request templates |
| `DrizzleDerivedDestinationRepo` | `DualWriteDerivedDestinationRepo` | Derived destinations, sweeps |
| `DrizzleTrezorAccountRepo` | `DualWriteTrezorAccountRepo` | Trezor accounts, derived addresses |
| `DrizzlePointsRepo` | `DualWritePointsRepo` | Point transactions |
| `DrizzleNotificationRepo` | `DualWriteNotificationRepo` | Notifications |
| `DrizzleDisputeRepo` | `DualWriteDisputeRepo` | Disputes |
| `DrizzleBlogRepo` | `DualWriteBlogRepo` | Blog posts |
| `DrizzleChatRepo` | _(none — no dual-write wrapper)_ | Chats (JSONB shim; Chat normalization is a blocker) |
| `DrizzleReleaseHoldRepo` | _(none — no dual-write wrapper)_ | Release holds (bridges payments + purchase_requests) |
| Drizzle Repo | Domain |
|---|---|
| `DrizzleUserRepo` | Users, passkeys, refresh tokens |
| `DrizzlePaymentRepo` | Payments, funds ledger |
| `DrizzleMarketplaceRepo` | Categories, purchase requests, seller offers, request templates |
| `DrizzleDerivedDestinationRepo` | Derived destinations, sweeps |
| `DrizzleTrezorAccountRepo` | Trezor accounts, derived addresses |
| `DrizzlePointsRepo` | Point transactions |
| `DrizzleNotificationRepo` | Notifications |
| `DrizzleDisputeRepo` | Disputes |
| `DrizzleBlogRepo` | Blog posts |
| `DrizzleChatRepo` | Chats (JSONB shim; Chat normalization is an optional future improvement) |
| `DrizzleReleaseHoldRepo` | Release holds (bridges payments + purchase_requests) |
Tables with schema but no dedicated Drizzle repo yet: `addresses` (handled via addressStore facade), `shop_settings` (handled via shopSettings facade), `config_settings` / `config_setting_history` (handled via config-store facade), `telegram_links` / `telegram_sessions` (handled via auth-store facade), `reviews` (handled via review-store facade).
Tables with schema but no dedicated Drizzle repo (handled via store facades): `addresses`, `shop_settings`, `config_settings` / `config_setting_history`, `telegram_links` / `telegram_sessions`, `reviews`.
### Migration Count
18 migrations landed: **0000 through 0017**.
19 migrations landed: **0000 through 0019**.
| Migration | Key change |
|---|---|
@@ -77,88 +89,86 @@ Tables with schema but no dedicated Drizzle repo yet: `addresses` (handled via a
| 0015 | Ledger immutability extended: UPDATE + DELETE triggers |
| 0016 | `address_type` enum + `addresses` table |
| 0017 | `guard` value added to `user_role` enum |
| 0018 | AI request fields |
| 0019 | `payment_provider` enum: added `escrow` |
## What Uses Postgres Now
All domains are PostgreSQL-only. The table below summarises the runtime topology for reference.
| Area | Runtime status | Notes |
|---|---|---|
| Postgres connection | Available when `PG_URL` is set | Store facades use `src/infrastructure/postgres/client.ts`; the broader `src/db/` Drizzle layer and repository factory are fully populated. |
| Runtime schema bootstrap | Implemented for auth, config, address, and reference stores | Auth tables bootstrapped from `src/services/auth/postgresAuthSchema.ts`; store facades bootstrap their own tables at startup when their `*_STORE=postgres` flag is enabled. |
| Health observability | Implemented in `/api/health` | `checks.postgres` reports `configured`, `required`, `storeModes`, `enabledStores`, and `enabledStoreCount`. Dev Gatus asserts all dev PG-backed store modes are `postgres`, including notifications. Mongoose health check is lazy-loaded and skipped when Mongo is optional under `MONGO_CONNECT_MODE=auto/never`. |
| Auth-owned user store | PG-backed in dev deployment; code opt-in with `AUTH_STORE=postgres` | Auth, passkey, Telegram auth/link/session/temp-verification, and `/api/user` profile paths use an auth-store facade. PG-mode users are mirrored back to Mongo through `legacy_object_id` for compatibility with still-Mongo services. |
| Confirmation-threshold runtime config | PG-backed in dev deployment; code opt-in with `CONFIG_STORE=postgres` | `ConfigSetting` / `ConfigSettingHistory` access for `/api/admin/settings/confirmation-thresholds` and transaction-safety confirmation thresholds uses a config-store facade. PG-mode writes mirror back to Mongo. Legacy models load only for Mongo fallback/backfill/mirror paths. |
| User addresses | PG-backed in dev deployment; code opt-in with `ADDRESS_STORE=postgres` | `/api/addresses` CRUD uses an address-store facade. PG mode enforces one primary address per user with a partial unique index and mirrors writes/deletes back to Mongo. |
| Marketplace categories | PG-backed in dev deployment; code opt-in with `CATEGORY_STORE=postgres` | `CategoryService` and the default `General` category path use a category-store facade. PG-mode writes mirror back to Mongo. Migration 0009 deactivated duplicate active category labels and enforces `categories_active_name_norm_uq` on `lower(btrim(name)) WHERE is_active = true`. |
| Level configuration | PG-backed in dev deployment; code opt-in with `LEVEL_CONFIG_STORE=postgres` | `PointsService` level reads use a level-config facade. `PointTransaction` and user points remain Mongo-backed. `LEVEL_STORE=postgres` accepted as compatibility alias. |
| Shop settings | PG-backed in dev deployment; code opt-in with `SHOP_SETTINGS_STORE=postgres` | Shop settings controller, seller payment rail resolution (`2.8.56` fixes seller shop lookup to handle both uuid and legacy id formats), and review enable/disable checks use a shop-settings facade. PG-mode writes mirror back to Mongo. |
| Marketplace reviews | PG-backed in dev deployment; code opt-in with `REVIEW_STORE=postgres` | Review list/summary/create routes use a review-store facade. PG-mode list responses still hydrate `reviewerId` from the user mirror to preserve frontend shape. |
| Notifications | PG-backed in dev deployment; code opt-in with `NOTIFICATION_STORE=postgres` or `REPO_NOTIFICATION=pg` | `NotificationService` uses `getNotificationRepo()` for create/list/read/delete/count paths. `2.8.55` fixes chat routes and delivers notifications in real time. `2.8.37` fixes repository mode aliasing so `postgres` resolves to the Drizzle notification repo. Backfill script (`npm run backfill:notification:postgres`) and smoke script (`scripts/smoke/notifications-postgres.sh`) are available. |
| Oracle quote persistence | Conditional runtime PG write | `/api/payment/request-network/intents` lazily imports `quoteRepo` only when `ORACLE_QUOTING_ENABLED=true`; it writes `payment_quotes` if the PG parent payment row exists, mirrors to Mongo `Payment.quote`, and records `pg_dualwrite_gaps` if PG is behind. |
| Funds ledger | Repository-backed, default Mongo | `appendFundsLedgerEntry` and `getFundsBalanceBy*` call `getPaymentRepo()`. Default is `MongoPaymentRepo`; `REPO_PAYMENT=dual`/`pg` exercises the Drizzle ledger after backfill/soak. |
| Backfill/verify scripts | Available as operator tooling | `MIGRATION_PG_URL` drives all backfill scripts; guards restrict allowed target hosts. Marketplace-core runner backfills users/categories, request templates, purchase requests, seller offers, and `selectedOfferId` remap in dependency order. Not run automatically at startup. |
| Guard user role | PG schema-ready | Migration 0017 adds `guard` to the `user_role` enum. `2.8.54` adds guard role support across auth and user management. |
| PG response serialization | Fixed in `2.8.51–2.8.53` | PG response serialization and id resolution in marketplace purchase-request paths corrected; user creation and purchase request unblocked from a PG FK constraint error. |
| Admin user management | PG-capable as of `2.8.50` | Admin user count queries route through postgres-capable stores; admin user management works end-to-end under PG. |
| Seeds | Postgres-capable as of `2.8.47` | Seeds in `src/seeds/*` are store-aware and idempotent; can seed fresh PG under `MONGO_CONNECT_MODE=never`. |
| Postgres connection | Required — `PG_URL` must be set | Store facades use `src/infrastructure/postgres/client.ts`; the broader `src/db/` Drizzle layer and repository factory are fully populated. |
| Runtime schema bootstrap | Implemented for auth, config, address, and reference stores | Auth tables bootstrapped from `src/services/auth/postgresAuthSchema.ts`; store facades bootstrap their own tables at startup. |
| Health observability | Implemented in `/api/health` | `checks.postgres` reports `configured`, `required`, `storeModes`, `enabledStores`, and `enabledStoreCount`. Mongoose health check is no longer present. |
| Auth-owned user store | PG-backed | Auth, passkey, Telegram auth/link/session/temp-verification, and `/api/user` profile paths use the auth-store facade pointing at Postgres. `legacy_object_id` column retained for id-map compatibility. |
| Confirmation-threshold runtime config | PG-backed | `ConfigSetting` / `ConfigSettingHistory` access routes through the config-store facade. |
| User addresses | PG-backed | `/api/addresses` CRUD uses the address-store facade. |
| Marketplace categories | PG-backed | `CategoryService` and the default `General` category path use the category-store facade. `categories_active_name_norm_uq` enforced. |
| Level configuration | PG-backed | `PointsService` level reads use the level-config facade. `LEVEL_STORE=postgres` accepted as compatibility alias. |
| Shop settings | PG-backed | Shop settings controller, seller payment rail resolution, and review enable/disable checks use the shop-settings facade. Seller shop lookup handles both uuid and legacy id formats. |
| Marketplace reviews | PG-backed | Review list/summary/create routes use the review-store facade. |
| Notifications | PG-backed | `NotificationService` uses `getNotificationRepo()` for create/list/read/delete/count paths. |
| Oracle quote persistence | PG write when `ORACLE_QUOTING_ENABLED=true` | `/api/payment/request-network/intents` writes `payment_quotes` to PG. Mongo mirror path removed. |
| Funds ledger | PG-backed | `appendFundsLedgerEntry` and `getFundsBalanceBy*` call `getPaymentRepo()` which resolves to `DrizzlePaymentRepo`. |
| Payments and escrow state | PG-backed | All payment services use Drizzle repos; Mongoose `Payment` model removed. |
| Derived destinations and sweeps | PG-backed | `getDerivedDestinationRepo()` resolves to `DrizzleDerivedDestinationRepo`. |
| Points/referrals/transactions | PG-backed | `getPointsRepo()` resolves to `DrizzlePointsRepo`. |
| Chat/messages | PG-backed (JSONB shim) | `getChatRepo()` resolves to `DrizzleChatRepo`. Participants/messages are stored as JSONB blobs; normalization into relational child tables is an optional future improvement. |
| Disputes/blog | PG-backed | Both resolve to Drizzle repos. |
| ReleaseHold | PG-backed | `getReleaseHoldRepo()` resolves to `DrizzleReleaseHoldRepo`. |
| Backfill/verify scripts | Available as operator tooling | `MIGRATION_PG_URL` drives all backfill scripts. Not run automatically at startup. |
| Guard user role | PG schema-ready | Migration 0017 adds `guard` to the `user_role` enum. |
| Seeds | Postgres-capable | Seeds in `src/seeds/*` are store-aware and idempotent under `MONGO_CONNECT_MODE=never`. |
## What Is Still Mongo-Backed
## What Was Mongo-Backed (Historical)
Writes across all dual-write domains go to both Mongo and Postgres. Reads remain Mongo-authoritative for every dual-write domain — read cutover has not been performed for any domain. Chat is repository-backed (Drizzle repo exists) but participants/messages are stored as JSONB blobs rather than normalized child tables; Chat normalization is the primary structural blocker for Chat read cutover.
All domains are now PostgreSQL-only as of v2.9.12. The following were the remaining Mongo-backed areas prior to the final cutover:
| Domain | Current live store | Why not Postgres yet |
|---|---|---|
| User reads | MongoDB authoritative | Auth-owned users can be PG-backed for writes, but reads remain Mongo-authoritative. Still-Mongo domains expect Mongo ObjectId user references; PG-mode writes maintain a Mongo mirror until all consumers cut over. |
| Admin cleanup / seed address tooling | MongoDB | User-facing address CRUD is PG-capable, but admin cleanup scripts still operate on Mongo first. Seed scripts backfill addresses to PG when `ADDRESS_STORE=postgres`. |
| Marketplace requests/offers/templates reads | Repository-backed writes; Mongo reads | `getMarketplaceRepo()` wires all marketplace writes. `REPO_MARKETPLACE` defaults to Mongo; full PG/dual read cutover needs smoke coverage before flipping. The legacy `marketplaceRouter` is detached from the service index (`2.8.36`). |
| Payments and escrow state reads | MongoDB primary | Request Network, AMN scanner, webhook, admin, release/refund, adapter, reconciliation, and legacy payment paths still create/update `Payment` Mongoose documents directly for reads. Payment Drizzle repo and dual-write repo exist; `REPO_PAYMENT=dual` enables dual-writes, but read paths remain Mongo. |
| Derived destinations and sweeps | Repository-backed writes; Mongo reads | `getDerivedDestinationRepo()` wires writes; `REPO_DERIVED_DESTINATION` defaults to Mongo and has not been flipped in dev. |
| Points/referrals/transactions | Repository-backed writes; Mongo reads | `getPointsRepo()` wires writes; `REPO_POINTS` defaults to Mongo. Level config is PG-capable for reads; point transaction and user-point flows are not flipped in dev. |
| Chat/messages | Repository-backed writes (JSONB shim); Mongo reads | `getChatRepo()` wires writes. `REPO_CHAT` / `CHAT_STORE` defaults to Mongo. `DrizzleChatRepo` uses JSONB blobs for participants and messages — Chat normalization into relational child tables is required before safe read cutover. No dual-write wrapper exists. |
| Disputes/blog | Repository-backed writes; Mongo reads | Both have Drizzle repos and dual-write wrappers. Code defaults resolve to Mongo until `REPO_DISPUTE`/`BLOG_STORE` are flipped. |
| ReleaseHold | Drizzle repo only; default Mongo | `getReleaseHoldRepo()` returns Drizzle or Mongo; no dual-write wrapper — `dual` mode silently uses Mongo. Separate cutover needed. |
| Runtime config outside confirmation thresholds | MongoDB | `ConfigSetting` and `ConfigSettingHistory` are PG-capable for confirmation thresholds only; other admin-editable settings need to route through the config-store boundary before counting as cut over. |
| Telegram link/session/temp verification reads | PG-backed writes in dev; Mongo reads in code default | These records move with `AUTH_STORE=postgres`. Dev compose defaults that flag to `postgres`; read cutover follows the auth-store flag. |
- **User reads** — Auth-owned users were PG-backed for writes but reads remained Mongo-authoritative until the full auth cutover.
- **Marketplace requests/offers/templates reads** — `REPO_MARKETPLACE` defaulted to Mongo; read cutover required smoke coverage.
- **Payments and escrow state reads** — Payment services called Mongoose documents directly for reads until the final payment-domain wiring was completed.
- **Derived destinations and sweeps** — `REPO_DERIVED_DESTINATION` defaulted to Mongo.
- **Points/referrals/transactions** — `REPO_POINTS` defaulted to Mongo.
- **Chat/messages** — `getChatRepo()` defaulted to Mongo; JSONB shim was the Drizzle path. No dual-write wrapper existed.
- **Disputes/blog** — Defaulted to Mongo until `REPO_DISPUTE`/`BLOG_STORE` were flipped.
- **ReleaseHold** — No dual-write wrapper; required explicit flip.
All of the above are now fully PostgreSQL-backed. MongoDB and Mongoose have been removed from the runtime.
## Env Flag Reality
The backend code defaults every store flag below to `mongo`. Dev deployment overrides eight PG-capable store flags to `postgres` in `deployment/docker-compose.yml`. Repository factory normalizes `postgres` and `pg` as equivalent.
All `*_STORE=mongo` and `REPO_*=mongo` env flags are obsolete — the repository factory only supports `postgres`/`pg` mode. `MONGO_URI` and `MONGO_CONNECT_MODE` have been removed from the runtime.
| Flag | Current meaning |
|---|---|
| `AUTH_STORE` | Code default `mongo`; dev deployment default `postgres`. Routes auth-owned users, refresh tokens, passkeys, Telegram links/sessions, and temp verifications through Postgres. |
| `CONFIG_STORE` | Code default `mongo`; dev deployment default `postgres`. Routes confirmation-threshold settings/history through Postgres. |
| `ADDRESS_STORE` | Code default `mongo`; dev deployment default `postgres`. Routes `/api/addresses` through Postgres. |
| `CATEGORY_STORE` | Code default `mongo`; dev deployment default `postgres`. Routes marketplace category reads/writes through Postgres. Active PG categories are unique by normalized visible name. |
| `LEVEL_CONFIG_STORE` | Code default `mongo`; dev deployment default `postgres`. Routes level configuration reads and seed replacement through Postgres. `LEVEL_STORE=postgres` accepted as compatibility alias. |
| `SHOP_SETTINGS_STORE` | Code default `mongo`; dev deployment default `postgres`. Routes shop settings, review gates, and seller payment rails through Postgres. `2.8.56` fixes seller shop lookup tolerance for uuid vs legacy id formats. |
| `REVIEW_STORE` | Code default `mongo`; dev deployment default `postgres`. Routes marketplace reviews through Postgres. |
| `NOTIFICATION_STORE` / `REPO_NOTIFICATION` | Code default `mongo`; dev deployment default `postgres`. Routes notification inbox create/list/read/delete/count through the Drizzle notification repo. `postgres` and `pg` both resolve correctly since `2.8.37`. |
| `PG_URL` | Makes PG code importable/reachable. Required for any `*_STORE=postgres` flag; does not cut over unrelated app domains by itself. |
| `MIGRATION_PG_URL` | Used by backfill scripts and migration runbooks; not part of normal request handling. Marketplace-core dry-run/non-dry backfills also require `MIGRATION_MONGO_URL`. |
| `REPO_PAYMENT` | Code default `mongo`. Funds ledger appends and balance reads route through this flag. `dual` mode enables dual-write for the ledger seam. Do not flip broad payment runtime to `pg` yet; most payment services still call Mongoose directly for reads. |
| `REPO_MARKETPLACE` | Code default `mongo`. All marketplace writes route through `getMarketplaceRepo()`. Full read cutover needs smoke coverage. |
| `REPO_USER`, `REPO_POINTS`, `REPO_DERIVED_DESTINATION`, `REPO_TREZOR` | Factory flags with full trio (mongo/dual/pg). Writes are wired; reads remain Mongo-authoritative until each flag is flipped and verified. |
| `REPO_DISPUTE` / `DISPUTE_STORE`, `REPO_BLOG` / `BLOG_STORE` | Code default `mongo`. Dual-write wrappers exist; flip per-domain after verification. |
| `REPO_CHAT` / `CHAT_STORE` | Code default `mongo`. Chat normalization (JSONB→relational) is a structural blocker; no dual-write wrapper. |
| `REPO_RELEASE_HOLD` / `RELEASE_HOLD_STORE` | Code default `mongo`. No dual-write wrapper; `dual` silently uses Mongo. Must be flipped explicitly. As of `2.8.37`, `REPO_DISPUTE=pg` no longer leaks into release hold mode. |
| `ORACLE_QUOTING_ENABLED` | Enables server-side quote computation and the `payment_quotes` PG write in checkout when a PG parent payment row exists. |
| `MONGO_CONNECT_MODE` | Handled in Mongoose connection setup (not in the repository factory). `auto`/`never` allow PG-only boot; lazy Mongoose health check skips when Mongo is optional. |
| `MONGO_URI` | REMOVED — MongoDB has been removed from the runtime. |
| `MONGO_CONNECT_MODE` | REMOVED — MongoDB has been removed from the runtime. |
| `AUTH_STORE` | OBSOLETE — only `postgres` is valid. Setting to `mongo` has no effect. |
| `CONFIG_STORE` | OBSOLETE — only `postgres` is valid. |
| `ADDRESS_STORE` | OBSOLETE — only `postgres` is valid. |
| `CATEGORY_STORE` | OBSOLETE — only `postgres` is valid. |
| `LEVEL_CONFIG_STORE` | OBSOLETE — only `postgres` is valid. `LEVEL_STORE=postgres` accepted as alias. |
| `SHOP_SETTINGS_STORE` | OBSOLETE — only `postgres` is valid. |
| `REVIEW_STORE` | OBSOLETE — only `postgres` is valid. |
| `NOTIFICATION_STORE` / `REPO_NOTIFICATION` | OBSOLETE — only `postgres`/`pg` is valid. |
| `PG_URL` | REQUIRED — PostgreSQL is the sole database. All store facades and repos require this. |
| `MIGRATION_PG_URL` | Used by backfill scripts and migration runbooks; not part of normal request handling. |
| `REPO_PAYMENT` | OBSOLETE — only `postgres` is valid. All payment services use `DrizzlePaymentRepo`. |
| `REPO_MARKETPLACE` | OBSOLETE — only `postgres` is valid. All marketplace writes and reads route through `DrizzleMarketplaceRepo`. |
| `REPO_USER`, `REPO_POINTS`, `REPO_DERIVED_DESTINATION`, `REPO_TREZOR` | OBSOLETE — only `postgres` is valid. All resolve to their respective Drizzle repos. |
| `REPO_DISPUTE` / `DISPUTE_STORE`, `REPO_BLOG` / `BLOG_STORE` | OBSOLETE — only `postgres` is valid. |
| `REPO_CHAT` / `CHAT_STORE` | OBSOLETE — only `postgres` is valid. `DrizzleChatRepo` is the sole chat repo. |
| `REPO_RELEASE_HOLD` / `RELEASE_HOLD_STORE` | OBSOLETE — only `postgres` is valid. |
| `ORACLE_QUOTING_ENABLED` | Enables server-side quote computation and the `payment_quotes` PG write in checkout. |
## Overall Migration Phase
## What's Next (Post-Migration)
| Phase | Status |
|---|---|
| Schema design | Complete — 32 tables, 18 migrations (0000–0017) |
| Drizzle repos | Complete — all 11 factory domains have a Drizzle repo |
| Dual-write wrappers | Mostly complete — 9 of 11 domains have a dual-write wrapper (Chat and ReleaseHold are exceptions) |
| Write cutover (dual-write active) | Not yet enabled by default — `REPO_DEFAULT` is still `mongo`; must be flipped per-domain with care |
| Read cutover | Not started for any domain — Mongo remains authoritative for all reads |
| Prod backfill | Not run — backfill scripts are operator-ready but not executed against production |
| Chat normalization | Blocked — participants/messages stored as JSONB; relational normalization required before Chat read cutover |
1. **Prod backfill** — If the production instance was running Mongo-backed data before the cutover, a one-time backfill from Mongo to Postgres under a maintenance window is required. Use `MIGRATION_PG_URL` + `MIGRATION_MONGO_URL` with the existing backfill scripts. Validate row counts before switching prod traffic.
2. **Chat normalization** — The `DrizzleChatRepo` currently stores participants and messages as JSONB blobs rather than normalized relational child tables. This is an optional future improvement; it does not block current operation but would enable richer querying and FK integrity on chat data.
3. **`payment_provider` enum `escrow` value** — Confirm migration 0019 has been applied on all target databases (adds `escrow` to the `payment_provider` enum). If not already run, apply it before using escrow-provider payment records.
Estimated overall: **schema and infrastructure phase complete; write-seam phase substantially complete; read cutover and backfill execution remain for every domain.**
## Recent Progress Since Last Update (2.8.37 → 2.8.79)
## Recent Progress Since Last Update (2.8.37 → 2.9.12)
- **2.8.38–2.8.46:** Complete dual-write repos for all remaining domains; Drizzle migrations pipeline finalized; TTL scheduler added; shop lookup bug-fixed.
- **2.8.47:** Seeds made Postgres-capable and idempotent for PG-only boot (`MONGO_CONNECT_MODE=never`).
@@ -181,34 +191,7 @@ Estimated overall: **schema and infrastructure phase complete; write-seam phase
- **2.8.77:** Telegram keep email code panel mounted after sending.
- **2.8.78:** Telegram system messages neutral + post-delivery seller review.
- **2.8.79:** Request template maxUsage made truly optional; template creation 500 fix.
- **2.8.51–2.8.53:** PG response serialization and id resolution corrected in marketplace; user creation and purchase request creation unblocked from PG FK constraint errors.
- **2.8.54:** `guard` user role added to `user_role` enum (migration 0017); guard role support across auth and user management.
- **2.8.55:** Chat routes fixed; notifications delivered in real time alongside chat.
- **2.8.56:** Seller shop lookup made tolerant of both uuid and legacy id formats; `dataCleanupService` guarded against `MONGO_CONNECT_MODE=never`.
## Next Cutover Work
1. Apply Drizzle migrations to the target Postgres database (0000–0017 must be in-order; 0002 is a reset migration — confirm idempotency on existing instances).
2. For dev/test data, run the existing backfills or reseed acceptable test data before relying on PG-backed stores. The deployment default flip does not move historical Mongo rows.
3. For auth cutover, run `PG_URL=... npm run backfill:auth:postgres`, verify counts, and confirm `AUTH_STORE=postgres` in the target runtime.
4. For confirmation-threshold config cutover, run `PG_URL=... npm run backfill:config:postgres`, verify counts/history, and confirm `CONFIG_STORE=postgres`.
5. For address cutover, run `PG_URL=... npm run backfill:address:postgres`, verify one-primary invariants, and confirm `ADDRESS_STORE=postgres`.
6. For reference-domain cutover, run:
- `PG_URL=... npm run backfill:category:postgres`
- `PG_URL=... npm run backfill:level-config:postgres`
- `PG_URL=... npm run backfill:shop-settings:postgres`
- `PG_URL=... npm run backfill:review:postgres`
7. Run `PG_URL=... scripts/smoke/categories-postgres-unique.sh` and `PG_URL=... MONGODB_URI=... scripts/smoke/reference-stores-postgres.sh`, then confirm reference store flags in non-prod.
8. For marketplace-core data, run `MIGRATION_MONGO_URL=... MIGRATION_PG_URL=... npm run backfill:marketplace-core:postgres:dry-run`, then the non-dry run. The group runs root dependencies, RequestTemplate rows, PurchaseRequest main rows, SellerOffer rows, then the selected-offer remap.
9. Run `scripts/smoke/marketplace-core-postgres-backfill.sh` with the same migration DSNs and record row-count/checksum results.
10. For notifications, run `PG_URL=... npm run backfill:notification:postgres` and `PG_URL=... scripts/smoke/notifications-postgres.sh` against dev to validate the current default.
11. Enable `REPO_MARKETPLACE=dual`, `REPO_PAYMENT=dual`, `REPO_POINTS=dual`, `REPO_DERIVED_DESTINATION=dual`, `REPO_TREZOR=dual`, `REPO_DISPUTE=dual`, `REPO_BLOG=dual` one domain at a time after backfill verification, and run a soak window before flipping reads.
12. Continue payment-domain wiring: add missing payment repo methods for provider lookups, transaction-hash/webhook lookups, metadata/blockchain patching, template duplicate cleanup, and quote updates before moving `paymentService`, `paymentCoordinator`, RN, or AMN scanner routes.
13. Add a derived-destination/sweep repository seam before payment PG read cutover; destination allocation is payment-address state and should not remain Mongo-only once payments become PG-backed for reads.
14. Resolve Chat normalization: design relational child tables for participants and messages; migrate `DrizzleChatRepo` away from JSONB blobs; add `DualWriteChatRepo`; flip `REPO_CHAT=dual` only after normalization.
15. Add `DualWriteReleaseHoldRepo`; flip `REPO_RELEASE_HOLD=dual` explicitly after wiring is proven.
16. Flip reads to `pg` per domain only after zero-diff shadow reads and a documented rollback plan are in place.
17. Run prod backfill under a maintenance window with `MIGRATION_PG_URL` pointing at prod; validate row counts before cutting over reads.
- **2.9.x:** Full MongoDB/Mongoose removal — all Mongoose models replaced by Drizzle repos, dual-write decommissioned, TypeScript compiles with 0 errors (2026-06-06).
## Related Docs