|
|
|
|
@@ -3,48 +3,63 @@ title: Postgres Runtime Cutover Status
|
|
|
|
|
tags: [data-model, postgres, migration, runtime-status]
|
|
|
|
|
aliases: [Postgres Status, PG Cutover Status, Mongo vs Postgres Runtime]
|
|
|
|
|
created: 2026-05-31
|
|
|
|
|
source: backend origin/integrate-main-into-development@cab0719
|
|
|
|
|
source: backend integrate-main-into-development@1757f1e
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
# Postgres Runtime Cutover Status
|
|
|
|
|
|
|
|
|
|
> **Current branch:** backend `origin/integrate-main-into-development` at `cab0719`, version `2.6.80`.
|
|
|
|
|
> **Current branch:** backend `integrate-main-into-development` at `1757f1e`, version `2.8.9`.
|
|
|
|
|
>
|
|
|
|
|
> **Bottom line:** this branch is **Postgres-capable**, not fully Postgres-backed. MongoDB remains the live source of truth for normal app traffic until services are actually rewired through the repository factory and cutover flags are verified in a running environment.
|
|
|
|
|
> **Bottom line:** this branch is **Postgres-capable**, not fully Postgres-backed. Auth-owned user data can run through Postgres when `AUTH_STORE=postgres`; confirmation-threshold runtime config/history can run through Postgres when `CONFIG_STORE=postgres`; user address CRUD can run through Postgres when `ADDRESS_STORE=postgres`; marketplace categories, level config, shop settings, and reviews can run through Postgres with their own store flags. All PG-backed stores require `PG_URL`. Mongo remains the default and the compatibility store for still-Mongo domains.
|
|
|
|
|
|
|
|
|
|
## What Uses Postgres Now
|
|
|
|
|
|
|
|
|
|
| Area | Runtime status | Notes |
|
|
|
|
|
|---|---|---|
|
|
|
|
|
| Postgres connection | Available when `PG_URL` is set | `src/db/client.ts` fail-fast requires an explicit DSN before importing PG code. |
|
|
|
|
|
| Schema/migrations | Implemented through migration `0008` | Drizzle schemas cover users, categories, purchase requests, seller offers, payments, funds ledger, derived destinations, trezor accounts, points, `id_map`, `pg_dualwrite_gaps`, and `payment_quotes`. |
|
|
|
|
|
| Repository implementations | Implemented but not broadly called | `src/db/repositories/{mongo,drizzle,dual}` and `factory.ts` exist for user, payment, points, and marketplace domains. |
|
|
|
|
|
| Postgres connection | Available when `PG_URL` is set | Current runtime code uses `src/infrastructure/postgres/client.ts`; there is no `src/db/` Drizzle layer in this checkout. |
|
|
|
|
|
| Runtime schema bootstrap | Implemented for auth, config, address, and reference stores | Auth tables are bootstrapped from `src/services/auth/postgresAuthSchema.ts`; store facades bootstrap their own tables at startup when their `*_STORE=postgres` flag is enabled. |
|
|
|
|
|
| Auth-owned user store | Opt-in with `AUTH_STORE=postgres` | Auth, passkey, Telegram auth/link/session/temp-verification, and `/api/user` profile paths use an auth-store facade. In PG mode, users are stored in Postgres and mirrored back to Mongo through `legacy_object_id` for compatibility with still-Mongo services. |
|
|
|
|
|
| Confirmation-threshold runtime config | Opt-in with `CONFIG_STORE=postgres` | `ConfigSetting` / `ConfigSettingHistory` access for `/api/admin/settings/confirmation-thresholds` and transaction-safety confirmation thresholds uses a config-store facade. PG-mode writes mirror back to Mongo for rollback. |
|
|
|
|
|
| User addresses | Opt-in with `ADDRESS_STORE=postgres` | `/api/addresses` CRUD uses an address-store facade. PG mode enforces one primary address per user with a partial unique index and mirrors writes/deletes back to Mongo for rollback. |
|
|
|
|
|
| Marketplace categories | Opt-in with `CATEGORY_STORE=postgres` | `CategoryService` and the default `General` category path use a category-store facade. PG-mode writes mirror back to Mongo for rollback and still-Mongo request/template references. |
|
|
|
|
|
| Level configuration | Opt-in with `LEVEL_CONFIG_STORE=postgres` | `PointsService` level reads use a level-config facade. `PointTransaction` and user points remain Mongo-backed. |
|
|
|
|
|
| Shop settings | Opt-in with `SHOP_SETTINGS_STORE=postgres` | Shop settings controller, seller payment rail resolution, and review enable/disable checks use a shop-settings facade. PG-mode writes mirror back to Mongo. |
|
|
|
|
|
| Marketplace reviews | Opt-in with `REVIEW_STORE=postgres` | Review list/summary/create routes use a review-store facade. PG-mode list responses still hydrate `reviewerId` from the user mirror to preserve frontend shape. |
|
|
|
|
|
| Repository implementations | Not present in current checkout | The old Drizzle plan still describes the target architecture, but this branch's actual cutover work is currently store-specific raw PG facades. |
|
|
|
|
|
| Oracle quote persistence | Conditional runtime PG write | `/api/payment/request-network/intents` lazily imports `quoteRepo` only when `ORACLE_QUOTING_ENABLED=true`; it writes `payment_quotes` if the PG parent payment row exists, mirrors to Mongo `Payment.quote`, and records `pg_dualwrite_gaps` if PG is behind. |
|
|
|
|
|
| Backfill/verify scripts | Available as operator tooling | `MIGRATION_PG_URL` drives backfill scripts; guards restrict allowed target hosts. These scripts are not run automatically by app startup. |
|
|
|
|
|
|
|
|
|
|
## What Is Still Mongo-Backed
|
|
|
|
|
|
|
|
|
|
The service layer still imports Mongoose models directly. A source scan on 2026-05-31 found no runtime calls to `createRepositories()` / `getPaymentRepo()` / `getMarketplaceRepo()` outside `src/db/repositories/factory.ts`; the only normal service import of `src/db/client.ts` is `priceOracle/quoteRepo.ts`.
|
|
|
|
|
Most of the service layer still imports Mongoose models directly. Auth-owned paths now have an auth-store boundary; confirmation-threshold config, user addresses, categories, level config, shop settings, and reviews have store boundaries. Broad marketplace requests/offers/templates, payment, funds ledger, points transactions, chat, notification, and admin paths remain Mongo-first.
|
|
|
|
|
|
|
|
|
|
| Domain | Current live store | Why not Postgres yet |
|
|
|
|
|
|---|---|---|
|
|
|
|
|
| Auth, users, passkeys, refresh tokens | MongoDB | Auth/user controllers call `User` and related Mongoose models directly. |
|
|
|
|
|
| Marketplace requests/offers/templates/categories/shop settings | MongoDB | Marketplace, checkout, and seller-offer services call `PurchaseRequest`, `SellerOffer`, `RequestTemplate`, `Category`, and `ShopSettings` directly. |
|
|
|
|
|
| Legacy/broad user consumers | MongoDB mirror | Auth-owned users can be PG-backed, but still-Mongo domains expect Mongo ObjectId user references. PG-mode writes therefore maintain a Mongo mirror until those domains are cut over. |
|
|
|
|
|
| Admin cleanup / seed address tooling | MongoDB | User-facing address CRUD is PG-capable, but admin cleanup and seed scripts still operate on Mongo first. Seed scripts backfill addresses to PG when `ADDRESS_STORE=postgres`. |
|
|
|
|
|
| Marketplace requests/offers/templates | MongoDB | Marketplace, checkout, and seller-offer services still call `PurchaseRequest`, `SellerOffer`, and `RequestTemplate` directly. Category and shop-settings reads/writes are PG-capable through facades, but request/template documents still hold Mongo ObjectId references. |
|
|
|
|
|
| Payments and escrow state | MongoDB primary | Request Network, AMN scanner, webhook, admin, release/refund, adapter, reconciliation, and legacy payment paths still create/update `Payment` Mongoose documents directly. |
|
|
|
|
|
| Funds ledger | MongoDB primary | `FundsLedgerEntry` remains the ledger used by current services; PG ledger tables exist but are not the live write target. |
|
|
|
|
|
| Derived destinations and sweeps | MongoDB | Wallet destination allocation and sweep services call `DerivedDestination` directly. |
|
|
|
|
|
| Points/referrals/levels | MongoDB | Points service uses `User`, `PointTransaction`, `LevelConfig`, and Mongo transactions. |
|
|
|
|
|
| Points/referrals/transactions | MongoDB | Level configuration is PG-capable, but `User` points fields, `PointTransaction`, referral aggregation, and Mongo transactions remain Mongo-backed. |
|
|
|
|
|
| Chat/messages | MongoDB | Chat intentionally remains document-shaped and is not part of the current PG cutover. |
|
|
|
|
|
| Notifications | MongoDB | Notification TTL/read-state paths remain Mongo-backed. |
|
|
|
|
|
| Disputes/reviews/blog/content/admin cleanup | MongoDB | These services still call their Mongoose models directly. |
|
|
|
|
|
| Runtime config | MongoDB | `ConfigSetting` and `ConfigSettingHistory` remain Mongo-backed for admin-editable configuration. |
|
|
|
|
|
| Telegram link/session/temp verification | MongoDB | These session/link/TTL records are not cut over to PG. |
|
|
|
|
|
| Disputes/blog/content/admin cleanup | MongoDB | Reviews are PG-capable; disputes, blog/content, and admin cleanup still call their Mongoose models directly. |
|
|
|
|
|
| Runtime config outside confirmation thresholds | MongoDB | `ConfigSetting` and `ConfigSettingHistory` are PG-capable for confirmation thresholds only; any future admin-editable settings need to route through the same config-store boundary before they count as cut over. |
|
|
|
|
|
| Telegram link/session/temp verification | Default MongoDB, PG-capable with auth store | These records move with `AUTH_STORE=postgres`; default runtime remains Mongo until the environment flag is flipped. |
|
|
|
|
|
|
|
|
|
|
## Env Flag Reality
|
|
|
|
|
|
|
|
|
|
| Flag | Current meaning |
|
|
|
|
|
|---|---|
|
|
|
|
|
| `PG_URL` | Makes PG code importable/reachable; does not cut over app domains by itself. |
|
|
|
|
|
| `AUTH_STORE` | `mongo` by default. Set `AUTH_STORE=postgres` to route auth-owned users, refresh tokens, passkeys, Telegram links/sessions, and temp verifications through Postgres. |
|
|
|
|
|
| `CONFIG_STORE` | `mongo` by default. Set `CONFIG_STORE=postgres` to route confirmation-threshold settings/history through Postgres. |
|
|
|
|
|
| `ADDRESS_STORE` | `mongo` by default. Set `ADDRESS_STORE=postgres` to route `/api/addresses` through Postgres. |
|
|
|
|
|
| `CATEGORY_STORE` | `mongo` by default. Set `CATEGORY_STORE=postgres` to route marketplace category reads/writes through Postgres. |
|
|
|
|
|
| `LEVEL_CONFIG_STORE` | `mongo` by default. Set `LEVEL_CONFIG_STORE=postgres` to route level configuration reads and seed replacement through Postgres. `LEVEL_STORE=postgres` is accepted as a compatibility alias. |
|
|
|
|
|
| `SHOP_SETTINGS_STORE` | `mongo` by default. Set `SHOP_SETTINGS_STORE=postgres` to route shop settings, review gates, and seller payment rails through Postgres. |
|
|
|
|
|
| `REVIEW_STORE` | `mongo` by default. Set `REVIEW_STORE=postgres` to route marketplace reviews through Postgres. |
|
|
|
|
|
| `PG_URL` | Makes PG code importable/reachable. Required for any `*_STORE=postgres` flag; does not cut over unrelated app domains by itself. |
|
|
|
|
|
| `MIGRATION_PG_URL` | Used by backfill scripts and migration runbooks; not part of normal request handling. |
|
|
|
|
|
| `REPO_USER`, `REPO_PAYMENT`, `REPO_POINTS`, `REPO_MARKETPLACE`, `REPO_DEFAULT` | Repository factory flags exist, but broad services are not yet wired through the factory. Treat them as migration controls that need integration verification before relying on them. |
|
|
|
|
|
| `ORACLE_QUOTING_ENABLED` | Enables server-side quote computation and the only current PG write path in normal checkout: `payment_quotes`, when a PG parent row can be resolved. |
|
|
|
|
|
@@ -52,11 +67,20 @@ The service layer still imports Mongoose models directly. A source scan on 2026-
|
|
|
|
|
## Next Cutover Work
|
|
|
|
|
|
|
|
|
|
1. Apply Drizzle migrations to the target Postgres database.
|
|
|
|
|
2. Run non-prod backfills in dependency order and record row-count/checksum results.
|
|
|
|
|
3. Wire services to repository interfaces one domain at a time.
|
|
|
|
|
4. Enable `dual` mode per domain only after wiring is proven by tests and smoke checks.
|
|
|
|
|
5. Run shadow-read/reconcile during a soak window.
|
|
|
|
|
6. Flip reads to `pg` per domain only after zero-diff shadow reads and a rollback plan are in place.
|
|
|
|
|
2. For auth cutover, run `PG_URL=... npm run backfill:auth:postgres`, verify counts, then set `AUTH_STORE=postgres`.
|
|
|
|
|
3. For confirmation-threshold config cutover, run `PG_URL=... npm run backfill:config:postgres`, verify counts/history, then set `CONFIG_STORE=postgres`.
|
|
|
|
|
4. For address cutover, run `PG_URL=... npm run backfill:address:postgres`, verify one-primary invariants, then set `ADDRESS_STORE=postgres`.
|
|
|
|
|
5. For reference-domain cutover, run:
|
|
|
|
|
- `PG_URL=... npm run backfill:category:postgres`
|
|
|
|
|
- `PG_URL=... npm run backfill:level-config:postgres`
|
|
|
|
|
- `PG_URL=... npm run backfill:shop-settings:postgres`
|
|
|
|
|
- `PG_URL=... npm run backfill:review:postgres`
|
|
|
|
|
6. Run `PG_URL=... MONGODB_URI=... scripts/smoke/reference-stores-postgres.sh`, then set `CATEGORY_STORE=postgres LEVEL_CONFIG_STORE=postgres SHOP_SETTINGS_STORE=postgres REVIEW_STORE=postgres` together in non-prod.
|
|
|
|
|
7. Run non-prod backfills for the remaining larger domains in dependency order and record row-count/checksum results.
|
|
|
|
|
8. Wire remaining services to repository interfaces one domain at a time.
|
|
|
|
|
9. Enable `dual` mode per large domain only after wiring is proven by tests and smoke checks.
|
|
|
|
|
10. Run shadow-read/reconcile during a soak window.
|
|
|
|
|
11. Flip reads to `pg` per domain only after zero-diff shadow reads and a rollback plan are in place.
|
|
|
|
|
|
|
|
|
|
## Related Docs
|
|
|
|
|
|
|
|
|
|
|
Siavash Sameni