11 KiB
title, tags, aliases, created, source
| title | tags | aliases | created | source | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Postgres Runtime Cutover Status |
|
|
2026-05-31 | backend integrate-main-into-development@6df113d |
Postgres Runtime Cutover Status
Current branch: backend
integrate-main-into-developmentat6df113d, version2.8.13.Bottom line: this branch is Postgres-capable, not fully Postgres-backed. Auth-owned user data can run through Postgres when
AUTH_STORE=postgres; confirmation-threshold runtime config/history can run through Postgres whenCONFIG_STORE=postgres; user address CRUD can run through Postgres whenADDRESS_STORE=postgres; marketplace categories, level config, shop settings, and reviews can run through Postgres with their own store flags. All PG-backed stores requirePG_URL. Mongo remains the default and the compatibility store for still-Mongo domains.
What Uses Postgres Now
| Area | Runtime status | Notes |
|---|---|---|
| Postgres connection | Available when PG_URL is set |
Current store facades use src/infrastructure/postgres/client.ts; the broader src/db/ Drizzle layer and repository factory exist, but most live services are not wired through that factory yet. |
| Runtime schema bootstrap | Implemented for auth, config, address, and reference stores | Auth tables are bootstrapped from src/services/auth/postgresAuthSchema.ts; store facades bootstrap their own tables at startup when their *_STORE=postgres flag is enabled. |
| Health observability | Implemented in /api/health |
checks.postgres reports configured, required, storeModes, enabledStores, and enabledStoreCount, so Gatus/operators can verify both PG reachability and which runtime stores are actively PG-backed. |
| Auth-owned user store | Opt-in with AUTH_STORE=postgres |
Auth, passkey, Telegram auth/link/session/temp-verification, and /api/user profile paths use an auth-store facade. In PG mode, users are stored in Postgres and mirrored back to Mongo through legacy_object_id for compatibility with still-Mongo services. |
| Confirmation-threshold runtime config | Opt-in with CONFIG_STORE=postgres |
ConfigSetting / ConfigSettingHistory access for /api/admin/settings/confirmation-thresholds and transaction-safety confirmation thresholds uses a config-store facade. PG-mode writes mirror back to Mongo for rollback. |
| User addresses | Opt-in with ADDRESS_STORE=postgres |
/api/addresses CRUD uses an address-store facade. PG mode enforces one primary address per user with a partial unique index and mirrors writes/deletes back to Mongo for rollback. |
| Marketplace categories | Opt-in with CATEGORY_STORE=postgres |
CategoryService and the default General category path use a category-store facade. PG-mode writes mirror back to Mongo for rollback and still-Mongo request/template references. |
| Level configuration | Opt-in with LEVEL_CONFIG_STORE=postgres |
PointsService level reads use a level-config facade. PointTransaction and user points remain Mongo-backed. |
| Shop settings | Opt-in with SHOP_SETTINGS_STORE=postgres |
Shop settings controller, seller payment rail resolution, and review enable/disable checks use a shop-settings facade. PG-mode writes mirror back to Mongo. |
| Marketplace reviews | Opt-in with REVIEW_STORE=postgres |
Review list/summary/create routes use a review-store facade. PG-mode list responses still hydrate reviewerId from the user mirror to preserve frontend shape. |
| Repository implementations | Present but partial runtime wiring | src/db/repositories/* and Drizzle schemas exist for the target architecture, but this branch's live cutover work is still mostly store-specific raw PG facades plus the conditional oracle quote write path. |
| Oracle quote persistence | Conditional runtime PG write | /api/payment/request-network/intents lazily imports quoteRepo only when ORACLE_QUOTING_ENABLED=true; it writes payment_quotes if the PG parent payment row exists, mirrors to Mongo Payment.quote, and records pg_dualwrite_gaps if PG is behind. |
| Backfill/verify scripts | Available as operator tooling | MIGRATION_PG_URL drives backfill scripts; guards restrict allowed target hosts. The marketplace-core runner group now backfills users/categories, purchase requests, seller offers, and the post-offer selectedOfferId remap in dependency order. These scripts are not run automatically by app startup. |
What Is Still Mongo-Backed
Most of the service layer still imports Mongoose models directly. Auth-owned paths now have an auth-store boundary; confirmation-threshold config, user addresses, categories, level config, shop settings, and reviews have store boundaries. Broad marketplace requests/offers/templates, payment, funds ledger, points transactions, chat, notification, and admin paths remain Mongo-first.
| Domain | Current live store | Why not Postgres yet |
|---|---|---|
| Legacy/broad user consumers | MongoDB mirror | Auth-owned users can be PG-backed, but still-Mongo domains expect Mongo ObjectId user references. PG-mode writes therefore maintain a Mongo mirror until those domains are cut over. |
| Admin cleanup / seed address tooling | MongoDB | User-facing address CRUD is PG-capable, but admin cleanup and seed scripts still operate on Mongo first. Seed scripts backfill addresses to PG when ADDRESS_STORE=postgres. |
| Marketplace requests/offers/templates | MongoDB | Marketplace, checkout, and seller-offer services still call PurchaseRequest, SellerOffer, and RequestTemplate directly. Category and shop-settings reads/writes are PG-capable through facades. PurchaseRequest/SellerOffer backfill tooling is now operator-ready, but runtime reads/writes remain Mongo-first and RequestTemplate still has no PG table/repository. |
| Payments and escrow state | MongoDB primary | Request Network, AMN scanner, webhook, admin, release/refund, adapter, reconciliation, and legacy payment paths still create/update Payment Mongoose documents directly. |
| Funds ledger | MongoDB primary | FundsLedgerEntry remains the ledger used by current services; PG ledger tables exist but are not the live write target. |
| Derived destinations and sweeps | MongoDB | Wallet destination allocation and sweep services call DerivedDestination directly. |
| Points/referrals/transactions | MongoDB | Level configuration is PG-capable, but User points fields, PointTransaction, referral aggregation, and Mongo transactions remain Mongo-backed. |
| Chat/messages | MongoDB | Chat intentionally remains document-shaped and is not part of the current PG cutover. |
| Notifications | MongoDB | Notification TTL/read-state paths remain Mongo-backed. |
| Disputes/blog/content/admin cleanup | MongoDB | Reviews are PG-capable; disputes, blog/content, and admin cleanup still call their Mongoose models directly. |
| Runtime config outside confirmation thresholds | MongoDB | ConfigSetting and ConfigSettingHistory are PG-capable for confirmation thresholds only; any future admin-editable settings need to route through the same config-store boundary before they count as cut over. |
| Telegram link/session/temp verification | Default MongoDB, PG-capable with auth store | These records move with AUTH_STORE=postgres; default runtime remains Mongo until the environment flag is flipped. |
Env Flag Reality
| Flag | Current meaning |
|---|---|
AUTH_STORE |
mongo by default. Set AUTH_STORE=postgres to route auth-owned users, refresh tokens, passkeys, Telegram links/sessions, and temp verifications through Postgres. |
CONFIG_STORE |
mongo by default. Set CONFIG_STORE=postgres to route confirmation-threshold settings/history through Postgres. |
ADDRESS_STORE |
mongo by default. Set ADDRESS_STORE=postgres to route /api/addresses through Postgres. |
CATEGORY_STORE |
mongo by default. Set CATEGORY_STORE=postgres to route marketplace category reads/writes through Postgres. |
LEVEL_CONFIG_STORE |
mongo by default. Set LEVEL_CONFIG_STORE=postgres to route level configuration reads and seed replacement through Postgres. LEVEL_STORE=postgres is accepted as a compatibility alias. |
SHOP_SETTINGS_STORE |
mongo by default. Set SHOP_SETTINGS_STORE=postgres to route shop settings, review gates, and seller payment rails through Postgres. |
REVIEW_STORE |
mongo by default. Set REVIEW_STORE=postgres to route marketplace reviews through Postgres. |
PG_URL |
Makes PG code importable/reachable. Required for any *_STORE=postgres flag; does not cut over unrelated app domains by itself. |
MIGRATION_PG_URL |
Used by backfill scripts and migration runbooks; not part of normal request handling. Marketplace-core dry-run/non-dry backfills also require MIGRATION_MONGO_URL. |
REPO_USER, REPO_PAYMENT, REPO_POINTS, REPO_MARKETPLACE, REPO_DEFAULT |
Repository factory flags exist, but broad services are not yet wired through the factory. Treat them as migration controls that need integration verification before relying on them. |
ORACLE_QUOTING_ENABLED |
Enables server-side quote computation and the only current PG write path in normal checkout: payment_quotes, when a PG parent row can be resolved. |
Next Cutover Work
- Apply Drizzle migrations to the target Postgres database.
- For auth cutover, run
PG_URL=... npm run backfill:auth:postgres, verify counts, then setAUTH_STORE=postgres. - For confirmation-threshold config cutover, run
PG_URL=... npm run backfill:config:postgres, verify counts/history, then setCONFIG_STORE=postgres. - For address cutover, run
PG_URL=... npm run backfill:address:postgres, verify one-primary invariants, then setADDRESS_STORE=postgres. - For reference-domain cutover, run:
PG_URL=... npm run backfill:category:postgresPG_URL=... npm run backfill:level-config:postgresPG_URL=... npm run backfill:shop-settings:postgresPG_URL=... npm run backfill:review:postgres
- Run
PG_URL=... MONGODB_URI=... scripts/smoke/reference-stores-postgres.sh, then setCATEGORY_STORE=postgres LEVEL_CONFIG_STORE=postgres SHOP_SETTINGS_STORE=postgres REVIEW_STORE=postgrestogether in non-prod. - For marketplace-core data, run
MIGRATION_MONGO_URL=... MIGRATION_PG_URL=... npm run backfill:marketplace-core:postgres:dry-run, then the non-drynpm run backfill:marketplace-core:postgresagainst non-prod. The group runs root dependencies, PurchaseRequest main rows, SellerOffer rows, then the selected-offer remap. - Run
scripts/smoke/marketplace-core-postgres-backfill.shwith the same migration DSNs and record row-count/checksum results. - Wire remaining services to repository interfaces one domain at a time.
- Enable
dualmode per large domain only after wiring is proven by tests and smoke checks. - Run shadow-read/reconcile during a soak window.
- Flip reads to
pgper domain only after zero-diff shadow reads and a rollback plan are in place.