nick-doc/02 - Data Models/Postgres Runtime Cutover Status.md

title, tags, aliases, created, updated, source

title	tags	aliases	created	updated	source
Postgres Runtime Cutover Status	data-model postgres migration runtime-status	Postgres Status PG Cutover Status Mongo vs Postgres Runtime	2026-05-31	2026-06-03	backend integrate-main-into-development@14d164c + deployment main@8764fdf

Postgres Runtime Cutover Status

Current branch: backend integrate-main-into-development at 14d164c, version 2.8.56; dev deployment main at 8764fdf.

Bottom line: the codebase is in active dual-write phase. All 11 repository domains in the factory now have Drizzle schemas, Drizzle repos, and dual-write wrappers (except Chat and ReleaseHold, which have Drizzle repos but no dual-write counterpart). 18 Drizzle migrations (0000–0017) have landed, covering every table in scope. Dev deployment defaults nine PG-capable stores to Postgres: auth-owned users/Telegram auth, confirmation-threshold config/history, user addresses, categories, level config, shop settings, reviews, notifications, and the oracle payment_quotes path. Code-level defaults remain mongo outside those deployment overrides. Repository factory normalizes postgres and pg as equivalent mode tokens. The unmounted legacy marketplace router is detached. As of 2.8.54–2.8.56, the guard user role is in PG schema, chat routes are fixed, notifications deliver in real time, and PG response serialization/id resolution in marketplace is corrected. Reads are still Mongo-authoritative across all dual-write domains — read cutover is the remaining gate for each domain. Chat normalization (participants/messages stored as JSONB blobs, not relational child tables) remains an open blocker for full Chat cutover.

Schema and Repository Coverage

Tables with Full Drizzle Schema

All tables below have a .ts schema file in src/db/schema/ and are covered by at least one migration:

Infrastructure: id_map, pg_dualwrite_gaps

Auth/Users: users, user_passkeys, user_refresh_tokens, telegram_links, telegram_sessions

Marketplace: categories, purchase_requests, purchase_request_delivery_info, purchase_request_delivery_address, purchase_request_seller_delivery_info, purchase_request_service_info, purchase_request_specifications, purchase_request_preferred_sellers, delivery_attempts, seller_offers, request_templates

Payments: payments, payment_quotes, funds_ledger_entries, derived_destinations, derived_destination_sweeps

Points/Wallet: point_transactions, trezor_accounts, trezor_derived_addresses

Config/Ops: config_settings, config_setting_history, shop_settings, addresses, reviews

Content/Social: blog_posts, notifications, disputes, chats

Total: 32 tables across 18 migrations (0000–0017).

Tables with a Drizzle Repository

Drizzle Repo	Dual-Write Repo	Domain
DrizzleUserRepo	DualWriteUserRepo	Users, passkeys, refresh tokens
DrizzlePaymentRepo	DualWritePaymentRepo	Payments, funds ledger
DrizzleMarketplaceRepo	DualWriteMarketplaceRepo	Categories, purchase requests, seller offers, request templates
DrizzleDerivedDestinationRepo	DualWriteDerivedDestinationRepo	Derived destinations, sweeps
DrizzleTrezorAccountRepo	DualWriteTrezorAccountRepo	Trezor accounts, derived addresses
DrizzlePointsRepo	DualWritePointsRepo	Point transactions
DrizzleNotificationRepo	DualWriteNotificationRepo	Notifications
DrizzleDisputeRepo	DualWriteDisputeRepo	Disputes
DrizzleBlogRepo	DualWriteBlogRepo	Blog posts
DrizzleChatRepo	(none — no dual-write wrapper)	Chats (JSONB shim; Chat normalization is a blocker)
DrizzleReleaseHoldRepo	(none — no dual-write wrapper)	Release holds (bridges payments + purchase_requests)

Tables with schema but no dedicated Drizzle repo yet: addresses (handled via addressStore facade), shop_settings (handled via shopSettings facade), config_settings / config_setting_history (handled via config-store facade), telegram_links / telegram_sessions (handled via auth-store facade), reviews (handled via review-store facade).

Migration Count

18 migrations landed: 0000 through 0017.

Migration	Key change
0000	Core enums + id_map + categories
0001	trezor_accounts + trezor_derived_addresses
0002	Schema reset (drops 0000/0001 tables, adds category self-FK)
0003	Full rebuild: all core domain tables (users, payments, marketplace, funds ledger, derived destinations, points, trezor)
0004 (×2)	Funds ledger immutability trigger; seller_offer physical FKs
0005	pg_dualwrite_gaps; payment FKs; legacy_object_id uniques; pending payment index fix
0006	budget_currency crypto-only CHECK on purchase_requests
0007	Drops 0006 constraint; sets USDT default
0008	offer_currency adds TRY; creates payment_quotes
0009	Active category deduplication; categories_active_name_norm_uq
0010	request_templates; purchase_request_specifications unique constraint
0011	chats + chat enums
0012	disputes
0013	Money-integrity CHECK constraints; ledger TRUNCATE guard; id_map composite PK
0014	Physical NOT VALID FKs across schema; validates all
0015	Ledger immutability extended: UPDATE + DELETE triggers
0016	address_type enum + addresses table
0017	guard value added to user_role enum

What Uses Postgres Now

Area	Runtime status	Notes
Postgres connection	Available when PG_URL is set	Store facades use src/infrastructure/postgres/client.ts; the broader src/db/ Drizzle layer and repository factory are fully populated.
Runtime schema bootstrap	Implemented for auth, config, address, and reference stores	Auth tables bootstrapped from src/services/auth/postgresAuthSchema.ts; store facades bootstrap their own tables at startup when their *_STORE=postgres flag is enabled.
Health observability	Implemented in /api/health	checks.postgres reports configured, required, storeModes, enabledStores, and enabledStoreCount. Dev Gatus asserts all dev PG-backed store modes are postgres, including notifications. Mongoose health check is lazy-loaded and skipped when Mongo is optional under MONGO_CONNECT_MODE=auto/never.
Auth-owned user store	PG-backed in dev deployment; code opt-in with AUTH_STORE=postgres	Auth, passkey, Telegram auth/link/session/temp-verification, and /api/user profile paths use an auth-store facade. PG-mode users are mirrored back to Mongo through legacy_object_id for compatibility with still-Mongo services.
Confirmation-threshold runtime config	PG-backed in dev deployment; code opt-in with CONFIG_STORE=postgres	ConfigSetting / ConfigSettingHistory access for /api/admin/settings/confirmation-thresholds and transaction-safety confirmation thresholds uses a config-store facade. PG-mode writes mirror back to Mongo. Legacy models load only for Mongo fallback/backfill/mirror paths.
User addresses	PG-backed in dev deployment; code opt-in with ADDRESS_STORE=postgres	/api/addresses CRUD uses an address-store facade. PG mode enforces one primary address per user with a partial unique index and mirrors writes/deletes back to Mongo.
Marketplace categories	PG-backed in dev deployment; code opt-in with CATEGORY_STORE=postgres	CategoryService and the default General category path use a category-store facade. PG-mode writes mirror back to Mongo. Migration 0009 deactivated duplicate active category labels and enforces categories_active_name_norm_uq on lower(btrim(name)) WHERE is_active = true.
Level configuration	PG-backed in dev deployment; code opt-in with LEVEL_CONFIG_STORE=postgres	PointsService level reads use a level-config facade. PointTransaction and user points remain Mongo-backed. LEVEL_STORE=postgres accepted as compatibility alias.
Shop settings	PG-backed in dev deployment; code opt-in with SHOP_SETTINGS_STORE=postgres	Shop settings controller, seller payment rail resolution (2.8.56 fixes seller shop lookup to handle both uuid and legacy id formats), and review enable/disable checks use a shop-settings facade. PG-mode writes mirror back to Mongo.
Marketplace reviews	PG-backed in dev deployment; code opt-in with REVIEW_STORE=postgres	Review list/summary/create routes use a review-store facade. PG-mode list responses still hydrate reviewerId from the user mirror to preserve frontend shape.
Notifications	PG-backed in dev deployment; code opt-in with NOTIFICATION_STORE=postgres or REPO_NOTIFICATION=pg	NotificationService uses getNotificationRepo() for create/list/read/delete/count paths. 2.8.55 fixes chat routes and delivers notifications in real time. 2.8.37 fixes repository mode aliasing so postgres resolves to the Drizzle notification repo. Backfill script (npm run backfill:notification:postgres) and smoke script (scripts/smoke/notifications-postgres.sh) are available.
Oracle quote persistence	Conditional runtime PG write	/api/payment/request-network/intents lazily imports quoteRepo only when ORACLE_QUOTING_ENABLED=true; it writes payment_quotes if the PG parent payment row exists, mirrors to Mongo Payment.quote, and records pg_dualwrite_gaps if PG is behind.
Funds ledger	Repository-backed, default Mongo	appendFundsLedgerEntry and getFundsBalanceBy* call getPaymentRepo(). Default is MongoPaymentRepo; REPO_PAYMENT=dual/pg exercises the Drizzle ledger after backfill/soak.
Backfill/verify scripts	Available as operator tooling	MIGRATION_PG_URL drives all backfill scripts; guards restrict allowed target hosts. Marketplace-core runner backfills users/categories, request templates, purchase requests, seller offers, and selectedOfferId remap in dependency order. Not run automatically at startup.
Guard user role	PG schema-ready	Migration 0017 adds guard to the user_role enum. 2.8.54 adds guard role support across auth and user management.
PG response serialization	Fixed in 2.8.51–2.8.53	PG response serialization and id resolution in marketplace purchase-request paths corrected; user creation and purchase request unblocked from a PG FK constraint error.
Admin user management	PG-capable as of 2.8.50	Admin user count queries route through postgres-capable stores; admin user management works end-to-end under PG.
Seeds	Postgres-capable as of 2.8.47	Seeds in src/seeds/* are store-aware and idempotent; can seed fresh PG under MONGO_CONNECT_MODE=never.

What Is Still Mongo-Backed

Writes across all dual-write domains go to both Mongo and Postgres. Reads remain Mongo-authoritative for every dual-write domain — read cutover has not been performed for any domain. Chat is repository-backed (Drizzle repo exists) but participants/messages are stored as JSONB blobs rather than normalized child tables; Chat normalization is the primary structural blocker for Chat read cutover.

Domain	Current live store	Why not Postgres yet
User reads	MongoDB authoritative	Auth-owned users can be PG-backed for writes, but reads remain Mongo-authoritative. Still-Mongo domains expect Mongo ObjectId user references; PG-mode writes maintain a Mongo mirror until all consumers cut over.
Admin cleanup / seed address tooling	MongoDB	User-facing address CRUD is PG-capable, but admin cleanup scripts still operate on Mongo first. Seed scripts backfill addresses to PG when ADDRESS_STORE=postgres.
Marketplace requests/offers/templates reads	Repository-backed writes; Mongo reads	getMarketplaceRepo() wires all marketplace writes. REPO_MARKETPLACE defaults to Mongo; full PG/dual read cutover needs smoke coverage before flipping. The legacy marketplaceRouter is detached from the service index (2.8.36).
Payments and escrow state reads	MongoDB primary	Request Network, AMN scanner, webhook, admin, release/refund, adapter, reconciliation, and legacy payment paths still create/update Payment Mongoose documents directly for reads. Payment Drizzle repo and dual-write repo exist; REPO_PAYMENT=dual enables dual-writes, but read paths remain Mongo.
Derived destinations and sweeps	Repository-backed writes; Mongo reads	getDerivedDestinationRepo() wires writes; REPO_DERIVED_DESTINATION defaults to Mongo and has not been flipped in dev.
Points/referrals/transactions	Repository-backed writes; Mongo reads	getPointsRepo() wires writes; REPO_POINTS defaults to Mongo. Level config is PG-capable for reads; point transaction and user-point flows are not flipped in dev.
Chat/messages	Repository-backed writes (JSONB shim); Mongo reads	getChatRepo() wires writes. REPO_CHAT / CHAT_STORE defaults to Mongo. DrizzleChatRepo uses JSONB blobs for participants and messages — Chat normalization into relational child tables is required before safe read cutover. No dual-write wrapper exists.
Disputes/blog	Repository-backed writes; Mongo reads	Both have Drizzle repos and dual-write wrappers. Code defaults resolve to Mongo until REPO_DISPUTE/BLOG_STORE are flipped.
ReleaseHold	Drizzle repo only; default Mongo	getReleaseHoldRepo() returns Drizzle or Mongo; no dual-write wrapper — dual mode silently uses Mongo. Separate cutover needed.
Runtime config outside confirmation thresholds	MongoDB	ConfigSetting and ConfigSettingHistory are PG-capable for confirmation thresholds only; other admin-editable settings need to route through the config-store boundary before counting as cut over.
Telegram link/session/temp verification reads	PG-backed writes in dev; Mongo reads in code default	These records move with AUTH_STORE=postgres. Dev compose defaults that flag to postgres; read cutover follows the auth-store flag.

Env Flag Reality

The backend code defaults every store flag below to mongo. Dev deployment overrides eight PG-capable store flags to postgres in deployment/docker-compose.yml. Repository factory normalizes postgres and pg as equivalent.

Flag	Current meaning
AUTH_STORE	Code default mongo; dev deployment default postgres. Routes auth-owned users, refresh tokens, passkeys, Telegram links/sessions, and temp verifications through Postgres.
CONFIG_STORE	Code default mongo; dev deployment default postgres. Routes confirmation-threshold settings/history through Postgres.
ADDRESS_STORE	Code default mongo; dev deployment default postgres. Routes /api/addresses through Postgres.
CATEGORY_STORE	Code default mongo; dev deployment default postgres. Routes marketplace category reads/writes through Postgres. Active PG categories are unique by normalized visible name.
LEVEL_CONFIG_STORE	Code default mongo; dev deployment default postgres. Routes level configuration reads and seed replacement through Postgres. LEVEL_STORE=postgres accepted as compatibility alias.
SHOP_SETTINGS_STORE	Code default mongo; dev deployment default postgres. Routes shop settings, review gates, and seller payment rails through Postgres. 2.8.56 fixes seller shop lookup tolerance for uuid vs legacy id formats.
REVIEW_STORE	Code default mongo; dev deployment default postgres. Routes marketplace reviews through Postgres.
NOTIFICATION_STORE / REPO_NOTIFICATION	Code default mongo; dev deployment default postgres. Routes notification inbox create/list/read/delete/count through the Drizzle notification repo. postgres and pg both resolve correctly since 2.8.37.
PG_URL	Makes PG code importable/reachable. Required for any *_STORE=postgres flag; does not cut over unrelated app domains by itself.
MIGRATION_PG_URL	Used by backfill scripts and migration runbooks; not part of normal request handling. Marketplace-core dry-run/non-dry backfills also require MIGRATION_MONGO_URL.
REPO_PAYMENT	Code default mongo. Funds ledger appends and balance reads route through this flag. dual mode enables dual-write for the ledger seam. Do not flip broad payment runtime to pg yet; most payment services still call Mongoose directly for reads.
REPO_MARKETPLACE	Code default mongo. All marketplace writes route through getMarketplaceRepo(). Full read cutover needs smoke coverage.
REPO_USER, REPO_POINTS, REPO_DERIVED_DESTINATION, REPO_TREZOR	Factory flags with full trio (mongo/dual/pg). Writes are wired; reads remain Mongo-authoritative until each flag is flipped and verified.
REPO_DISPUTE / DISPUTE_STORE, REPO_BLOG / BLOG_STORE	Code default mongo. Dual-write wrappers exist; flip per-domain after verification.
REPO_CHAT / CHAT_STORE	Code default mongo. Chat normalization (JSONB→relational) is a structural blocker; no dual-write wrapper.
REPO_RELEASE_HOLD / RELEASE_HOLD_STORE	Code default mongo. No dual-write wrapper; dual silently uses Mongo. Must be flipped explicitly. As of 2.8.37, REPO_DISPUTE=pg no longer leaks into release hold mode.
ORACLE_QUOTING_ENABLED	Enables server-side quote computation and the payment_quotes PG write in checkout when a PG parent payment row exists.
MONGO_CONNECT_MODE	Handled in Mongoose connection setup (not in the repository factory). auto/never allow PG-only boot; lazy Mongoose health check skips when Mongo is optional.

Overall Migration Phase

Phase	Status
Schema design	Complete — 32 tables, 18 migrations (0000–0017)
Drizzle repos	Complete — all 11 factory domains have a Drizzle repo
Dual-write wrappers	Mostly complete — 9 of 11 domains have a dual-write wrapper (Chat and ReleaseHold are exceptions)
Write cutover (dual-write active)	Not yet enabled by default — REPO_DEFAULT is still mongo; must be flipped per-domain with care
Read cutover	Not started for any domain — Mongo remains authoritative for all reads
Prod backfill	Not run — backfill scripts are operator-ready but not executed against production
Chat normalization	Blocked — participants/messages stored as JSONB; relational normalization required before Chat read cutover

Estimated overall: schema and infrastructure phase complete; write-seam phase substantially complete; read cutover and backfill execution remain for every domain.

Recent Progress Since Last Update (2.8.37 → 2.8.56)

• 2.8.38–2.8.46: Complete dual-write repos for all remaining domains; Drizzle migrations pipeline finalized; TTL scheduler added; shop lookup bug-fixed.
• 2.8.47: Seeds made Postgres-capable and idempotent for PG-only boot (MONGO_CONNECT_MODE=never).
• 2.8.48–2.8.49: Fresh-DB PG migrate + seed path corrected; 0013/0014 migrations made valid for a fresh drizzle-kit migrate run.
• 2.8.50: Admin user counts routed through postgres-capable stores; admin user management works end-to-end under PG.
• 2.8.51–2.8.53: PG response serialization and id resolution corrected in marketplace; user creation and purchase request creation unblocked from PG FK constraint errors.
• 2.8.54: guard user role added to user_role enum (migration 0017); guard role support across auth and user management.
• 2.8.55: Chat routes fixed; notifications delivered in real time alongside chat.
• 2.8.56: Seller shop lookup made tolerant of both uuid and legacy id formats; dataCleanupService guarded against MONGO_CONNECT_MODE=never.

Next Cutover Work

1. Apply Drizzle migrations to the target Postgres database (0000–0017 must be in-order; 0002 is a reset migration — confirm idempotency on existing instances).
2. For dev/test data, run the existing backfills or reseed acceptable test data before relying on PG-backed stores. The deployment default flip does not move historical Mongo rows.
3. For auth cutover, run PG_URL=... npm run backfill:auth:postgres, verify counts, and confirm AUTH_STORE=postgres in the target runtime.
4. For confirmation-threshold config cutover, run PG_URL=... npm run backfill:config:postgres, verify counts/history, and confirm CONFIG_STORE=postgres.
5. For address cutover, run PG_URL=... npm run backfill:address:postgres, verify one-primary invariants, and confirm ADDRESS_STORE=postgres.
6. For reference-domain cutover, run:
   • PG_URL=... npm run backfill:category:postgres
   • PG_URL=... npm run backfill:level-config:postgres
   • PG_URL=... npm run backfill:shop-settings:postgres
   • PG_URL=... npm run backfill:review:postgres
7. Run PG_URL=... scripts/smoke/categories-postgres-unique.sh and PG_URL=... MONGODB_URI=... scripts/smoke/reference-stores-postgres.sh, then confirm reference store flags in non-prod.
8. For marketplace-core data, run MIGRATION_MONGO_URL=... MIGRATION_PG_URL=... npm run backfill:marketplace-core:postgres:dry-run, then the non-dry run. The group runs root dependencies, RequestTemplate rows, PurchaseRequest main rows, SellerOffer rows, then the selected-offer remap.
9. Run scripts/smoke/marketplace-core-postgres-backfill.sh with the same migration DSNs and record row-count/checksum results.
10. For notifications, run PG_URL=... npm run backfill:notification:postgres and PG_URL=... scripts/smoke/notifications-postgres.sh against dev to validate the current default.
11. Enable REPO_MARKETPLACE=dual, REPO_PAYMENT=dual, REPO_POINTS=dual, REPO_DERIVED_DESTINATION=dual, REPO_TREZOR=dual, REPO_DISPUTE=dual, REPO_BLOG=dual one domain at a time after backfill verification, and run a soak window before flipping reads.
12. Continue payment-domain wiring: add missing payment repo methods for provider lookups, transaction-hash/webhook lookups, metadata/blockchain patching, template duplicate cleanup, and quote updates before moving paymentService, paymentCoordinator, RN, or AMN scanner routes.
13. Add a derived-destination/sweep repository seam before payment PG read cutover; destination allocation is payment-address state and should not remain Mongo-only once payments become PG-backed for reads.
14. Resolve Chat normalization: design relational child tables for participants and messages; migrate DrizzleChatRepo away from JSONB blobs; add DualWriteChatRepo; flip REPO_CHAT=dual only after normalization.
15. Add DualWriteReleaseHoldRepo; flip REPO_RELEASE_HOLD=dual explicitly after wiring is proven.
16. Flip reads to pg per domain only after zero-diff shadow reads and a documented rollback plan are in place.
17. Run prod backfill under a maintenance window with MIGRATION_PG_URL pointing at prod; validate row counts before cutting over reads.

Related Docs

• Database Strategy - Mongo vs Postgres Assessment
• MongoDB to PostgreSQL Migration Plan (Drizzle)
• Payment
• Payment API
• Environment Variables
• Database Operations