Files
nick-doc/02 - Data Models/Postgres Runtime Cutover Status.md

204 lines
14 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
---
title: Postgres Runtime Cutover Status
tags: [data-model, postgres, migration, runtime-status]
aliases: [Postgres Status, PG Cutover Status, Mongo vs Postgres Runtime]
created: 2026-05-31
updated: 2026-06-06
source: backend integrate-main-into-development@41087c7 + deployment main@8764fdf
---
# Postgres Runtime Cutover Status
> **Current branch:** backend `integrate-main-into-development`, version `2.9.12`.
>
> **Bottom line: Migration complete as of 2026-06-06, backend v2.9.12.** MongoDB and Mongoose have been fully removed from the runtime. PostgreSQL (Drizzle ORM) is the sole database. All 11 repository domains use DrizzleXxxRepo exclusively. No dual-write wrappers are active. TypeScript compilation: 0 errors.
## Migration Status
| Phase | Status |
|---|---|
| Schema design | Complete — 32 tables, 19 migrations (00000019) |
| Drizzle repos | Complete — all 11 factory domains have a DrizzleXxxRepo |
| Dual-write wrappers | Decommissioned — removed from runtime as of v2.9.12 |
| Write cutover | Complete — all writes go to PostgreSQL only |
| Read cutover | Complete — all reads from PostgreSQL |
| Mongoose removal | Complete — no Mongoose imports in runtime src/ |
| TypeScript compilation | 0 errors |
## Schema and Repository Coverage
### Tables with Full Drizzle Schema
All tables below have a `.ts` schema file in `src/db/schema/` and are covered by at least one migration:
**Infrastructure:** `id_map`, `pg_dualwrite_gaps`
**Auth/Users:** `users`, `user_passkeys`, `user_refresh_tokens`, `telegram_links`, `telegram_sessions`
**Marketplace:** `categories`, `purchase_requests`, `purchase_request_delivery_info`, `purchase_request_delivery_address`, `purchase_request_seller_delivery_info`, `purchase_request_service_info`, `purchase_request_specifications`, `purchase_request_preferred_sellers`, `delivery_attempts`, `seller_offers`, `request_templates`
**Payments:** `payments`, `payment_quotes`, `funds_ledger_entries`, `derived_destinations`, `derived_destination_sweeps`
**Points/Wallet:** `point_transactions`, `trezor_accounts`, `trezor_derived_addresses`
**Config/Ops:** `config_settings`, `config_setting_history`, `shop_settings`, `addresses`, `reviews`
**Content/Social:** `blog_posts`, `notifications`, `disputes`, `chats`
Total: **32 tables** across 19 migrations (00000019).
### Tables with a Drizzle Repository
| Drizzle Repo | Domain |
|---|---|
| `DrizzleUserRepo` | Users, passkeys, refresh tokens |
| `DrizzlePaymentRepo` | Payments, funds ledger |
| `DrizzleMarketplaceRepo` | Categories, purchase requests, seller offers, request templates |
| `DrizzleDerivedDestinationRepo` | Derived destinations, sweeps |
| `DrizzleTrezorAccountRepo` | Trezor accounts, derived addresses |
| `DrizzlePointsRepo` | Point transactions |
| `DrizzleNotificationRepo` | Notifications |
| `DrizzleDisputeRepo` | Disputes |
| `DrizzleBlogRepo` | Blog posts |
| `DrizzleChatRepo` | Chats (JSONB shim; Chat normalization is an optional future improvement) |
| `DrizzleReleaseHoldRepo` | Release holds (bridges payments + purchase_requests) |
Tables with schema but no dedicated Drizzle repo (handled via store facades): `addresses`, `shop_settings`, `config_settings` / `config_setting_history`, `telegram_links` / `telegram_sessions`, `reviews`.
### Migration Count
19 migrations landed: **0000 through 0019**.
| Migration | Key change |
|---|---|
| 0000 | Core enums + `id_map` + `categories` |
| 0001 | `trezor_accounts` + `trezor_derived_addresses` |
| 0002 | Schema reset (drops 0000/0001 tables, adds category self-FK) |
| 0003 | Full rebuild: all core domain tables (users, payments, marketplace, funds ledger, derived destinations, points, trezor) |
| 0004 (×2) | Funds ledger immutability trigger; seller_offer physical FKs |
| 0005 | `pg_dualwrite_gaps`; payment FKs; legacy_object_id uniques; pending payment index fix |
| 0006 | budget_currency crypto-only CHECK on purchase_requests |
| 0007 | Drops 0006 constraint; sets USDT default |
| 0008 | `offer_currency` adds TRY; creates `payment_quotes` |
| 0009 | Active category deduplication; `categories_active_name_norm_uq` |
| 0010 | `request_templates`; purchase_request_specifications unique constraint |
| 0011 | `chats` + chat enums |
| 0012 | `disputes` |
| 0013 | Money-integrity CHECK constraints; ledger TRUNCATE guard; id_map composite PK |
| 0014 | Physical NOT VALID FKs across schema; validates all |
| 0015 | Ledger immutability extended: UPDATE + DELETE triggers |
| 0016 | `address_type` enum + `addresses` table |
| 0017 | `guard` value added to `user_role` enum |
| 0018 | AI request fields |
| 0019 | `payment_provider` enum: added `escrow` |
## What Uses Postgres Now
All domains are PostgreSQL-only. The table below summarises the runtime topology for reference.
| Area | Runtime status | Notes |
|---|---|---|
| Postgres connection | Required — `PG_URL` must be set | Store facades use `src/infrastructure/postgres/client.ts`; the broader `src/db/` Drizzle layer and repository factory are fully populated. |
| Runtime schema bootstrap | Implemented for auth, config, address, and reference stores | Auth tables bootstrapped from `src/services/auth/postgresAuthSchema.ts`; store facades bootstrap their own tables at startup. |
| Health observability | Implemented in `/api/health` | `checks.postgres` reports `configured`, `required`, `storeModes`, `enabledStores`, and `enabledStoreCount`. Mongoose health check is no longer present. |
| Auth-owned user store | PG-backed | Auth, passkey, Telegram auth/link/session/temp-verification, and `/api/user` profile paths use the auth-store facade pointing at Postgres. `legacy_object_id` column retained for id-map compatibility. |
| Confirmation-threshold runtime config | PG-backed | `ConfigSetting` / `ConfigSettingHistory` access routes through the config-store facade. |
| User addresses | PG-backed | `/api/addresses` CRUD uses the address-store facade. |
| Marketplace categories | PG-backed | `CategoryService` and the default `General` category path use the category-store facade. `categories_active_name_norm_uq` enforced. |
| Level configuration | PG-backed | `PointsService` level reads use the level-config facade. `LEVEL_STORE=postgres` accepted as compatibility alias. |
| Shop settings | PG-backed | Shop settings controller, seller payment rail resolution, and review enable/disable checks use the shop-settings facade. Seller shop lookup handles both uuid and legacy id formats. |
| Marketplace reviews | PG-backed | Review list/summary/create routes use the review-store facade. |
| Notifications | PG-backed | `NotificationService` uses `getNotificationRepo()` for create/list/read/delete/count paths. |
| Oracle quote persistence | PG write when `ORACLE_QUOTING_ENABLED=true` | `/api/payment/request-network/intents` writes `payment_quotes` to PG. Mongo mirror path removed. |
| Funds ledger | PG-backed | `appendFundsLedgerEntry` and `getFundsBalanceBy*` call `getPaymentRepo()` which resolves to `DrizzlePaymentRepo`. |
| Payments and escrow state | PG-backed | All payment services use Drizzle repos; Mongoose `Payment` model removed. |
| Derived destinations and sweeps | PG-backed | `getDerivedDestinationRepo()` resolves to `DrizzleDerivedDestinationRepo`. |
| Points/referrals/transactions | PG-backed | `getPointsRepo()` resolves to `DrizzlePointsRepo`. |
| Chat/messages | PG-backed (JSONB shim) | `getChatRepo()` resolves to `DrizzleChatRepo`. Participants/messages are stored as JSONB blobs; normalization into relational child tables is an optional future improvement. |
| Disputes/blog | PG-backed | Both resolve to Drizzle repos. |
| ReleaseHold | PG-backed | `getReleaseHoldRepo()` resolves to `DrizzleReleaseHoldRepo`. |
| Backfill/verify scripts | Available as operator tooling | `MIGRATION_PG_URL` drives all backfill scripts. Not run automatically at startup. |
| Guard user role | PG schema-ready | Migration 0017 adds `guard` to the `user_role` enum. |
| Seeds | Postgres-capable | Seeds in `src/seeds/*` are store-aware and idempotent under `MONGO_CONNECT_MODE=never`. |
## What Was Mongo-Backed (Historical)
All domains are now PostgreSQL-only as of v2.9.12. The following were the remaining Mongo-backed areas prior to the final cutover:
- **User reads** — Auth-owned users were PG-backed for writes but reads remained Mongo-authoritative until the full auth cutover.
- **Marketplace requests/offers/templates reads** — `REPO_MARKETPLACE` defaulted to Mongo; read cutover required smoke coverage.
- **Payments and escrow state reads** — Payment services called Mongoose documents directly for reads until the final payment-domain wiring was completed.
- **Derived destinations and sweeps** — `REPO_DERIVED_DESTINATION` defaulted to Mongo.
- **Points/referrals/transactions** — `REPO_POINTS` defaulted to Mongo.
- **Chat/messages** — `getChatRepo()` defaulted to Mongo; JSONB shim was the Drizzle path. No dual-write wrapper existed.
- **Disputes/blog** — Defaulted to Mongo until `REPO_DISPUTE`/`BLOG_STORE` were flipped.
- **ReleaseHold** — No dual-write wrapper; required explicit flip.
All of the above are now fully PostgreSQL-backed. MongoDB and Mongoose have been removed from the runtime.
## Env Flag Reality
All `*_STORE=mongo` and `REPO_*=mongo` env flags are obsolete — the repository factory only supports `postgres`/`pg` mode. `MONGO_URI` and `MONGO_CONNECT_MODE` have been removed from the runtime.
| Flag | Current meaning |
|---|---|
| `MONGO_URI` | REMOVED — MongoDB has been removed from the runtime. |
| `MONGO_CONNECT_MODE` | REMOVED — MongoDB has been removed from the runtime. |
| `AUTH_STORE` | OBSOLETE — only `postgres` is valid. Setting to `mongo` has no effect. |
| `CONFIG_STORE` | OBSOLETE — only `postgres` is valid. |
| `ADDRESS_STORE` | OBSOLETE — only `postgres` is valid. |
| `CATEGORY_STORE` | OBSOLETE — only `postgres` is valid. |
| `LEVEL_CONFIG_STORE` | OBSOLETE — only `postgres` is valid. `LEVEL_STORE=postgres` accepted as alias. |
| `SHOP_SETTINGS_STORE` | OBSOLETE — only `postgres` is valid. |
| `REVIEW_STORE` | OBSOLETE — only `postgres` is valid. |
| `NOTIFICATION_STORE` / `REPO_NOTIFICATION` | OBSOLETE — only `postgres`/`pg` is valid. |
| `PG_URL` | REQUIRED — PostgreSQL is the sole database. All store facades and repos require this. |
| `MIGRATION_PG_URL` | Used by backfill scripts and migration runbooks; not part of normal request handling. |
| `REPO_PAYMENT` | OBSOLETE — only `postgres` is valid. All payment services use `DrizzlePaymentRepo`. |
| `REPO_MARKETPLACE` | OBSOLETE — only `postgres` is valid. All marketplace writes and reads route through `DrizzleMarketplaceRepo`. |
| `REPO_USER`, `REPO_POINTS`, `REPO_DERIVED_DESTINATION`, `REPO_TREZOR` | OBSOLETE — only `postgres` is valid. All resolve to their respective Drizzle repos. |
| `REPO_DISPUTE` / `DISPUTE_STORE`, `REPO_BLOG` / `BLOG_STORE` | OBSOLETE — only `postgres` is valid. |
| `REPO_CHAT` / `CHAT_STORE` | OBSOLETE — only `postgres` is valid. `DrizzleChatRepo` is the sole chat repo. |
| `REPO_RELEASE_HOLD` / `RELEASE_HOLD_STORE` | OBSOLETE — only `postgres` is valid. |
| `ORACLE_QUOTING_ENABLED` | Enables server-side quote computation and the `payment_quotes` PG write in checkout. |
## What's Next (Post-Migration)
1. **Prod backfill** — If the production instance was running Mongo-backed data before the cutover, a one-time backfill from Mongo to Postgres under a maintenance window is required. Use `MIGRATION_PG_URL` + `MIGRATION_MONGO_URL` with the existing backfill scripts. Validate row counts before switching prod traffic.
2. **Chat normalization** — The `DrizzleChatRepo` currently stores participants and messages as JSONB blobs rather than normalized relational child tables. This is an optional future improvement; it does not block current operation but would enable richer querying and FK integrity on chat data.
3. **`payment_provider` enum `escrow` value** — Confirm migration 0019 has been applied on all target databases (adds `escrow` to the `payment_provider` enum). If not already run, apply it before using escrow-provider payment records.
## Recent Progress Since Last Update (2.8.37 → 2.9.12)
- **2.8.382.8.46:** Complete dual-write repos for all remaining domains; Drizzle migrations pipeline finalized; TTL scheduler added; shop lookup bug-fixed.
- **2.8.47:** Seeds made Postgres-capable and idempotent for PG-only boot (`MONGO_CONNECT_MODE=never`).
- **2.8.482.8.49:** Fresh-DB PG migrate + seed path corrected; 0013/0014 migrations made valid for a fresh `drizzle-kit migrate` run.
- **2.8.50:** Admin user counts routed through postgres-capable stores; admin user management works end-to-end under PG.
- **2.8.512.8.53:** PG response serialization and id resolution in marketplace purchase-request paths corrected; user creation and purchase request unblocked.
- **2.8.54:** Guard user role added across auth and user management; migration 0017 adds guard to user_role enum.
- **2.8.55:** Chat routes fixed and notifications deliver in real time.
- **2.8.56:** Seller shop lookup made tolerant of uuid/legacy id formats.
- **2.8.572.8.60:** Telegram Mini App in-shell shop, account tab parity, shopping cart.
- **2.8.61:** Direct-transfer checkout option for non-web3 users.
- **2.8.622.8.64:** Points level boundary fixes, legacy 24-hex user id support, seller ratings from real published reviews.
- **2.8.65:** Chat participant names populated on Postgres path, participant canonicalization.
- **2.8.662.8.69:** Telegram: product-style template cards, in-shell template detail, web-app-parity templates, settings/addresses in-shell, theme/dark mode, solar-style icons, avatar upload, achievements.
- **2.8.70:** Telegram in-shell settings and addresses, theme from central config.
- **2.8.712.8.73:** Telegram: solar-style icons, avatar URL fixes, inline email verify, web links keep app alive, remove escrow-states.
- **2.8.74:** Telegram chat own-message detection, read-only email field.
- **2.8.75:** Self-contained email-change flow with visible code entry.
- **2.8.76:** Telegram send-code always reveals verify panel.
- **2.8.77:** Telegram keep email code panel mounted after sending.
- **2.8.78:** Telegram system messages neutral + post-delivery seller review.
- **2.8.79:** Request template maxUsage made truly optional; template creation 500 fix.
- **2.9.x:** Full MongoDB/Mongoose removal — all Mongoose models replaced by Drizzle repos, dual-write decommissioned, TypeScript compiles with 0 errors (2026-06-06).
## Related Docs
- [[Database Strategy - Mongo vs Postgres Assessment]]
- [[MongoDB to PostgreSQL Migration Plan (Drizzle)]]
- [[Payment]]
- [[Payment API]]
- [[Environment Variables]]
- [[Database Operations]]