nick/nick-doc
3
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

docs: sync from backend 6df113d — marketplace pg backfill

Browse Source
This commit is contained in:
Siavash Sameni
2026-06-01 14:53:35 +04:00
parent 5352a78e96
commit 78707c11a7
4 changed files with 29 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
02 - Data Models/MongoDB to PostgreSQL Migration Plan (Drizzle).md
View File

@@ -4,7 +4,7 @@ tags: [data-model, migration, postgres, drizzle, plan, runbook]
aliases: [Drizzle Migration Plan, PG Migration Plan]
created: 2026-05-31
companion: "[[MongoDB to PostgreSQL Migration Guide]]"
updated: 2026-06-01 for backend integrate-main-into-development@1757f1e backend 2.8.9
updated: 2026-06-01 for backend integrate-main-into-development@6df113d backend 2.8.13
---
# MongoDB → PostgreSQL Migration Plan (Drizzle)
@@ -17,7 +17,7 @@ updated: 2026-06-01 for backend integrate-main-into-development@1757f1e backend
> **Scope reminder:** partial migration (Phases 0–5) is the recommended stopping point — ≈16–28 engineer-weeks. Full migration of Chat/Notification/sessions is explicitly deferred.
> [!warning] Current implementation status
> Backend `2.8.9` has started the runtime cutover with store-specific raw Postgres facades: auth-owned users/Telegram auth records behind `AUTH_STORE=postgres`, confirmation-threshold config/history behind `CONFIG_STORE=postgres`, user address CRUD behind `ADDRESS_STORE=postgres`, and the first marketplace/reference domains behind `CATEGORY_STORE=postgres`, `LEVEL_CONFIG_STORE=postgres`, `SHOP_SETTINGS_STORE=postgres`, and `REVIEW_STORE=postgres`. This checkout does **not** currently contain the old `src/db/` Drizzle repository layer described below, so treat the Drizzle sections as the target architecture/runbook, not a statement of current code. Mongo remains authoritative unless a per-store flag is explicitly flipped. See [[Postgres Runtime Cutover Status]].
> Backend `2.8.13` has started the runtime cutover with store-specific raw Postgres facades: auth-owned users/Telegram auth records behind `AUTH_STORE=postgres`, confirmation-threshold config/history behind `CONFIG_STORE=postgres`, user address CRUD behind `ADDRESS_STORE=postgres`, and the first marketplace/reference domains behind `CATEGORY_STORE=postgres`, `LEVEL_CONFIG_STORE=postgres`, `SHOP_SETTINGS_STORE=postgres`, and `REVIEW_STORE=postgres`. It also contains the broader `src/db/` Drizzle schemas, repository implementations/factory, id-map bridge, and backfill runner described below, but the broad marketplace/payment/points services are still mostly not wired through that factory. Mongo remains authoritative unless a per-store flag is explicitly flipped. See [[Postgres Runtime Cutover Status]].
---
@@ -361,6 +361,7 @@ Same phases as the guide §2, here with Drizzle-concrete entry/exit gates. Each
### Phase 4 — Money core (6–10 wk) — *the point of the project*
- `PurchaseRequest`, `SellerOffer`, `Payment`, `FundsLedgerEntry`, `DerivedDestination`, `TrezorAccount`, `PointTransaction`.
- **Status 2026-06-01:** Drizzle schemas/repositories and backfill scripts exist for PurchaseRequest/SellerOffer. Backend `2.8.13` hardens the marketplace-core backfill path with `npm run backfill:marketplace-core:postgres`, fixed PurchaseRequest timestamp/preferred-seller writes, and a post-SellerOffer selected-offer remap step. Runtime marketplace services still call Mongoose directly and must not be flipped with `REPO_MARKETPLACE` until service wiring plus shadow-read checks land.
- Apply §4.1 (Mixed→discriminator+FK), §4.2 (offers/preferredSellers junctions, deliveryInfo/serviceInfo child tables), §4.5 (derivation counter).
- **Wrap in real PG transactions the multi-doc writes that today have none:** `raiseDispute` (PurchaseRequest + Payment), payment confirm + `FundsLedgerEntry` AML-fee insert, referral reward (points + referralStats), PointsService flows (migrate its 2 `withTransaction` sites to PG `BEGIN/COMMIT`).
- Preserve the `Payment` partial-unique idempotency index and `FundsLedgerEntry.idempotencyKey` uniqueness.
@@ -402,6 +403,7 @@ async function shadowRead(key, mongoFn, pgFn) {
## 8. Cutover & rollback runbook (per collection)
1. **Backfill** in batches with checkpointing; allocate uuids → `id_map`; remap FKs from already-migrated parents. Re-runnable (idempotent upserts).
- Marketplace-core operator path: `MIGRATION_MONGO_URL=... MIGRATION_PG_URL=... npm run backfill:marketplace-core:postgres:dry-run`, then `npm run backfill:marketplace-core:postgres` in non-prod. Run `scripts/smoke/marketplace-core-postgres-backfill.sh` with the same DSNs to exercise the static backfill invariants and dry-run.
2. **Enable `dual`** (flag) — writes go to both; shadow-read diffing on. Backfill the delta accumulated during step 1.
3. **Soak** until row-count + checksum + shadow-read are clean for the agreed window.
4. **Flip reads to `pg`** (flag). Keep dual-write on.