Files
nick-doc/09 - Audits/DB Migration Audit Report (2026-06-02).md
Siavash Sameni c98c31dc24 docs: sync documentation with latest codebase state (merged)
- Update Activity Log with 108 missing commits (48 backend + 60 frontend)
- Update version references: backend v2.8.79, frontend v2.8.94
- Update migration count: 18 migrations (0000-0017)
- Update Telegram Mini App Flow to v2.8.94
- Update Payment Flow - Scanner to 2026-06-05
- Update all architectural and database references
- Add MongoDB removal handoff document with updated versions

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
2026-06-05 07:51:00 +04:00

273 lines
24 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# DB Migration Audit Report — Amanat Escrow (Mongo→PG)
**Date:** 2026-06-02 | **Scope:** Full Mongo→PG migration audit — schemas, indexes, constraints, dual-write coverage, backfill, verify harness, and service-layer Mongo-idiomatic patterns
---
## Executive Summary
The migration is **~50% complete** and **NOT ready for PG-primary cutover**. Schema and backfill scaffolding are mature (all 13 in-scope Mongo collections have Drizzle tables and backfill scripts), but three categories block cutover:
1. **Migration correctness**`0004_funds_ledger_entries.sql` is unjournaled (silently skips `funds_ledger_entries` on fresh DBs); `shadowRead()` exists but is never called from any read path so the soak window is completely blind.
2. **Financial integrity gaps** — missing `CHECK (amount > 0)` / `fx_rate > 0` constraints, ~20 FK columns declared in `relations()` only (never as physical FKs), backfill that silently writes `amount = '0'` for NULL Mongo amounts.
3. **Service-layer rework is far bigger than the schema work** — the factory (`createRepositories`) has **zero callers**; 30+ services still import Mongoose directly and contain ~50 Mongo-idiomatic patterns (N+1 loops, full-fetch+JS-filter, read-modify-write without locking, multi-table writes with no transaction) that will cause real money errors and lost updates under concurrent load.
---
## Critical Issues (must fix before cutover)
| # | Dimension | Table/File → Issue | Fix |
|---|---|---|---|
| CR-1 | Migration | `0004_funds_ledger_entries.sql` has NO `_journal.json` entry; `funds_ledger_entries` DDL conflicts between 0003 and 0004 — silently skipped on fresh DB, `ALTER` fails with "constraint already exists" if run | Reduce 0004 to trigger-only DDL; register in journal |
| CR-2 | Arch | `shadowRead.ts` exists and is complete, but no `DualWrite*Repo` read method ever calls it — soak window measures zero signal | Wire `shadowRead()` + `ShadowReadMetrics` into all 4 DualWrite read paths |
| CR-3 | Arch | Factory `createRepositories` has **zero callers** outside `src/db/repositories/` — every `REPO_*=dual` env flag changes env but routes zero traffic | Inject factory into 30+ service files (`paymentController`, `paymentCoordinator`, marketplace/user/points services) |
| CR-4 | Backfill | `backfill-payments.ts`: `extractDecimalString(null)` returns `'0'` — NULL Mongo payment amount silently inserted as `amount = 0` (money integrity violation) | Make `extractDecimalString(null)` throw, or skip + warn |
| CR-5 | Backfill | `backfill-derivedDestinations.ts`: `require()` failure falls back to `strict:false` model — all fields become `undefined`, all rows skipped silently, exits 0 | Make import failure `throw` and exit 1; never fall back to schema-less model |
| CR-6 | Backfill | `backfill-fundsLedger.ts` + `backfill-pointTransactions.ts`: `upsertIdMap` + `INSERT` not in one transaction — interruption leaves orphan id_map rows; re-run `DO NOTHING` never inserts data row → unrecoverable | Wrap `upsertIdMap` + data INSERT in one PG transaction |
| CR-7 | Schema | `payment_quotes`: no `CHECK (offer_amount > 0 AND fx_rate > 0 AND token_price_usd > 0)` — zero/negative FX rate → divide-by-zero in settlement | Add three CHECK constraints |
| CR-8 | Schema | `payments`: no `CHECK (amount > 0)` | `ALTER TABLE payments ADD CONSTRAINT ck_payments_amount_pos CHECK (amount > 0)` |
| CR-9 | Verify | `checksums.ts`: `.catch(() => [])` silently returns `[]` on DB connection failure → `hasMismatch=false`, gate passes green | Propagate errors; never swallow in comparison path |
| CR-10 | Verify | `shadowRead.ts`: Decimal128 detection (`constructor.name === 'Decimal128'`) breaks on `.lean()` POJO results (`{$numberDecimal:...}`) — every numeric field appears equal, silent false-negative | Normalize amounts to strings before compare; detect `$numberDecimal` key |
| CR-11 | Verify | `migration-fk-idmap.test.ts`: `skipIfUnreachable` returns early without `test.skip()` — money-safety tests PASS when DB is unreachable; CI exit 0 | Call `test.skip()` when `!isReachable`; CI must assert `MONEY_SAFETY_TESTS_SKIPPED` absence |
| CR-12 | Verify | `rowCounts.ts`: id_map coverage check exists for `payments` only — dropped id_map entry for other collections → dangling FK silently | Add id_map coverage checks for users/purchaseRequests/sellerOffers/fundsLedger/pointTransactions |
| CR-13 | Code | `paymentCoordinator.ts` `executePaymentUpdate`: read `status` → JS guard → write — two concurrent webhooks both read `pending`, both write (lost update) | `UPDATE payments SET status=... WHERE id=... AND status NOT IN ('completed','cancelled','refunded') RETURNING *`; 0 rows → abort |
| CR-14 | Code | `paymentCoordinator.ts` dispute gate: `isReleaseBlockedById(prId)` read, then payment update — dispute raised in the gap bypasses gate | `SELECT ... FOR UPDATE` on PR row + payment update in one transaction |
| CR-15 | Code | `paymentCoordinator.ts` `executePaymentUpdate`: payment update + ledger append + PR backfill + `acceptOffer` + duplicate cancel + template delete run with **NO transaction** — step 3/4 failure leaves payment completed but offer not accepted | Wrap all side effects in one DB transaction |
| CR-16 | Code | `DisputeService.ts` `createDispute`: dispute create → chat create → save → `setChatId` with no transaction — partial failure → orphaned dispute/chat, UI crashes | Wrap all four ops in one PG transaction |
| CR-17 | Code | `SellerOfferService.ts` `withdrawOffer` + `marketplaceController.ts` `validateStatusTransition`: read-validate-write status machine with no atomic guard | `UPDATE ... WHERE id=... AND status=... RETURNING *`; 0 rows → 409 Conflict |
| CR-18 | Code | `PointsService.ts` `getReferrals`/`collectDeliveredReferralOrders`: per-referred-user `while(true)` skip/limit loop + per-row offer lookup → 510+ queries/user; 14+ s per leaderboard on WAN | Replace with single CTE: `LEFT JOIN purchase_requests/seller_offers/point_transactions ... GROUP BY u.id` |
| CR-19 | Code | `PurchaseRequestService.ts` `searchPurchaseRequests`: `findPurchaseRequests({limit:100})` then JS `.filter().slice(0,20)` — catastrophic at 10k rows | `WHERE title ILIKE $s OR description ILIKE $s LIMIT 20`, or `tsvector` generated column + GIN |
| CR-20 | Code | `Chat` model: `messages[]`/`participants[]`/`unreadCounts[]` as JSONB — no FK integrity, unbounded row bloat, non-indexable | Child tables `chat_messages`, `chat_participants`, `chat_message_reactions`; rewrite `ChatService` as SQL |
---
## High-Priority Issues
### Schema — Missing Physical FKs
All declared via Drizzle `relations()` only, never as `foreignKey()`/`.references()`. Zero referential integrity enforcement in DB.
- `users.referred_by_id` → add FK `ON DELETE SET NULL`
- `purchase_requests.buyer_id`, `category_id`, `selected_offer_id`
- All PR child tables (`purchase_request_delivery_info`, `_delivery_address`, `_seller_delivery_info`, `_service_info`, `_specifications`, `_preferred_sellers`) — `purchase_request_id`/`delivery_info_id`
- `delivery_attempts.delivery_info_id`, `seller_id`
- `derived_destinations.buyer_id`, `seller_id`, `seller_offer_id`
- `derived_destination_sweeps.destination_id`
- `trezor_accounts.user_id`; `trezor_derived_addresses.trezor_account_id`
- `funds_ledger_entries.purchase_request_id`, `payment_id` (deferred since 0003, never added)
- `point_transactions.user_id`, `referred_user_id`
### Schema — Other HIGH
- `users`: no `CHECK (points_available >= 0 AND points_available <= points_total)`
- `point_transactions`: no `CHECK (balance >= 0)`
- `payment_quotes`: no `CHECK (settle_amount >= raw_settle_amount)` (snap-up invariant unenforced)
- `purchase_request_preferred_sellers`: no composite PK, only uniqueIndex
- `seller_offers.price_amount numeric(18,8)` vs project-wide `numeric(38,18)` (precision gap in settlement)
### Migration — HIGH
- All 70+ `CREATE INDEX` are non-`CONCURRENTLY` (blocking SHARE lock on live data for all)
- All FK `ADD CONSTRAINT` run validating (no `NOT VALID` + later `VALIDATE`) — prolonged ACCESS EXCLUSIVE lock
- `blog_posts` and `notifications` exported from schema barrel but **no migration creates them**
- `disputes`/`chats` use `text` (not `uuid`) for FK columns — zero referential integrity
- Migration 0009: three sequential `UPDATE` DML steps not in `BEGIN/COMMIT` — partial failure leaves inconsistent category re-parenting
### Backfill — HIGH
- `String(number)` for `numeric` columns risks scientific notation in `backfill-purchaseRequests.ts` (budget), `backfill-sellerOffers.ts` (price.amount), `backfill-requestTemplates.ts` (budget, proposal.price)
- `backfill-users.ts`: `email ?? null` fails if `users.email` is NOT NULL for OAuth-only users
- `backfill-fundsLedger.ts`: missing `d.entryType` (no default) → NOT NULL violation
- `run-backfill.ts`: `requestTemplates` runs in Tier B but runbook documents it last (inconsistency)
### Verify — HIGH
- `reconcile.ts`: no double-refund detection; no `escrow_state`↔last-ledger-entry check; `LIMIT 1000` silently truncates
- `rowCounts.ts`: `estimatedDocumentCount()` is approximate — use `countDocuments({})`
- `checksums.ts`: no Mongo-side per-user points balance comparison during dual-write window
- `ledgerImmutability.ts`: `TRUNCATE` bypasses row-level trigger — add `BEFORE TRUNCATE` statement-level trigger
- Enum-value completeness verified nowhere
### Code — HIGH
- `SellerOfferService.ts` `acceptOffer`: per-rejected-seller `createNotification` loop (use `createNotificationsBulk`); multi-UPDATE repo needs transaction
- `RequestTemplateService.ts` `batchConvertTemplates`: ~50 sequential queries per 10-item cart; no transaction per item → orphan PRs with no offer, oversold templates
- `paymentService.ts` `createPaymentRecord`: `String(metadata?.sellerId || createLegacyObjectIdString())` injects random fake ObjectIds as FKs → PG FK violation
- `userController.ts` `getUsersList`: `$regex` on name/email → PG seqscan; needs `pg_trgm` GIN index + `ILIKE`
- `PurchaseRequestService.ts` `updatePurchaseRequestStatus` (completed): non-idempotent double-points risk; no transaction
---
## Medium Issues
**Schema:** dual unique indexes on `categories.name` (drop raw, keep partial `WHERE is_active`); missing `payments(purchase_request_id, status)` composite index; missing `seller_offers(seller_id,status)`, `derived_destinations(address, chain_id)`, `trezor_derived_addresses.address` indexes; `id_map` no PK and `new_id` no unique constraint; `request_templates` no `CHECK (usage_count <= max_usage)`.
**Migration:** `wallet_type` enum created but used in no column (dead DDL); `ALTER TYPE offer_currency ADD VALUE 'TRY'` requires PG 12+ in-transaction; `ck_pr_budget_currency_crypto` add(0006)/drop(0007) round-trip fails on rows with non-crypto values; `chats.participants` JSONB has no GIN index.
**Backfill:** enum default mismatches (`provider:'request.network'` vs `request_network`); `escrow_state ?? null` may hit NOT NULL; `derivedDestinations.lastKnownBalance` via JS Number loses precision above 2^53 for wei.
**Code:** `dataCleanupService.getCollectionStats` — 13 sequential `countDocuments()` (should be single subselect); `userController.updateUserProfile` writes arbitrary `profile.${key}` (whitelist needed); `paymentCoordinator` metadata read-spread-write overwrites concurrent keys (use `metadata || jsonb_build_object(...)`); skip/limit pagination in `getOffersBySeller`/`getUsersList`.
---
## Index & Constraint Punch List
| Table | Missing | Recommended DDL |
|---|---|---|
| payments | CHECK amount > 0 | `ALTER TABLE payments ADD CONSTRAINT ck_payments_amount_pos CHECK (amount > 0);` |
| payments | (purchase_request_id, status) | `CREATE INDEX CONCURRENTLY idx_payments_pr_status ON payments (purchase_request_id, status);` |
| payments | disputed partial | `CREATE INDEX CONCURRENTLY idx_payments_disputed ON payments (id) WHERE disputed = true;` |
| payment_quotes | CHECK money fields | `ALTER TABLE payment_quotes ADD CONSTRAINT ck_pq_pos CHECK (offer_amount > 0 AND fx_rate > 0 AND token_price_usd > 0);` |
| payment_quotes | CHECK snap-up | `ALTER TABLE payment_quotes ADD CONSTRAINT ck_pq_settle CHECK (settle_amount >= raw_settle_amount);` |
| users | referred_by_id FK | `ALTER TABLE users ADD CONSTRAINT users_referred_by_fk FOREIGN KEY (referred_by_id) REFERENCES users(id) ON DELETE SET NULL NOT VALID;` then `VALIDATE` |
| users | CHECK points | `ALTER TABLE users ADD CONSTRAINT ck_users_points CHECK (points_available >= 0 AND points_used >= 0 AND points_total >= 0 AND points_available <= points_total);` |
| point_transactions | CHECK balance | `ALTER TABLE point_transactions ADD CONSTRAINT ck_pt_balance CHECK (balance >= 0);` |
| funds_ledger_entries | FK pr + payment | `ALTER TABLE funds_ledger_entries ADD CONSTRAINT fle_pr_fk FOREIGN KEY (purchase_request_id) REFERENCES purchase_requests(id) NOT VALID;` then `VALIDATE` |
| funds_ledger_entries | TRUNCATE trigger | `CREATE TRIGGER funds_ledger_no_truncate BEFORE TRUNCATE ON funds_ledger_entries FOR EACH STATEMENT EXECUTE FUNCTION funds_ledger_immutable_fn();` |
| trezor_accounts | user_id FK | `ALTER TABLE trezor_accounts ADD CONSTRAINT ta_user_fk FOREIGN KEY (user_id) REFERENCES users(id) NOT VALID;` then `VALIDATE` |
| derived_destinations | buyer/seller/offer FK + (address,chain_id) | add 3 FKs `NOT VALID`; `CREATE INDEX CONCURRENTLY idx_dd_addr_chain ON derived_destinations (address, chain_id);` |
| purchase_requests | buyer/category/offer FK + (status,created_at) | add 3 FKs `NOT VALID`; `CREATE INDEX CONCURRENTLY idx_pr_status_created ON purchase_requests (status, created_at DESC);` |
| seller_offers | (seller_id,status) + (purchase_request_id,status) | `CREATE INDEX CONCURRENTLY idx_so_seller_status ON seller_offers (seller_id, status);` |
| id_map | PK + new_id unique | `ALTER TABLE id_map ADD PRIMARY KEY (collection, legacy_object_id); CREATE UNIQUE INDEX id_map_new_id_uq ON id_map (new_id);` |
| users (search) | trigram | `CREATE INDEX CONCURRENTLY idx_users_name_trgm ON users USING GIN (lower(first_name\|\|' '\|\|last_name\|\|' '\|\|coalesce(email,'')) gin_trgm_ops);` |
| purchase_requests | tags GIN | `CREATE INDEX CONCURRENTLY idx_pr_tags ON purchase_requests USING GIN (tags);` |
---
## Repository Coverage Matrix
| Interface | Drizzle Impl | Dual-Write | Status |
|---|---|---|---|
| PaymentRepo | Yes | Yes (shadow read NOT wired) | PARTIAL |
| UserRepo | Yes | Yes (shadow read NOT wired) | PARTIAL |
| MarketplaceRepo | Yes | Yes (shadow read NOT wired) | PARTIAL |
| PointsRepo | Yes | Yes (shadow read NOT wired) | PARTIAL |
| ReleaseHoldRepo | Yes | — | No dual-write |
| TrezorAccountRepo | Yes | — | No dual-write |
| DerivedDestinationRepo | Yes | — | No dual-write |
**Factory has zero application callers (CR-3) — most critical architecture gap.**
---
## Backfill Coverage Matrix
| Mongo Collection | Backfill Script | Ordering | Status |
|---|---|---|---|
| users | backfill-users.ts | Tier A | OK (email NOT NULL risk) |
| categories | backfill-categories.ts | Tier A | OK |
| requestTemplates | backfill-requestTemplates.ts | Tier B | OK (String() decimals; runbook order mismatch) |
| purchaseRequests | backfill-purchaseRequests.ts (2-pass) | Tier B | OK (String() decimals; silent preferred-seller skips) |
| sellerOffers | backfill-sellerOffers.ts | Tier B | OK (String() price.amount) |
| payments | backfill-payments.ts | Tier C | **RISK** — NULL amount → '0' (CR-4) |
| fundsLedger | backfill-fundsLedger.ts | Tier C | **RISK** — non-txn idMap (CR-6); entryType NOT NULL |
| derivedDestinations | backfill-derivedDestinations.ts | Tier C | **RISK** — schema-less fallback (CR-5); wei precision |
| trezorAccounts | backfill-trezorAccounts.ts | Tier C | OK |
| pointTransactions | backfill-pointTransactions.ts | Tier C | **RISK** — non-txn idMap (CR-6); String() decimals |
| id_map | (infra — `_idMap.ts`) | n/a | CORRECT |
| payment_quotes | (none — runtime-generated) | n/a | EXPECTED |
| pg_dualwrite_gaps | (none — operational log) | n/a | EXPECTED |
---
## Verification Coverage Matrix
| Concern | Covered By | Gap |
|---|---|---|
| Row-count parity | rowCounts.ts (9/~23 collections) | `estimatedDocumentCount()` approximate; id_map not counted |
| ID-mapping completeness | rowCounts.ts (payments only) | **CRITICAL** — no check for users/PR/sellerOffers/FLE/pointTransactions |
| FK integrity | rowCounts.ts (7 pairs) | `seller_offers.seller_id`, `trezor_accounts→users` missing |
| Money sum accuracy | checksums.ts | `.catch(()=>[])` silent pass on conn failure (CR-9) |
| Ledger reconciliation | reconcile.ts | No double-refund; no `escrow_state`↔last-entry; LIMIT 1000 truncation |
| Ledger immutability | ledgerImmutability.ts | TRUNCATE bypass; no schema filter on `pg_proc` |
| Shadow read fidelity | shadowRead.ts | Decimal128 lean false-negative (CR-10); not wired (CR-2) |
| Enum completeness | — | **Not covered anywhere** |
| Timestamp precision/TZ | — | Not covered |
| CI gate output | boolean only | No JSON stdout; tests pass-not-skip on unreachable DB (CR-11) |
---
## Models Not Yet in PG Schema
| Mongo Model | Fields | Actively Used | Effort | Notes |
|---|---|---|---|---|
| Dispute | ~15 + 3 embedded arrays | Yes (DisputeService, releaseHoldService) | **L** | `evidence[]`/`timeline[]`/`messages[]` → child tables; pre-save timeline hook → service |
| Notification | 11 | Yes (all services, high frequency) | S schema / M migration | `userId` as text → uuid FK backfill; TTL index → pg_cron |
| ShopSettings | ~12 | Yes (marketplace template pages) | S | `paymentConfig.allowedChains int[]`, `socialLinks` → 4 columns |
| ConfigSetting (+History) | 4 (+audit) | Yes (walletMonitor, scanner threshold) | S | key-value; history child table |
| LevelConfig | ~10 | Yes (PointsService) | S | flatten `benefits{}` to 4 columns |
| Address | 10 | Yes (dataCleanup, delivery flows) | S | `addressType` pgEnum; one-primary partial unique |
| Review | 9 | Admin CMS only | S | polymorphic `subjectId` → ref_kind discriminator |
| TelegramLink | ~12 | Yes (auth) | S | two unique constraints; (userId, status) idx |
| TelegramSession | ~10 | Yes (auth middleware) | S | TTL `expiresAt` → pg_cron |
| BlogPost | ~20 | Admin CMS only | S | `videos[]` child table; slug/publishedAt pre-save → service |
| TempVerification | 8 | Registration only | S | TTL cleanup |
---
## Mongo-Idiomatic Code Refactoring Tracker
| Pattern | File | Function | Severity | Fix |
|---|---|---|---|---|
| N+1 | PointsService.ts | getReferrals / collectDeliveredReferralOrders | CRITICAL | Single CTE with LEFT JOINs + GROUP BY |
| N+1 | SellerOfferService.ts | acceptOffer | HIGH | `createNotificationsBulk` + single seller-id query |
| N+1 | RequestTemplateService.ts | batchConvertTemplates | HIGH | Batch SELECT ANY($links), batch INSERT…VALUES, single usage UPDATE |
| N+1 | dataCleanupService.ts | getCollectionStats | MEDIUM | Single subselect count query |
| Full-fetch+filter | PurchaseRequestService.ts | searchPurchaseRequests | CRITICAL | ILIKE/tsvector WHERE + LIMIT 20 |
| Full-fetch+filter | PurchaseRequestService.ts | createPurchaseRequest (dup detect) | HIGH | WHERE buyer/title/description/created_at LIMIT 1 |
| Full-fetch+filter | paymentCoordinator.ts | executePaymentUpdate (template cleanup) | HIGH | Push JSONB conditions into WHERE/DELETE |
| Full-fetch+filter | userController.ts | getUsersList | HIGH | pg_trgm GIN + ILIKE |
| JSONB no join table | Chat | messages/participants/unreadCounts | CRITICAL | 3 child tables (CR-20) |
| JSONB no join table | Dispute | evidence/timeline/messages | HIGH | 3 child tables on migration |
| JSONB schemaless | Payment | metadata | HIGH | Promote `is_template_checkout`, `rn_request_id` to typed columns |
| In-memory agg | PointsService.ts | sumDeliveredReferralSpend | CRITICAL | SUM in CTE |
| In-memory agg | SellerOfferService.ts | getOfferStatistics | MEDIUM | COUNT(*) OVER() / ROLLUP |
| Lost update | paymentCoordinator.ts | executePaymentUpdate | CRITICAL | UPDATE…WHERE status NOT IN (terminal) RETURNING |
| Lost update | SellerOfferService.ts | updateOffer | HIGH | UPDATE…WHERE status='pending' RETURNING |
| TOCTOU | SellerOfferService.ts | withdrawOffer | CRITICAL | UPDATE…WHERE id AND seller AND status='pending' |
| TOCTOU | marketplaceController.ts | validateStatusTransition | CRITICAL | UPDATE…WHERE status=$expected; 0 rows → 409 |
| TOCTOU | paymentCoordinator.ts | dispute gate | CRITICAL | FOR UPDATE on PR + same txn |
| TOCTOU | PurchaseRequestService.ts | updatePurchaseRequestStatus | HIGH | UPDATE…WHERE status=$old RETURNING |
| Missing txn | paymentCoordinator.ts | executePaymentUpdate | CRITICAL | One txn for all side effects |
| Missing txn | DisputeService.ts | createDispute | CRITICAL | One txn for dispute+chat+link |
| Missing txn | SellerOfferService.ts | acceptOffer (repo) | HIGH | Txn for accept/reject/PR update |
| Missing txn | RequestTemplateService.ts | batchConvertTemplates | HIGH | Txn (or savepoint) per cart item |
| Missing txn | PurchaseRequestService.ts | updatePurchaseRequestStatus (completed) | HIGH | Txn or outbox for referral reward |
| Schemaless write | paymentService.ts | createPaymentRecord | HIGH | Remove fake-ObjectId FK fallback |
| Schemaless write | userController.ts | updateUserProfile | MEDIUM | Whitelist + jsonb \|\| merge |
| Skip/limit pagination | PointsService.ts | collectDeliveredReferralOrders | CRITICAL | Replace loop with aggregate |
| Skip/limit pagination | PurchaseRequestService.ts | searchPurchaseRequests | HIGH | Keyset on (created_at, id) |
| Skip/limit pagination | SellerOfferService.ts / userController.ts | getOffersBySeller / getUsersList | MEDIUM | Keyset + cap limit 100 |
| Virtual/hook | Chat | addMessage/markAsRead/getUnreadCount | CRITICAL | SQL ops in ChatRepository |
| Pre-save hook | FundsLedgerEntry | immutability | HIGH | Apply trigger DDL now |
---
## Migration Completion Assessment
| Layer | % |
|---|---|
| Schema (Drizzle tables vs Mongo collections) | 90% |
| Repository layer | 70% |
| Backfill scripts | 85% |
| Verification harness | 75% |
| **Service layer (Mongo→RDBMS patterns)** | **5%** |
| **Overall** | **~50%** |
### Top 5 Blockers for PG-Primary Cutover
1. **Service-layer rework not started + factory uncalled (CR-3)** — flag flips route zero traffic; ~50 patterns including lost-update/missing-txn money bugs
2. **Transaction + locking defects on payment/escrow paths (CR-1317)** — real money errors and lost updates under concurrent webhooks
3. **Shadow read unwired (CR-2)** — soak window is blind; cutover decision would be based on no signal
4. **Migration correctness: 0004 unjournaled + duplicate ledger DDL (CR-1)** — fresh-DB apply silently omits `funds_ledger_entries`
5. **Money-integrity gaps + verification silent-passes (CR-4/7/8/9/10/11/12)** — corruption can occur and pass green
---
## Recommended Next Actions
| # | Action | Files | Effort |
|---|---|---|---|
| 1 | Fix 0004 journal collision | `0004_funds_ledger_entries.sql`, `_journal.json` | S |
| 2 | Add money CHECK constraints + apply ledger TRUNCATE trigger | new migration on payments/payment_quotes/users/point_transactions/funds_ledger_entries | S |
| 3 | Fix backfill money/integrity defects (NULL amount, schema-less fallback, non-txn idMap, `_decimal.ts`) | backfill-payments/derivedDestinations/fundsLedger/pointTransactions/purchaseRequests | M |
| 4 | Close verification silent-passes (checksums, shadowRead, test.skip, id-map/enum/FK coverage, `--json` gate) | checksums.ts, shadowRead.ts, reconcile.ts, rowCounts.ts, migration-fk-idmap.test.ts | M |
| 5 | Add all deferred physical FKs `NOT VALID` + `VALIDATE`; rebuild blocking indexes `CONCURRENTLY` | new migration | M |
| 6 | Wire shadow reads into all 4 DualWrite read paths | DualWritePayment/User/Marketplace/PointsRepo | M |
| 7 | Inject factory into services + fix money/escrow concurrency (txn + `UPDATE…WHERE…RETURNING`) | paymentCoordinator, DisputeService, SellerOfferService, marketplaceController, PurchaseRequestService | **L** |
| 8 | Eliminate N+1 / full-fetch / skip-limit hotpaths | PointsService, searchPurchaseRequests, batchConvertTemplates, getUsersList | L |
| 9 | Schema + backfill for unmodeled active models | Dispute (L), Notification (M), ShopSettings/ConfigSetting/Address/Telegram* (S each) | L |