- Update Activity Log with 108 missing commits (48 backend + 60 frontend) - Update version references: backend v2.8.79, frontend v2.8.94 - Update migration count: 18 migrations (0000-0017) - Update Telegram Mini App Flow to v2.8.94 - Update Payment Flow - Scanner to 2026-06-05 - Update all architectural and database references - Add MongoDB removal handoff document with updated versions Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
273 lines
24 KiB
Markdown
273 lines
24 KiB
Markdown
# DB Migration Audit Report — Amanat Escrow (Mongo→PG)
|
||
**Date:** 2026-06-02 | **Scope:** Full Mongo→PG migration audit — schemas, indexes, constraints, dual-write coverage, backfill, verify harness, and service-layer Mongo-idiomatic patterns
|
||
|
||
---
|
||
|
||
## Executive Summary
|
||
|
||
The migration is **~50% complete** and **NOT ready for PG-primary cutover**. Schema and backfill scaffolding are mature (all 13 in-scope Mongo collections have Drizzle tables and backfill scripts), but three categories block cutover:
|
||
|
||
1. **Migration correctness** — `0004_funds_ledger_entries.sql` is unjournaled (silently skips `funds_ledger_entries` on fresh DBs); `shadowRead()` exists but is never called from any read path so the soak window is completely blind.
|
||
2. **Financial integrity gaps** — missing `CHECK (amount > 0)` / `fx_rate > 0` constraints, ~20 FK columns declared in `relations()` only (never as physical FKs), backfill that silently writes `amount = '0'` for NULL Mongo amounts.
|
||
3. **Service-layer rework is far bigger than the schema work** — the factory (`createRepositories`) has **zero callers**; 30+ services still import Mongoose directly and contain ~50 Mongo-idiomatic patterns (N+1 loops, full-fetch+JS-filter, read-modify-write without locking, multi-table writes with no transaction) that will cause real money errors and lost updates under concurrent load.
|
||
|
||
---
|
||
|
||
## Critical Issues (must fix before cutover)
|
||
|
||
| # | Dimension | Table/File → Issue | Fix |
|
||
|---|---|---|---|
|
||
| CR-1 | Migration | `0004_funds_ledger_entries.sql` has NO `_journal.json` entry; `funds_ledger_entries` DDL conflicts between 0003 and 0004 — silently skipped on fresh DB, `ALTER` fails with "constraint already exists" if run | Reduce 0004 to trigger-only DDL; register in journal |
|
||
| CR-2 | Arch | `shadowRead.ts` exists and is complete, but no `DualWrite*Repo` read method ever calls it — soak window measures zero signal | Wire `shadowRead()` + `ShadowReadMetrics` into all 4 DualWrite read paths |
|
||
| CR-3 | Arch | Factory `createRepositories` has **zero callers** outside `src/db/repositories/` — every `REPO_*=dual` env flag changes env but routes zero traffic | Inject factory into 30+ service files (`paymentController`, `paymentCoordinator`, marketplace/user/points services) |
|
||
| CR-4 | Backfill | `backfill-payments.ts`: `extractDecimalString(null)` returns `'0'` — NULL Mongo payment amount silently inserted as `amount = 0` (money integrity violation) | Make `extractDecimalString(null)` throw, or skip + warn |
|
||
| CR-5 | Backfill | `backfill-derivedDestinations.ts`: `require()` failure falls back to `strict:false` model — all fields become `undefined`, all rows skipped silently, exits 0 | Make import failure `throw` and exit 1; never fall back to schema-less model |
|
||
| CR-6 | Backfill | `backfill-fundsLedger.ts` + `backfill-pointTransactions.ts`: `upsertIdMap` + `INSERT` not in one transaction — interruption leaves orphan id_map rows; re-run `DO NOTHING` never inserts data row → unrecoverable | Wrap `upsertIdMap` + data INSERT in one PG transaction |
|
||
| CR-7 | Schema | `payment_quotes`: no `CHECK (offer_amount > 0 AND fx_rate > 0 AND token_price_usd > 0)` — zero/negative FX rate → divide-by-zero in settlement | Add three CHECK constraints |
|
||
| CR-8 | Schema | `payments`: no `CHECK (amount > 0)` | `ALTER TABLE payments ADD CONSTRAINT ck_payments_amount_pos CHECK (amount > 0)` |
|
||
| CR-9 | Verify | `checksums.ts`: `.catch(() => [])` silently returns `[]` on DB connection failure → `hasMismatch=false`, gate passes green | Propagate errors; never swallow in comparison path |
|
||
| CR-10 | Verify | `shadowRead.ts`: Decimal128 detection (`constructor.name === 'Decimal128'`) breaks on `.lean()` POJO results (`{$numberDecimal:...}`) — every numeric field appears equal, silent false-negative | Normalize amounts to strings before compare; detect `$numberDecimal` key |
|
||
| CR-11 | Verify | `migration-fk-idmap.test.ts`: `skipIfUnreachable` returns early without `test.skip()` — money-safety tests PASS when DB is unreachable; CI exit 0 | Call `test.skip()` when `!isReachable`; CI must assert `MONEY_SAFETY_TESTS_SKIPPED` absence |
|
||
| CR-12 | Verify | `rowCounts.ts`: id_map coverage check exists for `payments` only — dropped id_map entry for other collections → dangling FK silently | Add id_map coverage checks for users/purchaseRequests/sellerOffers/fundsLedger/pointTransactions |
|
||
| CR-13 | Code | `paymentCoordinator.ts` `executePaymentUpdate`: read `status` → JS guard → write — two concurrent webhooks both read `pending`, both write (lost update) | `UPDATE payments SET status=... WHERE id=... AND status NOT IN ('completed','cancelled','refunded') RETURNING *`; 0 rows → abort |
|
||
| CR-14 | Code | `paymentCoordinator.ts` dispute gate: `isReleaseBlockedById(prId)` read, then payment update — dispute raised in the gap bypasses gate | `SELECT ... FOR UPDATE` on PR row + payment update in one transaction |
|
||
| CR-15 | Code | `paymentCoordinator.ts` `executePaymentUpdate`: payment update + ledger append + PR backfill + `acceptOffer` + duplicate cancel + template delete run with **NO transaction** — step 3/4 failure leaves payment completed but offer not accepted | Wrap all side effects in one DB transaction |
|
||
| CR-16 | Code | `DisputeService.ts` `createDispute`: dispute create → chat create → save → `setChatId` with no transaction — partial failure → orphaned dispute/chat, UI crashes | Wrap all four ops in one PG transaction |
|
||
| CR-17 | Code | `SellerOfferService.ts` `withdrawOffer` + `marketplaceController.ts` `validateStatusTransition`: read-validate-write status machine with no atomic guard | `UPDATE ... WHERE id=... AND status=... RETURNING *`; 0 rows → 409 Conflict |
|
||
| CR-18 | Code | `PointsService.ts` `getReferrals`/`collectDeliveredReferralOrders`: per-referred-user `while(true)` skip/limit loop + per-row offer lookup → 510+ queries/user; 14+ s per leaderboard on WAN | Replace with single CTE: `LEFT JOIN purchase_requests/seller_offers/point_transactions ... GROUP BY u.id` |
|
||
| CR-19 | Code | `PurchaseRequestService.ts` `searchPurchaseRequests`: `findPurchaseRequests({limit:100})` then JS `.filter().slice(0,20)` — catastrophic at 10k rows | `WHERE title ILIKE $s OR description ILIKE $s LIMIT 20`, or `tsvector` generated column + GIN |
|
||
| CR-20 | Code | `Chat` model: `messages[]`/`participants[]`/`unreadCounts[]` as JSONB — no FK integrity, unbounded row bloat, non-indexable | Child tables `chat_messages`, `chat_participants`, `chat_message_reactions`; rewrite `ChatService` as SQL |
|
||
|
||
---
|
||
|
||
## High-Priority Issues
|
||
|
||
### Schema — Missing Physical FKs
|
||
All declared via Drizzle `relations()` only, never as `foreignKey()`/`.references()`. Zero referential integrity enforcement in DB.
|
||
|
||
- `users.referred_by_id` → add FK `ON DELETE SET NULL`
|
||
- `purchase_requests.buyer_id`, `category_id`, `selected_offer_id`
|
||
- All PR child tables (`purchase_request_delivery_info`, `_delivery_address`, `_seller_delivery_info`, `_service_info`, `_specifications`, `_preferred_sellers`) — `purchase_request_id`/`delivery_info_id`
|
||
- `delivery_attempts.delivery_info_id`, `seller_id`
|
||
- `derived_destinations.buyer_id`, `seller_id`, `seller_offer_id`
|
||
- `derived_destination_sweeps.destination_id`
|
||
- `trezor_accounts.user_id`; `trezor_derived_addresses.trezor_account_id`
|
||
- `funds_ledger_entries.purchase_request_id`, `payment_id` (deferred since 0003, never added)
|
||
- `point_transactions.user_id`, `referred_user_id`
|
||
|
||
### Schema — Other HIGH
|
||
- `users`: no `CHECK (points_available >= 0 AND points_available <= points_total)`
|
||
- `point_transactions`: no `CHECK (balance >= 0)`
|
||
- `payment_quotes`: no `CHECK (settle_amount >= raw_settle_amount)` (snap-up invariant unenforced)
|
||
- `purchase_request_preferred_sellers`: no composite PK, only uniqueIndex
|
||
- `seller_offers.price_amount numeric(18,8)` vs project-wide `numeric(38,18)` (precision gap in settlement)
|
||
|
||
### Migration — HIGH
|
||
- All 70+ `CREATE INDEX` are non-`CONCURRENTLY` (blocking SHARE lock on live data for all)
|
||
- All FK `ADD CONSTRAINT` run validating (no `NOT VALID` + later `VALIDATE`) — prolonged ACCESS EXCLUSIVE lock
|
||
- `blog_posts` and `notifications` exported from schema barrel but **no migration creates them**
|
||
- `disputes`/`chats` use `text` (not `uuid`) for FK columns — zero referential integrity
|
||
- Migration 0009: three sequential `UPDATE` DML steps not in `BEGIN/COMMIT` — partial failure leaves inconsistent category re-parenting
|
||
|
||
### Backfill — HIGH
|
||
- `String(number)` for `numeric` columns risks scientific notation in `backfill-purchaseRequests.ts` (budget), `backfill-sellerOffers.ts` (price.amount), `backfill-requestTemplates.ts` (budget, proposal.price)
|
||
- `backfill-users.ts`: `email ?? null` fails if `users.email` is NOT NULL for OAuth-only users
|
||
- `backfill-fundsLedger.ts`: missing `d.entryType` (no default) → NOT NULL violation
|
||
- `run-backfill.ts`: `requestTemplates` runs in Tier B but runbook documents it last (inconsistency)
|
||
|
||
### Verify — HIGH
|
||
- `reconcile.ts`: no double-refund detection; no `escrow_state`↔last-ledger-entry check; `LIMIT 1000` silently truncates
|
||
- `rowCounts.ts`: `estimatedDocumentCount()` is approximate — use `countDocuments({})`
|
||
- `checksums.ts`: no Mongo-side per-user points balance comparison during dual-write window
|
||
- `ledgerImmutability.ts`: `TRUNCATE` bypasses row-level trigger — add `BEFORE TRUNCATE` statement-level trigger
|
||
- Enum-value completeness verified nowhere
|
||
|
||
### Code — HIGH
|
||
- `SellerOfferService.ts` `acceptOffer`: per-rejected-seller `createNotification` loop (use `createNotificationsBulk`); multi-UPDATE repo needs transaction
|
||
- `RequestTemplateService.ts` `batchConvertTemplates`: ~50 sequential queries per 10-item cart; no transaction per item → orphan PRs with no offer, oversold templates
|
||
- `paymentService.ts` `createPaymentRecord`: `String(metadata?.sellerId || createLegacyObjectIdString())` injects random fake ObjectIds as FKs → PG FK violation
|
||
- `userController.ts` `getUsersList`: `$regex` on name/email → PG seqscan; needs `pg_trgm` GIN index + `ILIKE`
|
||
- `PurchaseRequestService.ts` `updatePurchaseRequestStatus` (completed): non-idempotent double-points risk; no transaction
|
||
|
||
---
|
||
|
||
## Medium Issues
|
||
|
||
**Schema:** dual unique indexes on `categories.name` (drop raw, keep partial `WHERE is_active`); missing `payments(purchase_request_id, status)` composite index; missing `seller_offers(seller_id,status)`, `derived_destinations(address, chain_id)`, `trezor_derived_addresses.address` indexes; `id_map` no PK and `new_id` no unique constraint; `request_templates` no `CHECK (usage_count <= max_usage)`.
|
||
|
||
**Migration:** `wallet_type` enum created but used in no column (dead DDL); `ALTER TYPE offer_currency ADD VALUE 'TRY'` requires PG 12+ in-transaction; `ck_pr_budget_currency_crypto` add(0006)/drop(0007) round-trip fails on rows with non-crypto values; `chats.participants` JSONB has no GIN index.
|
||
|
||
**Backfill:** enum default mismatches (`provider:'request.network'` vs `request_network`); `escrow_state ?? null` may hit NOT NULL; `derivedDestinations.lastKnownBalance` via JS Number loses precision above 2^53 for wei.
|
||
|
||
**Code:** `dataCleanupService.getCollectionStats` — 13 sequential `countDocuments()` (should be single subselect); `userController.updateUserProfile` writes arbitrary `profile.${key}` (whitelist needed); `paymentCoordinator` metadata read-spread-write overwrites concurrent keys (use `metadata || jsonb_build_object(...)`); skip/limit pagination in `getOffersBySeller`/`getUsersList`.
|
||
|
||
---
|
||
|
||
## Index & Constraint Punch List
|
||
|
||
| Table | Missing | Recommended DDL |
|
||
|---|---|---|
|
||
| payments | CHECK amount > 0 | `ALTER TABLE payments ADD CONSTRAINT ck_payments_amount_pos CHECK (amount > 0);` |
|
||
| payments | (purchase_request_id, status) | `CREATE INDEX CONCURRENTLY idx_payments_pr_status ON payments (purchase_request_id, status);` |
|
||
| payments | disputed partial | `CREATE INDEX CONCURRENTLY idx_payments_disputed ON payments (id) WHERE disputed = true;` |
|
||
| payment_quotes | CHECK money fields | `ALTER TABLE payment_quotes ADD CONSTRAINT ck_pq_pos CHECK (offer_amount > 0 AND fx_rate > 0 AND token_price_usd > 0);` |
|
||
| payment_quotes | CHECK snap-up | `ALTER TABLE payment_quotes ADD CONSTRAINT ck_pq_settle CHECK (settle_amount >= raw_settle_amount);` |
|
||
| users | referred_by_id FK | `ALTER TABLE users ADD CONSTRAINT users_referred_by_fk FOREIGN KEY (referred_by_id) REFERENCES users(id) ON DELETE SET NULL NOT VALID;` then `VALIDATE` |
|
||
| users | CHECK points | `ALTER TABLE users ADD CONSTRAINT ck_users_points CHECK (points_available >= 0 AND points_used >= 0 AND points_total >= 0 AND points_available <= points_total);` |
|
||
| point_transactions | CHECK balance | `ALTER TABLE point_transactions ADD CONSTRAINT ck_pt_balance CHECK (balance >= 0);` |
|
||
| funds_ledger_entries | FK pr + payment | `ALTER TABLE funds_ledger_entries ADD CONSTRAINT fle_pr_fk FOREIGN KEY (purchase_request_id) REFERENCES purchase_requests(id) NOT VALID;` then `VALIDATE` |
|
||
| funds_ledger_entries | TRUNCATE trigger | `CREATE TRIGGER funds_ledger_no_truncate BEFORE TRUNCATE ON funds_ledger_entries FOR EACH STATEMENT EXECUTE FUNCTION funds_ledger_immutable_fn();` |
|
||
| trezor_accounts | user_id FK | `ALTER TABLE trezor_accounts ADD CONSTRAINT ta_user_fk FOREIGN KEY (user_id) REFERENCES users(id) NOT VALID;` then `VALIDATE` |
|
||
| derived_destinations | buyer/seller/offer FK + (address,chain_id) | add 3 FKs `NOT VALID`; `CREATE INDEX CONCURRENTLY idx_dd_addr_chain ON derived_destinations (address, chain_id);` |
|
||
| purchase_requests | buyer/category/offer FK + (status,created_at) | add 3 FKs `NOT VALID`; `CREATE INDEX CONCURRENTLY idx_pr_status_created ON purchase_requests (status, created_at DESC);` |
|
||
| seller_offers | (seller_id,status) + (purchase_request_id,status) | `CREATE INDEX CONCURRENTLY idx_so_seller_status ON seller_offers (seller_id, status);` |
|
||
| id_map | PK + new_id unique | `ALTER TABLE id_map ADD PRIMARY KEY (collection, legacy_object_id); CREATE UNIQUE INDEX id_map_new_id_uq ON id_map (new_id);` |
|
||
| users (search) | trigram | `CREATE INDEX CONCURRENTLY idx_users_name_trgm ON users USING GIN (lower(first_name\|\|' '\|\|last_name\|\|' '\|\|coalesce(email,'')) gin_trgm_ops);` |
|
||
| purchase_requests | tags GIN | `CREATE INDEX CONCURRENTLY idx_pr_tags ON purchase_requests USING GIN (tags);` |
|
||
|
||
---
|
||
|
||
## Repository Coverage Matrix
|
||
|
||
| Interface | Drizzle Impl | Dual-Write | Status |
|
||
|---|---|---|---|
|
||
| PaymentRepo | Yes | Yes (shadow read NOT wired) | PARTIAL |
|
||
| UserRepo | Yes | Yes (shadow read NOT wired) | PARTIAL |
|
||
| MarketplaceRepo | Yes | Yes (shadow read NOT wired) | PARTIAL |
|
||
| PointsRepo | Yes | Yes (shadow read NOT wired) | PARTIAL |
|
||
| ReleaseHoldRepo | Yes | — | No dual-write |
|
||
| TrezorAccountRepo | Yes | — | No dual-write |
|
||
| DerivedDestinationRepo | Yes | — | No dual-write |
|
||
|
||
**Factory has zero application callers (CR-3) — most critical architecture gap.**
|
||
|
||
---
|
||
|
||
## Backfill Coverage Matrix
|
||
|
||
| Mongo Collection | Backfill Script | Ordering | Status |
|
||
|---|---|---|---|
|
||
| users | backfill-users.ts | Tier A | OK (email NOT NULL risk) |
|
||
| categories | backfill-categories.ts | Tier A | OK |
|
||
| requestTemplates | backfill-requestTemplates.ts | Tier B | OK (String() decimals; runbook order mismatch) |
|
||
| purchaseRequests | backfill-purchaseRequests.ts (2-pass) | Tier B | OK (String() decimals; silent preferred-seller skips) |
|
||
| sellerOffers | backfill-sellerOffers.ts | Tier B | OK (String() price.amount) |
|
||
| payments | backfill-payments.ts | Tier C | **RISK** — NULL amount → '0' (CR-4) |
|
||
| fundsLedger | backfill-fundsLedger.ts | Tier C | **RISK** — non-txn idMap (CR-6); entryType NOT NULL |
|
||
| derivedDestinations | backfill-derivedDestinations.ts | Tier C | **RISK** — schema-less fallback (CR-5); wei precision |
|
||
| trezorAccounts | backfill-trezorAccounts.ts | Tier C | OK |
|
||
| pointTransactions | backfill-pointTransactions.ts | Tier C | **RISK** — non-txn idMap (CR-6); String() decimals |
|
||
| id_map | (infra — `_idMap.ts`) | n/a | CORRECT |
|
||
| payment_quotes | (none — runtime-generated) | n/a | EXPECTED |
|
||
| pg_dualwrite_gaps | (none — operational log) | n/a | EXPECTED |
|
||
|
||
---
|
||
|
||
## Verification Coverage Matrix
|
||
|
||
| Concern | Covered By | Gap |
|
||
|---|---|---|
|
||
| Row-count parity | rowCounts.ts (9/~23 collections) | `estimatedDocumentCount()` approximate; id_map not counted |
|
||
| ID-mapping completeness | rowCounts.ts (payments only) | **CRITICAL** — no check for users/PR/sellerOffers/FLE/pointTransactions |
|
||
| FK integrity | rowCounts.ts (7 pairs) | `seller_offers.seller_id`, `trezor_accounts→users` missing |
|
||
| Money sum accuracy | checksums.ts | `.catch(()=>[])` silent pass on conn failure (CR-9) |
|
||
| Ledger reconciliation | reconcile.ts | No double-refund; no `escrow_state`↔last-entry; LIMIT 1000 truncation |
|
||
| Ledger immutability | ledgerImmutability.ts | TRUNCATE bypass; no schema filter on `pg_proc` |
|
||
| Shadow read fidelity | shadowRead.ts | Decimal128 lean false-negative (CR-10); not wired (CR-2) |
|
||
| Enum completeness | — | **Not covered anywhere** |
|
||
| Timestamp precision/TZ | — | Not covered |
|
||
| CI gate output | boolean only | No JSON stdout; tests pass-not-skip on unreachable DB (CR-11) |
|
||
|
||
---
|
||
|
||
## Models Not Yet in PG Schema
|
||
|
||
| Mongo Model | Fields | Actively Used | Effort | Notes |
|
||
|---|---|---|---|---|
|
||
| Dispute | ~15 + 3 embedded arrays | Yes (DisputeService, releaseHoldService) | **L** | `evidence[]`/`timeline[]`/`messages[]` → child tables; pre-save timeline hook → service |
|
||
| Notification | 11 | Yes (all services, high frequency) | S schema / M migration | `userId` as text → uuid FK backfill; TTL index → pg_cron |
|
||
| ShopSettings | ~12 | Yes (marketplace template pages) | S | `paymentConfig.allowedChains int[]`, `socialLinks` → 4 columns |
|
||
| ConfigSetting (+History) | 4 (+audit) | Yes (walletMonitor, scanner threshold) | S | key-value; history child table |
|
||
| LevelConfig | ~10 | Yes (PointsService) | S | flatten `benefits{}` to 4 columns |
|
||
| Address | 10 | Yes (dataCleanup, delivery flows) | S | `addressType` pgEnum; one-primary partial unique |
|
||
| Review | 9 | Admin CMS only | S | polymorphic `subjectId` → ref_kind discriminator |
|
||
| TelegramLink | ~12 | Yes (auth) | S | two unique constraints; (userId, status) idx |
|
||
| TelegramSession | ~10 | Yes (auth middleware) | S | TTL `expiresAt` → pg_cron |
|
||
| BlogPost | ~20 | Admin CMS only | S | `videos[]` child table; slug/publishedAt pre-save → service |
|
||
| TempVerification | 8 | Registration only | S | TTL cleanup |
|
||
|
||
---
|
||
|
||
## Mongo-Idiomatic Code Refactoring Tracker
|
||
|
||
| Pattern | File | Function | Severity | Fix |
|
||
|---|---|---|---|---|
|
||
| N+1 | PointsService.ts | getReferrals / collectDeliveredReferralOrders | CRITICAL | Single CTE with LEFT JOINs + GROUP BY |
|
||
| N+1 | SellerOfferService.ts | acceptOffer | HIGH | `createNotificationsBulk` + single seller-id query |
|
||
| N+1 | RequestTemplateService.ts | batchConvertTemplates | HIGH | Batch SELECT ANY($links), batch INSERT…VALUES, single usage UPDATE |
|
||
| N+1 | dataCleanupService.ts | getCollectionStats | MEDIUM | Single subselect count query |
|
||
| Full-fetch+filter | PurchaseRequestService.ts | searchPurchaseRequests | CRITICAL | ILIKE/tsvector WHERE + LIMIT 20 |
|
||
| Full-fetch+filter | PurchaseRequestService.ts | createPurchaseRequest (dup detect) | HIGH | WHERE buyer/title/description/created_at LIMIT 1 |
|
||
| Full-fetch+filter | paymentCoordinator.ts | executePaymentUpdate (template cleanup) | HIGH | Push JSONB conditions into WHERE/DELETE |
|
||
| Full-fetch+filter | userController.ts | getUsersList | HIGH | pg_trgm GIN + ILIKE |
|
||
| JSONB no join table | Chat | messages/participants/unreadCounts | CRITICAL | 3 child tables (CR-20) |
|
||
| JSONB no join table | Dispute | evidence/timeline/messages | HIGH | 3 child tables on migration |
|
||
| JSONB schemaless | Payment | metadata | HIGH | Promote `is_template_checkout`, `rn_request_id` to typed columns |
|
||
| In-memory agg | PointsService.ts | sumDeliveredReferralSpend | CRITICAL | SUM in CTE |
|
||
| In-memory agg | SellerOfferService.ts | getOfferStatistics | MEDIUM | COUNT(*) OVER() / ROLLUP |
|
||
| Lost update | paymentCoordinator.ts | executePaymentUpdate | CRITICAL | UPDATE…WHERE status NOT IN (terminal) RETURNING |
|
||
| Lost update | SellerOfferService.ts | updateOffer | HIGH | UPDATE…WHERE status='pending' RETURNING |
|
||
| TOCTOU | SellerOfferService.ts | withdrawOffer | CRITICAL | UPDATE…WHERE id AND seller AND status='pending' |
|
||
| TOCTOU | marketplaceController.ts | validateStatusTransition | CRITICAL | UPDATE…WHERE status=$expected; 0 rows → 409 |
|
||
| TOCTOU | paymentCoordinator.ts | dispute gate | CRITICAL | FOR UPDATE on PR + same txn |
|
||
| TOCTOU | PurchaseRequestService.ts | updatePurchaseRequestStatus | HIGH | UPDATE…WHERE status=$old RETURNING |
|
||
| Missing txn | paymentCoordinator.ts | executePaymentUpdate | CRITICAL | One txn for all side effects |
|
||
| Missing txn | DisputeService.ts | createDispute | CRITICAL | One txn for dispute+chat+link |
|
||
| Missing txn | SellerOfferService.ts | acceptOffer (repo) | HIGH | Txn for accept/reject/PR update |
|
||
| Missing txn | RequestTemplateService.ts | batchConvertTemplates | HIGH | Txn (or savepoint) per cart item |
|
||
| Missing txn | PurchaseRequestService.ts | updatePurchaseRequestStatus (completed) | HIGH | Txn or outbox for referral reward |
|
||
| Schemaless write | paymentService.ts | createPaymentRecord | HIGH | Remove fake-ObjectId FK fallback |
|
||
| Schemaless write | userController.ts | updateUserProfile | MEDIUM | Whitelist + jsonb \|\| merge |
|
||
| Skip/limit pagination | PointsService.ts | collectDeliveredReferralOrders | CRITICAL | Replace loop with aggregate |
|
||
| Skip/limit pagination | PurchaseRequestService.ts | searchPurchaseRequests | HIGH | Keyset on (created_at, id) |
|
||
| Skip/limit pagination | SellerOfferService.ts / userController.ts | getOffersBySeller / getUsersList | MEDIUM | Keyset + cap limit 100 |
|
||
| Virtual/hook | Chat | addMessage/markAsRead/getUnreadCount | CRITICAL | SQL ops in ChatRepository |
|
||
| Pre-save hook | FundsLedgerEntry | immutability | HIGH | Apply trigger DDL now |
|
||
|
||
---
|
||
|
||
## Migration Completion Assessment
|
||
|
||
| Layer | % |
|
||
|---|---|
|
||
| Schema (Drizzle tables vs Mongo collections) | 90% |
|
||
| Repository layer | 70% |
|
||
| Backfill scripts | 85% |
|
||
| Verification harness | 75% |
|
||
| **Service layer (Mongo→RDBMS patterns)** | **5%** |
|
||
| **Overall** | **~50%** |
|
||
|
||
### Top 5 Blockers for PG-Primary Cutover
|
||
|
||
1. **Service-layer rework not started + factory uncalled (CR-3)** — flag flips route zero traffic; ~50 patterns including lost-update/missing-txn money bugs
|
||
2. **Transaction + locking defects on payment/escrow paths (CR-13–17)** — real money errors and lost updates under concurrent webhooks
|
||
3. **Shadow read unwired (CR-2)** — soak window is blind; cutover decision would be based on no signal
|
||
4. **Migration correctness: 0004 unjournaled + duplicate ledger DDL (CR-1)** — fresh-DB apply silently omits `funds_ledger_entries`
|
||
5. **Money-integrity gaps + verification silent-passes (CR-4/7/8/9/10/11/12)** — corruption can occur and pass green
|
||
|
||
---
|
||
|
||
## Recommended Next Actions
|
||
|
||
| # | Action | Files | Effort |
|
||
|---|---|---|---|
|
||
| 1 | Fix 0004 journal collision | `0004_funds_ledger_entries.sql`, `_journal.json` | S |
|
||
| 2 | Add money CHECK constraints + apply ledger TRUNCATE trigger | new migration on payments/payment_quotes/users/point_transactions/funds_ledger_entries | S |
|
||
| 3 | Fix backfill money/integrity defects (NULL amount, schema-less fallback, non-txn idMap, `_decimal.ts`) | backfill-payments/derivedDestinations/fundsLedger/pointTransactions/purchaseRequests | M |
|
||
| 4 | Close verification silent-passes (checksums, shadowRead, test.skip, id-map/enum/FK coverage, `--json` gate) | checksums.ts, shadowRead.ts, reconcile.ts, rowCounts.ts, migration-fk-idmap.test.ts | M |
|
||
| 5 | Add all deferred physical FKs `NOT VALID` + `VALIDATE`; rebuild blocking indexes `CONCURRENTLY` | new migration | M |
|
||
| 6 | Wire shadow reads into all 4 DualWrite read paths | DualWritePayment/User/Marketplace/PointsRepo | M |
|
||
| 7 | Inject factory into services + fix money/escrow concurrency (txn + `UPDATE…WHERE…RETURNING`) | paymentCoordinator, DisputeService, SellerOfferService, marketplaceController, PurchaseRequestService | **L** |
|
||
| 8 | Eliminate N+1 / full-fetch / skip-limit hotpaths | PointsService, searchPurchaseRequests, batchConvertTemplates, getUsersList | L |
|
||
| 9 | Schema + backfill for unmodeled active models | Dispute (L), Notification (M), ShopSettings/ConfigSetting/Address/Telegram* (S each) | L |
|