docs: sync documentation with latest codebase state (merged)
- Update Activity Log with 108 missing commits (48 backend + 60 frontend) - Update version references: backend v2.8.79, frontend v2.8.94 - Update migration count: 18 migrations (0000-0017) - Update Telegram Mini App Flow to v2.8.94 - Update Payment Flow - Scanner to 2026-06-05 - Update all architectural and database references - Add MongoDB removal handoff document with updated versions Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
This commit is contained in:
272
09 - Audits/DB Migration Audit Report (2026-06-02).md
Normal file
272
09 - Audits/DB Migration Audit Report (2026-06-02).md
Normal file
@@ -0,0 +1,272 @@
|
||||
# DB Migration Audit Report — Amanat Escrow (Mongo→PG)
|
||||
**Date:** 2026-06-02 | **Scope:** Full Mongo→PG migration audit — schemas, indexes, constraints, dual-write coverage, backfill, verify harness, and service-layer Mongo-idiomatic patterns
|
||||
|
||||
---
|
||||
|
||||
## Executive Summary
|
||||
|
||||
The migration is **~50% complete** and **NOT ready for PG-primary cutover**. Schema and backfill scaffolding are mature (all 13 in-scope Mongo collections have Drizzle tables and backfill scripts), but three categories block cutover:
|
||||
|
||||
1. **Migration correctness** — `0004_funds_ledger_entries.sql` is unjournaled (silently skips `funds_ledger_entries` on fresh DBs); `shadowRead()` exists but is never called from any read path so the soak window is completely blind.
|
||||
2. **Financial integrity gaps** — missing `CHECK (amount > 0)` / `fx_rate > 0` constraints, ~20 FK columns declared in `relations()` only (never as physical FKs), backfill that silently writes `amount = '0'` for NULL Mongo amounts.
|
||||
3. **Service-layer rework is far bigger than the schema work** — the factory (`createRepositories`) has **zero callers**; 30+ services still import Mongoose directly and contain ~50 Mongo-idiomatic patterns (N+1 loops, full-fetch+JS-filter, read-modify-write without locking, multi-table writes with no transaction) that will cause real money errors and lost updates under concurrent load.
|
||||
|
||||
---
|
||||
|
||||
## Critical Issues (must fix before cutover)
|
||||
|
||||
| # | Dimension | Table/File → Issue | Fix |
|
||||
|---|---|---|---|
|
||||
| CR-1 | Migration | `0004_funds_ledger_entries.sql` has NO `_journal.json` entry; `funds_ledger_entries` DDL conflicts between 0003 and 0004 — silently skipped on fresh DB, `ALTER` fails with "constraint already exists" if run | Reduce 0004 to trigger-only DDL; register in journal |
|
||||
| CR-2 | Arch | `shadowRead.ts` exists and is complete, but no `DualWrite*Repo` read method ever calls it — soak window measures zero signal | Wire `shadowRead()` + `ShadowReadMetrics` into all 4 DualWrite read paths |
|
||||
| CR-3 | Arch | Factory `createRepositories` has **zero callers** outside `src/db/repositories/` — every `REPO_*=dual` env flag changes env but routes zero traffic | Inject factory into 30+ service files (`paymentController`, `paymentCoordinator`, marketplace/user/points services) |
|
||||
| CR-4 | Backfill | `backfill-payments.ts`: `extractDecimalString(null)` returns `'0'` — NULL Mongo payment amount silently inserted as `amount = 0` (money integrity violation) | Make `extractDecimalString(null)` throw, or skip + warn |
|
||||
| CR-5 | Backfill | `backfill-derivedDestinations.ts`: `require()` failure falls back to `strict:false` model — all fields become `undefined`, all rows skipped silently, exits 0 | Make import failure `throw` and exit 1; never fall back to schema-less model |
|
||||
| CR-6 | Backfill | `backfill-fundsLedger.ts` + `backfill-pointTransactions.ts`: `upsertIdMap` + `INSERT` not in one transaction — interruption leaves orphan id_map rows; re-run `DO NOTHING` never inserts data row → unrecoverable | Wrap `upsertIdMap` + data INSERT in one PG transaction |
|
||||
| CR-7 | Schema | `payment_quotes`: no `CHECK (offer_amount > 0 AND fx_rate > 0 AND token_price_usd > 0)` — zero/negative FX rate → divide-by-zero in settlement | Add three CHECK constraints |
|
||||
| CR-8 | Schema | `payments`: no `CHECK (amount > 0)` | `ALTER TABLE payments ADD CONSTRAINT ck_payments_amount_pos CHECK (amount > 0)` |
|
||||
| CR-9 | Verify | `checksums.ts`: `.catch(() => [])` silently returns `[]` on DB connection failure → `hasMismatch=false`, gate passes green | Propagate errors; never swallow in comparison path |
|
||||
| CR-10 | Verify | `shadowRead.ts`: Decimal128 detection (`constructor.name === 'Decimal128'`) breaks on `.lean()` POJO results (`{$numberDecimal:...}`) — every numeric field appears equal, silent false-negative | Normalize amounts to strings before compare; detect `$numberDecimal` key |
|
||||
| CR-11 | Verify | `migration-fk-idmap.test.ts`: `skipIfUnreachable` returns early without `test.skip()` — money-safety tests PASS when DB is unreachable; CI exit 0 | Call `test.skip()` when `!isReachable`; CI must assert `MONEY_SAFETY_TESTS_SKIPPED` absence |
|
||||
| CR-12 | Verify | `rowCounts.ts`: id_map coverage check exists for `payments` only — dropped id_map entry for other collections → dangling FK silently | Add id_map coverage checks for users/purchaseRequests/sellerOffers/fundsLedger/pointTransactions |
|
||||
| CR-13 | Code | `paymentCoordinator.ts` `executePaymentUpdate`: read `status` → JS guard → write — two concurrent webhooks both read `pending`, both write (lost update) | `UPDATE payments SET status=... WHERE id=... AND status NOT IN ('completed','cancelled','refunded') RETURNING *`; 0 rows → abort |
|
||||
| CR-14 | Code | `paymentCoordinator.ts` dispute gate: `isReleaseBlockedById(prId)` read, then payment update — dispute raised in the gap bypasses gate | `SELECT ... FOR UPDATE` on PR row + payment update in one transaction |
|
||||
| CR-15 | Code | `paymentCoordinator.ts` `executePaymentUpdate`: payment update + ledger append + PR backfill + `acceptOffer` + duplicate cancel + template delete run with **NO transaction** — step 3/4 failure leaves payment completed but offer not accepted | Wrap all side effects in one DB transaction |
|
||||
| CR-16 | Code | `DisputeService.ts` `createDispute`: dispute create → chat create → save → `setChatId` with no transaction — partial failure → orphaned dispute/chat, UI crashes | Wrap all four ops in one PG transaction |
|
||||
| CR-17 | Code | `SellerOfferService.ts` `withdrawOffer` + `marketplaceController.ts` `validateStatusTransition`: read-validate-write status machine with no atomic guard | `UPDATE ... WHERE id=... AND status=... RETURNING *`; 0 rows → 409 Conflict |
|
||||
| CR-18 | Code | `PointsService.ts` `getReferrals`/`collectDeliveredReferralOrders`: per-referred-user `while(true)` skip/limit loop + per-row offer lookup → 510+ queries/user; 14+ s per leaderboard on WAN | Replace with single CTE: `LEFT JOIN purchase_requests/seller_offers/point_transactions ... GROUP BY u.id` |
|
||||
| CR-19 | Code | `PurchaseRequestService.ts` `searchPurchaseRequests`: `findPurchaseRequests({limit:100})` then JS `.filter().slice(0,20)` — catastrophic at 10k rows | `WHERE title ILIKE $s OR description ILIKE $s LIMIT 20`, or `tsvector` generated column + GIN |
|
||||
| CR-20 | Code | `Chat` model: `messages[]`/`participants[]`/`unreadCounts[]` as JSONB — no FK integrity, unbounded row bloat, non-indexable | Child tables `chat_messages`, `chat_participants`, `chat_message_reactions`; rewrite `ChatService` as SQL |
|
||||
|
||||
---
|
||||
|
||||
## High-Priority Issues
|
||||
|
||||
### Schema — Missing Physical FKs
|
||||
All declared via Drizzle `relations()` only, never as `foreignKey()`/`.references()`. Zero referential integrity enforcement in DB.
|
||||
|
||||
- `users.referred_by_id` → add FK `ON DELETE SET NULL`
|
||||
- `purchase_requests.buyer_id`, `category_id`, `selected_offer_id`
|
||||
- All PR child tables (`purchase_request_delivery_info`, `_delivery_address`, `_seller_delivery_info`, `_service_info`, `_specifications`, `_preferred_sellers`) — `purchase_request_id`/`delivery_info_id`
|
||||
- `delivery_attempts.delivery_info_id`, `seller_id`
|
||||
- `derived_destinations.buyer_id`, `seller_id`, `seller_offer_id`
|
||||
- `derived_destination_sweeps.destination_id`
|
||||
- `trezor_accounts.user_id`; `trezor_derived_addresses.trezor_account_id`
|
||||
- `funds_ledger_entries.purchase_request_id`, `payment_id` (deferred since 0003, never added)
|
||||
- `point_transactions.user_id`, `referred_user_id`
|
||||
|
||||
### Schema — Other HIGH
|
||||
- `users`: no `CHECK (points_available >= 0 AND points_available <= points_total)`
|
||||
- `point_transactions`: no `CHECK (balance >= 0)`
|
||||
- `payment_quotes`: no `CHECK (settle_amount >= raw_settle_amount)` (snap-up invariant unenforced)
|
||||
- `purchase_request_preferred_sellers`: no composite PK, only uniqueIndex
|
||||
- `seller_offers.price_amount numeric(18,8)` vs project-wide `numeric(38,18)` (precision gap in settlement)
|
||||
|
||||
### Migration — HIGH
|
||||
- All 70+ `CREATE INDEX` are non-`CONCURRENTLY` (blocking SHARE lock on live data for all)
|
||||
- All FK `ADD CONSTRAINT` run validating (no `NOT VALID` + later `VALIDATE`) — prolonged ACCESS EXCLUSIVE lock
|
||||
- `blog_posts` and `notifications` exported from schema barrel but **no migration creates them**
|
||||
- `disputes`/`chats` use `text` (not `uuid`) for FK columns — zero referential integrity
|
||||
- Migration 0009: three sequential `UPDATE` DML steps not in `BEGIN/COMMIT` — partial failure leaves inconsistent category re-parenting
|
||||
|
||||
### Backfill — HIGH
|
||||
- `String(number)` for `numeric` columns risks scientific notation in `backfill-purchaseRequests.ts` (budget), `backfill-sellerOffers.ts` (price.amount), `backfill-requestTemplates.ts` (budget, proposal.price)
|
||||
- `backfill-users.ts`: `email ?? null` fails if `users.email` is NOT NULL for OAuth-only users
|
||||
- `backfill-fundsLedger.ts`: missing `d.entryType` (no default) → NOT NULL violation
|
||||
- `run-backfill.ts`: `requestTemplates` runs in Tier B but runbook documents it last (inconsistency)
|
||||
|
||||
### Verify — HIGH
|
||||
- `reconcile.ts`: no double-refund detection; no `escrow_state`↔last-ledger-entry check; `LIMIT 1000` silently truncates
|
||||
- `rowCounts.ts`: `estimatedDocumentCount()` is approximate — use `countDocuments({})`
|
||||
- `checksums.ts`: no Mongo-side per-user points balance comparison during dual-write window
|
||||
- `ledgerImmutability.ts`: `TRUNCATE` bypasses row-level trigger — add `BEFORE TRUNCATE` statement-level trigger
|
||||
- Enum-value completeness verified nowhere
|
||||
|
||||
### Code — HIGH
|
||||
- `SellerOfferService.ts` `acceptOffer`: per-rejected-seller `createNotification` loop (use `createNotificationsBulk`); multi-UPDATE repo needs transaction
|
||||
- `RequestTemplateService.ts` `batchConvertTemplates`: ~50 sequential queries per 10-item cart; no transaction per item → orphan PRs with no offer, oversold templates
|
||||
- `paymentService.ts` `createPaymentRecord`: `String(metadata?.sellerId || createLegacyObjectIdString())` injects random fake ObjectIds as FKs → PG FK violation
|
||||
- `userController.ts` `getUsersList`: `$regex` on name/email → PG seqscan; needs `pg_trgm` GIN index + `ILIKE`
|
||||
- `PurchaseRequestService.ts` `updatePurchaseRequestStatus` (completed): non-idempotent double-points risk; no transaction
|
||||
|
||||
---
|
||||
|
||||
## Medium Issues
|
||||
|
||||
**Schema:** dual unique indexes on `categories.name` (drop raw, keep partial `WHERE is_active`); missing `payments(purchase_request_id, status)` composite index; missing `seller_offers(seller_id,status)`, `derived_destinations(address, chain_id)`, `trezor_derived_addresses.address` indexes; `id_map` no PK and `new_id` no unique constraint; `request_templates` no `CHECK (usage_count <= max_usage)`.
|
||||
|
||||
**Migration:** `wallet_type` enum created but used in no column (dead DDL); `ALTER TYPE offer_currency ADD VALUE 'TRY'` requires PG 12+ in-transaction; `ck_pr_budget_currency_crypto` add(0006)/drop(0007) round-trip fails on rows with non-crypto values; `chats.participants` JSONB has no GIN index.
|
||||
|
||||
**Backfill:** enum default mismatches (`provider:'request.network'` vs `request_network`); `escrow_state ?? null` may hit NOT NULL; `derivedDestinations.lastKnownBalance` via JS Number loses precision above 2^53 for wei.
|
||||
|
||||
**Code:** `dataCleanupService.getCollectionStats` — 13 sequential `countDocuments()` (should be single subselect); `userController.updateUserProfile` writes arbitrary `profile.${key}` (whitelist needed); `paymentCoordinator` metadata read-spread-write overwrites concurrent keys (use `metadata || jsonb_build_object(...)`); skip/limit pagination in `getOffersBySeller`/`getUsersList`.
|
||||
|
||||
---
|
||||
|
||||
## Index & Constraint Punch List
|
||||
|
||||
| Table | Missing | Recommended DDL |
|
||||
|---|---|---|
|
||||
| payments | CHECK amount > 0 | `ALTER TABLE payments ADD CONSTRAINT ck_payments_amount_pos CHECK (amount > 0);` |
|
||||
| payments | (purchase_request_id, status) | `CREATE INDEX CONCURRENTLY idx_payments_pr_status ON payments (purchase_request_id, status);` |
|
||||
| payments | disputed partial | `CREATE INDEX CONCURRENTLY idx_payments_disputed ON payments (id) WHERE disputed = true;` |
|
||||
| payment_quotes | CHECK money fields | `ALTER TABLE payment_quotes ADD CONSTRAINT ck_pq_pos CHECK (offer_amount > 0 AND fx_rate > 0 AND token_price_usd > 0);` |
|
||||
| payment_quotes | CHECK snap-up | `ALTER TABLE payment_quotes ADD CONSTRAINT ck_pq_settle CHECK (settle_amount >= raw_settle_amount);` |
|
||||
| users | referred_by_id FK | `ALTER TABLE users ADD CONSTRAINT users_referred_by_fk FOREIGN KEY (referred_by_id) REFERENCES users(id) ON DELETE SET NULL NOT VALID;` then `VALIDATE` |
|
||||
| users | CHECK points | `ALTER TABLE users ADD CONSTRAINT ck_users_points CHECK (points_available >= 0 AND points_used >= 0 AND points_total >= 0 AND points_available <= points_total);` |
|
||||
| point_transactions | CHECK balance | `ALTER TABLE point_transactions ADD CONSTRAINT ck_pt_balance CHECK (balance >= 0);` |
|
||||
| funds_ledger_entries | FK pr + payment | `ALTER TABLE funds_ledger_entries ADD CONSTRAINT fle_pr_fk FOREIGN KEY (purchase_request_id) REFERENCES purchase_requests(id) NOT VALID;` then `VALIDATE` |
|
||||
| funds_ledger_entries | TRUNCATE trigger | `CREATE TRIGGER funds_ledger_no_truncate BEFORE TRUNCATE ON funds_ledger_entries FOR EACH STATEMENT EXECUTE FUNCTION funds_ledger_immutable_fn();` |
|
||||
| trezor_accounts | user_id FK | `ALTER TABLE trezor_accounts ADD CONSTRAINT ta_user_fk FOREIGN KEY (user_id) REFERENCES users(id) NOT VALID;` then `VALIDATE` |
|
||||
| derived_destinations | buyer/seller/offer FK + (address,chain_id) | add 3 FKs `NOT VALID`; `CREATE INDEX CONCURRENTLY idx_dd_addr_chain ON derived_destinations (address, chain_id);` |
|
||||
| purchase_requests | buyer/category/offer FK + (status,created_at) | add 3 FKs `NOT VALID`; `CREATE INDEX CONCURRENTLY idx_pr_status_created ON purchase_requests (status, created_at DESC);` |
|
||||
| seller_offers | (seller_id,status) + (purchase_request_id,status) | `CREATE INDEX CONCURRENTLY idx_so_seller_status ON seller_offers (seller_id, status);` |
|
||||
| id_map | PK + new_id unique | `ALTER TABLE id_map ADD PRIMARY KEY (collection, legacy_object_id); CREATE UNIQUE INDEX id_map_new_id_uq ON id_map (new_id);` |
|
||||
| users (search) | trigram | `CREATE INDEX CONCURRENTLY idx_users_name_trgm ON users USING GIN (lower(first_name\|\|' '\|\|last_name\|\|' '\|\|coalesce(email,'')) gin_trgm_ops);` |
|
||||
| purchase_requests | tags GIN | `CREATE INDEX CONCURRENTLY idx_pr_tags ON purchase_requests USING GIN (tags);` |
|
||||
|
||||
---
|
||||
|
||||
## Repository Coverage Matrix
|
||||
|
||||
| Interface | Drizzle Impl | Dual-Write | Status |
|
||||
|---|---|---|---|
|
||||
| PaymentRepo | Yes | Yes (shadow read NOT wired) | PARTIAL |
|
||||
| UserRepo | Yes | Yes (shadow read NOT wired) | PARTIAL |
|
||||
| MarketplaceRepo | Yes | Yes (shadow read NOT wired) | PARTIAL |
|
||||
| PointsRepo | Yes | Yes (shadow read NOT wired) | PARTIAL |
|
||||
| ReleaseHoldRepo | Yes | — | No dual-write |
|
||||
| TrezorAccountRepo | Yes | — | No dual-write |
|
||||
| DerivedDestinationRepo | Yes | — | No dual-write |
|
||||
|
||||
**Factory has zero application callers (CR-3) — most critical architecture gap.**
|
||||
|
||||
---
|
||||
|
||||
## Backfill Coverage Matrix
|
||||
|
||||
| Mongo Collection | Backfill Script | Ordering | Status |
|
||||
|---|---|---|---|
|
||||
| users | backfill-users.ts | Tier A | OK (email NOT NULL risk) |
|
||||
| categories | backfill-categories.ts | Tier A | OK |
|
||||
| requestTemplates | backfill-requestTemplates.ts | Tier B | OK (String() decimals; runbook order mismatch) |
|
||||
| purchaseRequests | backfill-purchaseRequests.ts (2-pass) | Tier B | OK (String() decimals; silent preferred-seller skips) |
|
||||
| sellerOffers | backfill-sellerOffers.ts | Tier B | OK (String() price.amount) |
|
||||
| payments | backfill-payments.ts | Tier C | **RISK** — NULL amount → '0' (CR-4) |
|
||||
| fundsLedger | backfill-fundsLedger.ts | Tier C | **RISK** — non-txn idMap (CR-6); entryType NOT NULL |
|
||||
| derivedDestinations | backfill-derivedDestinations.ts | Tier C | **RISK** — schema-less fallback (CR-5); wei precision |
|
||||
| trezorAccounts | backfill-trezorAccounts.ts | Tier C | OK |
|
||||
| pointTransactions | backfill-pointTransactions.ts | Tier C | **RISK** — non-txn idMap (CR-6); String() decimals |
|
||||
| id_map | (infra — `_idMap.ts`) | n/a | CORRECT |
|
||||
| payment_quotes | (none — runtime-generated) | n/a | EXPECTED |
|
||||
| pg_dualwrite_gaps | (none — operational log) | n/a | EXPECTED |
|
||||
|
||||
---
|
||||
|
||||
## Verification Coverage Matrix
|
||||
|
||||
| Concern | Covered By | Gap |
|
||||
|---|---|---|
|
||||
| Row-count parity | rowCounts.ts (9/~23 collections) | `estimatedDocumentCount()` approximate; id_map not counted |
|
||||
| ID-mapping completeness | rowCounts.ts (payments only) | **CRITICAL** — no check for users/PR/sellerOffers/FLE/pointTransactions |
|
||||
| FK integrity | rowCounts.ts (7 pairs) | `seller_offers.seller_id`, `trezor_accounts→users` missing |
|
||||
| Money sum accuracy | checksums.ts | `.catch(()=>[])` silent pass on conn failure (CR-9) |
|
||||
| Ledger reconciliation | reconcile.ts | No double-refund; no `escrow_state`↔last-entry; LIMIT 1000 truncation |
|
||||
| Ledger immutability | ledgerImmutability.ts | TRUNCATE bypass; no schema filter on `pg_proc` |
|
||||
| Shadow read fidelity | shadowRead.ts | Decimal128 lean false-negative (CR-10); not wired (CR-2) |
|
||||
| Enum completeness | — | **Not covered anywhere** |
|
||||
| Timestamp precision/TZ | — | Not covered |
|
||||
| CI gate output | boolean only | No JSON stdout; tests pass-not-skip on unreachable DB (CR-11) |
|
||||
|
||||
---
|
||||
|
||||
## Models Not Yet in PG Schema
|
||||
|
||||
| Mongo Model | Fields | Actively Used | Effort | Notes |
|
||||
|---|---|---|---|---|
|
||||
| Dispute | ~15 + 3 embedded arrays | Yes (DisputeService, releaseHoldService) | **L** | `evidence[]`/`timeline[]`/`messages[]` → child tables; pre-save timeline hook → service |
|
||||
| Notification | 11 | Yes (all services, high frequency) | S schema / M migration | `userId` as text → uuid FK backfill; TTL index → pg_cron |
|
||||
| ShopSettings | ~12 | Yes (marketplace template pages) | S | `paymentConfig.allowedChains int[]`, `socialLinks` → 4 columns |
|
||||
| ConfigSetting (+History) | 4 (+audit) | Yes (walletMonitor, scanner threshold) | S | key-value; history child table |
|
||||
| LevelConfig | ~10 | Yes (PointsService) | S | flatten `benefits{}` to 4 columns |
|
||||
| Address | 10 | Yes (dataCleanup, delivery flows) | S | `addressType` pgEnum; one-primary partial unique |
|
||||
| Review | 9 | Admin CMS only | S | polymorphic `subjectId` → ref_kind discriminator |
|
||||
| TelegramLink | ~12 | Yes (auth) | S | two unique constraints; (userId, status) idx |
|
||||
| TelegramSession | ~10 | Yes (auth middleware) | S | TTL `expiresAt` → pg_cron |
|
||||
| BlogPost | ~20 | Admin CMS only | S | `videos[]` child table; slug/publishedAt pre-save → service |
|
||||
| TempVerification | 8 | Registration only | S | TTL cleanup |
|
||||
|
||||
---
|
||||
|
||||
## Mongo-Idiomatic Code Refactoring Tracker
|
||||
|
||||
| Pattern | File | Function | Severity | Fix |
|
||||
|---|---|---|---|---|
|
||||
| N+1 | PointsService.ts | getReferrals / collectDeliveredReferralOrders | CRITICAL | Single CTE with LEFT JOINs + GROUP BY |
|
||||
| N+1 | SellerOfferService.ts | acceptOffer | HIGH | `createNotificationsBulk` + single seller-id query |
|
||||
| N+1 | RequestTemplateService.ts | batchConvertTemplates | HIGH | Batch SELECT ANY($links), batch INSERT…VALUES, single usage UPDATE |
|
||||
| N+1 | dataCleanupService.ts | getCollectionStats | MEDIUM | Single subselect count query |
|
||||
| Full-fetch+filter | PurchaseRequestService.ts | searchPurchaseRequests | CRITICAL | ILIKE/tsvector WHERE + LIMIT 20 |
|
||||
| Full-fetch+filter | PurchaseRequestService.ts | createPurchaseRequest (dup detect) | HIGH | WHERE buyer/title/description/created_at LIMIT 1 |
|
||||
| Full-fetch+filter | paymentCoordinator.ts | executePaymentUpdate (template cleanup) | HIGH | Push JSONB conditions into WHERE/DELETE |
|
||||
| Full-fetch+filter | userController.ts | getUsersList | HIGH | pg_trgm GIN + ILIKE |
|
||||
| JSONB no join table | Chat | messages/participants/unreadCounts | CRITICAL | 3 child tables (CR-20) |
|
||||
| JSONB no join table | Dispute | evidence/timeline/messages | HIGH | 3 child tables on migration |
|
||||
| JSONB schemaless | Payment | metadata | HIGH | Promote `is_template_checkout`, `rn_request_id` to typed columns |
|
||||
| In-memory agg | PointsService.ts | sumDeliveredReferralSpend | CRITICAL | SUM in CTE |
|
||||
| In-memory agg | SellerOfferService.ts | getOfferStatistics | MEDIUM | COUNT(*) OVER() / ROLLUP |
|
||||
| Lost update | paymentCoordinator.ts | executePaymentUpdate | CRITICAL | UPDATE…WHERE status NOT IN (terminal) RETURNING |
|
||||
| Lost update | SellerOfferService.ts | updateOffer | HIGH | UPDATE…WHERE status='pending' RETURNING |
|
||||
| TOCTOU | SellerOfferService.ts | withdrawOffer | CRITICAL | UPDATE…WHERE id AND seller AND status='pending' |
|
||||
| TOCTOU | marketplaceController.ts | validateStatusTransition | CRITICAL | UPDATE…WHERE status=$expected; 0 rows → 409 |
|
||||
| TOCTOU | paymentCoordinator.ts | dispute gate | CRITICAL | FOR UPDATE on PR + same txn |
|
||||
| TOCTOU | PurchaseRequestService.ts | updatePurchaseRequestStatus | HIGH | UPDATE…WHERE status=$old RETURNING |
|
||||
| Missing txn | paymentCoordinator.ts | executePaymentUpdate | CRITICAL | One txn for all side effects |
|
||||
| Missing txn | DisputeService.ts | createDispute | CRITICAL | One txn for dispute+chat+link |
|
||||
| Missing txn | SellerOfferService.ts | acceptOffer (repo) | HIGH | Txn for accept/reject/PR update |
|
||||
| Missing txn | RequestTemplateService.ts | batchConvertTemplates | HIGH | Txn (or savepoint) per cart item |
|
||||
| Missing txn | PurchaseRequestService.ts | updatePurchaseRequestStatus (completed) | HIGH | Txn or outbox for referral reward |
|
||||
| Schemaless write | paymentService.ts | createPaymentRecord | HIGH | Remove fake-ObjectId FK fallback |
|
||||
| Schemaless write | userController.ts | updateUserProfile | MEDIUM | Whitelist + jsonb \|\| merge |
|
||||
| Skip/limit pagination | PointsService.ts | collectDeliveredReferralOrders | CRITICAL | Replace loop with aggregate |
|
||||
| Skip/limit pagination | PurchaseRequestService.ts | searchPurchaseRequests | HIGH | Keyset on (created_at, id) |
|
||||
| Skip/limit pagination | SellerOfferService.ts / userController.ts | getOffersBySeller / getUsersList | MEDIUM | Keyset + cap limit 100 |
|
||||
| Virtual/hook | Chat | addMessage/markAsRead/getUnreadCount | CRITICAL | SQL ops in ChatRepository |
|
||||
| Pre-save hook | FundsLedgerEntry | immutability | HIGH | Apply trigger DDL now |
|
||||
|
||||
---
|
||||
|
||||
## Migration Completion Assessment
|
||||
|
||||
| Layer | % |
|
||||
|---|---|
|
||||
| Schema (Drizzle tables vs Mongo collections) | 90% |
|
||||
| Repository layer | 70% |
|
||||
| Backfill scripts | 85% |
|
||||
| Verification harness | 75% |
|
||||
| **Service layer (Mongo→RDBMS patterns)** | **5%** |
|
||||
| **Overall** | **~50%** |
|
||||
|
||||
### Top 5 Blockers for PG-Primary Cutover
|
||||
|
||||
1. **Service-layer rework not started + factory uncalled (CR-3)** — flag flips route zero traffic; ~50 patterns including lost-update/missing-txn money bugs
|
||||
2. **Transaction + locking defects on payment/escrow paths (CR-13–17)** — real money errors and lost updates under concurrent webhooks
|
||||
3. **Shadow read unwired (CR-2)** — soak window is blind; cutover decision would be based on no signal
|
||||
4. **Migration correctness: 0004 unjournaled + duplicate ledger DDL (CR-1)** — fresh-DB apply silently omits `funds_ledger_entries`
|
||||
5. **Money-integrity gaps + verification silent-passes (CR-4/7/8/9/10/11/12)** — corruption can occur and pass green
|
||||
|
||||
---
|
||||
|
||||
## Recommended Next Actions
|
||||
|
||||
| # | Action | Files | Effort |
|
||||
|---|---|---|---|
|
||||
| 1 | Fix 0004 journal collision | `0004_funds_ledger_entries.sql`, `_journal.json` | S |
|
||||
| 2 | Add money CHECK constraints + apply ledger TRUNCATE trigger | new migration on payments/payment_quotes/users/point_transactions/funds_ledger_entries | S |
|
||||
| 3 | Fix backfill money/integrity defects (NULL amount, schema-less fallback, non-txn idMap, `_decimal.ts`) | backfill-payments/derivedDestinations/fundsLedger/pointTransactions/purchaseRequests | M |
|
||||
| 4 | Close verification silent-passes (checksums, shadowRead, test.skip, id-map/enum/FK coverage, `--json` gate) | checksums.ts, shadowRead.ts, reconcile.ts, rowCounts.ts, migration-fk-idmap.test.ts | M |
|
||||
| 5 | Add all deferred physical FKs `NOT VALID` + `VALIDATE`; rebuild blocking indexes `CONCURRENTLY` | new migration | M |
|
||||
| 6 | Wire shadow reads into all 4 DualWrite read paths | DualWritePayment/User/Marketplace/PointsRepo | M |
|
||||
| 7 | Inject factory into services + fix money/escrow concurrency (txn + `UPDATE…WHERE…RETURNING`) | paymentCoordinator, DisputeService, SellerOfferService, marketplaceController, PurchaseRequestService | **L** |
|
||||
| 8 | Eliminate N+1 / full-fetch / skip-limit hotpaths | PointsService, searchPurchaseRequests, batchConvertTemplates, getUsersList | L |
|
||||
| 9 | Schema + backfill for unmodeled active models | Dispute (L), Notification (M), ShopSettings/ConfigSetting/Address/Telegram* (S each) | L |
|
||||
@@ -0,0 +1,872 @@
|
||||
{
|
||||
"generatedAt": "2026-05-31T14:29:51.927Z",
|
||||
"config": {
|
||||
"baseUrl": "https://dev.manwe.qzz.io",
|
||||
"sshHost": "root@5.78.213.189",
|
||||
"mongoContainer": "amanat-dev-mongodb",
|
||||
"mongoDb": "marketplace",
|
||||
"mongoAuthDb": "admin",
|
||||
"backendContainer": "amanat-dev-backend",
|
||||
"resetBackendLimiter": true,
|
||||
"containers": [
|
||||
"amanat-dev-nginx",
|
||||
"amanat-dev-backend",
|
||||
"amanat-dev-frontend",
|
||||
"amanat-dev-postgres",
|
||||
"amanat-dev-mongodb",
|
||||
"amanat-dev-redis",
|
||||
"amanat-dev-scanner"
|
||||
],
|
||||
"templateShareableLink": "logo-design-template",
|
||||
"outputDir": "/Users/manwe/CascadeProjects/escrow/nick-doc/09 - Audits/Mongo API Profiles/2026-05-31T14-26-19-969Z"
|
||||
},
|
||||
"results": [
|
||||
{
|
||||
"name": "health",
|
||||
"method": "GET",
|
||||
"path": "/api/health",
|
||||
"requestCount": 5,
|
||||
"rps": 2.5,
|
||||
"latency": {
|
||||
"averageMs": 327.2,
|
||||
"p50Ms": 233,
|
||||
"p90Ms": 707,
|
||||
"p95Ms": 707,
|
||||
"p99Ms": 707,
|
||||
"maxMs": 707
|
||||
},
|
||||
"non2xx": 0,
|
||||
"statusCodeStats": {
|
||||
"200": {
|
||||
"count": 5
|
||||
}
|
||||
},
|
||||
"mongoProfile": {
|
||||
"totalOperations": 0,
|
||||
"totalMillis": 0,
|
||||
"groups": []
|
||||
},
|
||||
"blockIoDelta": {
|
||||
"amanat-dev-nginx": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-backend": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-frontend": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-postgres": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-mongodb": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-redis": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-scanner": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 10000
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "categories",
|
||||
"method": "GET",
|
||||
"path": "/api/marketplace/categories",
|
||||
"requestCount": 10,
|
||||
"rps": 3.34,
|
||||
"latency": {
|
||||
"averageMs": 390.6,
|
||||
"p50Ms": 232,
|
||||
"p90Ms": 731,
|
||||
"p95Ms": 1308,
|
||||
"p99Ms": 1308,
|
||||
"maxMs": 1308
|
||||
},
|
||||
"non2xx": 0,
|
||||
"statusCodeStats": {
|
||||
"200": {
|
||||
"count": 10
|
||||
}
|
||||
},
|
||||
"mongoProfile": {
|
||||
"totalOperations": 0,
|
||||
"totalMillis": 0,
|
||||
"groups": []
|
||||
},
|
||||
"blockIoDelta": {
|
||||
"amanat-dev-nginx": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-backend": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-frontend": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-postgres": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-mongodb": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-redis": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-scanner": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "categories_tree",
|
||||
"method": "GET",
|
||||
"path": "/api/marketplace/categories/tree",
|
||||
"requestCount": 10,
|
||||
"rps": 5,
|
||||
"latency": {
|
||||
"averageMs": 342.5,
|
||||
"p50Ms": 240,
|
||||
"p90Ms": 742,
|
||||
"p95Ms": 752,
|
||||
"p99Ms": 752,
|
||||
"maxMs": 752
|
||||
},
|
||||
"non2xx": 0,
|
||||
"statusCodeStats": {
|
||||
"200": {
|
||||
"count": 10
|
||||
}
|
||||
},
|
||||
"mongoProfile": {
|
||||
"totalOperations": 10,
|
||||
"totalMillis": 0,
|
||||
"groups": [
|
||||
{
|
||||
"namespace": "marketplace.categories",
|
||||
"operation": "query",
|
||||
"command": "find",
|
||||
"collection": "categories",
|
||||
"planSummary": "IXSCAN { isActive: 1 }",
|
||||
"queryHash": "35A725FF",
|
||||
"planCacheKey": "80333596",
|
||||
"queryShape": "filter={isActive:boolean} sort={name:number,order:number}",
|
||||
"count": 10,
|
||||
"millisTotal": 0,
|
||||
"millisAverage": 0,
|
||||
"millisP50": 0,
|
||||
"millisP95": 0,
|
||||
"millisMax": 0,
|
||||
"docsExamined": 240,
|
||||
"keysExamined": 240,
|
||||
"nreturned": 240,
|
||||
"ninserted": 0,
|
||||
"nMatched": 0,
|
||||
"nModified": 0,
|
||||
"responseLength": 65670,
|
||||
"numYield": 0
|
||||
}
|
||||
]
|
||||
},
|
||||
"blockIoDelta": {
|
||||
"amanat-dev-nginx": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-backend": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-frontend": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-postgres": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-mongodb": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-redis": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-scanner": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "sellers",
|
||||
"method": "GET",
|
||||
"path": "/api/marketplace/sellers",
|
||||
"requestCount": 10,
|
||||
"rps": 5,
|
||||
"latency": {
|
||||
"averageMs": 341.6,
|
||||
"p50Ms": 245,
|
||||
"p90Ms": 729,
|
||||
"p95Ms": 733,
|
||||
"p99Ms": 733,
|
||||
"maxMs": 733
|
||||
},
|
||||
"non2xx": 0,
|
||||
"statusCodeStats": {
|
||||
"200": {
|
||||
"count": 10
|
||||
}
|
||||
},
|
||||
"mongoProfile": {
|
||||
"totalOperations": 10,
|
||||
"totalMillis": 0,
|
||||
"groups": [
|
||||
{
|
||||
"namespace": "marketplace.users",
|
||||
"operation": "query",
|
||||
"command": "find",
|
||||
"collection": "users",
|
||||
"planSummary": "IXSCAN { role: 1 }",
|
||||
"queryHash": "BA1E76D1",
|
||||
"planCacheKey": "0CB19E91",
|
||||
"queryShape": "filter={isEmailVerified:boolean,role:string} projection={_id:number,email:number,firstName:number,lastName:number,profile.avatar:number}",
|
||||
"count": 10,
|
||||
"millisTotal": 0,
|
||||
"millisAverage": 0,
|
||||
"millisP50": 0,
|
||||
"millisP95": 0,
|
||||
"millisMax": 0,
|
||||
"docsExamined": 20,
|
||||
"keysExamined": 20,
|
||||
"nreturned": 20,
|
||||
"ninserted": 0,
|
||||
"nMatched": 0,
|
||||
"nModified": 0,
|
||||
"responseLength": 3610,
|
||||
"numYield": 0
|
||||
}
|
||||
]
|
||||
},
|
||||
"blockIoDelta": {
|
||||
"amanat-dev-nginx": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-backend": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-frontend": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-postgres": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-mongodb": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-redis": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-scanner": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 20000
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "template_public",
|
||||
"method": "GET",
|
||||
"path": "/api/marketplace/request-templates/public/logo-design-template",
|
||||
"requestCount": 10,
|
||||
"rps": 5,
|
||||
"latency": {
|
||||
"averageMs": 340.3,
|
||||
"p50Ms": 241,
|
||||
"p90Ms": 734,
|
||||
"p95Ms": 740,
|
||||
"p99Ms": 740,
|
||||
"maxMs": 740
|
||||
},
|
||||
"non2xx": 0,
|
||||
"statusCodeStats": {
|
||||
"200": {
|
||||
"count": 10
|
||||
}
|
||||
},
|
||||
"mongoProfile": {
|
||||
"totalOperations": 30,
|
||||
"totalMillis": 0,
|
||||
"groups": [
|
||||
{
|
||||
"namespace": "marketplace.requesttemplates",
|
||||
"operation": "query",
|
||||
"command": "find",
|
||||
"collection": "requesttemplates",
|
||||
"planSummary": "IXSCAN { shareableLink: 1 }",
|
||||
"queryHash": "69A943C9",
|
||||
"planCacheKey": "7C668FB5",
|
||||
"queryShape": "filter={$or:[{expiresAt:null},{expiresAt:{$gt:{}}}],isActive:boolean,shareableLink:string}",
|
||||
"count": 10,
|
||||
"millisTotal": 0,
|
||||
"millisAverage": 0,
|
||||
"millisP50": 0,
|
||||
"millisP95": 0,
|
||||
"millisMax": 0,
|
||||
"docsExamined": 10,
|
||||
"keysExamined": 10,
|
||||
"nreturned": 10,
|
||||
"ninserted": 0,
|
||||
"nMatched": 0,
|
||||
"nModified": 0,
|
||||
"responseLength": 15470,
|
||||
"numYield": 0
|
||||
},
|
||||
{
|
||||
"namespace": "marketplace.users",
|
||||
"operation": "query",
|
||||
"command": "find",
|
||||
"collection": "users",
|
||||
"planSummary": "IXSCAN { _id: 1 }",
|
||||
"queryHash": "39E03FF8",
|
||||
"planCacheKey": "AED36A0D",
|
||||
"queryShape": "filter={_id:{$in:[ObjectId]}} projection={email:number,firstName:number,lastName:number}",
|
||||
"count": 10,
|
||||
"millisTotal": 0,
|
||||
"millisAverage": 0,
|
||||
"millisP50": 0,
|
||||
"millisP95": 0,
|
||||
"millisMax": 0,
|
||||
"docsExamined": 10,
|
||||
"keysExamined": 10,
|
||||
"nreturned": 10,
|
||||
"ninserted": 0,
|
||||
"nMatched": 0,
|
||||
"nModified": 0,
|
||||
"responseLength": 2180,
|
||||
"numYield": 0
|
||||
},
|
||||
{
|
||||
"namespace": "marketplace.categories",
|
||||
"operation": "query",
|
||||
"command": "find",
|
||||
"collection": "categories",
|
||||
"planSummary": "IXSCAN { _id: 1 }",
|
||||
"queryHash": "ABAD6477",
|
||||
"planCacheKey": "E494D204",
|
||||
"queryShape": "filter={_id:{$in:[ObjectId]}} projection={name:number,nameEn:number}",
|
||||
"count": 10,
|
||||
"millisTotal": 0,
|
||||
"millisAverage": 0,
|
||||
"millisP50": 0,
|
||||
"millisP95": 0,
|
||||
"millisMax": 0,
|
||||
"docsExamined": 10,
|
||||
"keysExamined": 10,
|
||||
"nreturned": 10,
|
||||
"ninserted": 0,
|
||||
"nMatched": 0,
|
||||
"nModified": 0,
|
||||
"responseLength": 1890,
|
||||
"numYield": 0
|
||||
}
|
||||
]
|
||||
},
|
||||
"blockIoDelta": {
|
||||
"amanat-dev-nginx": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-backend": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-frontend": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-postgres": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-mongodb": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-redis": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-scanner": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "payment_options_template",
|
||||
"method": "GET",
|
||||
"path": "/api/payment/request-network/options?currency=USD&amount=0.01&sellerId=6a1bfd1400e8b8205e86db9e&templateId=6a1c4512d07eb576c3509690",
|
||||
"requestCount": 50,
|
||||
"rps": 12.5,
|
||||
"latency": {
|
||||
"averageMs": 303.52,
|
||||
"p50Ms": 255,
|
||||
"p90Ms": 273,
|
||||
"p95Ms": 753,
|
||||
"p99Ms": 758,
|
||||
"maxMs": 758
|
||||
},
|
||||
"non2xx": 0,
|
||||
"statusCodeStats": {
|
||||
"200": {
|
||||
"count": 50
|
||||
}
|
||||
},
|
||||
"mongoProfile": {
|
||||
"totalOperations": 100,
|
||||
"totalMillis": 0,
|
||||
"groups": [
|
||||
{
|
||||
"namespace": "marketplace.requesttemplates",
|
||||
"operation": "query",
|
||||
"command": "find",
|
||||
"collection": "requesttemplates",
|
||||
"planSummary": "IDHACK",
|
||||
"queryHash": "3B008735",
|
||||
"planCacheKey": "",
|
||||
"queryShape": "filter={_id:ObjectId} projection={paymentConfig:number}",
|
||||
"count": 50,
|
||||
"millisTotal": 0,
|
||||
"millisAverage": 0,
|
||||
"millisP50": 0,
|
||||
"millisP95": 0,
|
||||
"millisMax": 0,
|
||||
"docsExamined": 50,
|
||||
"keysExamined": 50,
|
||||
"nreturned": 50,
|
||||
"ninserted": 0,
|
||||
"nMatched": 0,
|
||||
"nModified": 0,
|
||||
"responseLength": 12850,
|
||||
"numYield": 0
|
||||
},
|
||||
{
|
||||
"namespace": "marketplace.shopsettings",
|
||||
"operation": "query",
|
||||
"command": "find",
|
||||
"collection": "shopsettings",
|
||||
"planSummary": "IXSCAN { sellerId: 1 }",
|
||||
"queryHash": "BF51CF8A",
|
||||
"planCacheKey": "9CF87C58",
|
||||
"queryShape": "filter={sellerId:ObjectId} projection={paymentConfig:number}",
|
||||
"count": 50,
|
||||
"millisTotal": 0,
|
||||
"millisAverage": 0,
|
||||
"millisP50": 0,
|
||||
"millisP95": 0,
|
||||
"millisMax": 0,
|
||||
"docsExamined": 0,
|
||||
"keysExamined": 0,
|
||||
"nreturned": 0,
|
||||
"ninserted": 0,
|
||||
"nMatched": 0,
|
||||
"nModified": 0,
|
||||
"responseLength": 5650,
|
||||
"numYield": 0
|
||||
}
|
||||
]
|
||||
},
|
||||
"blockIoDelta": {
|
||||
"amanat-dev-nginx": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-backend": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-frontend": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-postgres": {
|
||||
"readBytes": 100000,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-mongodb": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-redis": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-scanner": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "addresses_me",
|
||||
"method": "GET",
|
||||
"path": "/api/addresses",
|
||||
"requestCount": 10,
|
||||
"rps": 5,
|
||||
"latency": {
|
||||
"averageMs": 330.9,
|
||||
"p50Ms": 239,
|
||||
"p90Ms": 707,
|
||||
"p95Ms": 715,
|
||||
"p99Ms": 715,
|
||||
"maxMs": 715
|
||||
},
|
||||
"non2xx": 0,
|
||||
"statusCodeStats": {
|
||||
"200": {
|
||||
"count": 10
|
||||
}
|
||||
},
|
||||
"mongoProfile": {
|
||||
"totalOperations": 10,
|
||||
"totalMillis": 0,
|
||||
"groups": [
|
||||
{
|
||||
"namespace": "marketplace.addresses",
|
||||
"operation": "query",
|
||||
"command": "find",
|
||||
"collection": "addresses",
|
||||
"planSummary": "IXSCAN { userId: 1 }",
|
||||
"queryHash": "6935090D",
|
||||
"planCacheKey": "C80BED60",
|
||||
"queryShape": "filter={userId:ObjectId} sort={createdAt:number,primary:number}",
|
||||
"count": 10,
|
||||
"millisTotal": 0,
|
||||
"millisAverage": 0,
|
||||
"millisP50": 0,
|
||||
"millisP95": 0,
|
||||
"millisMax": 0,
|
||||
"docsExamined": 30,
|
||||
"keysExamined": 30,
|
||||
"nreturned": 30,
|
||||
"ninserted": 0,
|
||||
"nMatched": 0,
|
||||
"nModified": 0,
|
||||
"responseLength": 13800,
|
||||
"numYield": 0
|
||||
}
|
||||
]
|
||||
},
|
||||
"blockIoDelta": {
|
||||
"amanat-dev-nginx": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-backend": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-frontend": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-postgres": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-mongodb": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-redis": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-scanner": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "purchase_requests_my",
|
||||
"method": "GET",
|
||||
"path": "/api/marketplace/purchase-requests/my",
|
||||
"requestCount": 10,
|
||||
"rps": 5,
|
||||
"latency": {
|
||||
"averageMs": 353.3,
|
||||
"p50Ms": 256,
|
||||
"p90Ms": 747,
|
||||
"p95Ms": 753,
|
||||
"p99Ms": 753,
|
||||
"maxMs": 753
|
||||
},
|
||||
"non2xx": 0,
|
||||
"statusCodeStats": {
|
||||
"200": {
|
||||
"count": 10
|
||||
}
|
||||
},
|
||||
"mongoProfile": {
|
||||
"totalOperations": 30,
|
||||
"totalMillis": 1,
|
||||
"groups": [
|
||||
{
|
||||
"namespace": "marketplace.purchaserequests",
|
||||
"operation": "query",
|
||||
"command": "find",
|
||||
"collection": "purchaserequests",
|
||||
"planSummary": "IXSCAN { createdAt: -1 }",
|
||||
"queryHash": "6F3C3F41",
|
||||
"planCacheKey": "A22CDD0E",
|
||||
"queryShape": "filter={buyerId:ObjectId} sort={createdAt:number}",
|
||||
"count": 10,
|
||||
"millisTotal": 1,
|
||||
"millisAverage": 0.1,
|
||||
"millisP50": 0,
|
||||
"millisP95": 1,
|
||||
"millisMax": 1,
|
||||
"docsExamined": 0,
|
||||
"keysExamined": 0,
|
||||
"nreturned": 0,
|
||||
"ninserted": 0,
|
||||
"nMatched": 0,
|
||||
"nModified": 0,
|
||||
"responseLength": 1170,
|
||||
"numYield": 0
|
||||
},
|
||||
{
|
||||
"namespace": "marketplace.purchaserequests",
|
||||
"operation": "command",
|
||||
"command": "aggregate",
|
||||
"collection": "purchaserequests",
|
||||
"planSummary": "COUNT_SCAN { buyerId: 1 }",
|
||||
"queryHash": "C22625EF",
|
||||
"planCacheKey": "BD75157B",
|
||||
"queryShape": "pipeline=[{$match:{buyerId:ObjectId}},{$group:{_id:number,n:{$sum:number}}}]",
|
||||
"count": 10,
|
||||
"millisTotal": 0,
|
||||
"millisAverage": 0,
|
||||
"millisP50": 0,
|
||||
"millisP95": 0,
|
||||
"millisMax": 0,
|
||||
"docsExamined": 0,
|
||||
"keysExamined": 10,
|
||||
"nreturned": 0,
|
||||
"ninserted": 0,
|
||||
"nMatched": 0,
|
||||
"nModified": 0,
|
||||
"responseLength": 1170,
|
||||
"numYield": 0
|
||||
},
|
||||
{
|
||||
"namespace": "marketplace.payments",
|
||||
"operation": "query",
|
||||
"command": "find",
|
||||
"collection": "payments",
|
||||
"planSummary": "IXSCAN { status: 1, createdAt: -1 }, IXSCAN { status: 1, createdAt: -1 }, IXSCAN { status: 1, createdAt: -1 }, IXSCAN { status: 1, createdAt: -1 }",
|
||||
"queryHash": "3B29FB2B",
|
||||
"planCacheKey": "8762DEE5",
|
||||
"queryShape": "filter={purchaseRequestId:{$in:[]},status:{$in:[string,string,string,string]}} sort={createdAt:number}",
|
||||
"count": 10,
|
||||
"millisTotal": 0,
|
||||
"millisAverage": 0,
|
||||
"millisP50": 0,
|
||||
"millisP95": 0,
|
||||
"millisMax": 0,
|
||||
"docsExamined": 0,
|
||||
"keysExamined": 0,
|
||||
"nreturned": 0,
|
||||
"ninserted": 0,
|
||||
"nMatched": 0,
|
||||
"nModified": 0,
|
||||
"responseLength": 1090,
|
||||
"numYield": 0
|
||||
}
|
||||
]
|
||||
},
|
||||
"blockIoDelta": {
|
||||
"amanat-dev-nginx": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-backend": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-frontend": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-postgres": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-mongodb": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-redis": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-scanner": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 20000
|
||||
}
|
||||
}
|
||||
},
|
||||
{
|
||||
"name": "auth_login",
|
||||
"method": "POST",
|
||||
"path": "/api/auth/login",
|
||||
"requestCount": 5,
|
||||
"rps": 1.25,
|
||||
"latency": {
|
||||
"averageMs": 724.2,
|
||||
"p50Ms": 636,
|
||||
"p90Ms": 1090,
|
||||
"p95Ms": 1090,
|
||||
"p99Ms": 1090,
|
||||
"maxMs": 1090
|
||||
},
|
||||
"non2xx": 0,
|
||||
"statusCodeStats": {
|
||||
"200": {
|
||||
"count": 5
|
||||
}
|
||||
},
|
||||
"mongoProfile": {
|
||||
"totalOperations": 15,
|
||||
"totalMillis": 0,
|
||||
"groups": [
|
||||
{
|
||||
"namespace": "marketplace.users",
|
||||
"operation": "query",
|
||||
"command": "find",
|
||||
"collection": "users",
|
||||
"planSummary": "IXSCAN { email: 1 }",
|
||||
"queryHash": "106ECB7C",
|
||||
"planCacheKey": "AB4716E0",
|
||||
"queryShape": "filter={email:string,status:string}",
|
||||
"count": 5,
|
||||
"millisTotal": 0,
|
||||
"millisAverage": 0,
|
||||
"millisP50": 0,
|
||||
"millisP95": 0,
|
||||
"millisMax": 0,
|
||||
"docsExamined": 5,
|
||||
"keysExamined": 5,
|
||||
"nreturned": 5,
|
||||
"ninserted": 0,
|
||||
"nMatched": 0,
|
||||
"nModified": 0,
|
||||
"responseLength": 17735,
|
||||
"numYield": 0
|
||||
},
|
||||
{
|
||||
"namespace": "marketplace.users",
|
||||
"operation": "update",
|
||||
"command": "q",
|
||||
"collection": "users",
|
||||
"planSummary": "IDHACK",
|
||||
"queryHash": "",
|
||||
"planCacheKey": "",
|
||||
"queryShape": "",
|
||||
"count": 5,
|
||||
"millisTotal": 0,
|
||||
"millisAverage": 0,
|
||||
"millisP50": 0,
|
||||
"millisP95": 0,
|
||||
"millisMax": 0,
|
||||
"docsExamined": 5,
|
||||
"keysExamined": 5,
|
||||
"nreturned": 0,
|
||||
"ninserted": 0,
|
||||
"nMatched": 5,
|
||||
"nModified": 5,
|
||||
"responseLength": 0,
|
||||
"numYield": 0
|
||||
},
|
||||
{
|
||||
"namespace": "marketplace.users",
|
||||
"operation": "update",
|
||||
"command": "q",
|
||||
"collection": "users",
|
||||
"planSummary": "IXSCAN { _id: 1 }",
|
||||
"queryHash": "E515C562",
|
||||
"planCacheKey": "5EA96075",
|
||||
"queryShape": "",
|
||||
"count": 5,
|
||||
"millisTotal": 0,
|
||||
"millisAverage": 0,
|
||||
"millisP50": 0,
|
||||
"millisP95": 0,
|
||||
"millisMax": 0,
|
||||
"docsExamined": 5,
|
||||
"keysExamined": 5,
|
||||
"nreturned": 0,
|
||||
"ninserted": 0,
|
||||
"nMatched": 5,
|
||||
"nModified": 5,
|
||||
"responseLength": 0,
|
||||
"numYield": 0
|
||||
}
|
||||
]
|
||||
},
|
||||
"blockIoDelta": {
|
||||
"amanat-dev-nginx": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-backend": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-frontend": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-postgres": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-mongodb": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-redis": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
},
|
||||
"amanat-dev-scanner": {
|
||||
"readBytes": 0,
|
||||
"writeBytes": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,119 @@
|
||||
# Mongo API Query Profile
|
||||
|
||||
Generated: 2026-05-31T14:29:51.927Z
|
||||
Base URL: `https://dev.manwe.qzz.io`
|
||||
Mongo: `amanat-dev-mongodb/marketplace`
|
||||
|
||||
This is a query-shape profile, not a max-throughput test. Request counts are intentionally small so the backend rate limiter does not dominate the profile.
|
||||
|
||||
## Endpoint Summary
|
||||
|
||||
| Endpoint | Requests | Avg | P95 | P99 | Non-2xx | Mongo ops | Top Mongo query |
|
||||
|---|---:|---:|---:|---:|---:|---:|---|
|
||||
| `GET /api/health` | 5 | 327.2ms | 707ms | 707ms | 0 | 0 | - |
|
||||
| `GET /api/marketplace/categories` | 10 | 390.6ms | 1308ms | 1308ms | 0 | 0 | - |
|
||||
| `GET /api/marketplace/categories/tree` | 10 | 342.5ms | 752ms | 752ms | 0 | 10 | `categories` find (10x, IXSCAN { isActive: 1 }) |
|
||||
| `GET /api/marketplace/sellers` | 10 | 341.6ms | 733ms | 733ms | 0 | 10 | `users` find (10x, IXSCAN { role: 1 }) |
|
||||
| `GET /api/marketplace/request-templates/public/logo-design-template` | 10 | 340.3ms | 740ms | 740ms | 0 | 30 | `requesttemplates` find (10x, IXSCAN { shareableLink: 1 }) |
|
||||
| `GET /api/payment/request-network/options?currency=USD&amount=0.01&sellerId=6a1bfd1400e8b8205e86db9e&templateId=6a1c4512d07eb576c3509690` | 50 | 303.52ms | 753ms | 758ms | 0 | 100 | `requesttemplates` find (50x, IDHACK) |
|
||||
| `GET /api/addresses` | 10 | 330.9ms | 715ms | 715ms | 0 | 10 | `addresses` find (10x, IXSCAN { userId: 1 }) |
|
||||
| `GET /api/marketplace/purchase-requests/my` | 10 | 353.3ms | 753ms | 753ms | 0 | 30 | `purchaserequests` find (10x, IXSCAN { createdAt: -1 }) |
|
||||
| `POST /api/auth/login` | 5 | 724.2ms | 1090ms | 1090ms | 0 | 15 | `users` find (5x, IXSCAN { email: 1 }) |
|
||||
|
||||
## Query Groups
|
||||
|
||||
### health
|
||||
|
||||
Path: `GET /api/health`
|
||||
Status codes: `{"200":{"count":5}}`
|
||||
|
||||
No Mongo operations captured in this endpoint window.
|
||||
|
||||
### categories
|
||||
|
||||
Path: `GET /api/marketplace/categories`
|
||||
Status codes: `{"200":{"count":10}}`
|
||||
|
||||
No Mongo operations captured in this endpoint window.
|
||||
|
||||
### categories_tree
|
||||
|
||||
Path: `GET /api/marketplace/categories/tree`
|
||||
Status codes: `{"200":{"count":10}}`
|
||||
|
||||
| Collection | Command | Count | Total ms | Avg ms | P95 ms | Plan | Docs | Keys | Returned | Shape |
|
||||
|---|---|---:|---:|---:|---:|---|---:|---:|---:|---|
|
||||
| `categories` | `find` | 10 | 0 | 0 | 0 | `IXSCAN { isActive: 1 }` | 240 | 240 | 240 | `filter={isActive:boolean} sort={name:number,order:number}` |
|
||||
|
||||
### sellers
|
||||
|
||||
Path: `GET /api/marketplace/sellers`
|
||||
Status codes: `{"200":{"count":10}}`
|
||||
|
||||
| Collection | Command | Count | Total ms | Avg ms | P95 ms | Plan | Docs | Keys | Returned | Shape |
|
||||
|---|---|---:|---:|---:|---:|---|---:|---:|---:|---|
|
||||
| `users` | `find` | 10 | 0 | 0 | 0 | `IXSCAN { role: 1 }` | 20 | 20 | 20 | `filter={isEmailVerified:boolean,role:string} projection={_id:number,email:number,firstName:number,lastName:number,profile.avatar:number}` |
|
||||
|
||||
### template_public
|
||||
|
||||
Path: `GET /api/marketplace/request-templates/public/logo-design-template`
|
||||
Status codes: `{"200":{"count":10}}`
|
||||
|
||||
| Collection | Command | Count | Total ms | Avg ms | P95 ms | Plan | Docs | Keys | Returned | Shape |
|
||||
|---|---|---:|---:|---:|---:|---|---:|---:|---:|---|
|
||||
| `requesttemplates` | `find` | 10 | 0 | 0 | 0 | `IXSCAN { shareableLink: 1 }` | 10 | 10 | 10 | `filter={$or:[{expiresAt:null},{expiresAt:{$gt:{}}}],isActive:boolean,shareableLink:string}` |
|
||||
| `users` | `find` | 10 | 0 | 0 | 0 | `IXSCAN { _id: 1 }` | 10 | 10 | 10 | `filter={_id:{$in:[ObjectId]}} projection={email:number,firstName:number,lastName:number}` |
|
||||
| `categories` | `find` | 10 | 0 | 0 | 0 | `IXSCAN { _id: 1 }` | 10 | 10 | 10 | `filter={_id:{$in:[ObjectId]}} projection={name:number,nameEn:number}` |
|
||||
|
||||
### payment_options_template
|
||||
|
||||
Path: `GET /api/payment/request-network/options?currency=USD&amount=0.01&sellerId=6a1bfd1400e8b8205e86db9e&templateId=6a1c4512d07eb576c3509690`
|
||||
Status codes: `{"200":{"count":50}}`
|
||||
|
||||
| Collection | Command | Count | Total ms | Avg ms | P95 ms | Plan | Docs | Keys | Returned | Shape |
|
||||
|---|---|---:|---:|---:|---:|---|---:|---:|---:|---|
|
||||
| `requesttemplates` | `find` | 50 | 0 | 0 | 0 | `IDHACK` | 50 | 50 | 50 | `filter={_id:ObjectId} projection={paymentConfig:number}` |
|
||||
| `shopsettings` | `find` | 50 | 0 | 0 | 0 | `IXSCAN { sellerId: 1 }` | 0 | 0 | 0 | `filter={sellerId:ObjectId} projection={paymentConfig:number}` |
|
||||
|
||||
### addresses_me
|
||||
|
||||
Path: `GET /api/addresses`
|
||||
Status codes: `{"200":{"count":10}}`
|
||||
|
||||
| Collection | Command | Count | Total ms | Avg ms | P95 ms | Plan | Docs | Keys | Returned | Shape |
|
||||
|---|---|---:|---:|---:|---:|---|---:|---:|---:|---|
|
||||
| `addresses` | `find` | 10 | 0 | 0 | 0 | `IXSCAN { userId: 1 }` | 30 | 30 | 30 | `filter={userId:ObjectId} sort={createdAt:number,primary:number}` |
|
||||
|
||||
### purchase_requests_my
|
||||
|
||||
Path: `GET /api/marketplace/purchase-requests/my`
|
||||
Status codes: `{"200":{"count":10}}`
|
||||
|
||||
| Collection | Command | Count | Total ms | Avg ms | P95 ms | Plan | Docs | Keys | Returned | Shape |
|
||||
|---|---|---:|---:|---:|---:|---|---:|---:|---:|---|
|
||||
| `purchaserequests` | `find` | 10 | 1 | 0.1 | 1 | `IXSCAN { createdAt: -1 }` | 0 | 0 | 0 | `filter={buyerId:ObjectId} sort={createdAt:number}` |
|
||||
| `purchaserequests` | `aggregate` | 10 | 0 | 0 | 0 | `COUNT_SCAN { buyerId: 1 }` | 0 | 10 | 0 | `pipeline=[{$match:{buyerId:ObjectId}},{$group:{_id:number,n:{$sum:number}}}]` |
|
||||
| `payments` | `find` | 10 | 0 | 0 | 0 | `IXSCAN { status: 1, createdAt: -1 }, IXSCAN { status: 1, createdAt: -1 }, IXSCAN { status: 1, createdAt: -1 }, IXSCAN { status: 1, createdAt: -1 }` | 0 | 0 | 0 | `filter={purchaseRequestId:{$in:[]},status:{$in:[string,string,string,string]}} sort={createdAt:number}` |
|
||||
|
||||
### auth_login
|
||||
|
||||
Path: `POST /api/auth/login`
|
||||
Status codes: `{"200":{"count":5}}`
|
||||
|
||||
| Collection | Command | Count | Total ms | Avg ms | P95 ms | Plan | Docs | Keys | Returned | Shape |
|
||||
|---|---|---:|---:|---:|---:|---|---:|---:|---:|---|
|
||||
| `users` | `find` | 5 | 0 | 0 | 0 | `IXSCAN { email: 1 }` | 5 | 5 | 5 | `filter={email:string,status:string}` |
|
||||
| `users` | `q` | 5 | 0 | 0 | 0 | `IDHACK` | 5 | 5 | 0 | `-` |
|
||||
| `users` | `q` | 5 | 0 | 0 | 0 | `IXSCAN { _id: 1 }` | 5 | 5 | 0 | `-` |
|
||||
|
||||
## Block I/O Deltas
|
||||
- health: amanat-dev-scanner: read 0 B, write 10 KB
|
||||
- categories: no container block I/O delta
|
||||
- categories_tree: no container block I/O delta
|
||||
- sellers: amanat-dev-scanner: read 0 B, write 20 KB
|
||||
- template_public: no container block I/O delta
|
||||
- payment_options_template: amanat-dev-postgres: read 100 KB, write 0 B
|
||||
- addresses_me: no container block I/O delta
|
||||
- purchase_requests_my: amanat-dev-scanner: read 0 B, write 20 KB
|
||||
- auth_login: no container block I/O delta
|
||||
|
||||
Reference in New Issue
Block a user