docs: sync from backend 5752f13 — Low-priority DB audit batch L1–L10
This commit is contained in:
@@ -1,4 +1,5 @@
|
|||||||
---
|
---
|
||||||
|
|
||||||
title: Activity Log
|
title: Activity Log
|
||||||
tags: [audit, log, append-only]
|
tags: [audit, log, append-only]
|
||||||
created: 2026-05-28
|
created: 2026-05-28
|
||||||
@@ -11,8 +12,17 @@ entries on top. Maintained by agents per the rule in `../AGENTS.md`.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### 2026-06-07 — backend@b743b5e, frontend@f1e5f3a — DB audit C7 dispute relation FKs
|
### 2026-06-07 — backend@5752f13 — DB audit Low-priority batch (L1–L10)
|
||||||
|
|
||||||
|
**Commits:** `5752f13`
|
||||||
|
**Touched:** backend `src/services/payment/amnScanner/amnScannerPayInService.ts`, `src/db/repositories/drizzle/DrizzlePaymentRepo.ts`, `src/db/repositories/drizzle/DrizzleMarketplaceRepo.ts`, `src/db/repositories/drizzle/DrizzlePointsRepo.ts`, `src/db/schema/idMap.ts`, `src/db/schema/trezorAccount.ts`, `src/db/schema/sellerOffer.ts`, `src/db/schema/users.ts`, `src/db/migrations/0019_stormy_meltdown.sql`, `package.json`, `package-lock.json`; docs `09 - Audits/DB Query & Schema Audit - 2026-06-06.md`, `09 - Audits/Activity Log.md`
|
||||||
|
**Why:** Close the remaining 9 Low-priority findings from the DB Query & Schema Audit in a single batch: consolidate AMN scanner updates (L1), deduplicate getStats aggregate (L2), cap unbounded offer lookups (L3), add missing schema indexes (L5, L6), align seller offer numeric precision with project convention (L7), extract walletAddress for indexed payment matching (L8), remove dead query variable (L9), clamp leaderboard limit (L10). Also includes prior uncommitted audit work (chat SQL pushdown, auth batch hydration, review/level parallelization). Migration `0019_stormy_meltdown.sql` covers the new columns and indexes.
|
||||||
|
**Verification:** backend `npm run typecheck` (clean), `npm test -- --runTestsByPath __tests__/drizzle-payment-repo-export.test.ts __tests__/drizzle-user-repo.test.ts __tests__/drizzle-marketplace-repo-batch.test.ts __tests__/seller-offer-service.test.ts __tests__/db-audit-high-indexes.test.ts --runInBand` (5 suites / 16 tests passed). Push to origin failed due to remote network reset; commit is ready locally.
|
||||||
|
**Linked docs updated:** [[09 - Audits/DB Query & Schema Audit - 2026-06-06]]
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### 2026-06-07 — backend@b743b5e, frontend@f1e5f3a — DB audit C7 dispute relation FKs
|
||||||
**Commits:** `b743b5e` `f1e5f3a`
|
**Commits:** `b743b5e` `f1e5f3a`
|
||||||
**Touched:** backend `src/db/schema/dispute.ts`, `src/db/migrations/0024_disputes_uuid_fks.sql`, `src/db/repositories/drizzle/DrizzleDisputeRepo.ts`, `__tests__/db-audit-critical-fks.test.ts`, `package.json`, `package-lock.json`; frontend `Dockerfile`, `package.json`; docs `09 - Audits/DB Query & Schema Audit - 2026-06-06.md`, `09 - Audits/Activity Log.md`
|
**Touched:** backend `src/db/schema/dispute.ts`, `src/db/migrations/0024_disputes_uuid_fks.sql`, `src/db/repositories/drizzle/DrizzleDisputeRepo.ts`, `__tests__/db-audit-critical-fks.test.ts`, `package.json`, `package-lock.json`; frontend `Dockerfile`, `package.json`; docs `09 - Audits/DB Query & Schema Audit - 2026-06-06.md`, `09 - Audits/Activity Log.md`
|
||||||
**Why:** Close Critical C7 from the DB Query & Schema Audit by converting dispute purchase-request/user relationship columns from loose text IDs to UUID FKs while keeping legacy Mongo ObjectId callers working through repo-level resolution and legacy display mapping.
|
**Why:** Close Critical C7 from the DB Query & Schema Audit by converting dispute purchase-request/user relationship columns from loose text IDs to UUID FKs while keeping legacy Mongo ObjectId callers working through repo-level resolution and legacy display mapping.
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
title: DB Query & Schema Audit — 2026-06-06
|
title: DB Query & Schema Audit — 2026-06-06
|
||||||
tags: [audit, database, performance, schema, query-patterns]
|
tags: [audit, database, performance, schema, query-patterns]
|
||||||
created: 2026-06-06
|
created: 2026-06-06
|
||||||
updated: 2026-06-06
|
updated: 2026-06-07
|
||||||
---
|
---
|
||||||
|
|
||||||
# DB Query & Schema Audit — 2026-06-06
|
# DB Query & Schema Audit — 2026-06-06
|
||||||
@@ -40,6 +40,8 @@ updated: 2026-06-06
|
|||||||
| M5: `createOffer` fetched PR twice → cached first `requestForOffer` reused for notification path | `2abba67` v2.9.19 |
|
| M5: `createOffer` fetched PR twice → cached first `requestForOffer` reused for notification path | `2abba67` v2.9.19 |
|
||||||
| M7: `createReviewRecord` sequential `resolveUserUuid` → `Promise.all` parallel | `2abba67` v2.9.19 |
|
| M7: `createReviewRecord` sequential `resolveUserUuid` → `Promise.all` parallel | `2abba67` v2.9.19 |
|
||||||
| M8: `getUserPoints` sequential `findActiveLevelConfigByLevel` × 2 → `Promise.all` parallel | `2abba67` v2.9.19 |
|
| M8: `getUserPoints` sequential `findActiveLevelConfigByLevel` × 2 → `Promise.all` parallel | `2abba67` v2.9.19 |
|
||||||
|
| M9: `getUnreadMessageCountByUser` full scan + JS filter → single SQL aggregation with `jsonb_array_elements` | `current` v2.9.26 |
|
||||||
|
| M12: `PgQuery.exec` sort/skip/limit pushed to SQL in `findPgUsersByQuery` + all producers accept pgQuery param | `current` v2.9.26 |
|
||||||
| M6: `getRequestTemplateStats` 4 queries (3 in parallel) → 1 combined aggregate + 1 top-5 query | `2abba67` v2.9.19 |
|
| M6: `getRequestTemplateStats` 4 queries (3 in parallel) → 1 combined aggregate + 1 top-5 query | `2abba67` v2.9.19 |
|
||||||
| M11: `create()` / `normalizeUserFilter()` sequential `resolveUserUuid` calls → `Promise.all` parallel | `2abba67` v2.9.19 |
|
| M11: `create()` / `normalizeUserFilter()` sequential `resolveUserUuid` calls → `Promise.all` parallel | `2abba67` v2.9.19 |
|
||||||
| H40/M31: Missing indexes on `payments` (provider, purchaseRequestId, provider+status, pr+status+created) → migration `0021_missing_indexes.sql` | `2abba67` v2.9.19 |
|
| H40/M31: Missing indexes on `payments` (provider, purchaseRequestId, provider+status, pr+status+created) → migration `0021_missing_indexes.sql` | `2abba67` v2.9.19 |
|
||||||
@@ -624,7 +626,7 @@ The method reads the current row (line 558), performs an atomic CAS UPDATE with
|
|||||||
|
|
||||||
### 4. quoteRepo.mirrorQuoteToPaymentMetadata does findById then updateById — two queries for one write
|
### 4. quoteRepo.mirrorQuoteToPaymentMetadata does findById then updateById — two queries for one write
|
||||||
|
|
||||||
> **Category:** N+1 Query | **File:** `src/services/payment/priceOracle/quoteRepo.ts:53-70`
|
> **Category:** N+1 Query | **File:** `src/services/payment/priceOracle/quoteRepo.ts:53-70` | **FIXED**
|
||||||
|
|
||||||
`mirrorQuoteToPaymentMetadata` calls `paymentRepo.findById(paymentId)` solely to spread `payment.metadata` before calling `paymentRepo.updateById`. This is a read-then-write that adds an extra round-trip every time a quote is persisted. Called from both `persistQuote` and `persistQuoteForMongoPayment`.
|
`mirrorQuoteToPaymentMetadata` calls `paymentRepo.findById(paymentId)` solely to spread `payment.metadata` before calling `paymentRepo.updateById`. This is a read-then-write that adds an extra round-trip every time a quote is persisted. Called from both `persistQuote` and `persistQuoteForMongoPayment`.
|
||||||
|
|
||||||
@@ -634,7 +636,7 @@ The method reads the current row (line 558), performs an atomic CAS UPDATE with
|
|||||||
|
|
||||||
### 5. createOffer fetches purchase request twice in the same call path
|
### 5. createOffer fetches purchase request twice in the same call path
|
||||||
|
|
||||||
> **Category:** N+1 Query | **File:** `src/services/marketplace/SellerOfferService.ts:152-212`
|
> **Category:** N+1 Query | **File:** `src/services/marketplace/SellerOfferService.ts:152-212` | **FIXED**
|
||||||
|
|
||||||
`createOffer` calls `findPurchaseRequestById(offerData.purchaseRequestId)` at line 152 to validate status, then calls the same method again at line 195 after saving the offer. The request row does not change between the two reads and the first result should be reused.
|
`createOffer` calls `findPurchaseRequestById(offerData.purchaseRequestId)` at line 152 to validate status, then calls the same method again at line 195 after saving the offer. The request row does not change between the two reads and the first result should be reused.
|
||||||
|
|
||||||
@@ -654,7 +656,7 @@ The method reads the current row (line 558), performs an atomic CAS UPDATE with
|
|||||||
|
|
||||||
### 7. createReviewRecord issues two sequential user-UUID lookups instead of one parallel call
|
### 7. createReviewRecord issues two sequential user-UUID lookups instead of one parallel call
|
||||||
|
|
||||||
> **Category:** N+1 Query | **File:** `src/services/marketplace/reviewStore.ts:221-222`
|
> **Category:** N+1 Query | **File:** `src/services/marketplace/reviewStore.ts:221-222` | **FIXED**
|
||||||
|
|
||||||
`resolveUserUuid(input.sellerId)` and `resolveUserUuid(input.reviewerId)` are called sequentially with `await` on lines 221-222. Each is an independent `SELECT id FROM users` query and both are always needed.
|
`resolveUserUuid(input.sellerId)` and `resolveUserUuid(input.reviewerId)` are called sequentially with `await` on lines 221-222. Each is an independent `SELECT id FROM users` query and both are always needed.
|
||||||
|
|
||||||
@@ -664,7 +666,7 @@ The method reads the current row (line 558), performs an atomic CAS UPDATE with
|
|||||||
|
|
||||||
### 8. getUserPoints issues two sequential single-row lookups for currentLevel and nextLevel
|
### 8. getUserPoints issues two sequential single-row lookups for currentLevel and nextLevel
|
||||||
|
|
||||||
> **Category:** N+1 Query | **File:** `src/services/points/PointsService.ts:158-165`
|
> **Category:** N+1 Query | **File:** `src/services/points/PointsService.ts:158-165` | **FIXED** `2abba67` v2.9.19
|
||||||
|
|
||||||
`findActiveLevelConfigByLevel(snapshot.points.level)` and `findActiveLevelConfigByLevel(snapshot.points.level + 1)` are two independent `SELECT … WHERE level = $1 LIMIT 1` queries on every GET /points request.
|
`findActiveLevelConfigByLevel(snapshot.points.level)` and `findActiveLevelConfigByLevel(snapshot.points.level + 1)` are two independent `SELECT … WHERE level = $1 LIMIT 1` queries on every GET /points request.
|
||||||
|
|
||||||
@@ -674,11 +676,11 @@ The method reads the current row (line 558), performs an atomic CAS UPDATE with
|
|||||||
|
|
||||||
### 9. getUnreadMessageCountByUser fetches all matching chat rows and iterates unreadCounts array in JS
|
### 9. getUnreadMessageCountByUser fetches all matching chat rows and iterates unreadCounts array in JS
|
||||||
|
|
||||||
> **Category:** N+1 Query | **File:** `src/db/repositories/drizzle/DrizzleChatRepo.ts:612-629`
|
> **Category:** N+1 Query | **File:** `src/db/repositories/drizzle/DrizzleChatRepo.ts:755-772` | **FIXED** `current` v2.9.26
|
||||||
|
|
||||||
The method calls `this.findRows({...})` (full table scan), deserialises every chat row, filters in JS, then scans the `unreadCounts` JSONB array for each matching row. This is O(N*M) where N = all chats, M = participants per chat. Called on every page load where the unread badge is shown.
|
The method calls `this.findRows({...})` (full table scan), deserialises every chat row, filters in JS, then scans the `unreadCounts` JSONB array for each matching row. This is O(N*M) where N = all chats, M = participants per chat. Called on every page load where the unread badge is shown.
|
||||||
|
|
||||||
**Fix:** Once a `chat_unread_counts` relational table exists (see wrong-schema finding), this becomes: `SELECT SUM(count) FROM chat_unread_counts WHERE user_id = $1 AND count > 0` — a single indexed aggregation.
|
**Fix:** Replaced with single SQL query using `jsonb_array_elements` and `SUM` aggregation. Filters participants and unreadCounts at the SQL level, eliminating JS iteration.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -704,11 +706,11 @@ The method calls `this.findRows({...})` (full table scan), deserialises every ch
|
|||||||
|
|
||||||
### 12. PgQuery.exec applies sort/skip/limit in-memory after a full unbounded fetch
|
### 12. PgQuery.exec applies sort/skip/limit in-memory after a full unbounded fetch
|
||||||
|
|
||||||
> **Category:** N+1 Query | **File:** `src/services/auth/authStore.ts:168-203`
|
> **Category:** N+1 Query | **File:** `src/services/auth/authStore.ts:168-203` | **FIXED** `current` v2.9.26
|
||||||
|
|
||||||
The PgQuery wrapper's `.sort()`, `.skip()`, `.limit()` methods are applied inside `exec()` in JavaScript after fetching ALL rows from Postgres. For admin user-list queries this means every row is transferred from the DB and sorted in Node.js, with DB indexes ignored.
|
The PgQuery wrapper's `.sort()`, `.skip()`, `.limit()` methods are applied inside `exec()` in JavaScript after fetching ALL rows from Postgres. For admin user-list queries this means every row is transferred from the DB and sorted in Node.js, with DB indexes ignored.
|
||||||
|
|
||||||
**Fix:** Push sort (ORDER BY), skip (OFFSET), and limit into the SQL queries generated by `findPgUsersByQuery`. Remove the in-memory sort/slice logic.
|
**Fix:** Push sort (ORDER BY), skip (OFFSET), and limit into the SQL queries generated by `findPgUsersByQuery`. Updated all 12 PgQuery producers to accept `_pgQuery` parameter. Removed in-memory sort/slice logic for array results.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -1034,7 +1036,7 @@ When `input.legacyObjectId` is set, the code does a SELECT to check existence (l
|
|||||||
|
|
||||||
## 🟢 Low (10)
|
## 🟢 Low (10)
|
||||||
|
|
||||||
### 1. amnScannerPayInService issues three sequential updateById calls on the same payment row
|
### 1. amnScannerPayInService issues three sequential updateById calls on the same payment row | **FIXED** `5752f13` v2.9.29
|
||||||
|
|
||||||
> **Category:** N+1 Query | **File:** `src/services/payment/amnScanner/amnScannerPayInService.ts:140-147, 159-169, 194-203`
|
> **Category:** N+1 Query | **File:** `src/services/payment/amnScanner/amnScannerPayInService.ts:140-147, 159-169, 194-203`
|
||||||
|
|
||||||
@@ -1044,7 +1046,7 @@ After finding or creating the payment, the service calls `paymentRepo.updateById
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### 2. getStats issues two aggregate queries over the same table that can be collapsed into one
|
### 2. getStats issues two aggregate queries over the same table that can be collapsed into one | **FIXED** `5752f13` v2.9.29
|
||||||
|
|
||||||
> **Category:** N+1 Query | **File:** `src/db/repositories/drizzle/DrizzlePaymentRepo.ts:661-680`
|
> **Category:** N+1 Query | **File:** `src/db/repositories/drizzle/DrizzlePaymentRepo.ts:661-680`
|
||||||
|
|
||||||
@@ -1054,7 +1056,7 @@ After finding or creating the payment, the service calls `paymentRepo.updateById
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### 3. findOffersByPurchaseRequest and findExpiredSellerOffers use SELECT * with no LIMIT
|
### 3. findOffersByPurchaseRequest and findExpiredSellerOffers use SELECT * with no LIMIT | **FIXED** `5752f13` v2.9.29
|
||||||
|
|
||||||
> **Category:** Unbounded Fetch | **File:** `src/db/repositories/drizzle/DrizzleMarketplaceRepo.ts:1588-1598, 1731-1744`
|
> **Category:** Unbounded Fetch | **File:** `src/db/repositories/drizzle/DrizzleMarketplaceRepo.ts:1588-1598, 1731-1744`
|
||||||
|
|
||||||
@@ -1074,7 +1076,7 @@ The schema defines `idx_funds_ledger_payment_created` on `(paymentId, createdAt)
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### 5. id_map missing index on newId for reverse-lookup (UUID to ObjectId)
|
### 5. id_map missing index on newId for reverse-lookup (UUID to ObjectId) | **FIXED** `5752f13` v2.9.29
|
||||||
|
|
||||||
> **Category:** Missing Index | **File:** `src/db/schema/idMap.ts:33`
|
> **Category:** Missing Index | **File:** `src/db/schema/idMap.ts:33`
|
||||||
|
|
||||||
@@ -1084,7 +1086,7 @@ The `id_map` table has a unique index on `(collection, legacyObjectId)` for forw
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### 6. trezor_derived_addresses missing index on (trezorAccountId, purpose)
|
### 6. trezor_derived_addresses missing index on (trezorAccountId, purpose) | **FIXED** `5752f13` v2.9.29
|
||||||
|
|
||||||
> **Category:** Missing Index | **File:** `src/db/schema/trezorAccount.ts:117-161`
|
> **Category:** Missing Index | **File:** `src/db/schema/trezorAccount.ts:117-161`
|
||||||
|
|
||||||
@@ -1094,7 +1096,7 @@ The PK on `(trezorAccountId, addressIndex)` covers account-scoped lookups but qu
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### 7. sellerOffers.priceAmount uses numeric(18,8) instead of project-wide numeric(38,18)
|
### 7. sellerOffers.priceAmount uses numeric(18,8) instead of project-wide numeric(38,18) | **FIXED** `5752f13` v2.9.29
|
||||||
|
|
||||||
> **Category:** Wrong Schema | **File:** `src/db/schema/sellerOffer.ts:147`
|
> **Category:** Wrong Schema | **File:** `src/db/schema/sellerOffer.ts:147`
|
||||||
|
|
||||||
@@ -1104,7 +1106,7 @@ The PK on `(trezorAccountId, addressIndex)` covers account-scoped lookups but qu
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### 8. users.profile JSONB embeds walletAddress used for payment matching without any index
|
### 8. users.profile JSONB embeds walletAddress used for payment matching without any index | **FIXED** `5752f13` v2.9.29
|
||||||
|
|
||||||
> **Category:** Wrong Schema | **File:** `src/db/schema/users.ts:98-109`
|
> **Category:** Wrong Schema | **File:** `src/db/schema/users.ts:98-109`
|
||||||
|
|
||||||
@@ -1114,7 +1116,7 @@ The `profile` JSONB blob contains `walletAddress`, `walletType`, and `walletProv
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### 9. Dead unused query builder variable in findByUserId wastes an allocation on every call
|
### 9. Dead unused query builder variable in findByUserId wastes an allocation on every call | **FIXED** `5752f13` v2.9.29
|
||||||
|
|
||||||
> **Category:** Other | **File:** `src/db/repositories/drizzle/DrizzlePaymentRepo.ts:606-615`
|
> **Category:** Other | **File:** `src/db/repositories/drizzle/DrizzlePaymentRepo.ts:606-615`
|
||||||
|
|
||||||
@@ -1124,7 +1126,7 @@ Line 606 assigns `let q = db.select().from(payments).where(conditions).orderBy(.
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
### 10. getLeaderboard passes caller-supplied limit directly to SQL with no upper-bound cap
|
### 10. getLeaderboard passes caller-supplied limit directly to SQL with no upper-bound cap | **FIXED** `5752f13` v2.9.29
|
||||||
|
|
||||||
> **Category:** Missing Index | **File:** `src/db/repositories/drizzle/DrizzlePointsRepo.ts:622-658`
|
> **Category:** Missing Index | **File:** `src/db/repositories/drizzle/DrizzlePointsRepo.ts:622-658`
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user