docs: sync from backend 8fc2309 — M43/M44 missing FKs + H37 dispute enums
This commit is contained in:
Siavash Sameni
308
PRD - Mongo Retirement (Full Nuke).md
Normal file
308
PRD - Mongo Retirement (Full Nuke).md
Normal file
@@ -0,0 +1,308 @@
|
||||
# PRD — Mongo Retirement: Full Code Nuke
|
||||
|
||||
**Status:** In Progress
|
||||
**Date:** 2026-06-06
|
||||
**Scope:** Remove every Mongoose/MongoDB reference from the codebase and replace Mongo-style ObjectId fields with plain UUID strings throughout.
|
||||
|
||||
---
|
||||
|
||||
## Context
|
||||
|
||||
The Mongo→Postgres migration scaffolding is complete:
|
||||
- 25 Drizzle schemas cover all 23 collections
|
||||
- 11 Drizzle repos, 11 Mongo repos, 9 DualWrite fan-out repos
|
||||
- Backfill scripts written (not yet run against production)
|
||||
- Factory defaults to `mongo`; reads not yet cut over
|
||||
- Chat uses a JSONB shim (acceptable for now — normalize later)
|
||||
|
||||
This PRD covers the **code-level retirement**: delete all Mongoose artifacts, flip the factory to Postgres-only, remove MongoDB from docker-compose and package.json, and replace `Types.ObjectId` with `string` (UUID) across every interface and schema.
|
||||
|
||||
Production data migration (backfill execution + read-cutover per domain) is a **separate, human-gated operation** and is NOT in scope here.
|
||||
|
||||
---
|
||||
|
||||
## Goals
|
||||
|
||||
1. Zero `mongoose` or `mongodb` imports anywhere in `backend/`
|
||||
2. Zero `Schema.Types.ObjectId` / `Types.ObjectId` field types
|
||||
3. Zero `_id: ObjectId` interface declarations (replaced with `id: string`)
|
||||
4. Zero `ObjectId(...)` constructors or `.toString()` coercions on IDs
|
||||
5. MongoDB service removed from all docker-compose files
|
||||
6. `mongoose` removed from `package.json`
|
||||
7. `MONGODB_URI` / `MONGO_*` env vars removed from config and docs
|
||||
8. Factory defaults to `pg`; Mongo and DualWrite modes deleted
|
||||
9. Seeds and scripts updated to target Postgres only
|
||||
10. Frontend: legacy ObjectId validation path removed
|
||||
|
||||
---
|
||||
|
||||
## Out of Scope
|
||||
|
||||
- Backfill execution against production data
|
||||
- Production env var changes (`REPO_*`, `MONGO_CONNECT_MODE`)
|
||||
- Chat normalization (JSONB → child tables) — tracked separately
|
||||
- Drizzle schema changes (schemas are already correct)
|
||||
- Any Drizzle repo logic changes
|
||||
|
||||
---
|
||||
|
||||
## Phased Work Plan
|
||||
|
||||
### Phase 1 — Factory Cutover & Repo Deletion (Day 1)
|
||||
|
||||
**1.1 Factory: delete Mongo + DualWrite modes**
|
||||
- File: `src/db/repositories/factory.ts`
|
||||
- Change: remove all `mongo` and `dual` branches; every domain returns its Drizzle repo directly
|
||||
- Remove `REPO_*` env flag checks (no longer needed)
|
||||
- Remove all imports of Mongo repos and DualWrite repos
|
||||
|
||||
**1.2 Delete Mongo repo files (11 files)**
|
||||
```
|
||||
src/db/repositories/mongo/MongoUserRepo.ts
|
||||
src/db/repositories/mongo/MongoPaymentRepo.ts
|
||||
src/db/repositories/mongo/MongoPointsRepo.ts
|
||||
src/db/repositories/mongo/MongoMarketplaceRepo.ts
|
||||
src/db/repositories/mongo/MongoBlogRepo.ts
|
||||
src/db/repositories/mongo/MongoNotificationRepo.ts
|
||||
src/db/repositories/mongo/MongoDisputeRepo.ts
|
||||
src/db/repositories/mongo/MongoTrezorAccountRepo.ts
|
||||
src/db/repositories/mongo/MongoDerivedDestinationRepo.ts
|
||||
src/db/repositories/mongo/MongoChatRepo.ts
|
||||
src/db/repositories/mongo/MongoReleaseHoldRepo.ts
|
||||
src/db/repositories/mongo/index.ts
|
||||
```
|
||||
|
||||
**1.3 Delete DualWrite repo files (9 files)**
|
||||
```
|
||||
src/db/repositories/dual/DualWriteUserRepo.ts
|
||||
src/db/repositories/dual/DualWritePaymentRepo.ts
|
||||
src/db/repositories/dual/DualWritePointsRepo.ts
|
||||
src/db/repositories/dual/DualWriteMarketplaceRepo.ts
|
||||
src/db/repositories/dual/DualWriteBlogRepo.ts
|
||||
src/db/repositories/dual/DualWriteNotificationRepo.ts
|
||||
src/db/repositories/dual/DualWriteDisputeRepo.ts
|
||||
src/db/repositories/dual/DualWriteTrezorAccountRepo.ts
|
||||
src/db/repositories/dual/DualWriteDerivedDestinationRepo.ts
|
||||
src/db/repositories/dual/index.ts
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Phase 2 — Mongoose Model Deletion & Interface Extraction (Day 1–2)
|
||||
|
||||
**2.1 Extract pure TS interfaces from each model file**
|
||||
|
||||
For each of the 24 model files in `src/models/`, the plan is:
|
||||
- Keep the `I<Name>` TypeScript interface (with `id: string` replacing `_id: Types.ObjectId`)
|
||||
- Delete the `new Schema(...)` definition, virtual fields, pre/post hooks
|
||||
- Delete the `mongoose.model<I<Name>>(...)` export
|
||||
- Delete the mongoose import
|
||||
|
||||
The interface files move to `src/types/models/` (or inline into the Drizzle schema files as inferred types).
|
||||
|
||||
**2.2 Convert all `_id` → `id: string` in interfaces**
|
||||
|
||||
Key changes per domain:
|
||||
- `IUser._id: Types.ObjectId` → `id: string`
|
||||
- `ICategory._id` → `id: string`, `parentId: string | null`
|
||||
- `IPurchaseRequest._id` → `id: string`, `buyerId/sellerId: string`
|
||||
- `ISellerOffer._id` → `id: string`, `sellerId/purchaseRequestId: string`
|
||||
- `IPayment._id` → `id: string`, polymorphic fields → `string`
|
||||
- `IChat._id` → `id: string`, `senderId/userId: string`
|
||||
- `IDispute._id` → `id: string`, all ref fields → `string`
|
||||
- All others follow same pattern
|
||||
|
||||
**2.3 Delete `src/models/` directory after interfaces extracted**
|
||||
|
||||
---
|
||||
|
||||
### Phase 3 — Mongoose Connection & Config Removal (Day 2)
|
||||
|
||||
**3.1 `src/infrastructure/database/connection.ts`**
|
||||
- Remove `mongoose.connect()` call and all Mongo connection logic
|
||||
- Remove `MONGO_CONNECT_MODE` handling
|
||||
- Keep only the Postgres pool initialization
|
||||
|
||||
**3.2 `src/shared/config/index.ts`**
|
||||
- Remove `mongoUri` field
|
||||
- Remove `MONGODB_URI` env var read
|
||||
|
||||
**3.3 `src/services/auth/authStore.ts`**
|
||||
- Remove `AUTH_FALLBACK_MONGO`, `AUTH_MIRROR_MONGO`, `MONGO_CONNECT_MODE` branches
|
||||
- Auth reads only from Postgres
|
||||
|
||||
**3.4 `src/services/health/healthCheckService.ts`**
|
||||
- Remove MongoDB health check
|
||||
- Remove `MONGO_CONNECT_MODE` reference
|
||||
|
||||
**3.5 `src/app.ts`**
|
||||
- Remove `mongoose.connect()` / `connectMongo()` call on startup
|
||||
- Remove `p.userId.toString()` ObjectId coercions (already string)
|
||||
|
||||
---
|
||||
|
||||
### Phase 4 — Seeds & Scripts Cleanup (Day 2)
|
||||
|
||||
**4.1 Seeds (`src/seeds/`)**
|
||||
- Remove `mongoose.connect()` from all 7 seed files
|
||||
- Seeds already have PG-aware paths; remove Mongo dual-path
|
||||
- `seedUsers.ts`, `seedLevels.ts`, `seedCategories.ts`, etc.
|
||||
|
||||
**4.2 Scripts (`src/scripts/`)**
|
||||
- Remove Mongoose imports from 13 scripts
|
||||
- Scripts that only operated on Mongo (e.g. `clearChats.ts`, `updateRequestStatus.ts`) — convert to Postgres queries or delete if obsolete
|
||||
|
||||
---
|
||||
|
||||
### Phase 5 — Backfill Infrastructure (Day 3)
|
||||
|
||||
**5.1 Archive backfill scripts** (don't delete — needed for production data migration)
|
||||
- Move `src/db/backfill/` → `src/db/backfill/_archive/`
|
||||
- OR keep as-is but add a `.nocompile` flag / remove from tsconfig paths
|
||||
- The backfill scripts import mongoose (to read from Mongo) — they're tools for ops, not app code
|
||||
|
||||
**5.2 Remove verification layer Mongo references**
|
||||
- `src/db/verify/shadowRead.ts` — remove Mongo comparison path
|
||||
- `src/db/verify/rowCounts.ts` — remove Mongo row count queries
|
||||
- `src/db/verify/checksums.ts` — remove Mongo checksum queries
|
||||
- `src/db/verify/reconcile.ts` — keep (handles `pg_dualwrite_gaps` replay, still useful)
|
||||
|
||||
**5.3 `_idMap.ts`** — keep as a utility for ops scripts only; remove from app imports
|
||||
|
||||
---
|
||||
|
||||
### Phase 6 — Docker & Dependencies (Day 3)
|
||||
|
||||
**6.1 `docker-compose.dev.yml`**
|
||||
- Remove `mongodb` service block
|
||||
- Remove `depends_on: mongodb` from backend service
|
||||
- Remove all `*_STORE=mongo` env vars (Auth, Config, Address, etc.)
|
||||
- Remove `mongodb_data` volume
|
||||
|
||||
**6.2 `docker-compose.production.yml`**
|
||||
- Same: remove mongodb service, depends_on, volume
|
||||
|
||||
**6.3 `deployment/docker-compose.yml`**
|
||||
- Same: remove mongodb service block
|
||||
|
||||
**6.4 `backend/package.json`**
|
||||
- Remove `mongoose` from dependencies
|
||||
- Remove `mongodb-memory-server` from devDependencies
|
||||
|
||||
**6.5 `.env.example` / `.env.development` / `.env.local`**
|
||||
- Remove `MONGODB_URI`, `MONGO_CONNECT_MODE`, `AUTH_FALLBACK_MONGO`, `AUTH_MIRROR_MONGO`
|
||||
- Remove `MONGO_INITDB_*` vars
|
||||
|
||||
---
|
||||
|
||||
### Phase 7 — Frontend ID Validation Cleanup (Day 3)
|
||||
|
||||
**7.1 Remove ObjectId validation branch**
|
||||
- `frontend/src/sections/request-template/view/seller-shop-view.tsx:78`
|
||||
- Remove the "legacy 24-hex Mongo ObjectId" validation path
|
||||
- Keep only UUID validation
|
||||
|
||||
**7.2 Audit frontend for other ObjectId references**
|
||||
- `grep -r 'ObjectId\|[0-9a-f]{24}' frontend/src/`
|
||||
- Remove any other legacy ID format handling
|
||||
|
||||
---
|
||||
|
||||
### Phase 8 — TypeScript Compilation Check (Day 3)
|
||||
|
||||
- `npm run tsc` — fix all remaining type errors from the migration
|
||||
- Common issues: `.toString()` on already-string IDs, `_id` vs `id` mismatches, missing UUID imports
|
||||
|
||||
---
|
||||
|
||||
## Files to Delete (Complete List)
|
||||
|
||||
### Mongoose Model Files (24)
|
||||
```
|
||||
backend/src/models/User.ts
|
||||
backend/src/models/Category.ts
|
||||
backend/src/models/PurchaseRequest.ts
|
||||
backend/src/models/SellerOffer.ts
|
||||
backend/src/models/Payment.ts
|
||||
backend/src/models/Chat.ts
|
||||
backend/src/models/Dispute.ts
|
||||
backend/src/models/Review.ts
|
||||
backend/src/models/Address.ts
|
||||
backend/src/models/Notification.ts
|
||||
backend/src/models/TelegramLink.ts
|
||||
backend/src/models/TelegramSession.ts
|
||||
backend/src/models/TempVerification.ts
|
||||
backend/src/models/TrezorAccount.ts
|
||||
backend/src/models/DerivedDestination.ts
|
||||
backend/src/models/FundsLedgerEntry.ts
|
||||
backend/src/models/PointTransaction.ts
|
||||
backend/src/models/RequestTemplate.ts
|
||||
backend/src/models/BlogPost.ts
|
||||
backend/src/models/ConfigSetting.ts
|
||||
backend/src/models/ConfigSettingHistory.ts
|
||||
backend/src/models/LevelConfig.ts
|
||||
backend/src/models/ShopSettings.ts
|
||||
backend/src/models/index.ts
|
||||
```
|
||||
|
||||
### Mongo Repos (11+index)
|
||||
```
|
||||
backend/src/db/repositories/mongo/ (entire directory)
|
||||
```
|
||||
|
||||
### DualWrite Repos (9+index)
|
||||
```
|
||||
backend/src/db/repositories/dual/ (entire directory)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## ID Migration Pattern
|
||||
|
||||
Every interface field that was `Types.ObjectId` or `Schema.Types.ObjectId` becomes `string` (UUID).
|
||||
|
||||
**Before:**
|
||||
```typescript
|
||||
interface ISellerOffer {
|
||||
_id: Types.ObjectId;
|
||||
sellerId: Types.ObjectId | string;
|
||||
purchaseRequestId: Types.ObjectId | string;
|
||||
}
|
||||
```
|
||||
|
||||
**After:**
|
||||
```typescript
|
||||
interface ISellerOffer {
|
||||
id: string; // UUID
|
||||
sellerId: string;
|
||||
purchaseRequestId: string;
|
||||
}
|
||||
```
|
||||
|
||||
Any code doing `someDoc._id.toString()` → `someDoc.id` (already a string).
|
||||
Any code doing `new mongoose.Types.ObjectId(value)` → just use `value` as string.
|
||||
|
||||
---
|
||||
|
||||
## Risk Register
|
||||
|
||||
| Risk | Mitigation |
|
||||
|---|---|
|
||||
| App breaks because Mongo isn't seeded in dev | Run `npm run seed:pg` before removing Mongo from docker-compose |
|
||||
| Type errors cascade from `_id` → `id` rename | Fix systematically: models first, then services/routes |
|
||||
| Backfill scripts break (they import mongoose) | Keep backfill dir outside tsconfig compilation scope |
|
||||
| Auth fallback to Mongo breaks login | Auth already has PG path; remove fallback gate |
|
||||
| Chat reads fail (JSONB shim) | JSONB shim already works; normalization is future work |
|
||||
|
||||
---
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [ ] `grep -r "mongoose" backend/src/ --include="*.ts"` returns zero hits (excluding backfill archive)
|
||||
- [ ] `grep -r "Types.ObjectId\|Schema.Types.ObjectId" backend/src/` returns zero hits
|
||||
- [ ] `grep -r "mongodb" backend/package.json` returns zero hits
|
||||
- [ ] `grep -r "MONGODB_URI\|MONGO_CONNECT_MODE" backend/src/` returns zero hits
|
||||
- [ ] `npm run tsc` exits 0
|
||||
- [ ] Backend starts with `MONGO_CONNECT_MODE=never` (or removed) and `REPO_*=pg` (or removed)
|
||||
- [ ] Seed scripts populate Postgres successfully
|
||||
- [ ] All docker-compose files have no `mongodb` service
|
||||
Reference in New Issue
Block a user