# MongoDB Runtime Removal Handoff Date: 2026-06-02 Workspace: `/Users/manwe/CascadeProjects/escrow` Goal: remove MongoDB as a runtime dependency by migrating remaining Mongo-backed backend domains and cutover paths to Postgres-compatible repositories. ## Current State No commits or pushes were made for the current WIP. The work is local only. Repo heads at handoff time: | Repo | Branch | HEAD | State | | --- | --- | --- | --- | | `backend` | `integrate-main-into-development` | `cf59726` | dirty WIP | | `frontend` | `integrate-main-into-development` | `a2b972b` | dirty version bump only | | `nick-doc` | `main` | `345c585` | dirty unrelated local docs/profile files | | `deployment` | `main` | `8764fdf` | dirty unrelated `.env` and `docker-compose.yml` | Backend/frontend package versions: backend at `2.8.79`, frontend at `2.8.94`. Important repo rules: - Any backend/frontend product change requires patch version bump in both repos. - Before any backend push, run the relevant focused tests and smoke script. - After every backend push, sync `nick-doc`: - append `09 - Audits/Activity Log.md` - update relevant data/architecture docs - commit as `docs: sync from backend — ` - push `nick-doc` - Do not stage unrelated dirty files in `nick-doc` or `deployment`. ## Dirty Files Backend dirty files: ```text package-lock.json package.json src/db/repositories/drizzle/DrizzleMarketplaceRepo.ts src/db/repositories/drizzle/DrizzlePaymentRepo.ts src/db/repositories/dual/DualWriteMarketplaceRepo.ts src/db/repositories/dual/DualWritePaymentRepo.ts src/db/repositories/interfaces/IMarketplaceRepo.ts src/db/repositories/interfaces/IPaymentRepo.ts src/db/repositories/mongo/MongoMarketplaceRepo.ts src/db/repositories/mongo/MongoPaymentRepo.ts src/services/auth/authStore.ts src/services/user/userController.ts __tests__/auth-store-pg-query.test.ts __tests__/user-dependencies-repo.test.ts scripts/smoke/user-admin-postgres.sh scripts/smoke/user-dependencies.sh ``` Frontend dirty files: ```text package.json ``` ## What This WIP Changes ### 1. Admin User Dependencies Endpoint Endpoint affected: ```text GET /api/user/admin/:userId/dependencies ``` Before this WIP, `src/services/user/userController.ts` dynamically imported Mongo models and counted dependencies directly: - `RequestTemplate.countDocuments(...)` - `PurchaseRequest.countDocuments(...)` - `Payment.countDocuments(...)` - `Chat.countDocuments(...)` This WIP replaces that direct model access with repository calls: - `getMarketplaceRepo().getUserDependencyCounts(userId)` - `getPaymentRepo().countByParticipant(userId)` - `getChatRepo().count({ 'participants.userId': userId, 'participants.isActive': true })` New/extended repository contract methods: - `IMarketplaceRepo.getUserDependencyCounts(userId)` - `IPaymentRepo.countByParticipant(userId)` Implementations added: - `MongoMarketplaceRepo.getUserDependencyCounts` - `DrizzleMarketplaceRepo.getUserDependencyCounts` - `DualWriteMarketplaceRepo.getUserDependencyCounts` - `MongoPaymentRepo.countByParticipant` - `DrizzlePaymentRepo.countByParticipant` - `DualWritePaymentRepo.countByParticipant` Behavior note: - Postgres counts seller-side marketplace dependencies by joining `purchase_requests.selected_offer_id` to `seller_offers.id` and checking `seller_offers.seller_id`. - Mongo implementation supports selected-offer-id style and also keeps compatibility with legacy embedded `selectedOffer.sellerId`. New test: ```text __tests__/user-dependencies-repo.test.ts ``` New smoke script: ```text scripts/smoke/user-dependencies.sh ``` Smoke usage: ```bash BASE_URL=https://dev.amn.gg ADMIN_TOKEN= USER_ID= scripts/smoke/user-dependencies.sh ``` The smoke script checks: - HTTP 200 - `success === true` - `data.user` exists - dependency counters are non-negative numbers - `total` equals the sum of component counters ### 2. AuthUser Postgres Query Facade Hardening Files affected: ```text src/services/auth/authStore.ts __tests__/auth-store-pg-query.test.ts ``` The mounted user/admin list and stats routes already use `AuthUser` from `authStore`, not raw Mongoose imports. However, the Postgres `PgQuery` wrapper only sorted arrays by `createdAt`; admin list accepts arbitrary `sortBy`. This WIP hardens the Postgres query wrapper so `AuthUser.find(...).select(...).sort(...).skip(...).limit(...).lean()` behaves more like the existing Mongoose chain: - generic sorting by requested field - nested path sorting support, e.g. `profile.avatar` - date, number, boolean, and string comparison - multi-field sort support - keeps existing skip/limit/select/lean chain behavior It also adds alias support in `buildUserWhere`: - `isActive: true` maps to `status = 'active'` - `isActive: false` maps to `status <> 'active'` - `isVerified` maps to `is_email_verified` This matters because: - `src/services/user/userController.ts` builds filters with `isActive` / `isVerified` - `src/services/user/userRoutes.ts` builds filters with `status` / `isEmailVerified` - both now work in Postgres auth mode New test: ```text __tests__/auth-store-pg-query.test.ts ``` New smoke script: ```text scripts/smoke/user-admin-postgres.sh ``` Smoke usage: ```bash BASE_URL=https://dev.amn.gg ADMIN_TOKEN= scripts/smoke/user-admin-postgres.sh ``` The smoke script checks: - `GET /api/user/admin/list?page=1&limit=5&sortBy=firstName&sortOrder=asc` - `GET /api/users/admin/stats` - response shape and numeric stats ## Verification Already Run Passed: ```bash npm test -- --runTestsByPath __tests__/auth-store-pg-query.test.ts __tests__/user-dependencies-repo.test.ts __tests__/repository-factory-modes.test.ts __tests__/health-check-service.test.ts __tests__/marketplace-runtime-import-surface.test.ts --runInBand ``` Result: ```text Test Suites: 5 passed, 5 total Tests: 11 passed, 11 total ``` Passed: ```bash npm run typecheck ``` Passed: ```bash git diff --check ``` for both backend and frontend. Passed syntax checks: ```bash bash -n scripts/smoke/user-admin-postgres.sh bash -n scripts/smoke/user-dependencies.sh ``` Not run end-to-end: - `scripts/smoke/user-dependencies.sh` - `scripts/smoke/user-admin-postgres.sh` Reason: both require an `ADMIN_TOKEN`; `user-dependencies.sh` also requires `USER_ID`. ## How To Pick This Up Start with: ```bash cd /Users/manwe/CascadeProjects/escrow/backend git status --short --branch git diff --stat ``` Review the exact WIP: ```bash git diff -- src/services/user/userController.ts src/services/auth/authStore.ts git diff -- src/db/repositories/interfaces/IMarketplaceRepo.ts src/db/repositories/interfaces/IPaymentRepo.ts git diff -- src/db/repositories/mongo/MongoMarketplaceRepo.ts src/db/repositories/drizzle/DrizzleMarketplaceRepo.ts src/db/repositories/dual/DualWriteMarketplaceRepo.ts git diff -- src/db/repositories/mongo/MongoPaymentRepo.ts src/db/repositories/drizzle/DrizzlePaymentRepo.ts src/db/repositories/dual/DualWritePaymentRepo.ts git diff -- __tests__/auth-store-pg-query.test.ts __tests__/user-dependencies-repo.test.ts git diff -- scripts/smoke/user-admin-postgres.sh scripts/smoke/user-dependencies.sh ``` Re-run local verification: ```bash npm test -- --runTestsByPath __tests__/auth-store-pg-query.test.ts __tests__/user-dependencies-repo.test.ts __tests__/repository-factory-modes.test.ts __tests__/health-check-service.test.ts __tests__/marketplace-runtime-import-surface.test.ts --runInBand npm run typecheck git diff --check ``` If you have a dev admin token: ```bash BASE_URL=https://dev.amn.gg ADMIN_TOKEN= USER_ID= scripts/smoke/user-dependencies.sh BASE_URL=https://dev.amn.gg ADMIN_TOKEN= scripts/smoke/user-admin-postgres.sh ``` If you commit this WIP, suggested commit shape: Backend: ```bash git add package.json package-lock.json \ src/db/repositories/interfaces/IMarketplaceRepo.ts \ src/db/repositories/interfaces/IPaymentRepo.ts \ src/db/repositories/mongo/MongoMarketplaceRepo.ts \ src/db/repositories/drizzle/DrizzleMarketplaceRepo.ts \ src/db/repositories/dual/DualWriteMarketplaceRepo.ts \ src/db/repositories/mongo/MongoPaymentRepo.ts \ src/db/repositories/drizzle/DrizzlePaymentRepo.ts \ src/db/repositories/dual/DualWritePaymentRepo.ts \ src/services/user/userController.ts \ src/services/auth/authStore.ts \ __tests__/auth-store-pg-query.test.ts \ __tests__/user-dependencies-repo.test.ts \ scripts/smoke/user-admin-postgres.sh \ scripts/smoke/user-dependencies.sh git commit -m "fix: route admin user counts through postgres-capable stores" ``` Frontend: ```bash cd ../frontend git add package.json git commit -m "chore: sync frontend version to 2.8.38" ``` Do not push without doing the `nick-doc` sync afterward. ## Remaining Runtime Mongo Scan Latest scan command: ```bash rg -n --pcre2 "^import (?!type).*from ['\"]mongoose['\"]|^import (?!type).*from ['\"][^'\"]*models/|await import\(['\"][^'\"]*models/|countDocuments\(|deleteMany\(|findByIdAndDelete\(" \ src/services src/routes src/app.ts src/infrastructure src/db/repositories/factory.ts \ --glob '!**/*.test.ts' \ --glob '!src/services/marketplace/routes.ts' ``` Important results and interpretation: ### Auth/User Routes Paths still visible in scans: ```text src/app.ts:672 AuthUser.countDocuments({ role: { $ne: 'admin' } }) src/services/user/userController.ts:306 User.countDocuments(filter) src/services/user/userRoutes.ts multiple User.countDocuments(...) src/services/auth/authStore.ts AuthUser.countDocuments implementation ``` Interpretation: - These are mostly through the `AuthUser` facade, not direct Mongoose imports. - In `AUTH_STORE=postgres` mode, `AuthUser.countDocuments` uses Postgres. - This WIP improved the Postgres query-chain behavior and filter aliases. - Future work should reduce the noisy model-shaped facade API over time, but these are not necessarily active Mongo runtime blockers when auth store is Postgres and Mongo fallback/mirroring are disabled. Recommended follow-up: - Add a `UserAdminRepo` or explicit auth-store helper methods for admin list/stats to replace model-shaped route code. - After that, route `src/services/user/userController.ts` and `src/services/user/userRoutes.ts` through helper methods and remove the remaining `User.countDocuments` call sites from route/controller code. ### Admin Data Cleanup Service High-priority blocker: ```text src/services/admin/dataCleanupService.ts ``` Scan hits include dynamic model counting/deleting: ```text Model.countDocuments(query) Model.deleteMany(query) User.countDocuments() PurchaseRequest.countDocuments() SellerOffer.countDocuments() Payment.countDocuments() Chat.countDocuments() Notification.countDocuments() RequestTemplate.countDocuments() Address.countDocuments() Category.countDocuments() TempVerification.countDocuments() User.findByIdAndDelete(userId) ``` Interpretation: - This is the biggest remaining direct runtime Mongo/model surface. - It likely imports or dynamically resolves models and is unsuitable for `MONGO_CONNECT_MODE=never`. Recommended migration: - Replace cleanup stats with repo-backed counts: - auth/user counts from AuthUser/Postgres helper - marketplace counts from MarketplaceRepo - payments from PaymentRepo - notifications from NotificationRepo - chat from ChatRepo - addresses/categories/reviews/temp verification through their Postgres-capable stores - For destructive cleanup operations, either: - implement explicit Postgres cleanup repo methods with strong safety guards, or - disable Mongo-only cleanup actions when Mongo is disabled and return a clear `501`/unsupported result. ### Store Facades Still Exposing Model-Style Methods Scan hits: ```text src/services/points/levelConfigStore.ts deleteMany(...) src/services/address/addressStore.ts findByIdAndDelete(...), countDocuments(...) src/services/marketplace/reviewStore.ts countDocuments(...) src/services/auth/authController.ts TempVerification.findByIdAndDelete(...) src/services/auth/authStore.ts TempVerification findByIdAndDelete implementation ``` Interpretation: - Some of these are behind Postgres-capable store facades. - They still show up because they preserve a Mongoose-shaped API. - For full Mongo removal, these facades must be audited under: - `MONGO_CONNECT_MODE=never` - store env set to `postgres` - fallback/mirror disabled where relevant Recommended migration: - Convert each store facade from "model-like object with Mongo fallback" to explicit repository functions. - Add tests that set the store env to `postgres` and assert no Mongo model getter is called. ### Payment Coordinator Scan hit: ```text src/services/payment/paymentCoordinator.ts:510 paymentRepo.deleteMany({ idIn: duplicateIds }) ``` Interpretation: - This is already repository-routed, not raw Mongoose. - Confirm `DrizzlePaymentRepo.deleteMany` exists and works for the duplicate cleanup path. ## Suggested Next Work Order 1. Finish and commit this WIP after review. 2. Run the two new smoke scripts with real dev admin credentials. 3. Update `nick-doc` after pushing backend/frontend. 4. Migrate `src/services/admin/dataCleanupService.ts`. 5. Replace user/admin list/stats route `User.countDocuments` calls with explicit auth/user helper methods so scans no longer flag route-level model-shaped calls. 6. Audit `addressStore`, `reviewStore`, `levelConfigStore`, and `TempVerification` under `MONGO_CONNECT_MODE=never`. 7. Once all runtime paths are Postgres-capable, set health logic so Mongo is optional and then remove startup Mongo requirement. 8. Final pass: - scan for non-test runtime `mongoose` / `models` imports - run typecheck - run focused Jest suites - run smoke scripts against local/dev - verify `/api/health` reports Mongo optional or absent and Postgres healthy ## Environment Notes For Cutover Testing Useful envs for a no-Mongo runtime test: ```text MONGO_CONNECT_MODE=never AUTH_STORE=postgres USER_STORE=postgres AUTH_FALLBACK_MONGO=false AUTH_MIRROR_MONGO=false ``` Also ensure all existing repo/store envs that support Postgres are set to `postgres` or `pg` consistently, including marketplace/payment/dispute/release-hold/notification/blog/address/category/review/level-config/shop-settings style stores. Do not rely only on `rg` results: some model-shaped methods are already routed through Postgres facades. Prove no-Mongo runtime by running the backend with `MONGO_CONNECT_MODE=never` and exercising the API smoke scripts. ## Known Caveats - The new smoke scripts need real admin credentials and were not executed end-to-end. - The current WIP is not pushed and has no `nick-doc` sync yet. - `nick-doc` and `deployment` have unrelated dirty files; do not stage them accidentally. - The full goal is not complete. MongoDB is still a runtime dependency until the remaining service/store paths above are migrated and verified under `MONGO_CONNECT_MODE=never`.