nick-doc/08 - Operations/MONGODB_REMOVAL_HANDOFF_2026-06-02.md

# MongoDB Runtime Removal Handoff

Date: 2026-06-02  
Workspace: `/Users/manwe/CascadeProjects/escrow`  
Goal: remove MongoDB as a runtime dependency by migrating remaining Mongo-backed backend domains and cutover paths to Postgres-compatible repositories.

## Current State

No commits or pushes were made for the current WIP. The work is local only.

Repo heads at handoff time:

| Repo | Branch | HEAD | State |
| --- | --- | --- | --- |
| `backend` | `integrate-main-into-development` | `cf59726` | dirty WIP |
| `frontend` | `integrate-main-into-development` | `a2b972b` | dirty version bump only |
| `nick-doc` | `main` | `345c585` | dirty unrelated local docs/profile files |
| `deployment` | `main` | `8764fdf` | dirty unrelated `.env` and `docker-compose.yml` |

Backend/frontend package versions: backend at `2.8.79`, frontend at `2.8.94`.

Important repo rules:

- Any backend/frontend product change requires patch version bump in both repos.
- Before any backend push, run the relevant focused tests and smoke script.
- After every backend push, sync `nick-doc`:
  - append `09 - Audits/Activity Log.md`
  - update relevant data/architecture docs
  - commit as `docs: sync from backend <short-sha> — <summary>`
  - push `nick-doc`
- Do not stage unrelated dirty files in `nick-doc` or `deployment`.

## Dirty Files

Backend dirty files:

```text
package-lock.json
package.json
src/db/repositories/drizzle/DrizzleMarketplaceRepo.ts
src/db/repositories/drizzle/DrizzlePaymentRepo.ts
src/db/repositories/dual/DualWriteMarketplaceRepo.ts
src/db/repositories/dual/DualWritePaymentRepo.ts
src/db/repositories/interfaces/IMarketplaceRepo.ts
src/db/repositories/interfaces/IPaymentRepo.ts
src/db/repositories/mongo/MongoMarketplaceRepo.ts
src/db/repositories/mongo/MongoPaymentRepo.ts
src/services/auth/authStore.ts
src/services/user/userController.ts
__tests__/auth-store-pg-query.test.ts
__tests__/user-dependencies-repo.test.ts
scripts/smoke/user-admin-postgres.sh
scripts/smoke/user-dependencies.sh
```

Frontend dirty files:

```text
package.json
```

## What This WIP Changes

### 1. Admin User Dependencies Endpoint

Endpoint affected:

```text
GET /api/user/admin/:userId/dependencies
```

Before this WIP, `src/services/user/userController.ts` dynamically imported Mongo models and counted dependencies directly:

- `RequestTemplate.countDocuments(...)`
- `PurchaseRequest.countDocuments(...)`
- `Payment.countDocuments(...)`
- `Chat.countDocuments(...)`

This WIP replaces that direct model access with repository calls:

- `getMarketplaceRepo().getUserDependencyCounts(userId)`
- `getPaymentRepo().countByParticipant(userId)`
- `getChatRepo().count({ 'participants.userId': userId, 'participants.isActive': true })`

New/extended repository contract methods:

- `IMarketplaceRepo.getUserDependencyCounts(userId)`
- `IPaymentRepo.countByParticipant(userId)`

Implementations added:

- `MongoMarketplaceRepo.getUserDependencyCounts`
- `DrizzleMarketplaceRepo.getUserDependencyCounts`
- `DualWriteMarketplaceRepo.getUserDependencyCounts`
- `MongoPaymentRepo.countByParticipant`
- `DrizzlePaymentRepo.countByParticipant`
- `DualWritePaymentRepo.countByParticipant`

Behavior note:

- Postgres counts seller-side marketplace dependencies by joining `purchase_requests.selected_offer_id` to `seller_offers.id` and checking `seller_offers.seller_id`.
- Mongo implementation supports selected-offer-id style and also keeps compatibility with legacy embedded `selectedOffer.sellerId`.

New test:

```text
__tests__/user-dependencies-repo.test.ts
```

New smoke script:

```text
scripts/smoke/user-dependencies.sh
```

Smoke usage:

```bash
BASE_URL=https://dev.amn.gg ADMIN_TOKEN=<admin-jwt> USER_ID=<target-user-id> scripts/smoke/user-dependencies.sh
```

The smoke script checks:

- HTTP 200
- `success === true`
- `data.user` exists
- dependency counters are non-negative numbers
- `total` equals the sum of component counters

### 2. AuthUser Postgres Query Facade Hardening

Files affected:

```text
src/services/auth/authStore.ts
__tests__/auth-store-pg-query.test.ts
```

The mounted user/admin list and stats routes already use `AuthUser` from `authStore`, not raw Mongoose imports. However, the Postgres `PgQuery` wrapper only sorted arrays by `createdAt`; admin list accepts arbitrary `sortBy`.

This WIP hardens the Postgres query wrapper so `AuthUser.find(...).select(...).sort(...).skip(...).limit(...).lean()` behaves more like the existing Mongoose chain:

- generic sorting by requested field
- nested path sorting support, e.g. `profile.avatar`
- date, number, boolean, and string comparison
- multi-field sort support
- keeps existing skip/limit/select/lean chain behavior

It also adds alias support in `buildUserWhere`:

- `isActive: true` maps to `status = 'active'`
- `isActive: false` maps to `status <> 'active'`
- `isVerified` maps to `is_email_verified`

This matters because:

- `src/services/user/userController.ts` builds filters with `isActive` / `isVerified`
- `src/services/user/userRoutes.ts` builds filters with `status` / `isEmailVerified`
- both now work in Postgres auth mode

New test:

```text
__tests__/auth-store-pg-query.test.ts
```

New smoke script:

```text
scripts/smoke/user-admin-postgres.sh
```

Smoke usage:

```bash
BASE_URL=https://dev.amn.gg ADMIN_TOKEN=<admin-jwt> scripts/smoke/user-admin-postgres.sh
```

The smoke script checks:

- `GET /api/user/admin/list?page=1&limit=5&sortBy=firstName&sortOrder=asc`
- `GET /api/users/admin/stats`
- response shape and numeric stats

## Verification Already Run

Passed:

```bash
npm test -- --runTestsByPath __tests__/auth-store-pg-query.test.ts __tests__/user-dependencies-repo.test.ts __tests__/repository-factory-modes.test.ts __tests__/health-check-service.test.ts __tests__/marketplace-runtime-import-surface.test.ts --runInBand
```

Result:

```text
Test Suites: 5 passed, 5 total
Tests: 11 passed, 11 total
```

Passed:

```bash
npm run typecheck
```

Passed:

```bash
git diff --check
```

for both backend and frontend.

Passed syntax checks:

```bash
bash -n scripts/smoke/user-admin-postgres.sh
bash -n scripts/smoke/user-dependencies.sh
```

Not run end-to-end:

- `scripts/smoke/user-dependencies.sh`
- `scripts/smoke/user-admin-postgres.sh`

Reason: both require an `ADMIN_TOKEN`; `user-dependencies.sh` also requires `USER_ID`.

## How To Pick This Up

Start with:

```bash
cd /Users/manwe/CascadeProjects/escrow/backend
git status --short --branch
git diff --stat
```

Review the exact WIP:

```bash
git diff -- src/services/user/userController.ts src/services/auth/authStore.ts
git diff -- src/db/repositories/interfaces/IMarketplaceRepo.ts src/db/repositories/interfaces/IPaymentRepo.ts
git diff -- src/db/repositories/mongo/MongoMarketplaceRepo.ts src/db/repositories/drizzle/DrizzleMarketplaceRepo.ts src/db/repositories/dual/DualWriteMarketplaceRepo.ts
git diff -- src/db/repositories/mongo/MongoPaymentRepo.ts src/db/repositories/drizzle/DrizzlePaymentRepo.ts src/db/repositories/dual/DualWritePaymentRepo.ts
git diff -- __tests__/auth-store-pg-query.test.ts __tests__/user-dependencies-repo.test.ts
git diff -- scripts/smoke/user-admin-postgres.sh scripts/smoke/user-dependencies.sh
```

Re-run local verification:

```bash
npm test -- --runTestsByPath __tests__/auth-store-pg-query.test.ts __tests__/user-dependencies-repo.test.ts __tests__/repository-factory-modes.test.ts __tests__/health-check-service.test.ts __tests__/marketplace-runtime-import-surface.test.ts --runInBand
npm run typecheck
git diff --check
```

If you have a dev admin token:

```bash
BASE_URL=https://dev.amn.gg ADMIN_TOKEN=<admin-jwt> USER_ID=<target-user-id> scripts/smoke/user-dependencies.sh
BASE_URL=https://dev.amn.gg ADMIN_TOKEN=<admin-jwt> scripts/smoke/user-admin-postgres.sh
```

If you commit this WIP, suggested commit shape:

Backend:

```bash
git add package.json package-lock.json \
  src/db/repositories/interfaces/IMarketplaceRepo.ts \
  src/db/repositories/interfaces/IPaymentRepo.ts \
  src/db/repositories/mongo/MongoMarketplaceRepo.ts \
  src/db/repositories/drizzle/DrizzleMarketplaceRepo.ts \
  src/db/repositories/dual/DualWriteMarketplaceRepo.ts \
  src/db/repositories/mongo/MongoPaymentRepo.ts \
  src/db/repositories/drizzle/DrizzlePaymentRepo.ts \
  src/db/repositories/dual/DualWritePaymentRepo.ts \
  src/services/user/userController.ts \
  src/services/auth/authStore.ts \
  __tests__/auth-store-pg-query.test.ts \
  __tests__/user-dependencies-repo.test.ts \
  scripts/smoke/user-admin-postgres.sh \
  scripts/smoke/user-dependencies.sh

git commit -m "fix: route admin user counts through postgres-capable stores"
```

Frontend:

```bash
cd ../frontend
git add package.json
git commit -m "chore: sync frontend version to 2.8.38"
```

Do not push without doing the `nick-doc` sync afterward.

## Remaining Runtime Mongo Scan

Latest scan command:

```bash
rg -n --pcre2 "^import (?!type).*from ['\"]mongoose['\"]|^import (?!type).*from ['\"][^'\"]*models/|await import\(['\"][^'\"]*models/|countDocuments\(|deleteMany\(|findByIdAndDelete\(" \
  src/services src/routes src/app.ts src/infrastructure src/db/repositories/factory.ts \
  --glob '!**/*.test.ts' \
  --glob '!src/services/marketplace/routes.ts'
```

Important results and interpretation:

### Auth/User Routes

Paths still visible in scans:

```text
src/app.ts:672 AuthUser.countDocuments({ role: { $ne: 'admin' } })
src/services/user/userController.ts:306 User.countDocuments(filter)
src/services/user/userRoutes.ts multiple User.countDocuments(...)
src/services/auth/authStore.ts AuthUser.countDocuments implementation
```

Interpretation:

- These are mostly through the `AuthUser` facade, not direct Mongoose imports.
- In `AUTH_STORE=postgres` mode, `AuthUser.countDocuments` uses Postgres.
- This WIP improved the Postgres query-chain behavior and filter aliases.
- Future work should reduce the noisy model-shaped facade API over time, but these are not necessarily active Mongo runtime blockers when auth store is Postgres and Mongo fallback/mirroring are disabled.

Recommended follow-up:

- Add a `UserAdminRepo` or explicit auth-store helper methods for admin list/stats to replace model-shaped route code.
- After that, route `src/services/user/userController.ts` and `src/services/user/userRoutes.ts` through helper methods and remove the remaining `User.countDocuments` call sites from route/controller code.

### Admin Data Cleanup Service

High-priority blocker:

```text
src/services/admin/dataCleanupService.ts
```

Scan hits include dynamic model counting/deleting:

```text
Model.countDocuments(query)
Model.deleteMany(query)
User.countDocuments()
PurchaseRequest.countDocuments()
SellerOffer.countDocuments()
Payment.countDocuments()
Chat.countDocuments()
Notification.countDocuments()
RequestTemplate.countDocuments()
Address.countDocuments()
Category.countDocuments()
TempVerification.countDocuments()
User.findByIdAndDelete(userId)
```

Interpretation:

- This is the biggest remaining direct runtime Mongo/model surface.
- It likely imports or dynamically resolves models and is unsuitable for `MONGO_CONNECT_MODE=never`.

Recommended migration:

- Replace cleanup stats with repo-backed counts:
  - auth/user counts from AuthUser/Postgres helper
  - marketplace counts from MarketplaceRepo
  - payments from PaymentRepo
  - notifications from NotificationRepo
  - chat from ChatRepo
  - addresses/categories/reviews/temp verification through their Postgres-capable stores
- For destructive cleanup operations, either:
  - implement explicit Postgres cleanup repo methods with strong safety guards, or
  - disable Mongo-only cleanup actions when Mongo is disabled and return a clear `501`/unsupported result.

### Store Facades Still Exposing Model-Style Methods

Scan hits:

```text
src/services/points/levelConfigStore.ts deleteMany(...)
src/services/address/addressStore.ts findByIdAndDelete(...), countDocuments(...)
src/services/marketplace/reviewStore.ts countDocuments(...)
src/services/auth/authController.ts TempVerification.findByIdAndDelete(...)
src/services/auth/authStore.ts TempVerification findByIdAndDelete implementation
```

Interpretation:

- Some of these are behind Postgres-capable store facades.
- They still show up because they preserve a Mongoose-shaped API.
- For full Mongo removal, these facades must be audited under:
  - `MONGO_CONNECT_MODE=never`
  - store env set to `postgres`
  - fallback/mirror disabled where relevant

Recommended migration:

- Convert each store facade from "model-like object with Mongo fallback" to explicit repository functions.
- Add tests that set the store env to `postgres` and assert no Mongo model getter is called.

### Payment Coordinator

Scan hit:

```text
src/services/payment/paymentCoordinator.ts:510 paymentRepo.deleteMany({ idIn: duplicateIds })
```

Interpretation:

- This is already repository-routed, not raw Mongoose.
- Confirm `DrizzlePaymentRepo.deleteMany` exists and works for the duplicate cleanup path.

## Suggested Next Work Order

1. Finish and commit this WIP after review.
2. Run the two new smoke scripts with real dev admin credentials.
3. Update `nick-doc` after pushing backend/frontend.
4. Migrate `src/services/admin/dataCleanupService.ts`.
5. Replace user/admin list/stats route `User.countDocuments` calls with explicit auth/user helper methods so scans no longer flag route-level model-shaped calls.
6. Audit `addressStore`, `reviewStore`, `levelConfigStore`, and `TempVerification` under `MONGO_CONNECT_MODE=never`.
7. Once all runtime paths are Postgres-capable, set health logic so Mongo is optional and then remove startup Mongo requirement.
8. Final pass:
   - scan for non-test runtime `mongoose` / `models` imports
   - run typecheck
   - run focused Jest suites
   - run smoke scripts against local/dev
   - verify `/api/health` reports Mongo optional or absent and Postgres healthy

## Environment Notes For Cutover Testing

Useful envs for a no-Mongo runtime test:

```text
MONGO_CONNECT_MODE=never
AUTH_STORE=postgres
USER_STORE=postgres
AUTH_FALLBACK_MONGO=false
AUTH_MIRROR_MONGO=false
```

Also ensure all existing repo/store envs that support Postgres are set to `postgres` or `pg` consistently, including marketplace/payment/dispute/release-hold/notification/blog/address/category/review/level-config/shop-settings style stores.

Do not rely only on `rg` results: some model-shaped methods are already routed through Postgres facades. Prove no-Mongo runtime by running the backend with `MONGO_CONNECT_MODE=never` and exercising the API smoke scripts.

## Known Caveats

- The new smoke scripts need real admin credentials and were not executed end-to-end.
- The current WIP is not pushed and has no `nick-doc` sync yet.
- `nick-doc` and `deployment` have unrelated dirty files; do not stage them accidentally.
- The full goal is not complete. MongoDB is still a runtime dependency until the remaining service/store paths above are migrated and verified under `MONGO_CONNECT_MODE=never`.