nick-doc/08 - Operations/MONGODB_REMOVAL_HANDOFF_2026-06-02.md

MongoDB Runtime Removal Handoff

Date: 2026-06-02
Workspace: /Users/manwe/CascadeProjects/escrow
Goal: remove MongoDB as a runtime dependency by migrating remaining Mongo-backed backend domains and cutover paths to Postgres-compatible repositories.

Current State

No commits or pushes were made for the current WIP. The work is local only.

Repo heads at handoff time:

Repo	Branch	HEAD	State
backend	integrate-main-into-development	cf59726	dirty WIP
frontend	integrate-main-into-development	a2b972b	dirty version bump only
nick-doc	main	345c585	dirty unrelated local docs/profile files
deployment	main	8764fdf	dirty unrelated .env and docker-compose.yml

Backend/frontend package versions: backend at 2.8.79, frontend at 2.8.94.

Important repo rules:

• Any backend/frontend product change requires patch version bump in both repos.
• Before any backend push, run the relevant focused tests and smoke script.
• After every backend push, sync nick-doc:
  • append 09 - Audits/Activity Log.md
  • update relevant data/architecture docs
  • commit as docs: sync from backend <short-sha> — <summary>
  • push nick-doc
• Do not stage unrelated dirty files in nick-doc or deployment.

Dirty Files

Backend dirty files:

package-lock.json
package.json
src/db/repositories/drizzle/DrizzleMarketplaceRepo.ts
src/db/repositories/drizzle/DrizzlePaymentRepo.ts
src/db/repositories/dual/DualWriteMarketplaceRepo.ts
src/db/repositories/dual/DualWritePaymentRepo.ts
src/db/repositories/interfaces/IMarketplaceRepo.ts
src/db/repositories/interfaces/IPaymentRepo.ts
src/db/repositories/mongo/MongoMarketplaceRepo.ts
src/db/repositories/mongo/MongoPaymentRepo.ts
src/services/auth/authStore.ts
src/services/user/userController.ts
__tests__/auth-store-pg-query.test.ts
__tests__/user-dependencies-repo.test.ts
scripts/smoke/user-admin-postgres.sh
scripts/smoke/user-dependencies.sh

Frontend dirty files:

package.json

What This WIP Changes

1. Admin User Dependencies Endpoint

Endpoint affected:

GET /api/user/admin/:userId/dependencies

Before this WIP, src/services/user/userController.ts dynamically imported Mongo models and counted dependencies directly:

• RequestTemplate.countDocuments(...)
• PurchaseRequest.countDocuments(...)
• Payment.countDocuments(...)
• Chat.countDocuments(...)

This WIP replaces that direct model access with repository calls:

• getMarketplaceRepo().getUserDependencyCounts(userId)
• getPaymentRepo().countByParticipant(userId)
• getChatRepo().count({ 'participants.userId': userId, 'participants.isActive': true })

New/extended repository contract methods:

• IMarketplaceRepo.getUserDependencyCounts(userId)
• IPaymentRepo.countByParticipant(userId)

Implementations added:

• MongoMarketplaceRepo.getUserDependencyCounts
• DrizzleMarketplaceRepo.getUserDependencyCounts
• DualWriteMarketplaceRepo.getUserDependencyCounts
• MongoPaymentRepo.countByParticipant
• DrizzlePaymentRepo.countByParticipant
• DualWritePaymentRepo.countByParticipant

Behavior note:

• Postgres counts seller-side marketplace dependencies by joining purchase_requests.selected_offer_id to seller_offers.id and checking seller_offers.seller_id.
• Mongo implementation supports selected-offer-id style and also keeps compatibility with legacy embedded selectedOffer.sellerId.

New test:

__tests__/user-dependencies-repo.test.ts

New smoke script:

scripts/smoke/user-dependencies.sh

Smoke usage:

BASE_URL=https://dev.amn.gg ADMIN_TOKEN=<admin-jwt> USER_ID=<target-user-id> scripts/smoke/user-dependencies.sh

The smoke script checks:

• HTTP 200
• success === true
• data.user exists
• dependency counters are non-negative numbers
• total equals the sum of component counters

2. AuthUser Postgres Query Facade Hardening

Files affected:

src/services/auth/authStore.ts
__tests__/auth-store-pg-query.test.ts

The mounted user/admin list and stats routes already use AuthUser from authStore, not raw Mongoose imports. However, the Postgres PgQuery wrapper only sorted arrays by createdAt; admin list accepts arbitrary sortBy.

This WIP hardens the Postgres query wrapper so AuthUser.find(...).select(...).sort(...).skip(...).limit(...).lean() behaves more like the existing Mongoose chain:

• generic sorting by requested field
• nested path sorting support, e.g. profile.avatar
• date, number, boolean, and string comparison
• multi-field sort support
• keeps existing skip/limit/select/lean chain behavior

It also adds alias support in buildUserWhere:

• isActive: true maps to status = 'active'
• isActive: false maps to status <> 'active'
• isVerified maps to is_email_verified

This matters because:

• src/services/user/userController.ts builds filters with isActive / isVerified
• src/services/user/userRoutes.ts builds filters with status / isEmailVerified
• both now work in Postgres auth mode

New test:

__tests__/auth-store-pg-query.test.ts

New smoke script:

scripts/smoke/user-admin-postgres.sh

Smoke usage:

BASE_URL=https://dev.amn.gg ADMIN_TOKEN=<admin-jwt> scripts/smoke/user-admin-postgres.sh

The smoke script checks:

• GET /api/user/admin/list?page=1&limit=5&sortBy=firstName&sortOrder=asc
• GET /api/users/admin/stats
• response shape and numeric stats

Verification Already Run

Passed:

npm test -- --runTestsByPath __tests__/auth-store-pg-query.test.ts __tests__/user-dependencies-repo.test.ts __tests__/repository-factory-modes.test.ts __tests__/health-check-service.test.ts __tests__/marketplace-runtime-import-surface.test.ts --runInBand

Result:

Test Suites: 5 passed, 5 total
Tests: 11 passed, 11 total

Passed:

npm run typecheck

Passed:

git diff --check

for both backend and frontend.

Passed syntax checks:

bash -n scripts/smoke/user-admin-postgres.sh
bash -n scripts/smoke/user-dependencies.sh

Not run end-to-end:

• scripts/smoke/user-dependencies.sh
• scripts/smoke/user-admin-postgres.sh

Reason: both require an ADMIN_TOKEN; user-dependencies.sh also requires USER_ID.

How To Pick This Up

Start with:

cd /Users/manwe/CascadeProjects/escrow/backend
git status --short --branch
git diff --stat

Review the exact WIP:

git diff -- src/services/user/userController.ts src/services/auth/authStore.ts
git diff -- src/db/repositories/interfaces/IMarketplaceRepo.ts src/db/repositories/interfaces/IPaymentRepo.ts
git diff -- src/db/repositories/mongo/MongoMarketplaceRepo.ts src/db/repositories/drizzle/DrizzleMarketplaceRepo.ts src/db/repositories/dual/DualWriteMarketplaceRepo.ts
git diff -- src/db/repositories/mongo/MongoPaymentRepo.ts src/db/repositories/drizzle/DrizzlePaymentRepo.ts src/db/repositories/dual/DualWritePaymentRepo.ts
git diff -- __tests__/auth-store-pg-query.test.ts __tests__/user-dependencies-repo.test.ts
git diff -- scripts/smoke/user-admin-postgres.sh scripts/smoke/user-dependencies.sh

Re-run local verification:

npm test -- --runTestsByPath __tests__/auth-store-pg-query.test.ts __tests__/user-dependencies-repo.test.ts __tests__/repository-factory-modes.test.ts __tests__/health-check-service.test.ts __tests__/marketplace-runtime-import-surface.test.ts --runInBand
npm run typecheck
git diff --check

If you have a dev admin token:

BASE_URL=https://dev.amn.gg ADMIN_TOKEN=<admin-jwt> USER_ID=<target-user-id> scripts/smoke/user-dependencies.sh
BASE_URL=https://dev.amn.gg ADMIN_TOKEN=<admin-jwt> scripts/smoke/user-admin-postgres.sh

If you commit this WIP, suggested commit shape:

Backend:

git add package.json package-lock.json \
  src/db/repositories/interfaces/IMarketplaceRepo.ts \
  src/db/repositories/interfaces/IPaymentRepo.ts \
  src/db/repositories/mongo/MongoMarketplaceRepo.ts \
  src/db/repositories/drizzle/DrizzleMarketplaceRepo.ts \
  src/db/repositories/dual/DualWriteMarketplaceRepo.ts \
  src/db/repositories/mongo/MongoPaymentRepo.ts \
  src/db/repositories/drizzle/DrizzlePaymentRepo.ts \
  src/db/repositories/dual/DualWritePaymentRepo.ts \
  src/services/user/userController.ts \
  src/services/auth/authStore.ts \
  __tests__/auth-store-pg-query.test.ts \
  __tests__/user-dependencies-repo.test.ts \
  scripts/smoke/user-admin-postgres.sh \
  scripts/smoke/user-dependencies.sh

git commit -m "fix: route admin user counts through postgres-capable stores"

Frontend:

cd ../frontend
git add package.json
git commit -m "chore: sync frontend version to 2.8.38"

Do not push without doing the nick-doc sync afterward.

Remaining Runtime Mongo Scan

Latest scan command:

rg -n --pcre2 "^import (?!type).*from ['\"]mongoose['\"]|^import (?!type).*from ['\"][^'\"]*models/|await import\(['\"][^'\"]*models/|countDocuments\(|deleteMany\(|findByIdAndDelete\(" \
  src/services src/routes src/app.ts src/infrastructure src/db/repositories/factory.ts \
  --glob '!**/*.test.ts' \
  --glob '!src/services/marketplace/routes.ts'

Important results and interpretation:

Auth/User Routes

Paths still visible in scans:

src/app.ts:672 AuthUser.countDocuments({ role: { $ne: 'admin' } })
src/services/user/userController.ts:306 User.countDocuments(filter)
src/services/user/userRoutes.ts multiple User.countDocuments(...)
src/services/auth/authStore.ts AuthUser.countDocuments implementation

Interpretation:

• These are mostly through the AuthUser facade, not direct Mongoose imports.
• In AUTH_STORE=postgres mode, AuthUser.countDocuments uses Postgres.
• This WIP improved the Postgres query-chain behavior and filter aliases.
• Future work should reduce the noisy model-shaped facade API over time, but these are not necessarily active Mongo runtime blockers when auth store is Postgres and Mongo fallback/mirroring are disabled.

Recommended follow-up:

• Add a UserAdminRepo or explicit auth-store helper methods for admin list/stats to replace model-shaped route code.
• After that, route src/services/user/userController.ts and src/services/user/userRoutes.ts through helper methods and remove the remaining User.countDocuments call sites from route/controller code.

Admin Data Cleanup Service

High-priority blocker:

src/services/admin/dataCleanupService.ts

Scan hits include dynamic model counting/deleting:

Model.countDocuments(query)
Model.deleteMany(query)
User.countDocuments()
PurchaseRequest.countDocuments()
SellerOffer.countDocuments()
Payment.countDocuments()
Chat.countDocuments()
Notification.countDocuments()
RequestTemplate.countDocuments()
Address.countDocuments()
Category.countDocuments()
TempVerification.countDocuments()
User.findByIdAndDelete(userId)

Interpretation:

• This is the biggest remaining direct runtime Mongo/model surface.
• It likely imports or dynamically resolves models and is unsuitable for MONGO_CONNECT_MODE=never.

Recommended migration:

• Replace cleanup stats with repo-backed counts:
  • auth/user counts from AuthUser/Postgres helper
  • marketplace counts from MarketplaceRepo
  • payments from PaymentRepo
  • notifications from NotificationRepo
  • chat from ChatRepo
  • addresses/categories/reviews/temp verification through their Postgres-capable stores
• For destructive cleanup operations, either:
  • implement explicit Postgres cleanup repo methods with strong safety guards, or
  • disable Mongo-only cleanup actions when Mongo is disabled and return a clear 501/unsupported result.

Store Facades Still Exposing Model-Style Methods

Scan hits:

src/services/points/levelConfigStore.ts deleteMany(...)
src/services/address/addressStore.ts findByIdAndDelete(...), countDocuments(...)
src/services/marketplace/reviewStore.ts countDocuments(...)
src/services/auth/authController.ts TempVerification.findByIdAndDelete(...)
src/services/auth/authStore.ts TempVerification findByIdAndDelete implementation

Interpretation:

• Some of these are behind Postgres-capable store facades.
• They still show up because they preserve a Mongoose-shaped API.
• For full Mongo removal, these facades must be audited under:
  • MONGO_CONNECT_MODE=never
  • store env set to postgres
  • fallback/mirror disabled where relevant

Recommended migration:

• Convert each store facade from "model-like object with Mongo fallback" to explicit repository functions.
• Add tests that set the store env to postgres and assert no Mongo model getter is called.

Payment Coordinator

Scan hit:

src/services/payment/paymentCoordinator.ts:510 paymentRepo.deleteMany({ idIn: duplicateIds })

Interpretation:

• This is already repository-routed, not raw Mongoose.
• Confirm DrizzlePaymentRepo.deleteMany exists and works for the duplicate cleanup path.

Suggested Next Work Order

1. Finish and commit this WIP after review.
2. Run the two new smoke scripts with real dev admin credentials.
3. Update nick-doc after pushing backend/frontend.
4. Migrate src/services/admin/dataCleanupService.ts.
5. Replace user/admin list/stats route User.countDocuments calls with explicit auth/user helper methods so scans no longer flag route-level model-shaped calls.
6. Audit addressStore, reviewStore, levelConfigStore, and TempVerification under MONGO_CONNECT_MODE=never.
7. Once all runtime paths are Postgres-capable, set health logic so Mongo is optional and then remove startup Mongo requirement.
8. Final pass:
   • scan for non-test runtime mongoose / models imports
   • run typecheck
   • run focused Jest suites
   • run smoke scripts against local/dev
   • verify /api/health reports Mongo optional or absent and Postgres healthy

Environment Notes For Cutover Testing

Useful envs for a no-Mongo runtime test:

MONGO_CONNECT_MODE=never
AUTH_STORE=postgres
USER_STORE=postgres
AUTH_FALLBACK_MONGO=false
AUTH_MIRROR_MONGO=false

Also ensure all existing repo/store envs that support Postgres are set to postgres or pg consistently, including marketplace/payment/dispute/release-hold/notification/blog/address/category/review/level-config/shop-settings style stores.

Do not rely only on rg results: some model-shaped methods are already routed through Postgres facades. Prove no-Mongo runtime by running the backend with MONGO_CONNECT_MODE=never and exercising the API smoke scripts.

Known Caveats

• The new smoke scripts need real admin credentials and were not executed end-to-end.
• The current WIP is not pushed and has no nick-doc sync yet.
• nick-doc and deployment have unrelated dirty files; do not stage them accidentally.
• The full goal is not complete. MongoDB is still a runtime dependency until the remaining service/store paths above are migrated and verified under MONGO_CONNECT_MODE=never.