nick-doc/01 - Architecture/Backend Architecture.md

title, tags, created, updated

title	tags	created	updated
Backend Architecture	architecture backend	2026-05-23	2026-06-03

Backend Architecture

Module-level architecture of the Express 5 + TypeScript backend. The system is mid-migration: MongoDB/Mongoose remains the authoritative read store for most domains, with PostgreSQL (Drizzle ORM) running in dual-write mode across 17 landed migrations. The repository factory pattern (src/db/repositories/factory.ts) controls which backend each domain reads and writes through env flags.

[!info] Repo: git@git.manko.yoga:222/nick/backend.git · Current version: 2.8.56 · 17 Drizzle migrations landed · Dual-write active across all major domains

---

1. Folder tree

backend/src/
├── app.ts                      # Express bootstrap, middleware chain, route registration
├── config/                     # Per-feature config (legacy — most moved to shared/config)
├── controllers/                # HTTP request handlers (slim — delegate to services)
├── infrastructure/
│   ├── database/               # Mongoose connection, retries, graceful shutdown
│   └── socket/socketService.ts # Socket.IO server, rooms, emit helpers
├── models/                     # Mongoose models — see 02 - Data Models/
├── db/                         # Drizzle/Postgres layer: schemas, migrations, repos, backfill, verify
│   ├── schema/                 # Per-table Drizzle schema files + index.ts barrel
│   ├── migrations/             # 17 numbered SQL migration files
│   └── repositories/           # Drizzle repos, dual-write wrappers, factory.ts
├── routes/                     # Express Router definitions (mounted in app.ts)
├── scripts/                    # CLI utilities (seed:users, seed:categories, ...)
├── seeds/                      # Seed data fixtures (Postgres-capable as of v2.8.47)
├── services/
│   ├── ai/                     # OpenAI integration (descriptions, moderation)
│   ├── auth/                   # JWT, OAuth, Passkey, password reset
│   ├── blockchain/             # Web3 read/verify helpers
│   ├── blog/                   # Posts, categories, comments
│   ├── chat/                   # Conversations, messages, attachments
│   ├── dispute/                # Dispute lifecycle, evidence, mediator
│   ├── file/                   # Multer uploads, MIME validation
│   ├── marketplace/            # PurchaseRequest, SellerOffer, Template, Shop
│   ├── notification/           # Templates, delivery, mark-as-read
│   ├── payment/                # Payment orchestration + provider adapters + ledger
│   │   ├── adapters/           # Provider-neutral adapter interface + registry
│   │   ├── ledger/             # Internal funds ledger (available / held / releasable)
│   │   ├── reconciliation/     # Webhook + status reconciliation per provider
│   │   ├── migration/          # Legacy data backfill utilities
│   │   ├── observability/      # Logging and incident controls
│   │   ├── requestNetwork/     # Request Network pay-in, routes, webhook signature
│   │   ├── safety/             # Transaction Safety Provider + confirmation thresholds
│   │   └── wallets/            # Derived destination wallets + sweep orchestration
│   ├── points/                 # Loyalty points, levels, redemption
│   ├── redis/                  # Redis client, cache helpers
│   ├── telegram/               # Bot webhook, Mini App session, identity linking, notifications
│   ├── user/                   # Profile, preferences, addresses
│   ├── admin/                  # Admin-only operations
│   └── email/                  # Nodemailer transport + templates
├── shared/
│   ├── config/index.ts         # Centralised env-var loader (typed)
│   ├── middleware/             # auth, errorHandler, validators
│   ├── types/                  # Cross-cutting TypeScript types
│   └── utils/response-handler.ts # Standard success/error response envelope
└── utils/                      # Pure utility fns (logger, currencyUtils, etc.)

[!warning] Reads still go to Mongo for all dual-write domains Even when REPO_*=dual, Mongo is the authoritative read source. The dual-write seam exists and is exercised in production, but no domain has been cut over to PG reads yet. See Postgres Runtime Cutover Status before assuming a REPO_* flag changes live read behavior.

Tip

Service folders are self-contained: each typically has <feature>Service.ts, <feature>Controller.ts, <feature>Routes.ts, <feature>Validation.ts. This makes each service movable to a microservice later with minimal coupling.

---

2. Bootstrap — src/app.ts

The bootstrap is intentionally linear and easy to audit. Execution order:

1. Imports & env load — dotenv (if used), then import { config } from './shared/config'.
2. Express app construction — const app = express();
3. Trust proxy — app.set('trust proxy', config.trustProxy) so X-Forwarded-For works behind Traefik.
4. Security headers — app.use(helmet({ ... })).
5. CORS — cors({ origin: config.frontendUrl, credentials: true, methods: [...] }).
6. Body parsers — express.json({ limit: '10mb' }), express.urlencoded({ extended: true }).
7. Static uploads — app.use('/uploads', express.static(uploadDir)).
8. Health endpoint — GET /health for Docker healthcheck and external monitors. Now surfaces active Postgres store modes.
9. Route mounting — every /api/* route registered before the error handler.
10. 404 handler — catches unmatched /api/*.
11. Error handler — central errorHandler middleware formats responses via response-handler.ts.
12. HTTP server creation — const server = http.createServer(app).
13. Socket.IO attach — initSocket(server, corsOptions) (see Real-time Layer).
14. DB connect — controlled by MONGO_CONNECT_MODE:
    • always (default) — connects Mongoose (Mongo) and PostgreSQL (via PG_URL) on boot.
    • never — skips Mongo entirely; Postgres is the only persistence layer. Seeds are Postgres-capable in this mode.
    • optional — connects Postgres; Mongo is attempted but failures are non-fatal.
15. Redis connect — await connectRedis().
16. Listen — server.listen(config.port, ...).
17. Graceful shutdown — SIGTERM/SIGINT handlers close server, drain sockets, close Mongoose, close Redis.
18. Optional dev seeding — when NODE_ENV === 'development' and SEED_USERS !== 'false', the bootstrap calls the seed scripts to provision default test users. Seeds are store-aware and run correctly against both Mongo and PG.

---

3. Middleware chain

Order	Middleware	Where	Purpose
1	helmet	global	Sets security headers (CSP, X-Frame-Options, ...).
2	cors	global	Origin allow-list = config.frontendUrl, credentials enabled.
3	express.json / express.urlencoded	global	Body parsers (10MB limit).
4	morgan (dev only)	global	HTTP request log to stdout.
5	requestId	global	Adds X-Request-Id for log correlation.
6	authMiddleware	per-route	Verifies JWT, attaches req.user. Mounted only on protected routes.
7	roleGuard('admin'|'seller'|'guard'|...)	per-route	RBAC check after auth. Roles: admin, buyer, seller, resolver, guard.
8	validate(schema)	per-route	express-validator + zod inputs.
9	controllerFn	per-route	Delegates to service layer.
10	notFound	tail	Returns 404 envelope for unmatched routes.
11	errorHandler	tail	Catches thrown errors, formats response.

Note

Rate-limit middleware is active: auth 10 req/15 min, payment 30/15 min, AI 20/15 min, global 100/15 min. GET /api/payment/:id is exempt from the payment limiter (polling route). Request Network and Telegram webhooks are exempt from the global limiter. Counters are in-memory — a Redis adapter is planned for distributed deployments.

---

4. Route registration

The full route table mounted by app.ts:

Mount path	Module	Auth	Notes
/api/auth	services/auth/authRoutes.ts	mixed	login, register, refresh, OAuth, passkey
/api/user	services/user/userRoutes.ts	JWT	profile, preferences
/api/address	services/user/addressRoutes.ts	JWT	CRUD addresses
/api/marketplace/requests	services/marketplace/controllerRoutes.ts	JWT	PurchaseRequest CRUD
/api/marketplace/offers	services/marketplace/controllerRoutes.ts	JWT (seller)	SellerOffer CRUD
/api/marketplace/templates	services/marketplace/controllerRoutes.ts	JWT (seller)	RequestTemplate CRUD
/api/marketplace/categories	services/marketplace/controllerRoutes.ts	public read	Category list
/api/marketplace/shop-settings	services/marketplace/shopSettingsController.ts	JWT (seller)	Shop profile; lookup tolerant of uuid/legacy id formats
/api/payment	services/payment/paymentControllerRoutes.ts + paymentRoutes.ts	JWT	Payment CRUD, health, export
/api/payment/decentralized	services/payment/decentralizedPaymentRoutes.ts	mixed	Legacy/manual Web3 save, verify, receiver
/api/payment/request-network	services/payment/requestNetwork/requestNetworkRoutes.ts	mixed + HMAC sig on webhook	Request Network pay-in creation, in-house checkout rehydrate, webhooks
/api/payment/derived-destinations	services/payment/wallets/derivedDestinationRoutes.ts	JWT (admin)	Derived address list, sweeps, cron, config health
/api/admin/rn/networks	services/payment/requestNetwork/networkRegistryRoutes.ts	JWT (admin)	Supported RN chain/token registry
/api/admin/settings/confirmation-thresholds	services/admin/confirmationThresholdRoutes.ts	JWT (admin)	Runtime min-confirmation thresholds
/api/admin/payments/awaiting-confirmation	services/admin/awaitingConfirmationRoutes.ts	JWT (admin)	Payments blocked on safety confirmations
/api/telegram	services/telegram/telegramRoutes.ts	mixed (some JWT, webhook uses secret-token)	Mini App verify/session, identity link/unlink, bot webhook; notifications delivered via Telegram as of v2.8.56
/api/chat	services/chat/chatRoutes.ts	JWT	Conversations, messages
/api/notification	services/notification/notificationRoutes.ts + notificationControllerRouter	JWT	List, mark read
/api/disputes	routes/disputeRoutes.ts + services/dispute/disputeRoutes.ts	JWT	Dispute CRUD plus release-hold helpers
/api/blog	services/blog/blogRoutes.ts	mixed	Public reads, admin writes
/api/admin/cleanup	services/admin/dataCleanupRoutes.ts	JWT (admin)	Data cleanup; scoped by provider to avoid wiping RN/multi-seller records
/api/points	services/points/pointsRoutes.ts	JWT	Points, levels, referrals
/api/ai	services/ai/aiRoutes.ts	JWT	OpenAI-backed helpers
/api/files	services/file/fileRoutes.ts	JWT	Multipart upload
/api/email	services/email/emailRoutes.ts	JWT	Email dispatch
/api/trezor	services/trezor/trezorRoutes.ts	JWT	Trezor hardware-wallet ops
/api/users	services/user/userRoutes.ts	JWT	Legacy user profile routes

Full per-endpoint details → 03 - API Reference/API Overview and the service-specific reference docs.

---

5. Service layer pattern

Every service module follows this contract:

// services/<feature>/<feature>Service.ts
export class FeatureService {
  static async createX(input, ctx): Promise<X> { /* business logic */ }
  static async getX(id, ctx): Promise<X | null> { /* ... */ }
  static async listX(filter, ctx): Promise<X[]> { /* ... */ }
  static async updateX(id, patch, ctx): Promise<X> { /* ... */ }
}

• Controllers are thin — they validate request shape, call the service, format the response.
• Services own business logic, side effects (DB writes, socket emits, email sends).
• Models are pure schema — only Mongoose definitions + virtuals/hooks.

Cross-service calls are direct imports — no event bus yet. When the system grows, the seam between services is a natural place to introduce a message queue.

---

6. Dependency map (simplified)

flowchart TB
    auth[auth]
    user[user]
    market[marketplace]
    pay[payment]
    chat[chat]
    notify[notification]
    dispute[dispute]
    points[points]
    file[file]
    email[email]
    socket[socket]
    telegram[telegram]

    auth --> user
    auth --> notify
    auth --> telegram
    market --> notify
    market --> chat
    market --> file
    pay --> market
    pay --> notify
    pay --> socket
    telegram --> notify
    telegram --> auth
    dispute -.-> market
    dispute -.-> chat
    dispute -.-> notify
    points -.-> notify
    notify --> socket
    notify --> email
    notify --> telegram

Note

socket, email, and telegram are leaf notification sinks — every notification path funnels through them. Mocking these three in tests covers most side-effect verification. Telegram notification delivery was added in v2.8.56.

---

7. Error handling

All thrown errors are caught by the central error handler. The expected shape:

class AppError extends Error {
  statusCode: number;   // HTTP 4xx/5xx
  code: string;         // app-specific code, e.g. "PAYMENT_ALREADY_REFUNDED"
  details?: unknown;    // optional debug payload
}

Response envelope (success path is {success:true,data:...}):

{
  "success": false,
  "error": {
    "code": "VALIDATION_FAILED",
    "message": "email is required",
    "details": [{ "path": "email", "msg": "required" }]
  }
}

See backend/src/shared/utils/response-handler.ts and backend/src/shared/middleware/errorHandler.ts.

---

8. Configuration

Single source of truth for env vars: src/shared/config/index.ts. It exports a typed config object — anywhere you would write process.env.X, instead import config.x.

Full table in Environment Variables. Critical ones:

Key	Default	Notes
PORT	5001	Listen port
MONGODB_URI	mongodb://localhost:27017/nickapp	Includes db name; not required when MONGO_CONNECT_MODE=never
MONGO_CONNECT_MODE	always	always | never | optional — controls whether Mongoose connects on boot
PG_URL	required for PG	PostgreSQL connection string for Drizzle; required when any REPO_*=pg|dual
REDIS_URI	redis://localhost:6379	+ REDIS_PASSWORD
JWT_SECRET	required	≥32 chars
JWT_EXPIRES_IN	7d
REFRESH_TOKEN_EXPIRES_IN	30d
FRONTEND_URL	http://localhost:3000	CORS origin
REPO_DEFAULT	mongo	Global fallback store mode for all domains (mongo | dual | pg)
REPO_USER	inherits REPO_DEFAULT	Per-domain override for user store
REPO_PAYMENT	inherits REPO_DEFAULT	Per-domain override for payment store
REPO_POINTS	inherits REPO_DEFAULT	Per-domain override for points store
REPO_MARKETPLACE	inherits REPO_DEFAULT	Per-domain override for marketplace store
REPO_TREZOR	inherits REPO_DEFAULT	Per-domain override for trezor store
REPO_DERIVED_DESTINATION	inherits REPO_DEFAULT	Per-domain override for derived destination store
REPO_BLOG | BLOG_STORE	inherits REPO_DEFAULT	Per-domain override for blog store
REPO_NOTIFICATION | NOTIFICATION_STORE	inherits REPO_DEFAULT	Per-domain override for notification store
REPO_DISPUTE | DISPUTE_STORE	inherits REPO_DEFAULT	Per-domain override for dispute store
REPO_CHAT | CHAT_STORE	inherits REPO_DEFAULT	Chat dual-write not implemented; dual silently uses Mongo
REPO_RELEASE_HOLD | RELEASE_HOLD_STORE	inherits REPO_DEFAULT	Release-hold dual-write not implemented; dual silently uses Mongo
REQUEST_NETWORK_API_BASE_URL	https://api.request.network	Request Network API
REQUEST_NETWORK_API_KEY	required	Request Network API credential
REQUEST_NETWORK_WEBHOOK_SECRET	required	Webhook HMAC key
PAYMENT_LEDGER_ENFORCEMENT	false	Target true before launch-scale releases
TRANSACTION_SAFETY_*	required for payments	Confirmation, transfer-match, and AML controls
DERIVED_DESTINATION_SWEEP_SIGNER	build-only	Target hardware/Safe-backed signer
SMTP_*	required	Nodemailer
OPENAI_API_KEY	required
ORACLE_QUOTING_ENABLED	false	Enables oracle-based depeg-protected payment quotes; requires PG_URL

---

9. Database & connection management

The backend runs a dual-database architecture during the Mongo→Postgres migration. Both stores may be active simultaneously; which one serves each domain is controlled by REPO_* env flags.

MongoDB / Mongoose

• ODM: Mongoose. Connection in src/infrastructure/database/.
• Connection options enable retryable writes, exponential backoff on reconnect.
• Indexes defined on each model and auto-created on connect (autoIndex: true in dev; recommend false in prod with explicit migration scripts).
• Remains the authoritative read store for all dual-write domains until read cutover is explicitly executed per domain.
• See Data Model Overview for the relational map and per-model docs.

PostgreSQL / Drizzle

• ORM: Drizzle. Schemas in src/db/schema/, migrations in src/db/migrations/ (17 migrations landed as of 2026-06-03).
• Managed via drizzle-kit migrate — never edit migration files manually.
• Connects lazily when any PG-capable store is imported, or eagerly on boot when MONGO_CONNECT_MODE=never.
• Every migrated table carries a legacy_object_id text column with a partial-unique index for idempotent backfill upserts.
• Money columns use numeric(38,18) (except seller_offers: numeric(18,8)). Blockchain balance columns use numeric(78,0) to hold uint256 without overflow.
• See Drizzle Schema Reference for the full per-table breakdown.

Repository factory — src/db/repositories/factory.ts

The factory is the single routing layer between service code and the underlying store. It exposes per-domain getters and resolves the mode (mongo | dual | pg) in this order:

1. Per-domain env flag (e.g. REPO_PAYMENT)
2. REPO_DEFAULT (global staging-wide fallback)
3. Hardcoded default: mongo

Unrecognized values silently fall back to mongo — intentional safety net against typos on money writes.

Domain	Getter	Dual-write	PG-only
user	getUserRepo	Yes (full trio)	Yes
payment	getPaymentRepo	Yes (full trio)	Yes
points	getPointsRepo	Yes (full trio)	Yes
marketplace	getMarketplaceRepo	Yes (full trio)	Yes
trezor	getTrezorRepo	Yes (full trio)	Yes
derivedDestination	getDerivedDestinationRepo	Yes (full trio)	Yes
blog	getBlogRepo	Yes (full trio)	Yes
notification	getNotificationRepo	Yes (full trio)	Yes
dispute	getDisputeRepo	Yes (full trio)	Yes
releaseHold	getReleaseHoldRepo	No — dual silently uses Mongo	Yes
chat	getChatRepo	No — dual silently uses Mongo	Yes

[!warning] MONGO_CONNECT_MODE is not handled by the factory MONGO_CONNECT_MODE is consumed by the Mongoose connection module, not by factory.ts. The factory only reads REPO_* flags. These two controls are orthogonal: MONGO_CONNECT_MODE=never prevents Mongoose from connecting, while REPO_*=pg prevents the factory from routing to Mongo. For a full PG-only boot, set both.

Migration phase status (as of 2026-06-03)

Phase	Status
Schema / migrations	Done — 17 migrations landed, all domain tables exist in PG
Dual-write seam	Done — active for all major domains via factory
Backfill tooling	Done — backfill + verification harness in src/db/
Reads cutover	Not started — all reads still served from Mongo
Chat normalization	Blocked — Chat stored as JSONB blobs; normalization required before PG read cutover
Mongo retirement	Future — blocked on per-domain read cutover completion

Infrastructure / bridge tables (PG-only)

• id_map — ObjectId → UUID bridge; every migrated entity upserts here during backfill/dual-write.
• pg_dualwrite_gaps — Append-only reconciliation log for failed PG dual-writes; includes severity, resolver notes, and error stack.
• payment_quotes — Oracle-based depeg-protected quote snapshots (1:1 with payments); PG-only, no Mongo equivalent. Only active when ORACLE_QUOTING_ENABLED=true.

Redis

Redis client (in src/services/redis/) provides:

• Session caching (login attempts, lockout counters)
• Rate-limit counters (when middleware is enabled)
• Hot-path caches (category list, level configs)

---

10. Background work

The codebase has no dedicated queue runner — scheduled / async work is triggered inline from request handlers and uses setTimeout / setInterval patterns where needed (e.g., delayed retries). Consider introducing Bull / BullMQ if you grow:

• Request Network webhook replay/reconciliation and derived-destination balance checks
• Notification email digests
• Auto-release escrow timers
• Token / refresh-token cleanup

---

11. Testing

Jest test suites in backend/__tests__/:

File	Covers
models.test.ts	Schema validation, virtuals, hooks
payment-services.test.ts	Payment orchestration logic
complete-backend.test.ts	Cross-service integration
request-network-webhook.test.ts	Request Network webhook signature and processing
request-network-adapter.test.ts	Request Network payment adapter
payment-ledger.service.test.ts	Ledger append/reconciliation behavior
payment-release-refund-orchestration.test.ts	Release/refund instruction orchestration

Run with npm run test:all. CI runs the same. Reach for npm run test:models, npm run test:payment, etc. when iterating on a slice.

---

12. Notable files for orientation

File	Why it matters
src/app.ts	Bootstrap — read once to understand wiring
src/shared/config/index.ts	All env vars, typed
src/shared/utils/response-handler.ts	Standard response shape
src/shared/middleware/auth.ts	JWT verify + RBAC
src/infrastructure/socket/socketService.ts	All socket plumbing
src/db/repositories/factory.ts	Store routing — which backend each domain uses
src/db/schema/index.ts	Drizzle schema barrel — all 25+ PG tables
src/services/payment/requestNetwork/requestNetworkRoutes.ts	Request Network checkout and webhook route
src/services/payment/ledger/fundsLedgerService.ts	Immutable payment ledger writes
src/services/marketplace/PurchaseRequestService.ts	Core marketplace state machine
src/services/auth/authService.ts	Auth flows, lockout, hashing
src/models/User.ts	Central entity with role/preferences
openapi.json	Generated API spec — definitive endpoint list

---

Related

• System Architecture — full system topology
• Frontend Architecture — how the FE talks to this BE
• Real-time Layer — Socket.IO room model
• Security Architecture — JWT, passkeys, webhook HMAC
• Data Model Overview — entity-relationship map (Mongoose)
• Drizzle Schema Reference — PostgreSQL table definitions, enums, migration status
• Postgres Runtime Cutover Status — per-domain read cutover tracker
• Authentication Flow · Escrow Flow · Dispute Flow