nick-doc/01 - Architecture/Backend Architecture.md

---
title: Backend Architecture
tags: [architecture, backend]
created: 2026-05-23
updated: 2026-06-03
---

# Backend Architecture

Module-level architecture of the Express 5 + TypeScript backend. The system is mid-migration: MongoDB/Mongoose remains the authoritative read store for most domains, with PostgreSQL (Drizzle ORM) running in dual-write mode across 18 landed migrations (0000–0017). The repository factory pattern (`src/db/repositories/factory.ts`) controls which backend each domain reads and writes through env flags.

> [!info]
> Repo: `git@git.manko.yoga:222/nick/backend.git` · Current version: `2.8.79` · 18 Drizzle migrations landed · Dual-write active across all major domains

---

## 1. Folder tree

```
backend/src/
├── app.ts                      # Express bootstrap, middleware chain, route registration
├── config/                     # Per-feature config (legacy — most moved to shared/config)
├── controllers/                # HTTP request handlers (slim — delegate to services)
├── infrastructure/
│   ├── database/               # Mongoose connection, retries, graceful shutdown
│   └── socket/socketService.ts # Socket.IO server, rooms, emit helpers
├── models/                     # Mongoose models — see 02 - Data Models/
├── db/                         # Drizzle/Postgres layer: schemas, migrations, repos, backfill, verify
│   ├── schema/                 # Per-table Drizzle schema files + index.ts barrel
│   ├── migrations/             # 18 numbered SQL migration files (0000–0017)
│   └── repositories/           # Drizzle repos, dual-write wrappers, factory.ts
├── routes/                     # Express Router definitions (mounted in app.ts)
├── scripts/                    # CLI utilities (seed:users, seed:categories, ...)
├── seeds/                      # Seed data fixtures (Postgres-capable as of v2.8.47)
├── services/
│   ├── ai/                     # OpenAI integration (descriptions, moderation)
│   ├── auth/                   # JWT, OAuth, Passkey, password reset
│   ├── blockchain/             # Web3 read/verify helpers
│   ├── blog/                   # Posts, categories, comments
│   ├── chat/                   # Conversations, messages, attachments
│   ├── dispute/                # Dispute lifecycle, evidence, mediator
│   ├── file/                   # Multer uploads, MIME validation
│   ├── marketplace/            # PurchaseRequest, SellerOffer, Template, Shop
│   ├── notification/           # Templates, delivery, mark-as-read
│   ├── payment/                # Payment orchestration + provider adapters + ledger
│   │   ├── adapters/           # Provider-neutral adapter interface + registry
│   │   ├── ledger/             # Internal funds ledger (available / held / releasable)
│   │   ├── reconciliation/     # Webhook + status reconciliation per provider
│   │   ├── migration/          # Legacy data backfill utilities
│   │   ├── observability/      # Logging and incident controls
│   │   ├── requestNetwork/     # Request Network pay-in, routes, webhook signature
│   │   ├── safety/             # Transaction Safety Provider + confirmation thresholds
│   │   └── wallets/            # Derived destination wallets + sweep orchestration
│   ├── points/                 # Loyalty points, levels, redemption
│   ├── redis/                  # Redis client, cache helpers
│   ├── telegram/               # Bot webhook, Mini App session, identity linking, notifications
│   ├── user/                   # Profile, preferences, addresses
│   ├── admin/                  # Admin-only operations
│   └── email/                  # Nodemailer transport + templates
├── shared/
│   ├── config/index.ts         # Centralised env-var loader (typed)
│   ├── middleware/             # auth, errorHandler, validators
│   ├── types/                  # Cross-cutting TypeScript types
│   └── utils/response-handler.ts # Standard success/error response envelope
└── utils/                      # Pure utility fns (logger, currencyUtils, etc.)
```

> [!warning] Reads still go to Mongo for all dual-write domains
> Even when `REPO_*=dual`, Mongo is the authoritative read source. The dual-write seam exists and is exercised in production, but no domain has been cut over to PG reads yet. See [[Postgres Runtime Cutover Status]] before assuming a `REPO_*` flag changes live read behavior.

> [!tip]
> Service folders are self-contained: each typically has `<feature>Service.ts`, `<feature>Controller.ts`, `<feature>Routes.ts`, `<feature>Validation.ts`. This makes each service movable to a microservice later with minimal coupling.

---

## 2. Bootstrap — `src/app.ts`

The bootstrap is intentionally linear and easy to audit. Execution order:

1. **Imports & env load** — `dotenv` (if used), then `import { config } from './shared/config'`.
2. **Express app construction** — `const app = express();`
3. **Trust proxy** — `app.set('trust proxy', config.trustProxy)` so X-Forwarded-For works behind Traefik.
4. **Security headers** — `app.use(helmet({ ... }))`.
5. **CORS** — `cors({ origin: config.frontendUrl, credentials: true, methods: [...] })`.
6. **Body parsers** — `express.json({ limit: '10mb' })`, `express.urlencoded({ extended: true })`.
7. **Static uploads** — `app.use('/uploads', express.static(uploadDir))`.
8. **Health endpoint** — `GET /health` for Docker healthcheck and external monitors. Now surfaces active Postgres store modes.
9. **Route mounting** — every `/api/*` route registered before the error handler.
10. **404 handler** — catches unmatched `/api/*`.
11. **Error handler** — central `errorHandler` middleware formats responses via `response-handler.ts`.
12. **HTTP server creation** — `const server = http.createServer(app)`.
13. **Socket.IO attach** — `initSocket(server, corsOptions)` (see [[Real-time Layer]]).
14. **DB connect** — controlled by `MONGO_CONNECT_MODE`:
    - `always` (default) — connects Mongoose (Mongo) and PostgreSQL (via `PG_URL`) on boot.
    - `never` — skips Mongo entirely; Postgres is the only persistence layer. Seeds are Postgres-capable in this mode.
    - `optional` — connects Postgres; Mongo is attempted but failures are non-fatal.
15. **Redis connect** — `await connectRedis()`.
16. **Listen** — `server.listen(config.port, ...)`.
17. **Graceful shutdown** — SIGTERM/SIGINT handlers close server, drain sockets, close Mongoose, close Redis.
18. **Optional dev seeding** — when `NODE_ENV === 'development'` and `SEED_USERS !== 'false'`, the bootstrap calls the seed scripts to provision default test users. Seeds are store-aware and run correctly against both Mongo and PG.

---

## 3. Middleware chain

| Order | Middleware | Where | Purpose |
|---|---|---|---|
| 1 | `helmet` | global | Sets security headers (CSP, X-Frame-Options, ...). |
| 2 | `cors` | global | Origin allow-list = `config.frontendUrl`, credentials enabled. |
| 3 | `express.json` / `express.urlencoded` | global | Body parsers (10MB limit). |
| 4 | `morgan` (dev only) | global | HTTP request log to stdout. |
| 5 | `requestId` | global | Adds `X-Request-Id` for log correlation. |
| 6 | `authMiddleware` | per-route | Verifies JWT, attaches `req.user`. Mounted only on protected routes. |
| 7 | `roleGuard('admin'\|'seller'\|'guard'\|...)` | per-route | RBAC check after auth. Roles: `admin`, `buyer`, `seller`, `resolver`, `guard`. |
| 8 | `validate(schema)` | per-route | express-validator + zod inputs. |
| 9 | `controllerFn` | per-route | Delegates to service layer. |
| 10 | `notFound` | tail | Returns 404 envelope for unmatched routes. |
| 11 | `errorHandler` | tail | Catches thrown errors, formats response. |

> [!note]
> Rate-limit middleware is **active**: auth 10 req/15 min, payment 30/15 min, AI 20/15 min, global 100/15 min. `GET /api/payment/:id` is exempt from the payment limiter (polling route). Request Network and Telegram webhooks are exempt from the global limiter. Counters are in-memory — a Redis adapter is planned for distributed deployments.

---

## 4. Route registration

The full route table mounted by `app.ts`:

| Mount path | Module | Auth | Notes |
|---|---|---|---|
| `/api/auth` | `services/auth/authRoutes.ts` | mixed | login, register, refresh, OAuth, passkey |
| `/api/user` | `services/user/userRoutes.ts` | JWT | profile, preferences |
| `/api/address` | `services/user/addressRoutes.ts` | JWT | CRUD addresses |
| `/api/marketplace/requests` | `services/marketplace/controllerRoutes.ts` | JWT | PurchaseRequest CRUD |
| `/api/marketplace/offers` | `services/marketplace/controllerRoutes.ts` | JWT (seller) | SellerOffer CRUD |
| `/api/marketplace/templates` | `services/marketplace/controllerRoutes.ts` | JWT (seller) | RequestTemplate CRUD |
| `/api/marketplace/categories` | `services/marketplace/controllerRoutes.ts` | public read | Category list |
| `/api/marketplace/shop-settings` | `services/marketplace/shopSettingsController.ts` | JWT (seller) | Shop profile; lookup tolerant of uuid/legacy id formats |
| `/api/payment` | `services/payment/paymentControllerRoutes.ts` + `paymentRoutes.ts` | JWT | Payment CRUD, health, export |
| `/api/payment/decentralized` | `services/payment/decentralizedPaymentRoutes.ts` | mixed | Legacy/manual Web3 save, verify, receiver |
| `/api/payment/request-network` | `services/payment/requestNetwork/requestNetworkRoutes.ts` | mixed + HMAC sig on webhook | Request Network pay-in creation, in-house checkout rehydrate, webhooks |
| `/api/payment/derived-destinations` | `services/payment/wallets/derivedDestinationRoutes.ts` | JWT (admin) | Derived address list, sweeps, cron, config health |
| `/api/admin/rn/networks` | `services/payment/requestNetwork/networkRegistryRoutes.ts` | JWT (admin) | Supported RN chain/token registry |
| `/api/admin/settings/confirmation-thresholds` | `services/admin/confirmationThresholdRoutes.ts` | JWT (admin) | Runtime min-confirmation thresholds |
| `/api/admin/payments/awaiting-confirmation` | `services/admin/awaitingConfirmationRoutes.ts` | JWT (admin) | Payments blocked on safety confirmations |
| `/api/telegram` | `services/telegram/telegramRoutes.ts` | mixed (some JWT, webhook uses secret-token) | Mini App verify/session, identity link/unlink, bot webhook; notifications delivered via Telegram as of v2.8.56 |
| `/api/chat` | `services/chat/chatRoutes.ts` | JWT | Conversations, messages |
| `/api/notification` | `services/notification/notificationRoutes.ts` + `notificationControllerRouter` | JWT | List, mark read |
| `/api/disputes` | `routes/disputeRoutes.ts` + `services/dispute/disputeRoutes.ts` | JWT | Dispute CRUD plus release-hold helpers |
| `/api/blog` | `services/blog/blogRoutes.ts` | mixed | Public reads, admin writes |
| `/api/admin/cleanup` | `services/admin/dataCleanupRoutes.ts` | JWT (admin) | Data cleanup; scoped by provider to avoid wiping RN/multi-seller records |
| `/api/points` | `services/points/pointsRoutes.ts` | JWT | Points, levels, referrals |
| `/api/ai` | `services/ai/aiRoutes.ts` | JWT | OpenAI-backed helpers |
| `/api/files` | `services/file/fileRoutes.ts` | JWT | Multipart upload |
| `/api/email` | `services/email/emailRoutes.ts` | JWT | Email dispatch |
| `/api/trezor` | `services/trezor/trezorRoutes.ts` | JWT | Trezor hardware-wallet ops |
| `/api/users` | `services/user/userRoutes.ts` | JWT | Legacy user profile routes |

Full per-endpoint details → [[03 - API Reference/API Overview]] and the service-specific reference docs.

---

## 5. Service layer pattern

Every service module follows this contract:

```ts
// services/<feature>/<feature>Service.ts
export class FeatureService {
  static async createX(input, ctx): Promise<X> { /* business logic */ }
  static async getX(id, ctx): Promise<X | null> { /* ... */ }
  static async listX(filter, ctx): Promise<X[]> { /* ... */ }
  static async updateX(id, patch, ctx): Promise<X> { /* ... */ }
}
```

- Controllers are **thin** — they validate request shape, call the service, format the response.
- Services own **business logic**, side effects (DB writes, socket emits, email sends).
- Models are **pure schema** — only Mongoose definitions + virtuals/hooks.

Cross-service calls are direct imports — no event bus yet. When the system grows, the seam between services is a natural place to introduce a message queue.

---

## 6. Dependency map (simplified)

```mermaid
flowchart TB
    auth[auth]
    user[user]
    market[marketplace]
    pay[payment]
    chat[chat]
    notify[notification]
    dispute[dispute]
    points[points]
    file[file]
    email[email]
    socket[socket]
    telegram[telegram]

    auth --> user
    auth --> notify
    auth --> telegram
    market --> notify
    market --> chat
    market --> file
    pay --> market
    pay --> notify
    pay --> socket
    telegram --> notify
    telegram --> auth
    dispute -.-> market
    dispute -.-> chat
    dispute -.-> notify
    points -.-> notify
    notify --> socket
    notify --> email
    notify --> telegram
```

> [!note]
> `socket`, `email`, and `telegram` are leaf notification sinks — every notification path funnels through them. Mocking these three in tests covers most side-effect verification. Telegram notification delivery was added in v2.8.56.

---

## 7. Error handling

All thrown errors are caught by the central error handler. The expected shape:

```ts
class AppError extends Error {
  statusCode: number;   // HTTP 4xx/5xx
  code: string;         // app-specific code, e.g. "PAYMENT_ALREADY_REFUNDED"
  details?: unknown;    // optional debug payload
}
```

Response envelope (success path is `{success:true,data:...}`):

```json
{
  "success": false,
  "error": {
    "code": "VALIDATION_FAILED",
    "message": "email is required",
    "details": [{ "path": "email", "msg": "required" }]
  }
}
```

See `backend/src/shared/utils/response-handler.ts` and `backend/src/shared/middleware/errorHandler.ts`.

---

## 8. Configuration

Single source of truth for env vars: `src/shared/config/index.ts`. It exports a typed `config` object — anywhere you would write `process.env.X`, instead import `config.x`.

Full table in [[Environment Variables]]. Critical ones:

| Key | Default | Notes |
|---|---|---|
| `PORT` | `5001` | Listen port |
| `MONGODB_URI` | `mongodb://localhost:27017/nickapp` | Includes db name; not required when `MONGO_CONNECT_MODE=never` |
| `MONGO_CONNECT_MODE` | `always` | `always` \| `never` \| `optional` — controls whether Mongoose connects on boot |
| `PG_URL` | required for PG | PostgreSQL connection string for Drizzle; required when any `REPO_*=pg\|dual` |
| `REDIS_URI` | `redis://localhost:6379` | + `REDIS_PASSWORD` |
| `JWT_SECRET` | required | ≥32 chars |
| `JWT_EXPIRES_IN` | `7d` | |
| `REFRESH_TOKEN_EXPIRES_IN` | `30d` | |
| `FRONTEND_URL` | `http://localhost:3000` | CORS origin |
| `REPO_DEFAULT` | `mongo` | Global fallback store mode for all domains (`mongo` \| `dual` \| `pg`) |
| `REPO_USER` | inherits `REPO_DEFAULT` | Per-domain override for user store |
| `REPO_PAYMENT` | inherits `REPO_DEFAULT` | Per-domain override for payment store |
| `REPO_POINTS` | inherits `REPO_DEFAULT` | Per-domain override for points store |
| `REPO_MARKETPLACE` | inherits `REPO_DEFAULT` | Per-domain override for marketplace store |
| `REPO_TREZOR` | inherits `REPO_DEFAULT` | Per-domain override for trezor store |
| `REPO_DERIVED_DESTINATION` | inherits `REPO_DEFAULT` | Per-domain override for derived destination store |
| `REPO_BLOG` \| `BLOG_STORE` | inherits `REPO_DEFAULT` | Per-domain override for blog store |
| `REPO_NOTIFICATION` \| `NOTIFICATION_STORE` | inherits `REPO_DEFAULT` | Per-domain override for notification store |
| `REPO_DISPUTE` \| `DISPUTE_STORE` | inherits `REPO_DEFAULT` | Per-domain override for dispute store |
| `REPO_CHAT` \| `CHAT_STORE` | inherits `REPO_DEFAULT` | Chat dual-write not implemented; `dual` silently uses Mongo |
| `REPO_RELEASE_HOLD` \| `RELEASE_HOLD_STORE` | inherits `REPO_DEFAULT` | Release-hold dual-write not implemented; `dual` silently uses Mongo |
| `REQUEST_NETWORK_API_BASE_URL` | `https://api.request.network` | Request Network API |
| `REQUEST_NETWORK_API_KEY` | required | Request Network API credential |
| `REQUEST_NETWORK_WEBHOOK_SECRET` | required | Webhook HMAC key |
| `PAYMENT_LEDGER_ENFORCEMENT` | `false` | Target `true` before launch-scale releases |
| `TRANSACTION_SAFETY_*` | required for payments | Confirmation, transfer-match, and AML controls |
| `DERIVED_DESTINATION_SWEEP_SIGNER` | `build-only` | Target hardware/Safe-backed signer |
| `SMTP_*` | required | Nodemailer |
| `OPENAI_API_KEY` | required | |
| `ORACLE_QUOTING_ENABLED` | `false` | Enables oracle-based depeg-protected payment quotes; requires `PG_URL` |

---

## 9. Database & connection management

The backend runs a **dual-database architecture** during the Mongo→Postgres migration. Both stores may be active simultaneously; which one serves each domain is controlled by `REPO_*` env flags.

### MongoDB / Mongoose

- ODM: Mongoose. Connection in `src/infrastructure/database/`.
- Connection options enable retryable writes, exponential backoff on reconnect.
- Indexes defined on each model and auto-created on connect (`autoIndex: true` in dev; recommend `false` in prod with explicit migration scripts).
- Remains the **authoritative read store** for all dual-write domains until read cutover is explicitly executed per domain.
- See [[Data Model Overview]] for the relational map and per-model docs.

### PostgreSQL / Drizzle

- ORM: Drizzle. Schemas in `src/db/schema/`, migrations in `src/db/migrations/` (18 migrations landed: 0000–0017 as of 2026-06-05).
- Managed via `drizzle-kit migrate` — never edit migration files manually.
- Connects lazily when any PG-capable store is imported, or eagerly on boot when `MONGO_CONNECT_MODE=never`.
- Every migrated table carries a `legacy_object_id text` column with a partial-unique index for idempotent backfill upserts.
- Money columns use `numeric(38,18)` (except `seller_offers`: `numeric(18,8)`). Blockchain balance columns use `numeric(78,0)` to hold uint256 without overflow.
- See [[Drizzle Schema Reference]] for the full per-table breakdown.

### Repository factory — `src/db/repositories/factory.ts`

The factory is the single routing layer between service code and the underlying store. It exposes per-domain getters and resolves the mode (`mongo` | `dual` | `pg`) in this order:

1. Per-domain env flag (e.g. `REPO_PAYMENT`)
2. `REPO_DEFAULT` (global staging-wide fallback)
3. Hardcoded default: `mongo`

Unrecognized values silently fall back to `mongo` — intentional safety net against typos on money writes.

| Domain | Getter | Dual-write | PG-only |
|---|---|---|---|
| user | `getUserRepo` | Yes (full trio) | Yes |
| payment | `getPaymentRepo` | Yes (full trio) | Yes |
| points | `getPointsRepo` | Yes (full trio) | Yes |
| marketplace | `getMarketplaceRepo` | Yes (full trio) | Yes |
| trezor | `getTrezorRepo` | Yes (full trio) | Yes |
| derivedDestination | `getDerivedDestinationRepo` | Yes (full trio) | Yes |
| blog | `getBlogRepo` | Yes (full trio) | Yes |
| notification | `getNotificationRepo` | Yes (full trio) | Yes |
| dispute | `getDisputeRepo` | Yes (full trio) | Yes |
| releaseHold | `getReleaseHoldRepo` | No — `dual` silently uses Mongo | Yes |
| chat | `getChatRepo` | No — `dual` silently uses Mongo | Yes |

> [!warning] `MONGO_CONNECT_MODE` is not handled by the factory
> `MONGO_CONNECT_MODE` is consumed by the Mongoose connection module, not by `factory.ts`. The factory only reads `REPO_*` flags. These two controls are orthogonal: `MONGO_CONNECT_MODE=never` prevents Mongoose from connecting, while `REPO_*=pg` prevents the factory from routing to Mongo. For a full PG-only boot, set **both**.

### Migration phase status (as of 2026-06-03)

| Phase | Status |
|---|---|
| Schema / migrations | Done — 18 migrations landed (0000–0017), all domain tables exist in PG |
| Dual-write seam | Done — active for all major domains via factory |
| Backfill tooling | Done — backfill + verification harness in `src/db/` |
| Reads cutover | Not started — all reads still served from Mongo |
| Chat normalization | Blocked — Chat stored as JSONB blobs; normalization required before PG read cutover |
| Mongo retirement | Future — blocked on per-domain read cutover completion |

### Infrastructure / bridge tables (PG-only)

- **`id_map`** — ObjectId → UUID bridge; every migrated entity upserts here during backfill/dual-write.
- **`pg_dualwrite_gaps`** — Append-only reconciliation log for failed PG dual-writes; includes severity, resolver notes, and error stack.
- **`payment_quotes`** — Oracle-based depeg-protected quote snapshots (1:1 with payments); PG-only, no Mongo equivalent. Only active when `ORACLE_QUOTING_ENABLED=true`.

### Redis

Redis client (in `src/services/redis/`) provides:
- Session caching (login attempts, lockout counters)
- Rate-limit counters (when middleware is enabled)
- Hot-path caches (category list, level configs)

---

## 10. Background work

The codebase has no dedicated queue runner — scheduled / async work is triggered inline from request handlers and uses `setTimeout` / `setInterval` patterns where needed (e.g., delayed retries). Consider introducing Bull / BullMQ if you grow:

- Request Network webhook replay/reconciliation and derived-destination balance checks
- Notification email digests
- Auto-release escrow timers
- Token / refresh-token cleanup

---

## 11. Testing

Jest test suites in `backend/__tests__/`:

| File | Covers |
|---|---|
| `models.test.ts` | Schema validation, virtuals, hooks |
| `payment-services.test.ts` | Payment orchestration logic |
| `complete-backend.test.ts` | Cross-service integration |
| `request-network-webhook.test.ts` | Request Network webhook signature and processing |
| `request-network-adapter.test.ts` | Request Network payment adapter |
| `payment-ledger.service.test.ts` | Ledger append/reconciliation behavior |
| `payment-release-refund-orchestration.test.ts` | Release/refund instruction orchestration |

Run with `npm run test:all`. CI runs the same. Reach for `npm run test:models`, `npm run test:payment`, etc. when iterating on a slice.

---

## 12. Notable files for orientation

| File | Why it matters |
|---|---|
| `src/app.ts` | Bootstrap — read once to understand wiring |
| `src/shared/config/index.ts` | All env vars, typed |
| `src/shared/utils/response-handler.ts` | Standard response shape |
| `src/shared/middleware/auth.ts` | JWT verify + RBAC |
| `src/infrastructure/socket/socketService.ts` | All socket plumbing |
| `src/db/repositories/factory.ts` | Store routing — which backend each domain uses |
| `src/db/schema/index.ts` | Drizzle schema barrel — all 25+ PG tables |
| `src/services/payment/requestNetwork/requestNetworkRoutes.ts` | Request Network checkout and webhook route |
| `src/services/payment/ledger/fundsLedgerService.ts` | Immutable payment ledger writes |
| `src/services/marketplace/PurchaseRequestService.ts` | Core marketplace state machine |
| `src/services/auth/authService.ts` | Auth flows, lockout, hashing |
| `src/models/User.ts` | Central entity with role/preferences |
| `openapi.json` | Generated API spec — definitive endpoint list |

---

## Related

- [[System Architecture]] — full system topology
- [[Frontend Architecture]] — how the FE talks to this BE
- [[Real-time Layer]] — Socket.IO room model
- [[Security Architecture]] — JWT, passkeys, webhook HMAC
- [[Data Model Overview]] — entity-relationship map (Mongoose)
- [[Drizzle Schema Reference]] — PostgreSQL table definitions, enums, migration status
- [[Postgres Runtime Cutover Status]] — per-domain read cutover tracker
- [[Authentication Flow]] · [[Escrow Flow]] · [[Dispute Flow]]