nick-doc/01 - Architecture/Backend Architecture.md

title, tags, created

title	tags	created
Backend Architecture	architecture backend	2026-05-23

Backend Architecture

Module-level architecture of the Express 5 + TypeScript backend. MongoDB/Mongoose is still the primary runtime persistence layer; the integrate-main-into-development backend also contains the Drizzle/Postgres migration layer.

[!info] Repo: git@git.manko.yoga:222/nick/backend.git · Active integration branch: integrate-main-into-development · Current baseline: backend 2.6.79 at 3a50dc4

---

1. Folder tree

backend/src/
├── app.ts                      # Express bootstrap, middleware chain, route registration
├── config/                     # Per-feature config (legacy — most moved to shared/config)
├── controllers/                # HTTP request handlers (slim — delegate to services)
├── infrastructure/
│   ├── database/               # Mongoose connection, retries, graceful shutdown
│   └── socket/socketService.ts # Socket.IO server, rooms, emit helpers
├── models/                     # Mongoose models — see 02 - Data Models/
├── db/                         # Drizzle/Postgres migration layer: schemas, migrations, repos, backfill, verify
├── routes/                     # Express Router definitions (mounted in app.ts)
├── scripts/                    # CLI utilities (seed:users, seed:categories, ...)
├── seeds/                      # Seed data fixtures
├── services/
│   ├── ai/                     # OpenAI integration (descriptions, moderation)
│   ├── auth/                   # JWT, OAuth, Passkey, password reset
│   ├── blockchain/             # Web3 read/verify helpers
│   ├── blog/                   # Posts, categories, comments
│   ├── chat/                   # Conversations, messages, attachments
│   ├── dispute/                # Dispute lifecycle, evidence, mediator
│   ├── file/                   # Multer uploads, MIME validation
│   ├── marketplace/            # PurchaseRequest, SellerOffer, Template, Shop
│   ├── notification/           # Templates, delivery, mark-as-read
│   ├── payment/                # Payment orchestration + provider adapters + ledger
│   │   ├── adapters/           # Provider-neutral adapter interface + registry
│   │   ├── ledger/             # Internal funds ledger (available / held / releasable)
│   │   ├── reconciliation/     # Webhook + status reconciliation per provider
│   │   ├── migration/          # Legacy data backfill utilities
│   │   ├── observability/      # Logging and incident controls
│   │   ├── requestNetwork/     # Request Network pay-in, routes, webhook signature
│   │   ├── safety/             # Transaction Safety Provider + confirmation thresholds
│   │   └── wallets/            # Derived destination wallets + sweep orchestration
│   ├── points/                 # Loyalty points, levels, redemption
│   ├── redis/                  # Redis client, cache helpers
│   ├── telegram/               # Bot webhook, Mini App session, identity linking, notifications
│   ├── user/                   # Profile, preferences, addresses
│   ├── admin/                  # Admin-only operations
│   └── email/                  # Nodemailer transport + templates
├── shared/
│   ├── config/index.ts         # Centralised env-var loader (typed)
│   ├── middleware/             # auth, errorHandler, validators
│   ├── types/                  # Cross-cutting TypeScript types
│   └── utils/response-handler.ts # Standard success/error response envelope
└── utils/                      # Pure utility fns (logger, currencyUtils, etc.)

[!warning] Postgres is not the default runtime store yet src/db/repositories/factory.ts can select mongo, dual, or pg implementations for user, payment, points, and marketplace domains, but the broad service layer still imports Mongoose models directly. A code scan on 2026-05-31 found no runtime calls to createRepositories() / getPaymentRepo() / getMarketplaceRepo() outside the factory itself. See Postgres Runtime Cutover Status before assuming a REPO_* flag changes live behavior.

Tip

Service folders are self-contained: each typically has <feature>Service.ts, <feature>Controller.ts, <feature>Routes.ts, <feature>Validation.ts. This makes each service movable to a microservice later with minimal coupling.

---

2. Bootstrap — src/app.ts

The bootstrap is intentionally linear and easy to audit. Execution order:

1. Imports & env load — dotenv (if used), then import { config } from './shared/config'.
2. Express app construction — const app = express();
3. Trust proxy — app.set('trust proxy', config.trustProxy) so X-Forwarded-For works behind Nginx.
4. Security headers — app.use(helmet({ ... })).
5. CORS — cors({ origin: config.frontendUrl, credentials: true, methods: [...] }).
6. Body parsers — express.json({ limit: '10mb' }), express.urlencoded({ extended: true }).
7. Static uploads — app.use('/uploads', express.static(uploadDir)).
8. Health endpoint — GET /health for Docker healthcheck and external monitors.
9. Route mounting — every /api/* route registered before the error handler.
10. 404 handler — catches unmatched /api/*.
11. Error handler — central errorHandler middleware formats responses via response-handler.ts.
12. HTTP server creation — const server = http.createServer(app).
13. Socket.IO attach — initSocket(server, corsOptions) (see Real-time Layer).
14. DB connect — await connectDatabase() for MongoDB/Mongoose. Postgres connects lazily only when PG modules are imported (for example oracle quote persistence with ORACLE_QUOTING_ENABLED=true) and requires PG_URL.
15. Redis connect — await connectRedis().
16. Listen — server.listen(config.port, ...).
17. Graceful shutdown — SIGTERM/SIGINT handlers close server, drain sockets, close Mongoose, close Redis.
18. Optional dev seeding — when NODE_ENV === 'development' and SEED_USERS !== 'false', the bootstrap calls the seed scripts to provision default test users.

---

3. Middleware chain

Order	Middleware	Where	Purpose
1	helmet	global	Sets security headers (CSP, X-Frame-Options, ...).
2	cors	global	Origin allow-list = config.frontendUrl, credentials enabled.
3	express.json / express.urlencoded	global	Body parsers (10MB limit).
4	morgan (dev only)	global	HTTP request log to stdout.
5	requestId	global	Adds X-Request-Id for log correlation.
6	authMiddleware	per-route	Verifies JWT, attaches req.user. Mounted only on protected routes.
7	`roleGuard('admin'	'seller'	...)`
8	validate(schema)	per-route	express-validator + zod inputs.
9	controllerFn	per-route	Delegates to service layer.
10	notFound	tail	Returns 404 envelope for unmatched routes.
11	errorHandler	tail	Catches thrown errors, formats response.

Note

Rate-limit middleware is active as of 2026-05-24: auth 10 req/15 min, payment 30/15 min, AI 20/15 min, global 100/15 min. Request Network and Telegram webhooks are exempt from the global limiter. Counters are in-memory — a Redis adapter is planned for distributed deployments.

---

4. Route registration

The full route table mounted by app.ts:

Mount path	Module	Auth	Notes
/api/auth	services/auth/authRoutes.ts	mixed	login, register, refresh, OAuth, passkey
/api/user	services/user/userRoutes.ts	JWT	profile, preferences
/api/address	services/user/addressRoutes.ts	JWT	CRUD addresses
/api/marketplace/requests	services/marketplace/controllerRoutes.ts	JWT	PurchaseRequest CRUD
/api/marketplace/offers	services/marketplace/controllerRoutes.ts	JWT (seller)	SellerOffer CRUD
/api/marketplace/templates	services/marketplace/controllerRoutes.ts	JWT (seller)	RequestTemplate CRUD
/api/marketplace/categories	services/marketplace/controllerRoutes.ts	public read	Category list
/api/marketplace/shop-settings	services/marketplace/shopSettingsController.ts	JWT (seller)	Shop profile
/api/payment	services/payment/paymentControllerRoutes.ts + paymentRoutes.ts	JWT	Payment CRUD, health, export
/api/payment/decentralized	services/payment/decentralizedPaymentRoutes.ts	mixed	Legacy/manual Web3 save, verify, receiver
/api/payment/request-network	services/payment/requestNetwork/requestNetworkRoutes.ts	mixed + HMAC sig on webhook	Request Network pay-in creation, in-house checkout rehydrate, webhooks
/api/payment/derived-destinations	services/payment/wallets/derivedDestinationRoutes.ts	JWT (admin)	Derived address list, sweeps, cron, config health
/api/admin/rn/networks	services/payment/requestNetwork/networkRegistryRoutes.ts	JWT (admin)	Supported RN chain/token registry
/api/admin/settings/confirmation-thresholds	services/admin/confirmationThresholdRoutes.ts	JWT (admin)	Runtime min-confirmation thresholds
/api/admin/payments/awaiting-confirmation	services/admin/awaitingConfirmationRoutes.ts	JWT (admin)	Payments blocked on safety confirmations
/api/telegram	services/telegram/telegramRoutes.ts	mixed (some JWT, webhook uses secret-token)	Mini App verify/session, identity link/unlink, bot webhook
/api/chat	services/chat/chatRoutes.ts	JWT	Conversations, messages
/api/notification	services/notification/notificationRoutes.ts + notificationControllerRouter	JWT	List, mark read
/api/disputes	routes/disputeRoutes.ts + services/dispute/disputeRoutes.ts	JWT	Dispute CRUD plus release-hold helpers
/api/blog	services/blog/blogRoutes.ts	mixed	Public reads, admin writes
/api/admin/cleanup	services/admin/dataCleanupRoutes.ts	JWT (admin)	Data cleanup operations
/api/points	services/points/pointsRoutes.ts	JWT	Points, levels, referrals
/api/ai	services/ai/aiRoutes.ts	JWT	OpenAI-backed helpers
/api/files	services/file/fileRoutes.ts	JWT	Multipart upload
/api/email	services/email/emailRoutes.ts	JWT	Email dispatch
/api/trezor	services/trezor/trezorRoutes.ts	JWT	Trezor hardware-wallet ops
/api/users	services/user/userRoutes.ts	JWT	Legacy user profile routes

Full per-endpoint details → 03 - API Reference/API Overview and the service-specific reference docs.

---

5. Service layer pattern

Every service module follows this contract:

// services/<feature>/<feature>Service.ts
export class FeatureService {
  static async createX(input, ctx): Promise<X> { /* business logic */ }
  static async getX(id, ctx): Promise<X | null> { /* ... */ }
  static async listX(filter, ctx): Promise<X[]> { /* ... */ }
  static async updateX(id, patch, ctx): Promise<X> { /* ... */ }
}

• Controllers are thin — they validate request shape, call the service, format the response.
• Services own business logic, side effects (DB writes, socket emits, email sends).
• Models are pure schema — only Mongoose definitions + virtuals/hooks.

Cross-service calls are direct imports — no event bus yet. When the system grows, the seam between services is a natural place to introduce a message queue.

---

6. Dependency map (simplified)

flowchart TB
    auth[auth]
    user[user]
    market[marketplace]
    pay[payment]
    chat[chat]
    notify[notification]
    dispute[dispute]
    points[points]
    file[file]
    email[email]
    socket[socket]
    telegram[telegram]

    auth --> user
    auth --> notify
    auth --> telegram
    market --> notify
    market --> chat
    market --> file
    pay --> market
    pay --> notify
    pay --> socket
    telegram --> notify
    telegram --> auth
    dispute -.-> market
    dispute -.-> chat
    dispute -.-> notify
    points -.-> notify
    notify --> socket
    notify --> email

Note

socket and email are leaf services — every notification path funnels through them. Mocking these two in tests covers most side-effect verification.

---

7. Error handling

All thrown errors are caught by the central error handler. The expected shape:

class AppError extends Error {
  statusCode: number;   // HTTP 4xx/5xx
  code: string;         // app-specific code, e.g. "PAYMENT_ALREADY_REFUNDED"
  details?: unknown;    // optional debug payload
}

Response envelope (success path is {success:true,data:...}):

{
  "success": false,
  "error": {
    "code": "VALIDATION_FAILED",
    "message": "email is required",
    "details": [{ "path": "email", "msg": "required" }]
  }
}

See backend/src/shared/utils/response-handler.ts and backend/src/shared/middleware/errorHandler.ts.

---

8. Configuration

Single source of truth for env vars: src/shared/config/index.ts. It exports a typed config object — anywhere you would write process.env.X, instead import config.x.

Full table in Environment Variables. Critical ones:

Key	Default	Notes
PORT	5001	Listen port
MONGODB_URI	mongodb://localhost:27017/nickapp	Includes db name
REDIS_URI	redis://localhost:6379	+ REDIS_PASSWORD
JWT_SECRET	required	≥32 chars
JWT_EXPIRES_IN	7d
REFRESH_TOKEN_EXPIRES_IN	30d
FRONTEND_URL	http://localhost:3000	CORS origin
REQUEST_NETWORK_API_BASE_URL	https://api.request.network	Request Network API
REQUEST_NETWORK_API_KEY	required	Request Network API credential
REQUEST_NETWORK_WEBHOOK_SECRET	required	Webhook HMAC key
PAYMENT_LEDGER_ENFORCEMENT	false	Target true before launch-scale releases
TRANSACTION_SAFETY_*	required for payments	Confirmation, transfer-match, and AML controls
DERIVED_DESTINATION_SWEEP_SIGNER	build-only	Target hardware/Safe-backed signer
SMTP_*	required	Nodemailer
OPENAI_API_KEY	required

---

9. Database & connection management

• Mongoose is the ODM. Connection in src/infrastructure/database/.
• Connection options enable retryable writes, exponential backoff on reconnect.
• Indexes are defined on each model and auto-created on connect (Mongoose autoIndex: true in dev, recommend false in prod with explicit migration).
• See Data Model Overview for the relational map and per-model docs.

Redis client (in src/services/redis/) provides:

• Session caching (login attempts, lockout counters)
• Rate-limit counters (when middleware is enabled)
• Hot-path caches (category list, level configs)

---

10. Background work

The codebase has no dedicated queue runner — scheduled / async work is triggered inline from request handlers and uses setTimeout / setInterval patterns where needed (e.g., delayed retries). Consider introducing Bull / BullMQ if you grow:

• Request Network webhook replay/reconciliation and derived-destination balance checks
• Notification email digests
• Auto-release escrow timers
• Token / refresh-token cleanup

---

11. Testing

Jest test suites in backend/__tests__/:

File	Covers
models.test.ts	Schema validation, virtuals, hooks
payment-services.test.ts	Payment orchestration logic
complete-backend.test.ts	Cross-service integration
request-network-webhook.test.ts	Request Network webhook signature and processing
request-network-adapter.test.ts	Request Network payment adapter
payment-ledger.service.test.ts	Ledger append/reconciliation behavior
payment-release-refund-orchestration.test.ts	Release/refund instruction orchestration

Run with npm run test:all. CI runs the same. Reach for npm run test:models, npm run test:payment, etc. when iterating on a slice.

---

12. Notable files for orientation

File	Why it matters
src/app.ts	Bootstrap — read once to understand wiring
src/shared/config/index.ts	All env vars, typed
src/shared/utils/response-handler.ts	Standard response shape
src/shared/middleware/auth.ts	JWT verify + RBAC
src/infrastructure/socket/socketService.ts	All socket plumbing
src/services/payment/requestNetwork/requestNetworkRoutes.ts	Request Network checkout and webhook route
src/services/payment/ledger/fundsLedgerService.ts	Immutable payment ledger writes
src/services/marketplace/PurchaseRequestService.ts	Core marketplace state machine
src/services/auth/authService.ts	Auth flows, lockout, hashing
src/models/User.ts	Central entity with role/preferences
openapi.json	Generated API spec — definitive endpoint list

---

Related

• System Architecture — full system topology
• Frontend Architecture — how the FE talks to this BE
• Real-time Layer — Socket.IO room model
• Security Architecture — JWT, passkeys, webhook HMAC
• Data Model Overview — entity-relationship map
• Authentication Flow · Escrow Flow · Dispute Flow