343 lines
17 KiB
Markdown
343 lines
17 KiB
Markdown
---
|
|
title: Backend Architecture
|
|
tags: [architecture, backend]
|
|
created: 2026-05-23
|
|
---
|
|
|
|
# Backend Architecture
|
|
|
|
Module-level architecture of the Express 5 + TypeScript backend. MongoDB/Mongoose is still the primary runtime persistence layer; the `integrate-main-into-development` backend also contains the Drizzle/Postgres migration layer.
|
|
|
|
> [!info]
|
|
> Repo: `git@git.manko.yoga:222/nick/backend.git` · Active integration branch: `integrate-main-into-development` · Current baseline: backend `2.6.79` at `3a50dc4`
|
|
|
|
---
|
|
|
|
## 1. Folder tree
|
|
|
|
```
|
|
backend/src/
|
|
├── app.ts # Express bootstrap, middleware chain, route registration
|
|
├── config/ # Per-feature config (legacy — most moved to shared/config)
|
|
├── controllers/ # HTTP request handlers (slim — delegate to services)
|
|
├── infrastructure/
|
|
│ ├── database/ # Mongoose connection, retries, graceful shutdown
|
|
│ └── socket/socketService.ts # Socket.IO server, rooms, emit helpers
|
|
├── models/ # Mongoose models — see 02 - Data Models/
|
|
├── db/ # Drizzle/Postgres migration layer: schemas, migrations, repos, backfill, verify
|
|
├── routes/ # Express Router definitions (mounted in app.ts)
|
|
├── scripts/ # CLI utilities (seed:users, seed:categories, ...)
|
|
├── seeds/ # Seed data fixtures
|
|
├── services/
|
|
│ ├── ai/ # OpenAI integration (descriptions, moderation)
|
|
│ ├── auth/ # JWT, OAuth, Passkey, password reset
|
|
│ ├── blockchain/ # Web3 read/verify helpers
|
|
│ ├── blog/ # Posts, categories, comments
|
|
│ ├── chat/ # Conversations, messages, attachments
|
|
│ ├── dispute/ # Dispute lifecycle, evidence, mediator
|
|
│ ├── file/ # Multer uploads, MIME validation
|
|
│ ├── marketplace/ # PurchaseRequest, SellerOffer, Template, Shop
|
|
│ ├── notification/ # Templates, delivery, mark-as-read
|
|
│ ├── payment/ # Payment orchestration + provider adapters + ledger
|
|
│ │ ├── adapters/ # Provider-neutral adapter interface + registry
|
|
│ │ ├── ledger/ # Internal funds ledger (available / held / releasable)
|
|
│ │ ├── reconciliation/ # Webhook + status reconciliation per provider
|
|
│ │ ├── migration/ # Legacy data backfill utilities
|
|
│ │ ├── observability/ # Logging and incident controls
|
|
│ │ ├── requestNetwork/ # Request Network pay-in, routes, webhook signature
|
|
│ │ ├── safety/ # Transaction Safety Provider + confirmation thresholds
|
|
│ │ └── wallets/ # Derived destination wallets + sweep orchestration
|
|
│ ├── points/ # Loyalty points, levels, redemption
|
|
│ ├── redis/ # Redis client, cache helpers
|
|
│ ├── telegram/ # Bot webhook, Mini App session, identity linking, notifications
|
|
│ ├── user/ # Profile, preferences, addresses
|
|
│ ├── admin/ # Admin-only operations
|
|
│ └── email/ # Nodemailer transport + templates
|
|
├── shared/
|
|
│ ├── config/index.ts # Centralised env-var loader (typed)
|
|
│ ├── middleware/ # auth, errorHandler, validators
|
|
│ ├── types/ # Cross-cutting TypeScript types
|
|
│ └── utils/response-handler.ts # Standard success/error response envelope
|
|
└── utils/ # Pure utility fns (logger, currencyUtils, etc.)
|
|
```
|
|
|
|
> [!warning] Postgres is not the default runtime store yet
|
|
> `src/db/repositories/factory.ts` can select `mongo`, `dual`, or `pg` implementations for user, payment, points, and marketplace domains, but the broad service layer still imports Mongoose models directly. A code scan on 2026-05-31 found no runtime calls to `createRepositories()` / `getPaymentRepo()` / `getMarketplaceRepo()` outside the factory itself. See [[Postgres Runtime Cutover Status]] before assuming a `REPO_*` flag changes live behavior.
|
|
|
|
> [!tip]
|
|
> Service folders are self-contained: each typically has `<feature>Service.ts`, `<feature>Controller.ts`, `<feature>Routes.ts`, `<feature>Validation.ts`. This makes each service movable to a microservice later with minimal coupling.
|
|
|
|
---
|
|
|
|
## 2. Bootstrap — `src/app.ts`
|
|
|
|
The bootstrap is intentionally linear and easy to audit. Execution order:
|
|
|
|
1. **Imports & env load** — `dotenv` (if used), then `import { config } from './shared/config'`.
|
|
2. **Express app construction** — `const app = express();`
|
|
3. **Trust proxy** — `app.set('trust proxy', config.trustProxy)` so X-Forwarded-For works behind Nginx.
|
|
4. **Security headers** — `app.use(helmet({ ... }))`.
|
|
5. **CORS** — `cors({ origin: config.frontendUrl, credentials: true, methods: [...] })`.
|
|
6. **Body parsers** — `express.json({ limit: '10mb' })`, `express.urlencoded({ extended: true })`.
|
|
7. **Static uploads** — `app.use('/uploads', express.static(uploadDir))`.
|
|
8. **Health endpoint** — `GET /health` for Docker healthcheck and external monitors.
|
|
9. **Route mounting** — every `/api/*` route registered before the error handler.
|
|
10. **404 handler** — catches unmatched `/api/*`.
|
|
11. **Error handler** — central `errorHandler` middleware formats responses via `response-handler.ts`.
|
|
12. **HTTP server creation** — `const server = http.createServer(app)`.
|
|
13. **Socket.IO attach** — `initSocket(server, corsOptions)` (see [[Real-time Layer]]).
|
|
14. **DB connect** — `await connectDatabase()` for MongoDB/Mongoose. Postgres connects lazily only when PG modules are imported (for example oracle quote persistence with `ORACLE_QUOTING_ENABLED=true`) and requires `PG_URL`.
|
|
15. **Redis connect** — `await connectRedis()`.
|
|
16. **Listen** — `server.listen(config.port, ...)`.
|
|
17. **Graceful shutdown** — SIGTERM/SIGINT handlers close server, drain sockets, close Mongoose, close Redis.
|
|
18. **Optional dev seeding** — when `NODE_ENV === 'development'` and `SEED_USERS !== 'false'`, the bootstrap calls the seed scripts to provision default test users.
|
|
|
|
---
|
|
|
|
## 3. Middleware chain
|
|
|
|
| Order | Middleware | Where | Purpose |
|
|
|---|---|---|---|
|
|
| 1 | `helmet` | global | Sets security headers (CSP, X-Frame-Options, ...). |
|
|
| 2 | `cors` | global | Origin allow-list = `config.frontendUrl`, credentials enabled. |
|
|
| 3 | `express.json` / `express.urlencoded` | global | Body parsers (10MB limit). |
|
|
| 4 | `morgan` (dev only) | global | HTTP request log to stdout. |
|
|
| 5 | `requestId` | global | Adds `X-Request-Id` for log correlation. |
|
|
| 6 | `authMiddleware` | per-route | Verifies JWT, attaches `req.user`. Mounted only on protected routes. |
|
|
| 7 | `roleGuard('admin'|'seller'|...)` | per-route | RBAC check after auth. |
|
|
| 8 | `validate(schema)` | per-route | express-validator + zod inputs. |
|
|
| 9 | `controllerFn` | per-route | Delegates to service layer. |
|
|
| 10 | `notFound` | tail | Returns 404 envelope for unmatched routes. |
|
|
| 11 | `errorHandler` | tail | Catches thrown errors, formats response. |
|
|
|
|
> [!note]
|
|
> Rate-limit middleware is **active** as of 2026-05-24: auth 10 req/15 min, payment 30/15 min, AI 20/15 min, global 100/15 min. Request Network and Telegram webhooks are exempt from the global limiter. Counters are in-memory — a Redis adapter is planned for distributed deployments.
|
|
|
|
---
|
|
|
|
## 4. Route registration
|
|
|
|
The full route table mounted by `app.ts`:
|
|
|
|
| Mount path | Module | Auth | Notes |
|
|
|---|---|---|---|
|
|
| `/api/auth` | `services/auth/authRoutes.ts` | mixed | login, register, refresh, OAuth, passkey |
|
|
| `/api/user` | `services/user/userRoutes.ts` | JWT | profile, preferences |
|
|
| `/api/address` | `services/user/addressRoutes.ts` | JWT | CRUD addresses |
|
|
| `/api/marketplace/requests` | `services/marketplace/controllerRoutes.ts` | JWT | PurchaseRequest CRUD |
|
|
| `/api/marketplace/offers` | `services/marketplace/controllerRoutes.ts` | JWT (seller) | SellerOffer CRUD |
|
|
| `/api/marketplace/templates` | `services/marketplace/controllerRoutes.ts` | JWT (seller) | RequestTemplate CRUD |
|
|
| `/api/marketplace/categories` | `services/marketplace/controllerRoutes.ts` | public read | Category list |
|
|
| `/api/marketplace/shop-settings` | `services/marketplace/shopSettingsController.ts` | JWT (seller) | Shop profile |
|
|
| `/api/payment` | `services/payment/paymentControllerRoutes.ts` + `paymentRoutes.ts` | JWT | Payment CRUD, health, export |
|
|
| `/api/payment/decentralized` | `services/payment/decentralizedPaymentRoutes.ts` | mixed | Legacy/manual Web3 save, verify, receiver |
|
|
| `/api/payment/request-network` | `services/payment/requestNetwork/requestNetworkRoutes.ts` | mixed + HMAC sig on webhook | Request Network pay-in creation, in-house checkout rehydrate, webhooks |
|
|
| `/api/payment/derived-destinations` | `services/payment/wallets/derivedDestinationRoutes.ts` | JWT (admin) | Derived address list, sweeps, cron, config health |
|
|
| `/api/admin/rn/networks` | `services/payment/requestNetwork/networkRegistryRoutes.ts` | JWT (admin) | Supported RN chain/token registry |
|
|
| `/api/admin/settings/confirmation-thresholds` | `services/admin/confirmationThresholdRoutes.ts` | JWT (admin) | Runtime min-confirmation thresholds |
|
|
| `/api/admin/payments/awaiting-confirmation` | `services/admin/awaitingConfirmationRoutes.ts` | JWT (admin) | Payments blocked on safety confirmations |
|
|
| `/api/telegram` | `services/telegram/telegramRoutes.ts` | mixed (some JWT, webhook uses secret-token) | Mini App verify/session, identity link/unlink, bot webhook |
|
|
| `/api/chat` | `services/chat/chatRoutes.ts` | JWT | Conversations, messages |
|
|
| `/api/notification` | `services/notification/notificationRoutes.ts` + `notificationControllerRouter` | JWT | List, mark read |
|
|
| `/api/disputes` | `routes/disputeRoutes.ts` + `services/dispute/disputeRoutes.ts` | JWT | Dispute CRUD plus release-hold helpers |
|
|
| `/api/blog` | `services/blog/blogRoutes.ts` | mixed | Public reads, admin writes |
|
|
| `/api/admin/cleanup` | `services/admin/dataCleanupRoutes.ts` | JWT (admin) | Data cleanup operations |
|
|
| `/api/points` | `services/points/pointsRoutes.ts` | JWT | Points, levels, referrals |
|
|
| `/api/ai` | `services/ai/aiRoutes.ts` | JWT | OpenAI-backed helpers |
|
|
| `/api/files` | `services/file/fileRoutes.ts` | JWT | Multipart upload |
|
|
| `/api/email` | `services/email/emailRoutes.ts` | JWT | Email dispatch |
|
|
| `/api/trezor` | `services/trezor/trezorRoutes.ts` | JWT | Trezor hardware-wallet ops |
|
|
| `/api/users` | `services/user/userRoutes.ts` | JWT | Legacy user profile routes |
|
|
|
|
Full per-endpoint details → [[03 - API Reference/API Overview]] and the service-specific reference docs.
|
|
|
|
---
|
|
|
|
## 5. Service layer pattern
|
|
|
|
Every service module follows this contract:
|
|
|
|
```ts
|
|
// services/<feature>/<feature>Service.ts
|
|
export class FeatureService {
|
|
static async createX(input, ctx): Promise<X> { /* business logic */ }
|
|
static async getX(id, ctx): Promise<X | null> { /* ... */ }
|
|
static async listX(filter, ctx): Promise<X[]> { /* ... */ }
|
|
static async updateX(id, patch, ctx): Promise<X> { /* ... */ }
|
|
}
|
|
```
|
|
|
|
- Controllers are **thin** — they validate request shape, call the service, format the response.
|
|
- Services own **business logic**, side effects (DB writes, socket emits, email sends).
|
|
- Models are **pure schema** — only Mongoose definitions + virtuals/hooks.
|
|
|
|
Cross-service calls are direct imports — no event bus yet. When the system grows, the seam between services is a natural place to introduce a message queue.
|
|
|
|
---
|
|
|
|
## 6. Dependency map (simplified)
|
|
|
|
```mermaid
|
|
flowchart TB
|
|
auth[auth]
|
|
user[user]
|
|
market[marketplace]
|
|
pay[payment]
|
|
chat[chat]
|
|
notify[notification]
|
|
dispute[dispute]
|
|
points[points]
|
|
file[file]
|
|
email[email]
|
|
socket[socket]
|
|
telegram[telegram]
|
|
|
|
auth --> user
|
|
auth --> notify
|
|
auth --> telegram
|
|
market --> notify
|
|
market --> chat
|
|
market --> file
|
|
pay --> market
|
|
pay --> notify
|
|
pay --> socket
|
|
telegram --> notify
|
|
telegram --> auth
|
|
dispute -.-> market
|
|
dispute -.-> chat
|
|
dispute -.-> notify
|
|
points -.-> notify
|
|
notify --> socket
|
|
notify --> email
|
|
```
|
|
|
|
> [!note]
|
|
> `socket` and `email` are leaf services — every notification path funnels through them. Mocking these two in tests covers most side-effect verification.
|
|
|
|
---
|
|
|
|
## 7. Error handling
|
|
|
|
All thrown errors are caught by the central error handler. The expected shape:
|
|
|
|
```ts
|
|
class AppError extends Error {
|
|
statusCode: number; // HTTP 4xx/5xx
|
|
code: string; // app-specific code, e.g. "PAYMENT_ALREADY_REFUNDED"
|
|
details?: unknown; // optional debug payload
|
|
}
|
|
```
|
|
|
|
Response envelope (success path is `{success:true,data:...}`):
|
|
|
|
```json
|
|
{
|
|
"success": false,
|
|
"error": {
|
|
"code": "VALIDATION_FAILED",
|
|
"message": "email is required",
|
|
"details": [{ "path": "email", "msg": "required" }]
|
|
}
|
|
}
|
|
```
|
|
|
|
See `backend/src/shared/utils/response-handler.ts` and `backend/src/shared/middleware/errorHandler.ts`.
|
|
|
|
---
|
|
|
|
## 8. Configuration
|
|
|
|
Single source of truth for env vars: `src/shared/config/index.ts`. It exports a typed `config` object — anywhere you would write `process.env.X`, instead import `config.x`.
|
|
|
|
Full table in [[Environment Variables]]. Critical ones:
|
|
|
|
| Key | Default | Notes |
|
|
|---|---|---|
|
|
| `PORT` | `5001` | Listen port |
|
|
| `MONGODB_URI` | `mongodb://localhost:27017/nickapp` | Includes db name |
|
|
| `REDIS_URI` | `redis://localhost:6379` | + `REDIS_PASSWORD` |
|
|
| `JWT_SECRET` | required | ≥32 chars |
|
|
| `JWT_EXPIRES_IN` | `7d` | |
|
|
| `REFRESH_TOKEN_EXPIRES_IN` | `30d` | |
|
|
| `FRONTEND_URL` | `http://localhost:3000` | CORS origin |
|
|
| `REQUEST_NETWORK_API_BASE_URL` | `https://api.request.network` | Request Network API |
|
|
| `REQUEST_NETWORK_API_KEY` | required | Request Network API credential |
|
|
| `REQUEST_NETWORK_WEBHOOK_SECRET` | required | Webhook HMAC key |
|
|
| `PAYMENT_LEDGER_ENFORCEMENT` | `false` | Target `true` before launch-scale releases |
|
|
| `TRANSACTION_SAFETY_*` | required for payments | Confirmation, transfer-match, and AML controls |
|
|
| `DERIVED_DESTINATION_SWEEP_SIGNER` | `build-only` | Target hardware/Safe-backed signer |
|
|
| `SMTP_*` | required | Nodemailer |
|
|
| `OPENAI_API_KEY` | required | |
|
|
|
|
---
|
|
|
|
## 9. Database & connection management
|
|
|
|
- **Mongoose** is the ODM. Connection in `src/infrastructure/database/`.
|
|
- Connection options enable retryable writes, exponential backoff on reconnect.
|
|
- Indexes are defined on each model and auto-created on connect (Mongoose `autoIndex: true` in dev, recommend `false` in prod with explicit migration).
|
|
- See [[Data Model Overview]] for the relational map and per-model docs.
|
|
|
|
Redis client (in `src/services/redis/`) provides:
|
|
- Session caching (login attempts, lockout counters)
|
|
- Rate-limit counters (when middleware is enabled)
|
|
- Hot-path caches (category list, level configs)
|
|
|
|
---
|
|
|
|
## 10. Background work
|
|
|
|
The codebase has no dedicated queue runner — scheduled / async work is triggered inline from request handlers and uses `setTimeout` / `setInterval` patterns where needed (e.g., delayed retries). Consider introducing Bull / BullMQ if you grow:
|
|
|
|
- Request Network webhook replay/reconciliation and derived-destination balance checks
|
|
- Notification email digests
|
|
- Auto-release escrow timers
|
|
- Token / refresh-token cleanup
|
|
|
|
---
|
|
|
|
## 11. Testing
|
|
|
|
Jest test suites in `backend/__tests__/`:
|
|
|
|
| File | Covers |
|
|
|---|---|
|
|
| `models.test.ts` | Schema validation, virtuals, hooks |
|
|
| `payment-services.test.ts` | Payment orchestration logic |
|
|
| `complete-backend.test.ts` | Cross-service integration |
|
|
| `request-network-webhook.test.ts` | Request Network webhook signature and processing |
|
|
| `request-network-adapter.test.ts` | Request Network payment adapter |
|
|
| `payment-ledger.service.test.ts` | Ledger append/reconciliation behavior |
|
|
| `payment-release-refund-orchestration.test.ts` | Release/refund instruction orchestration |
|
|
|
|
Run with `npm run test:all`. CI runs the same. Reach for `npm run test:models`, `npm run test:payment`, etc. when iterating on a slice.
|
|
|
|
---
|
|
|
|
## 12. Notable files for orientation
|
|
|
|
| File | Why it matters |
|
|
|---|---|
|
|
| `src/app.ts` | Bootstrap — read once to understand wiring |
|
|
| `src/shared/config/index.ts` | All env vars, typed |
|
|
| `src/shared/utils/response-handler.ts` | Standard response shape |
|
|
| `src/shared/middleware/auth.ts` | JWT verify + RBAC |
|
|
| `src/infrastructure/socket/socketService.ts` | All socket plumbing |
|
|
| `src/services/payment/requestNetwork/requestNetworkRoutes.ts` | Request Network checkout and webhook route |
|
|
| `src/services/payment/ledger/fundsLedgerService.ts` | Immutable payment ledger writes |
|
|
| `src/services/marketplace/PurchaseRequestService.ts` | Core marketplace state machine |
|
|
| `src/services/auth/authService.ts` | Auth flows, lockout, hashing |
|
|
| `src/models/User.ts` | Central entity with role/preferences |
|
|
| `openapi.json` | Generated API spec — definitive endpoint list |
|
|
|
|
---
|
|
|
|
## Related
|
|
|
|
- [[System Architecture]] — full system topology
|
|
- [[Frontend Architecture]] — how the FE talks to this BE
|
|
- [[Real-time Layer]] — Socket.IO room model
|
|
- [[Security Architecture]] — JWT, passkeys, webhook HMAC
|
|
- [[Data Model Overview]] — entity-relationship map
|
|
- [[Authentication Flow]] · [[Escrow Flow]] · [[Dispute Flow]]
|