nick/nick-doc
3
0
Fork 0
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Initial commit: nick docs

Browse Source
This commit is contained in:
moojttaba
2026-05-23 20:35:34 +03:30
commit 0da235ae27
90 changed files with 18268 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

259
08 - Operations/CI-CD Pipeline.md Normal file
View File

@@ -0,0 +1,259 @@
---
title: CI-CD Pipeline
tags: [operations]
---
# CI/CD Pipeline
How code goes from a push to a running container in production. The CI is **Gitea Actions** running on the same Gitea instance that hosts the repos. The CD is **Watchtower** on the production host (covered in [[Deployment]]).
---
## 1. Where workflows live
| Repo | Path | Files |
|------|------|-------|
| Backend | `.gitea/workflows/` | `docker-build-simple.yml`, `docker-build-dev.yml`, `docker-build-no-cache.yml` |
| Frontend | `.gitea/workflows/` | `deploy.yml`, `devDeploy.yml` |
Gitea Actions speaks the same YAML dialect as GitHub Actions — most third-party actions (`actions/checkout@v4`, `docker/login-action@v3`, `docker/build-push-action@v5`) work unchanged.
---
## 2. Required secrets
Configured per repo at **Settings → Actions → Secrets**.
| Secret | Repo | Purpose |
|--------|------|---------|
| `GITEATOKEN` | both | Personal access token for the `manawenuz` user with `write:packages` scope. Used by every workflow to log into the container registry at `git.manko.yoga`. |
| `SENTRY_AUTH_TOKEN` | frontend | (Optional) For source-map upload during Next.js build. Skipped if absent. |
The registry itself is implicit: `git.manko.yoga` with `manawenuz` as the user. Image paths are `git.manko.yoga/manawenuz/<image>`.
> [!warning] If `GITEATOKEN` expires or is rotated, all workflows fail at the `docker/login-action` step. Rotate proactively (annual reminder).
---
## 3. Backend workflows
### `docker-build-simple.yml` — manual build
```yaml
name: Manual Build and Push Docker Image
on:
workflow_dispatch:
inputs:
version:
description: 'Version to build (leave empty for package.json)'
required: false
type: string
```
- **Trigger.** Manual only (via Gitea UI → Actions → "Run workflow").
- **Steps.** Checkout → buildx → `docker login` → read version (input or `package.json`) → build `Dockerfile.prod` → push tags `:<version>` and `:dev` → echo result.
- **When to use.** Cutting an ad-hoc build of a specific commit without merging to a branch. The `:dev` tag is overwritten — production (`:latest`) is **not** touched.
- **Cache.** Uses `type=gha` cache to speed up subsequent runs.
### `docker-build-dev.yml` — dev branch auto-build
```yaml
on:
push:
branches: [ development ]
tags: [ 'v*' ]
```
- **Trigger.** Every push to `development` and every tag matching `v*`.
- **Tags pushed.** `:dev-<package-version>` + moving `:dev`.
- **Effect.** Refreshes the dev image. The production Watchtower **does not** watch `:dev`, so this is safe to push as often as you want.
### `docker-build-no-cache.yml` — production build
```yaml
on:
push:
branches: [ main, master ]
tags: [ 'v*' ]
```
- **Trigger.** Every push to `main` (or `master`) and every `v*` tag.
- **Tags pushed.** `:<package-version>` + moving `:latest`.
- **Effect.** Watchtower polls `:latest`, detects the new digest, restarts `nickapp-backend` on the production host. See [[Deployment#routine-deploy]].
- **No cache.** The file is named "No Cache" but actually does not pass `cache-from`/`cache-to`, so each build is from scratch. Slower (~5–8 min) but eliminates a class of stale-layer bugs. The `simple` workflow uses GHA cache for speed.
> [!tip] If you need to invalidate a cached layer in the `simple` workflow, run `no-cache` once — the resulting tag overwrites the registry digest and `simple`'s next run will start from a cleaner base.
---
## 4. Frontend workflows
Both workflows share the same shape: spin up a `node:22` container, run a deploy shell script that does `docker login + build + push`.
### `deploy.yml` — production
```yaml
on:
push:
branches: [ main, master ]
workflow_dispatch:
```
Calls `./scripts/deploy.sh` — see [[Scripts#deployment]]. The script:
1. Reads `package.json` version.
2. `docker login git.manko.yoga -u manawenuz -p $GITEATOKEN`.
3. Builds `git.manko.yoga/manawenuz/escrow-frontend:<version>` and `:latest` from `Dockerfile`.
4. Pushes both tags.
`:latest` is what production Watchtower watches → live deploy follows automatically.
### `devDeploy.yml` — development branch
Same as `deploy.yml` but triggered on `development` and runs `./scripts/deployDev.sh`, which pushes only `:dev`.
---
## 5. End-to-end timeline (production deploy)
```
t=0 Developer merges PR → main
t+5s Gitea webhook fires
t+10s Gitea Actions runner pulls repo, starts container
t+30s docker/setup-buildx-action initialised
t+45s docker/login-action authenticated
t+2-5m docker/build-push-action builds Dockerfile.prod
t+5m Push to git.manko.yoga/manawenuz/escrow-backend:latest
t+5m+ Watchtower (next poll, up to 5 min) detects new digest
t+10m Watchtower stops old container, starts new one
t+10m40s start_period=40s elapses, healthcheck passes
t+11m Nginx routes traffic to the new container
```
**Typical SLA: 10–12 minutes from merge to live.** For an emergency rollback see [[Deployment#roll-back]].
---
## 6. Versioning automation
Tied to `backend/scripts/auto-version.sh` + `ai-enhanced.sh` (and the frontend mirror). Full reference in [[Git Workflow#versioning]] and [[Scripts#auto-version-sh]].
In short:
```bash
# Developer side, on the branch they're releasing:
npm run smart-release
# → AI analyses last commit, picks bump (major/minor/patch/skip)
# → bumps package.json
# → commits "chore: bump version to vX.Y.Z"
# → tags vX.Y.Z
# → git push && git push --tags
```
The push to `main` (or the `v*` tag) then triggers `docker-build-no-cache.yml`, which:
- Reads the new version from `package.json` (`node -p "require('./package.json').version"`)
- Builds and pushes `:<version>` + `:latest`
So both the **image tag** and the **git tag** carry the same `vX.Y.Z` — easy to correlate when investigating an issue.
---
## 7. Adding tests to the pipeline
The workflows today only build + push; they do **not** run Jest or Playwright. To gate releases on tests, add a `test` job before the build:
```yaml
jobs:
test:
runs-on: ubuntu-latest
container: node:22
steps:
- uses: actions/checkout@v4
- run: yarn install --frozen-lockfile
- run: yarn lint
- run: yarn test --ci --runInBand
- run: yarn test:e2e # if a service container is available
build-and-push:
needs: test
runs-on: ubuntu-latest
# ...existing steps...
```
Or run lint + typecheck as a pre-gate using a separate workflow that triggers on PR opened/synchronised.
---
## 8. Inspecting a build
In Gitea: **Actions → workflow → run** to see real-time logs.
Useful CLI for the registry from your laptop:
```bash
# List images and tags
curl -s -u "manawenuz:$GITEATOKEN" \
"https://git.manko.yoga/v2/manawenuz/escrow-backend/tags/list" | jq
# Pull a specific tag
docker login git.manko.yoga -u manawenuz
docker pull git.manko.yoga/manawenuz/escrow-backend:2.6.3
```
---
## 9. Self-hosted runner notes
Gitea Actions can use either built-in `act_runner` or your own. Currently the workflows are written for `runs-on: ubuntu-latest`, which the act_runner supplies via a generic Ubuntu container. If you need:
- More CPU/RAM for builds → register a beefier self-hosted runner and change `runs-on:` to its label.
- A Docker-in-Docker setup (frontend `deploy.yml` does this with `options: --privileged`) — confirm the runner trusts the workflow.
---
## 10. Failure modes & remediation
| Failure | Most likely cause | Fix |
|---------|------------------|-----|
| `unauthorized: authentication required` at push | `GITEATOKEN` expired or lacks `write:packages` | Rotate the token, update the repo secret |
| `Cannot perform an interactive login from a non TTY device` | Old docker-login-action version | Bump to `docker/login-action@v3` |
| Build hangs at `yarn install` | npm registry timeout | Increase `network-timeout` (already 600000); re-run |
| Image pushed but Watchtower doesn't roll | Watchtower can't reach the registry | `docker logs watchtower`; verify `/root/.docker/config.json` is mounted into the container |
| New container fails healthcheck | App crash on boot | `docker logs nickapp-backend`; check env vars, follow [[Incident Response]] |
| Multi-arch warnings about platform | Build runner is arm64 but prod is amd64 | Add `--platform=linux/amd64` to `docker/build-push-action` inputs |
| Image size grew suddenly | Dev dep crept into prod stage | Audit `Dockerfile.prod` for missing `--production` flag in the runtime stage |
---
## 11. Pipeline diagram
```
Push to development Push to main
│ │
▼ ▼
┌───────────────────────────┐ ┌───────────────────────────┐
│ docker-build-dev.yml │ │ docker-build-no-cache.yml │
│ (backend) │ │ (backend) │
│ devDeploy.yml (frontend) │ │ deploy.yml (frontend) │
└───────────────┬───────────┘ └───────────────┬───────────┘
│ │
push :<version>,:dev push :<version>,:latest
│ │
▼ ▼
git.manko.yoga/manawenuz/...:dev git.manko.yoga/manawenuz/...:latest
│ │
│ ▼
│ ┌──────────────────────────┐
│ │ Watchtower │
│ │ (poll every 5 minutes) │
│ └──────────────┬───────────┘
│ │
manual pull on staging restart containers
│
▼
Production live
```
Cross-links: [[Deployment]] for what happens on the host, [[Git Workflow]] for what happens upstream, [[Scripts]] for the deploy shell scripts.