manawenuz/featherChat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: friend list, bot API, ETH addressing, deep links, docs overhaul

Browse Source 
Tier 1 — New features:
- E2E encrypted friend list: server stores opaque blob (POST/GET /v1/friends),
  protocol-level encrypt/decrypt with HKDF-derived key, 4 tests
- Telegram Bot API compatibility: /bot/register, /bot/:token/getUpdates,
  sendMessage, getMe — TG-style Update objects with proper message mapping
- ETH address resolution: GET /v1/resolve/:address (0x.../alias/@.../fp),
  bidirectional ETH↔fp mapping stored on key registration
- Seed recovery: /seed command in TUI + web client
- URL deep links: /message/@alias, /message/0xABC, /group/#ops
- Group members with online status in GET /groups/:name/members

Tier 2 — UX polish:
- TUI: /friend, /friend <addr>, /unfriend <addr> with presence checking
- Web: friend commands, showGroupMembers() on group join
- Web: ETH address in header, clickable addresses (click→peer or copy)
- Bot: full WireMessage→TG Update mapping (encrypted base64, CallSignal,
  FileHeader, bot_message JSON)

Documentation:
- USAGE.md rewritten: complete user guide with all commands
- SERVER.md rewritten: full admin guide with all 50+ endpoints
- CLIENT.md rewritten: architecture, commands, keyboard, storage
- LLM_HELP.md created: 1083-word token-optimized reference for helper LLM

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Siavash Sameni
2026-03-29 07:31:54 +04:00
parent dbf5d136cf
commit 7b72f7cba5
15 changed files with 2181 additions and 1023 deletions
Download Patch File Download Diff File Expand all files Collapse all files

656
warzone/docs/SERVER.md
View File

@@ -1,37 +1,63 @@
# Warzone Server -- Operation & Administration
# Warzone Server -- Administration Guide
**Version 0.0.21**
---
## 1. Building
The server is part of the Cargo workspace. From the workspace root:
### Local Build
From the workspace root:
```bash
# Debug build
# Debug
cargo build -p warzone-server
# Release build (recommended for deployment)
# Release (recommended for deployment)
cargo build -p warzone-server --release
```
The resulting binary is at `target/release/warzone-server` (or
`target/debug/warzone-server`). It is a single statically-linked binary with
no runtime dependencies beyond libc.
Binary output: `target/release/warzone-server`.
### Cross-Compile for Linux (x86_64)
The `scripts/build-linux.sh` script spins up a Hetzner Cloud VPS, builds
Linux release binaries, and pulls them back to `target/linux-x86_64/`.
```bash
# Full pipeline: build + deploy to all production servers + destroy VM
./scripts/build-linux.sh --ship
# Step-by-step:
./scripts/build-linux.sh --prepare # create VM, install deps, upload source
./scripts/build-linux.sh --build # compile release binaries on the VM
./scripts/build-linux.sh --transfer # download binaries to target/linux-x86_64/
./scripts/build-linux.sh --destroy # delete the VM
# Or all three build steps at once (VM persists):
./scripts/build-linux.sh --all
```
### Minimum Rust Version
Rust 1.75 or later (set via `rust-version = "1.75"` in `Cargo.toml`).
Rust 1.75 or later (`rust-version = "1.75"` in `Cargo.toml`).
---
## 2. Running
### Basic
```bash
# Default: bind 0.0.0.0:7700, data in ./warzone-data
# Defaults: bind 0.0.0.0:7700, data in ./warzone-data
./warzone-server
# Custom bind address and data directory
./warzone-server --bind 127.0.0.1:8080 --data-dir /var/lib/warzone
./warzone-server --bind 0.0.0.0:7700 --data-dir ./data
# With federation enabled
./warzone-server --federation federation.json
```
### CLI Flags
@@ -39,214 +65,339 @@ Rust 1.75 or later (set via `rust-version = "1.75"` in `Cargo.toml`).
| Flag | Short | Default | Description |
|------|-------|---------|-------------|
| `--bind` | `-b` | `0.0.0.0:7700` | Address and port to listen on |
| `--data-dir` | `-d` | `./warzone-data` | Directory for sled database files |
| `--data-dir` | `-d` | `./warzone-data` | Directory for the sled database |
| `--federation` | `-f` | *(none)* | Path to federation JSON config file |
### Logging
### Environment Variables
The server uses `tracing-subscriber`. Control log level with the `RUST_LOG`
environment variable:
| Variable | Default | Description |
|----------|---------|-------------|
| `RUST_LOG` | `warn` (production) | Log filter. Examples: `info`, `warzone_server=debug`, `trace` |
| `WZP_RELAY_ADDR` | *(none)* | WZP voice relay address advertised to clients |
### systemd Service
A production-ready unit file is provided at `deploy/warzone-server.service`:
```ini
[Unit]
Description=Warzone Messenger Server (featherChat)
After=network-online.target
Wants=network-online.target
[Service]
Type=simple
User=warzone
Group=warzone
WorkingDirectory=/home/warzone
ExecStart=/home/warzone/warzone-server --bind 0.0.0.0:7700 --data-dir /home/warzone/data --federation /home/warzone/federation.json
Restart=always
RestartSec=3
LimitNOFILE=65536
# Security hardening
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=read-only
ReadWritePaths=/home/warzone/data
PrivateTmp=yes
Environment=RUST_LOG=warn,warzone_server::federation=info
[Install]
WantedBy=multi-user.target
```
Install and enable:
```bash
RUST_LOG=info ./warzone-server
RUST_LOG=warzone_server=debug ./warzone-server
RUST_LOG=trace ./warzone-server # very verbose
sudo cp deploy/warzone-server.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable --now warzone-server
```
---
## 3. API Reference
## 3. Configuration
All API endpoints are under the `/v1` prefix. The web UI is served at `/`.
### Federation JSON
### Health Check
Enable federation by passing `--federation <path>` on startup. The config
file specifies the local server identity, peer connection details, and a
shared secret for authentication.
```
GET /v1/health
```
**Format** (see `federation.example.json`):
**Response:**
```json
{
"status": "ok",
"version": "0.1.0"
"server_id": "alpha",
"shared_secret": "change-me-to-a-long-random-string-shared-between-both-servers",
"peer": {
"id": "bravo",
"url": "http://10.0.0.2:7700"
},
"presence_interval_secs": 5
}
```
Use this for monitoring, load balancer health probes, and uptime checks.
| Field | Description |
|-------|-------------|
| `server_id` | Unique name for this server (e.g. `"alpha"`) |
| `shared_secret` | Pre-shared secret; must match on both sides |
| `peer.id` | The remote server's `server_id` |
| `peer.url` | HTTP base URL of the remote server |
| `presence_interval_secs` | How often to broadcast online-user lists (default 5) |
---
### Register Key Bundle
## 4. Federation
```
POST /v1/keys/register
Content-Type: application/json
```
Federation connects two Warzone servers over a persistent WebSocket so
their users can communicate transparently.
### How It Works
- On startup, each server opens an outgoing WebSocket to its peer at
`/v1/federation/ws` and authenticates with the shared secret.
- The connection auto-reconnects on failure.
- Presence (online fingerprints) is synced on the configured interval.
- Messages to users on the remote server are forwarded automatically.
### Federated Features
| Feature | Behavior |
|---------|----------|
| **Key lookup proxy** | If a key bundle is not found locally, the server queries the peer |
| **Message forwarding** | Messages addressed to a remote fingerprint are relayed over the WS |
| **Alias resolution** | `/v1/resolve/:address` checks the peer if the alias is not local |
| **Presence sync** | Each server broadcasts its online fingerprints to the peer |
### Two-Server Setup
**Server A** (`alpha`, e.g. `mequ`):
**Request body:**
```json
{
"fingerprint": "a3f8:c912:44be:7d01",
"bundle": [/* bincode-serialized PreKeyBundle as byte array */]
"server_id": "alpha",
"shared_secret": "s3cret-shared-between-both",
"peer": { "id": "bravo", "url": "http://bravo-host:7700" },
"presence_interval_secs": 5
}
```
The `bundle` field is a JSON array of unsigned bytes (the raw bincode
serialization of a `PreKeyBundle`).
**Server B** (`bravo`, e.g. `kh3rad3ree`):
**Response:**
```json
{
"ok": true
"server_id": "bravo",
"shared_secret": "s3cret-shared-between-both",
"peer": { "id": "alpha", "url": "http://alpha-host:7700" },
"presence_interval_secs": 5
}
```
**Behavior:** stores the bundle in the `keys` sled tree, keyed by the
fingerprint string. Overwrites any existing bundle for the same fingerprint.
Both files use the same `shared_secret`. Each server's `peer.id` matches
the other server's `server_id`.
### Federation Status Endpoint
```bash
curl http://localhost:7700/v1/federation/status
```
Returns JSON with connection state, peer info, and presence data.
---
### Fetch Key Bundle
## 5. API Reference
```
GET /v1/keys/{fingerprint}
```
All endpoints are prefixed with `/v1`. The web UI is served at `/`.
**Path parameter:** the fingerprint string, e.g. `a3f8:c912:44be:7d01`.
### Notation
**Response (200):**
```json
{
"fingerprint": "a3f8:c912:44be:7d01",
"bundle": "base64-encoded-bincode-bytes..."
}
```
The `bundle` value is standard base64-encoded bincode. The client decodes
base64, then deserializes with bincode to recover the `PreKeyBundle`.
**Response (404):** returned if no bundle is registered for the fingerprint.
- **Auth** = requires `Authorization: Bearer <token>` header (write routes).
- **Public** = no authentication needed (read routes).
---
### Send Message
### Health
```
POST /v1/messages/send
Content-Type: application/json
```
**Request body:**
```json
{
"to": "b7d1:e845:0022:9f3a",
"message": [/* bincode-serialized WireMessage as byte array */]
}
```
**Response:**
```json
{
"ok": true
}
```
**Behavior:** the message bytes are stored in the `messages` sled tree under
the key `queue:{recipient_fingerprint}:{uuid}`. The UUID is generated
server-side to ensure unique keys.
The server does NOT parse, validate, or inspect the message contents. It is an
opaque blob.
| Method | Path | Auth | Description |
|--------|------|------|-------------|
| GET | `/v1/health` | Public | Health check; returns `{"status":"ok"}` |
---
### Poll Messages
### Keys
```
GET /v1/messages/poll/{fingerprint}
```
**Response (200):**
```json
[
"base64-encoded-message-1",
"base64-encoded-message-2"
]
```
Returns a JSON array of base64-encoded message blobs. Each blob is a
bincode-serialized `WireMessage`. An empty array means no messages.
**Behavior:** scans the `messages` sled tree for all keys prefixed with
`queue:{fingerprint}`. Messages are NOT deleted by polling; they remain until
explicitly acknowledged.
| Method | Path | Auth | Description |
|--------|------|------|-------------|
| POST | `/v1/keys/register` | Public | Register a pre-key bundle |
| POST | `/v1/keys/replenish` | Public | Upload additional one-time pre-keys |
| GET | `/v1/keys/:fingerprint` | Public | Fetch a key bundle (falls back to federation peer) |
| GET | `/v1/keys/list` | Public | List all registered fingerprints |
| GET | `/v1/keys/:fingerprint/otpk-count` | Public | Remaining one-time pre-key count |
| GET | `/v1/keys/:fingerprint/devices` | Public | List devices for a fingerprint |
---
### Acknowledge Message
### Messages
```
DELETE /v1/messages/{id}/ack
```
**Path parameter:** the message storage key (currently the full sled key
including the `queue:` prefix and UUID).
**Response:**
```json
{
"ok": true
}
```
**Behavior:** removes the message from the `messages` tree.
**Note:** the current implementation requires knowing the exact sled key to
acknowledge. A proper message-ID-based index is planned for Phase 2.
| Method | Path | Auth | Description |
|--------|------|------|-------------|
| POST | `/v1/messages/send` | Auth | Send an encrypted message blob |
| GET | `/v1/messages/poll/:fingerprint` | Public | Poll queued messages |
| DELETE | `/v1/messages/:id/ack` | Public | Acknowledge (delete) a message |
---
## 4. Web UI
### Groups
The server serves a single-page web client at the root path `/`.
```
GET /
```
Returns an HTML page with embedded CSS and JavaScript. The web client provides:
- **Identity generation:** generates a random 32-byte seed in the browser
using `crypto.getRandomValues()`.
- **Identity recovery:** paste a hex-encoded seed to recover.
- **Fingerprint display:** shows the user's fingerprint in the header.
- **Key registration:** automatically registers a public key with the server
on entry.
- **Message polling:** polls `/v1/messages/poll/{fingerprint}` every 5 seconds.
- **Slash commands:** `/help`, `/info`, `/seed`.
### Web Client Limitations
- Uses ECDH P-256 (Web Crypto API) instead of X25519. Cross-client
compatibility with the CLI is not yet implemented. (Phase 2)
- Does not use BIP39 mnemonics; seed is displayed as hex.
- Message decryption is not yet wired (Double Ratchet in JS is TODO).
- The seed is stored in `localStorage` (unencrypted).
| Method | Path | Auth | Description |
|--------|------|------|-------------|
| POST | `/v1/groups/create` | Auth | Create a group |
| POST | `/v1/groups/:name/join` | Auth | Join a group |
| POST | `/v1/groups/:name/send` | Auth | Send a message to a group |
| POST | `/v1/groups/:name/leave` | Auth | Leave a group |
| POST | `/v1/groups/:name/kick` | Auth | Kick a member from a group |
| GET | `/v1/groups` | Public | List all groups |
| GET | `/v1/groups/:name` | Public | Get group details |
| GET | `/v1/groups/:name/members` | Public | List members (includes online status) |
---
## 5. Database
### Aliases
The server uses **sled** (embedded key-value store). All data lives under the
directory specified by `--data-dir`.
| Method | Path | Auth | Description |
|--------|------|------|-------------|
| POST | `/v1/alias/register` | Auth | Register a human-readable alias |
| POST | `/v1/alias/unregister` | Auth | Remove your alias |
| POST | `/v1/alias/recover` | Auth | Transfer alias to a new fingerprint |
| POST | `/v1/alias/renew` | Auth | Renew alias expiry |
| POST | `/v1/alias/admin-remove` | Auth | Admin-remove an alias |
| GET | `/v1/alias/resolve/:name` | Public | Resolve alias to fingerprint |
| GET | `/v1/alias/list` | Public | List all registered aliases |
| GET | `/v1/alias/whois/:fingerprint` | Public | Reverse-lookup: fingerprint to alias |
### Trees (Tables)
---
| Tree | Key format | Value | Purpose |
|------|-----------|-------|---------|
| `keys` | fingerprint string (UTF-8 bytes) | bincode `PreKeyBundle` | Pre-key bundle storage |
| `messages` | `queue:{fingerprint}:{uuid}` (UTF-8 bytes) | bincode `WireMessage` | Message queue |
| `otpks` | (reserved) | (reserved) | One-time pre-key tracking (not yet used server-side) |
### Calls (WZP)
| Method | Path | Auth | Description |
|--------|------|------|-------------|
| POST | `/v1/calls/initiate` | Auth | Start a 1:1 call |
| POST | `/v1/calls/:id/end` | Auth | End an active call |
| POST | `/v1/calls/missed` | Auth | Get missed calls for a fingerprint |
| POST | `/v1/groups/:name/call` | Auth | Initiate a group call |
| GET | `/v1/calls/:id` | Public | Get call details |
| GET | `/v1/calls/active` | Public | List active calls |
---
### Devices
| Method | Path | Auth | Description |
|--------|------|------|-------------|
| POST | `/v1/devices/:id/kick` | Auth | Disconnect a specific device |
| POST | `/v1/devices/revoke-all` | Auth | Disconnect all devices (optional keep one) |
| GET | `/v1/devices` | Auth | List your connected devices |
---
### Presence
| Method | Path | Auth | Description |
|--------|------|------|-------------|
| POST | `/v1/presence/batch` | Auth | Batch-query presence for multiple fingerprints |
| GET | `/v1/presence/:fingerprint` | Public | Check if a fingerprint is online |
---
### Friends
| Method | Path | Auth | Description |
|--------|------|------|-------------|
| POST | `/v1/friends` | Auth | Save friend list (encrypted blob) |
| GET | `/v1/friends` | Auth | Retrieve saved friend list |
---
### Resolve
| Method | Path | Auth | Description |
|--------|------|------|-------------|
| GET | `/v1/resolve/:address` | Public | Universal resolve: ETH address, alias, or fingerprint. Checks federation peer if not found locally. |
---
### WZP Voice Relay
| Method | Path | Auth | Description |
|--------|------|------|-------------|
| GET | `/v1/wzp/relay-config` | Public | Get the WZP relay address for voice calls |
---
### Federation
| Method | Path | Auth | Description |
|--------|------|------|-------------|
| GET | `/v1/federation/status` | Public | Federation connection status and peer info |
| GET | `/v1/federation/ws` | Internal | WebSocket endpoint for server-to-server communication |
---
### Bot API
| Method | Path | Auth | Description |
|--------|------|------|-------------|
| POST | `/v1/bot/register` | Auth | Register a bot; returns an API token |
| GET | `/v1/bot/:token/getMe` | Token | Bot identity info |
| POST | `/v1/bot/:token/getUpdates` | Token | Long-poll for new messages (Telegram-compatible) |
| POST | `/v1/bot/:token/sendMessage` | Token | Send a message as the bot (Telegram-compatible) |
Bot tokens are scoped to the bot's fingerprint. The `getUpdates` and
`sendMessage` endpoints follow the Telegram Bot API conventions so existing
Telegram bot libraries can be adapted with minimal changes.
---
### WebSocket
| Path | Description |
|------|-------------|
| `/v1/ws/:fingerprint` | Real-time message delivery. Clients receive instant push of new messages. |
---
### Web UI
| Path | Description |
|------|-------------|
| `/` | Single-page WASM web client |
| `/wasm/warzone_wasm.js` | WASM JavaScript bindings |
| `/wasm/warzone_wasm_bg.wasm` | WASM binary |
---
## 6. Database
The server uses **sled** (embedded key-value store). All data lives under
the `--data-dir` directory.
### Trees
| Tree | Purpose |
|------|---------|
| `keys` | Pre-key bundles (public keys only) |
| `messages` | Queued encrypted message blobs |
| `groups` | Group metadata and membership |
| `aliases` | Human-readable alias mappings |
| `tokens` | Authentication tokens (device sessions) |
| `calls` | Call records (1:1 and group) |
| `missed_calls` | Missed call notifications |
| `friends` | Encrypted friend lists |
| `eth_addresses` | Ethereum address to fingerprint mappings |
### Data Directory Structure
@@ -254,161 +405,123 @@ directory specified by `--data-dir`.
warzone-data/
db # sled database file
conf # sled config
blobs/ # sled blob storage (if any)
blobs/ # sled blob storage
snap.*/ # sled snapshots
```
The exact file layout is managed by sled internally. The entire directory
should be treated as a unit for backup.
### What the Server Stores
- **Pre-key bundles:** public keys only. The server never holds private keys.
- **Encrypted message blobs:** opaque binary data. The server cannot read
message contents.
- **Metadata visible to server:** sender fingerprint, recipient fingerprint,
message size, timestamps (implicit from storage order).
The entire directory should be treated as a unit for backup. Stop the server
before copying, or use filesystem-level snapshots (LVM, ZFS, btrfs).
---
## 6. Deployment
## 7. Security
### Single Binary
### Auth Middleware
The recommended deployment is a single `warzone-server` binary behind a
reverse proxy for TLS termination.
All write (POST) endpoints require a bearer token in the `Authorization`
header. Tokens are issued during key registration and tied to a fingerprint.
Read (GET) endpoints are public.
### Reverse Proxy (nginx)
### Rate Limiting
```nginx
server {
listen 443 ssl http2;
server_name wz.example.com;
- **200 concurrent requests** (tower `ConcurrencyLimitLayer`)
- **5 WebSocket connections per fingerprint** (multi-device cap)
ssl_certificate /etc/letsencrypt/live/wz.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/wz.example.com/privkey.pem;
### Device Management
location / {
proxy_pass http://127.0.0.1:7700;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
Users can list connected devices, kick individual devices, or revoke all
sessions via the `/v1/devices` endpoints. The `revoke-all` endpoint accepts
an optional `keep_device_id` to keep the current device active.
# WebSocket support (for future real-time push)
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
}
```
### What the Server Can See
When using a reverse proxy, bind the server to localhost only:
| Data | Visible |
|------|---------|
| Message plaintext | No (E2E encrypted blobs) |
| Sender/recipient fingerprints | Yes |
| Message size and timing | Yes |
| Public pre-key bundles | Yes (public by design) |
| IP addresses | Yes (from HTTP) |
---
## 8. Monitoring
### Logging
Control verbosity with `RUST_LOG`:
```bash
./warzone-server --bind 127.0.0.1:7700
RUST_LOG=warn ./warzone-server # production default
RUST_LOG=info ./warzone-server # request-level logging
RUST_LOG=warzone_server=debug ./warzone-server # server internals
RUST_LOG=trace ./warzone-server # everything
```
### systemd Service
With systemd:
```ini
[Unit]
Description=Warzone Messenger Server
After=network.target
[Service]
Type=simple
User=warzone
ExecStart=/usr/local/bin/warzone-server --bind 127.0.0.1:7700 --data-dir /var/lib/warzone
Restart=always
RestartSec=5
Environment=RUST_LOG=info
[Install]
WantedBy=multi-user.target
```bash
journalctl -u warzone-server -f
```
---
## 7. Monitoring
### Health Endpoint
### Health Check
```bash
curl http://localhost:7700/v1/health
# {"status":"ok","version":"0.1.0"}
```
Use this for:
- Load balancer health checks
- Uptime monitoring (e.g., with `uptime-kuma`, Prometheus blackbox exporter)
- Deployment verification
### Federation Status
### Logs
```bash
curl http://localhost:7700/v1/federation/status
```
All request activity is logged via `tracing`. In production, pipe to a log
aggregator or use `journalctl -u warzone-server`.
Returns connection state, peer identity, and synced presence data.
---
## 8. Security Considerations
## 9. Deploy Scripts
### The Server Is a Dumb Relay
The `scripts/build-linux.sh` script handles the full build and deploy
lifecycle via Hetzner Cloud VMs.
The server never sees plaintext message content. It stores and forwards
opaque encrypted blobs. Even if the server is fully compromised, an attacker
gains:
### Key Commands
- **Encrypted message blobs** (useless without recipient's private keys)
- **Public pre-key bundles** (public by design)
- **Metadata:** who is messaging whom, when, and how often
| Command | Description |
|---------|-------------|
| `--ship` | Full pipeline: build on VM, deploy to all production servers, destroy VM |
| `--update-all` | Upload pre-built binaries to all production servers and restart |
| `--update <user@host>` | Update a single production server |
| `--status` | Check service status and federation on all production servers |
| `--logs [user@host]` | Tail `journalctl` logs (defaults to first production server) |
### What the Server CAN See
### Typical Deploy Workflow
| Data | Visible to server |
|------|-------------------|
| Message plaintext | No |
| Sender fingerprint | Yes (in `WireMessage`) |
| Recipient fingerprint | Yes (used for routing) |
| Message size | Yes |
| Timing | Yes |
| IP addresses | Yes (from HTTP) |
| Pre-key bundles (public keys) | Yes |
```bash
# One command: build, deploy everywhere, clean up
./scripts/build-linux.sh --ship
### Mitigations for Metadata (Future)
- **Sealed sender** (Phase 6): hide sender identity from the server.
- **Padding:** fixed-size messages to prevent size-based analysis.
- **Onion routing** (Phase 6): hide IP addresses via relay chains.
### Access Control
The current server has **no authentication**. Anyone can:
- Register a key bundle for any fingerprint
- Poll messages for any fingerprint
- Send messages to any fingerprint
**TODO (Phase 2):** authentication via Ed25519 challenge-response. Clients
sign requests to prove they own the fingerprint they claim.
# Or step by step:
./scripts/build-linux.sh --all # build (VM persists)
./scripts/build-linux.sh --update-all # deploy binaries
./scripts/build-linux.sh --destroy # clean up VM
./scripts/build-linux.sh --status # verify
```
---
## 9. Backup and Recovery
## 10. Backup and Recovery
### Database Backup
The sled database can be backed up by copying the entire data directory while
the server is stopped:
### Backup
```bash
systemctl stop warzone-server
cp -r /var/lib/warzone /backup/warzone-$(date +%Y%m%d)
cp -r /home/warzone/data /backup/warzone-$(date +%Y%m%d)
systemctl start warzone-server
```
**Warning:** copying the sled directory while the server is running may
produce an inconsistent snapshot. Stop the server first or use filesystem-level
snapshots (LVM, ZFS, btrfs).
Do not copy the sled directory while the server is running without
filesystem-level snapshots.
### Recovery
@@ -416,14 +529,5 @@ snapshots (LVM, ZFS, btrfs).
2. Replace the data directory with the backup.
3. Start the server.
Messages queued after the backup was taken will be lost. Since all messages
are E2E encrypted, there is no way to recover them from any other source.
### Data Loss Impact
- **Lost key bundles:** users must re-register. No security impact (public
data).
- **Lost message queue:** undelivered messages are permanently lost. Senders
will not know delivery failed (no delivery receipts yet).
- **Corrupted database:** sled includes crash recovery. If the database is
corrupt beyond recovery, delete it and start fresh. Users re-register.
Messages queued after the backup was taken are permanently lost. All
messages are E2E encrypted and cannot be recovered from any other source.