T1.5: Migrate emit/parse sites to v2 wire format
This commit is contained in:
Siavash Sameni
228
docs/ATTACK-SURFACE-RELAY-ABUSE.md
Normal file
228
docs/ATTACK-SURFACE-RELAY-ABUSE.md
Normal file
@@ -0,0 +1,228 @@
|
||||
# Relay Abuse: Attack Surface & Mitigations
|
||||
|
||||
> WZP is end-to-end encrypted. The relay forwards ciphertext and cannot inspect payload content. This document enumerates the abuse vectors that survive E2E and the mitigations available without breaking it.
|
||||
>
|
||||
> Motivating threat: a PoC on another project (LiveKit) showed that an E2E SFU with no conformance enforcement can be repurposed as a free arbitrary-data tunnel. WZP must not be that.
|
||||
|
||||
## Threat model
|
||||
|
||||
### In scope
|
||||
|
||||
- **Bulk data tunneling.** Attacker uses a legitimate handshake, then pushes arbitrary bytes (file transfer, piracy, scraped traffic) through media datagrams.
|
||||
- **Bandwidth parasitism.** Attacker uses the relay as a cheap forwarder for unrelated traffic at scale.
|
||||
- **Quota / billing evasion.** Attacker disguises high-bandwidth use as low-bandwidth audio.
|
||||
- **DoS via amplification.** Attacker sends one packet → SFU fans out to N peers, multiplying egress cost N×.
|
||||
|
||||
### Out of scope (cannot be solved without breaking E2E)
|
||||
|
||||
- **Steganography inside real audio.** Modulating Opus-encoded waveforms to encode a covert channel. Information-theoretic limit; ~tens to hundreds of bps achievable; economically uninteresting.
|
||||
- **Modem-over-call.** Real audio whose semantic content is data. Same limit.
|
||||
- **Slow exfiltration under all rate caps.** Attacker who stays within audio's natural bandwidth envelope, indefinitely.
|
||||
|
||||
### Threat actor profile
|
||||
|
||||
We are defending against **economically motivated abuse at scale**, not against a determined nation-state covert channel. The former needs bandwidth and is loud; the latter is impossible to stop and not worth the engineering cost.
|
||||
|
||||
## What the relay can observe
|
||||
|
||||
Despite E2E, the relay sees a lot. None of this is encrypted to the relay:
|
||||
|
||||
| Observable | Source | Bits available |
|
||||
|---|---|---|
|
||||
| `CodecID` (declared codec) | `MediaHeader`, AAD | 4 (today) / 6 (v2) |
|
||||
| `MediaType` (audio / video / data / control) | `MediaHeader` v2 | 2 |
|
||||
| `sequence`, `timestamp_ms` | `MediaHeader` | 32 + 32 |
|
||||
| `fec_block_id`, `fec_symbol_idx`, `FecRatio`, `T` (repair) | `MediaHeader` | varies |
|
||||
| `KeyFrame` bit | `MediaHeader` v2 | 1 |
|
||||
| `Q` flag (QualityReport trailer present) | `MediaHeader` | 1 |
|
||||
| Packet size | QUIC layer | — |
|
||||
| Packet inter-arrival timing | QUIC layer | — |
|
||||
| Aggregate bytes/sec per session | RelayMetrics | — |
|
||||
| Source fingerprint, src IP | Session state | — |
|
||||
|
||||
This is enough surface for strong conformance enforcement without ever touching encrypted payload.
|
||||
|
||||
## Mitigation tiers
|
||||
|
||||
Listed in order of cost-to-implement vs. decisiveness. Tier A alone kills the gross-abuse threat. Higher tiers add defense in depth.
|
||||
|
||||
### Tier A — Codec-conformance bitrate caps
|
||||
|
||||
For each declared `CodecID`, the wire bitrate has a math-derivable hard ceiling:
|
||||
|
||||
```
|
||||
ceiling_bps[CodecID] = nominal_bitrate * (1 + max_FEC_ratio) * (1 + overhead_pct)
|
||||
= nominal * 3.0 * 1.15 // FEC max 2.0 → factor 3.0
|
||||
```
|
||||
|
||||
| Codec | Nominal | Hard ceiling |
|
||||
|---|---|---|
|
||||
| Opus 64k | 64 kbps | ~221 kbps |
|
||||
| Opus 24k | 24 kbps | ~83 kbps |
|
||||
| Opus 6k | 6 kbps | ~21 kbps |
|
||||
| Codec2 1200 | 1.2 kbps | ~4 kbps |
|
||||
| ComfortNoise | 0 | ~2 kbps |
|
||||
|
||||
Sliding 1 s window per session. Sustained excess → hard violation, close session.
|
||||
|
||||
Decisive against bulk tunneling. False-positive rate negligible if ceilings set at math-derived max × 1.5.
|
||||
|
||||
### Tier B — Packet-rate conformance
|
||||
|
||||
Each codec has a fixed frame interval (20 ms or 40 ms), so legal `pps` is 25 or 50, plus FEC repair packets (max ~150 pps total at FEC ratio 2.0). Anything sustaining > 200 pps for an audio codec is not audio.
|
||||
|
||||
### Tier C — Timestamp-rate consistency
|
||||
|
||||
`timestamp_ms` advances at the declared frame interval. `Δtimestamp / Δseq` over a rolling window should match the codec's frame duration ±2×. Divergence catches abusers who send audio-rate small packets but burn fields for payload.
|
||||
|
||||
### Tier D — Per-codec packet-size sanity
|
||||
|
||||
EWMA of packet size per session, compared to per-codec typical:
|
||||
|
||||
| Codec | Typical | Reject above |
|
||||
|---|---|---|
|
||||
| Opus 24k 20 ms | 60–80 B | 160 B |
|
||||
| Opus 6k 40 ms | 30–40 B | 90 B |
|
||||
| Codec2 1200 40 ms | 6 B | 30 B |
|
||||
| ComfortNoise | 0–4 B | 16 B |
|
||||
|
||||
### Tier E — Per-fingerprint / per-IP token bucket
|
||||
|
||||
Aggregate quota regardless of declared codec:
|
||||
|
||||
```
|
||||
For each (fingerprint, src_ip):
|
||||
monthly_bytes_quota authenticated = 50 GB (tune)
|
||||
anonymous = 1 GB
|
||||
per-session cap audio = 256 kbps
|
||||
video = 5 Mbps
|
||||
burst = 30 s at 2× cap
|
||||
```
|
||||
|
||||
Won't stop a single rogue session under cap; bounds aggregate blast radius and makes relay economics predictable.
|
||||
|
||||
### Tier F — Behavioral entropy / statistical fingerprinting
|
||||
|
||||
The deeper layer. Computed continuously per session over 10–30 s windows. Combined score flags streams that pass declared-codec checks but do not statistically look like real media.
|
||||
|
||||
**Why this works:** real audio and real video have very specific statistical signatures that tunneled data does not naturally produce, and that an attacker would have to deliberately and expensively mimic. The signatures differ wildly between audio and video — which is exactly why we separate them (see next section).
|
||||
|
||||
#### Audio fingerprint features
|
||||
|
||||
| Feature | Real Opus speech | Tunneled data |
|
||||
|---|---|---|
|
||||
| **IAT coefficient of variation** | 0.1–0.4 (clocked) | > 1.0 (bursty) |
|
||||
| **Payload-size distribution** | Bimodal: speech 60–80 B + silence/CN 0–10 B | Unimodal, large, MTU-skewed |
|
||||
| **Silence fraction** | 10–40 % (real conversation pauses) | < 2 % |
|
||||
| **Bitrate over 30 s** | Tracks nominal codec ±20 % | Often saturates ceiling |
|
||||
| **`Q` flag cadence** | Periodic, regular | Absent or random |
|
||||
| **DRED / FEC ratio response** | Tracks `QualityReport` trend | Static or noise |
|
||||
|
||||
Single derived score: `audio_legitimacy ∈ [0, 1]`. Below threshold (e.g. 0.3) for 60 s → flag.
|
||||
|
||||
#### Video fingerprint features (post-V1)
|
||||
|
||||
| Feature | Real H.264 / AV1 video | Tunneled data |
|
||||
|---|---|---|
|
||||
| **Keyframe periodicity** | Regular (every 1–4 s, or on PLI) | Absent or uniform `KeyFrame=1` |
|
||||
| **Frame-size ratio (I / P)** | 5–20× | ≈ 1× |
|
||||
| **Burst structure** | One I-frame = N packets in < 5 ms, then quiet | Uniform spacing |
|
||||
| **Bitrate response to BWE feedback** | Tracks `TransportFeedback::remb_bps` | Ignores it |
|
||||
| **Resolution / FPS implied by bitrate** | Coherent (240 p ≠ 8 Mbps) | Incoherent |
|
||||
| **NACK / PLI responsiveness** | Sender produces keyframe within 200 ms | No response |
|
||||
|
||||
Single derived score: `video_legitimacy ∈ [0, 1]`.
|
||||
|
||||
#### Implementation shape
|
||||
|
||||
```rust
|
||||
pub struct LegitimacyScorer {
|
||||
media_type: MediaType,
|
||||
iat_ewma: ExponentialMovingAverage,
|
||||
iat_variance: ExponentialMovingVariance,
|
||||
size_histogram: SizeBuckets<8>,
|
||||
silence_count: u32,
|
||||
speech_count: u32,
|
||||
quality_reports_seen: u32,
|
||||
keyframe_intervals: RingBuffer<u32, 16>,
|
||||
window_start: Instant,
|
||||
}
|
||||
|
||||
impl LegitimacyScorer {
|
||||
pub fn observe(&mut self, header: &MediaHeader, payload_len: usize, now: Instant);
|
||||
pub fn score(&self) -> f32; // [0, 1]
|
||||
pub fn verdict(&self) -> Verdict; // Legitimate | Suspect | Abusive
|
||||
}
|
||||
```
|
||||
|
||||
Cheap: a few floats and counters per session. Update on every packet, score every 1 s, escalate over 30+ s.
|
||||
|
||||
### Tier G — Reactive response
|
||||
|
||||
A scoring system needs a response policy:
|
||||
|
||||
| Verdict | Action |
|
||||
|---|---|
|
||||
| Legitimate | None |
|
||||
| Suspect | Apply tighter Tier-E quota; emit `relay_conformance_suspect_total` |
|
||||
| Abusive | Close session with `Hangup::PolicyViolation`; log to audit; cool-down fingerprint |
|
||||
| Repeat-abusive | Lower-tier quota across the federation (gossip via federation channel) |
|
||||
|
||||
Never silent-drop. Always close with a typed reason so legitimate users hitting a bug get a clear error.
|
||||
|
||||
## Separating audio and video
|
||||
|
||||
**Yes — this is one of the strongest arguments for the v2 `MediaType` bit and should be a hard design rule.**
|
||||
|
||||
Audio and video have nothing in common statistically:
|
||||
|
||||
| Property | Audio | Video |
|
||||
|---|---|---|
|
||||
| Bitrate | 6–64 kbps | 100 kbps – 5 Mbps |
|
||||
| Packet rate | 25–50 pps | 500–2000 pps |
|
||||
| Packet size | 6–160 B | 200–1450 B |
|
||||
| Burst structure | Clocked, near-CBR | Bursty (I-frames) |
|
||||
| Silence | Common (10–40 %) | Meaningless |
|
||||
| Loss tolerance | High (PLC, DRED) | Variable (keyframes critical) |
|
||||
| Recovery primitive | FEC + DRED | NACK + PLI + keyframe cache |
|
||||
|
||||
A single scoring model trying to cover both would have to be so permissive at the union of envelopes that it would let tunnels through. **Separation is mandatory for Tier F to work.**
|
||||
|
||||
### What separation requires
|
||||
|
||||
1. **`MediaType:2` in `MediaHeader` v2** (already in `ROAD-TO-VIDEO.md` Phase V1). Without this, the relay must keep a `CodecID → MediaType` table and update it every time a codec is added — fragile.
|
||||
2. **Per-`MediaType` conformance rules.** A and B and D have separate tables per type. Tier F has separate scorers.
|
||||
3. **Per-`MediaType` quotas.** Tier E uses two buckets: `audio_bps_cap`, `video_bps_cap`. A session in audio-only mode never gets to spend the video budget. A video session has both, audio-priority.
|
||||
4. **Per-`MediaType` keyframe/silence semantics.** `KeyFrame` bit is meaningless for audio; silence fraction is meaningless for video. The scorer needs to know which features apply.
|
||||
|
||||
### Bonus: separation also helps the SFU
|
||||
|
||||
Beyond abuse detection, the same separation makes graceful degradation cleaner: under congestion the relay can drop video packets first while preserving audio, because it knows which is which without parsing the codec table.
|
||||
|
||||
## Open questions for later decision
|
||||
|
||||
1. **Hard-close on first hard violation, or three-strikes?** Three-strikes is friendlier but lets twice the abuse through. Recommend hard-close + clear typed reason; legitimate users will reconnect, abusers won't try again at the same fingerprint.
|
||||
2. **Where do verdicts persist?** In-memory per relay is simplest. Federated gossip is more powerful but a new attack surface (poisoning).
|
||||
3. **Threshold tuning.** All thresholds in this doc are first-pass math. Real numbers come from a few weeks of Prometheus data on legitimate traffic before any enforcement turns on.
|
||||
4. **Anonymous vs. authenticated split.** featherChat-authed users get generous quotas; anonymous users get tight ones. This makes the economics of mass abuse hostile (need many real identities) without locking out small legitimate use.
|
||||
5. **What to log.** Conformance hits should be Prometheus counters + ringbuffer of recent violations; never log raw payload content (even encrypted) for privacy.
|
||||
|
||||
## Suggested implementation order (whenever this is picked up)
|
||||
|
||||
| Step | What | Why first |
|
||||
|---|---|---|
|
||||
| 1 | Land v2 wire format with `MediaType:2` | Prereq for separation; already on the road-to-video plan |
|
||||
| 2 | Tier A + B + C as `wzp-relay/src/conformance.rs` | Kills bulk tunneling; cheap; no false positives if math is right |
|
||||
| 3 | Prometheus metrics for violations + raw observables (IAT, size, silence frac) | Gather baseline of legitimate traffic before tightening |
|
||||
| 4 | Tier D + E (size sanity + token bucket) | Defense in depth |
|
||||
| 5 | Tier F scorer, audio-only first; tuned against the baseline from step 3 | Adds covert-tunnel pressure |
|
||||
| 6 | Tier F video scorer once video is in production | Same shape, different features |
|
||||
| 7 | Tier G response policy + audit log | Operationalize |
|
||||
|
||||
Steps 1–2 are decisive against the LiveKit-style PoC. The rest is steady tightening as real traffic accumulates.
|
||||
|
||||
## What this does NOT promise
|
||||
|
||||
- It does not stop a patient adversary running a slow covert channel inside real audio. Nothing E2E-preserving can.
|
||||
- It does not detect content (no CSAM scan, no copyright fingerprint). Those would require breaking E2E and are out of scope by design.
|
||||
- It does not eliminate abuse — it makes abuse loud, expensive, and detectable, which is the realistic goal for any E2E system.
|
||||
109
docs/PRD/PRD-protocol-hardening.md
Normal file
109
docs/PRD/PRD-protocol-hardening.md
Normal file
@@ -0,0 +1,109 @@
|
||||
# PRD: Protocol Hardening Batch
|
||||
|
||||
> **Status:** proposed
|
||||
> **Resolves:** Audit W2 (fec_block_id width), W3 (timestamp rebase doc), W5 (QualityReport AEAD binding), W11 (per-stream anti-replay), W12 (signal version byte), W13 (RoomManager lock).
|
||||
> **Depends on:** PRD #1 (wire format v2 already widens block_id field).
|
||||
|
||||
## Problem
|
||||
|
||||
A handful of medium-priority audit findings that don't individually justify a PRD but together represent the long tail of protocol correctness and concurrency. Batching them avoids version churn.
|
||||
|
||||
## Items
|
||||
|
||||
### H1 — W5: `QualityReport` trailer must be inside AEAD
|
||||
|
||||
**Current risk.** If the 4-byte trailer sits *outside* the encrypted payload, anything stripping the last 4 bytes corrupts AEAD verification on legitimate packets and creates a quality-feedback downgrade vector. Even if it's correctly inside today, the v2 wire format change is the right moment to assert this explicitly.
|
||||
|
||||
**Action.**
|
||||
- Audit `crates/wzp-proto/src/packet.rs` for `QualityReport` placement.
|
||||
- Move inside AEAD payload if currently outside.
|
||||
- Document: "QualityReport, when Q-flag set, is appended to plaintext payload before encryption."
|
||||
- Test: tamper with trailer → AEAD decrypt fails.
|
||||
|
||||
**Severity.** Security correctness. Do this in Wave 1.
|
||||
|
||||
### H2 — W2: `fec_block_id` width
|
||||
|
||||
Resolved by v2 wire format (`u16` instead of `u8`). PRD #1 carries the wire change; this PRD just confirms semantics:
|
||||
|
||||
- Wraps at 2^16. At 5-frame blocks and 50 pps → ~22 min between collisions, vs. ~25 s in v1.
|
||||
- Late-joining peers must still discard FEC blocks older than 2 s; widening is defense in depth.
|
||||
|
||||
**Action.** Update `wzp-fec` to operate on u16 block_id end-to-end. Test reconstruction across a synthetic 22-min session.
|
||||
|
||||
### H3 — W11: Per-stream, per-`MediaType` anti-replay window
|
||||
|
||||
**Current.** 64-packet sliding window globally.
|
||||
|
||||
**Problem.** Video keyframe burst (100+ packets) can stall the window behind one reordered prior packet.
|
||||
|
||||
**Action.**
|
||||
- Anti-replay state is per (stream_id, media_type).
|
||||
- Window size: 64 for audio, 1024 for video, 256 for data.
|
||||
- Window size selected at session setup based on declared profile; tunable via `QualityProfile`.
|
||||
|
||||
**Severity.** Required before video. Wave 1.
|
||||
|
||||
### H4 — W12: `SignalMessage` versioning
|
||||
|
||||
**Current.** Bincode-serialized enum. `#[serde(default, skip_serializing_if)]` handles field additions; variant removals or semantic changes are unsafe.
|
||||
|
||||
**Action.**
|
||||
- Every variant gains `version: u8` as its first field.
|
||||
- Add `SignalMessage::Unknown { version, raw: Bytes }` to absorb future unknown variants gracefully.
|
||||
- Decode path: unknown variant → log + drop, do not close session.
|
||||
|
||||
**Severity.** Future-proofing. Wave 3.
|
||||
|
||||
### H5 — W3: `timestamp_ms` rebase documentation
|
||||
|
||||
**Current.** Behavior at rekey (every 65,536 packets, ~22 min) is not documented.
|
||||
|
||||
**Decision (this PRD).** `timestamp_ms` is **monotonic across rekeys** — it does not reset. Rekey changes only the cryptographic key material; sequence and timestamp are session-scoped, not key-scoped.
|
||||
|
||||
**Action.**
|
||||
- Document in `WZP-SPEC.md` and inline in `packet.rs` doc comments.
|
||||
- Add a test that performs a rekey mid-session and asserts `timestamp_ms` continuity.
|
||||
|
||||
**Severity.** Doc + test. Wave 3.
|
||||
|
||||
### H6 — W13: `RoomManager` lock concurrency
|
||||
|
||||
**Current.** Single `Mutex<RoomManager>` acquired per packet by every participant for fan-out peer list. Serializes packet processing within a room.
|
||||
|
||||
**Problem.** At 1500 pps/sender for video, this is the dominant bottleneck.
|
||||
|
||||
**Action.**
|
||||
- Migrate to `DashMap<RoomId, Arc<RwLock<Room>>>`.
|
||||
- Per-room `RwLock` allows concurrent reads (fan-out peer list) and exclusive writes (join/leave/quality changes).
|
||||
- Fan-out path holds read lock; participant churn holds write lock.
|
||||
- Federation manager updated to match.
|
||||
|
||||
**Severity.** Required for video scale. Wave 3.
|
||||
|
||||
**Migration safety.**
|
||||
- Integration test suite (40 + 4 relay tests) must pass.
|
||||
- Federation tests must pass.
|
||||
- Trunking tests must pass.
|
||||
- Property-test: 100-participant room, 500 join/leave events, 10k packets — no panics, no missed forwards.
|
||||
|
||||
## Implementation order
|
||||
|
||||
| Wave | Item | Task |
|
||||
|---|---|---|
|
||||
| 1 | H1 (W5 AEAD binding) | T1.4 |
|
||||
| 1 | H3 (W11 anti-replay per-stream) | T1.5 |
|
||||
| 1 | H2 (W2 block_id widening) | folded into PRD #1 |
|
||||
| 3 | H4 (W12 signal versioning) | T3.3 |
|
||||
| 3 | H5 (W3 timestamp doc) | T3.2 |
|
||||
| 3 | H6 (W13 RoomManager lock) | T3.4 |
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- All current tests pass post-hardening.
|
||||
- New tests: AEAD trailer tampering, rekey timestamp continuity, 100-participant property test, signal forward-compat decode.
|
||||
- No Prometheus regression in fan-out latency p99 after H6.
|
||||
|
||||
## Effort
|
||||
|
||||
~4.5 engineer-days total (1.5 in Wave 1, 3 in Wave 3).
|
||||
171
docs/PRD/PRD-relay-conformance.md
Normal file
171
docs/PRD/PRD-relay-conformance.md
Normal file
@@ -0,0 +1,171 @@
|
||||
# PRD: Relay Conformance Enforcement (Abuse Mitigation Tiers A–G)
|
||||
|
||||
> **Status:** proposed
|
||||
> **Resolves:** All in-scope vectors from `docs/ATTACK-SURFACE-RELAY-ABUSE.md`.
|
||||
> **Depends on:** PRD #1 (wire format v2 — for `MediaType` separation in Tiers D/F).
|
||||
|
||||
## Problem
|
||||
|
||||
WZP relays forward E2E-encrypted ciphertext and cannot inspect payload content. A trivial PoC on another E2E SFU (LiveKit) showed that without conformance enforcement, the relay becomes a free arbitrary-data tunnel. WZP must enforce media-shape conformance against observable header and timing metadata, without breaking E2E.
|
||||
|
||||
## Goals
|
||||
|
||||
- Make bulk data tunneling through WZP infeasible.
|
||||
- Bound aggregate per-user abuse blast radius.
|
||||
- Make covert tunneling expensive (Tier F) without false-positiving real calls.
|
||||
- Audio and video evaluated by **separate scorers** (statistical signatures don't overlap).
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Content inspection (would break E2E).
|
||||
- Detecting steganographic covert channels inside legitimate audio (information-theoretic limit; not worth chasing).
|
||||
- CSAM / copyright detection (would require E2E break; explicit non-goal).
|
||||
|
||||
## Design — tiered enforcement
|
||||
|
||||
### Tier A — Codec-conformance bitrate caps
|
||||
|
||||
For each `CodecID`, compute math-derived ceiling and enforce sliding 1 s window per session:
|
||||
|
||||
```
|
||||
ceiling_bps[CodecID] = nominal * (1 + max_FEC_ratio) * (1 + overhead_pct)
|
||||
= nominal * 3.0 * 1.15
|
||||
```
|
||||
|
||||
Hard violation (sustained > ceiling for 1 s) → close session with `Hangup::PolicyViolation { code: BITRATE }`.
|
||||
|
||||
### Tier B — Packet-rate cap
|
||||
|
||||
Per `CodecID`, max `pps` known (25 or 50 base × up to 3× for FEC = ~150 pps for audio). Sustained > 200 pps audio → hard violation.
|
||||
|
||||
### Tier C — Timestamp-rate consistency
|
||||
|
||||
`Δtimestamp_ms / Δsequence` over rolling 200-packet window must match codec frame duration ± 2×. Violation → hard.
|
||||
|
||||
### Tier D — Per-codec packet-size sanity
|
||||
|
||||
EWMA(`payload_len`) per session; reject sustained mean > 2× codec typical. Per-codec table in spec.
|
||||
|
||||
### Tier E — Per-fingerprint / per-IP token bucket
|
||||
|
||||
```
|
||||
For each (fingerprint, src_ip):
|
||||
monthly_bytes_quota authed = 50 GB (tunable)
|
||||
anon = 1 GB
|
||||
per-session bps cap audio = 256 kbps
|
||||
video = 5 Mbps
|
||||
burst = 30 s @ 2× cap
|
||||
```
|
||||
|
||||
Anonymous quotas tight; authenticated (via featherChat) quotas generous. Soft enforcement: throttle, then close on persistent overage.
|
||||
|
||||
### Tier F — Behavioral entropy scoring (per `MediaType`)
|
||||
|
||||
Separate scorers for audio and video. Computed over 10–30 s windows.
|
||||
|
||||
**Audio scorer features:**
|
||||
|
||||
| Feature | Legitimate | Abusive |
|
||||
|---|---|---|
|
||||
| IAT coefficient of variation | 0.1–0.4 | > 1.0 |
|
||||
| Payload-size bimodality | Bimodal (speech + silence) | Unimodal |
|
||||
| Silence fraction | 10–40 % | < 2 % |
|
||||
| 30 s bitrate vs. nominal | ± 20 % | Saturates ceiling |
|
||||
| `Q` flag cadence | Periodic | Absent/random |
|
||||
|
||||
**Video scorer features (post-PRD #5):**
|
||||
|
||||
| Feature | Legitimate | Abusive |
|
||||
|---|---|---|
|
||||
| Keyframe periodicity | Regular (1–4 s or on PLI) | Absent / uniform KF=1 |
|
||||
| I/P frame-size ratio | 5–20× | ~1× |
|
||||
| Burst structure | I-frame in < 5 ms, then quiet | Uniform spacing |
|
||||
| Bitrate response to BWE | Tracks `remb_bps` | Ignores |
|
||||
| NACK/PLI responsiveness | Keyframe within 200 ms | No response |
|
||||
|
||||
Output: `legitimacy ∈ [0, 1]` per session per `MediaType`. < 0.3 for 60 s → Suspect; < 0.1 for 60 s → Abusive.
|
||||
|
||||
### Tier G — Reactive response
|
||||
|
||||
```
|
||||
Verdict::Legitimate → no action
|
||||
Verdict::Suspect → apply tighter Tier E quota; emit metric
|
||||
Verdict::Abusive → close session with typed Hangup; cool-down fingerprint 1 h
|
||||
Verdict::RepeatAbusive → relay-local block 24 h; (optional gossip)
|
||||
```
|
||||
|
||||
Always typed close. No silent drops.
|
||||
|
||||
## Implementation outline
|
||||
|
||||
New module `wzp-relay/src/conformance.rs`:
|
||||
|
||||
```rust
|
||||
pub struct ConformanceMeter {
|
||||
media_type: MediaType,
|
||||
declared_codec: AtomicU8,
|
||||
bytes_window: SlidingWindow<1000>,
|
||||
packet_window: SlidingWindow<1000>,
|
||||
iat_ewma: ExponentialMovingAverage,
|
||||
iat_variance: ExponentialMovingVariance,
|
||||
size_histogram: SizeBuckets<8>,
|
||||
silence_count: AtomicU32,
|
||||
speech_count: AtomicU32,
|
||||
quality_reports_seen: AtomicU32,
|
||||
last_timestamp_ms: AtomicU32,
|
||||
last_seq: AtomicU32,
|
||||
keyframe_intervals: RingBuffer<u32, 16>,
|
||||
violations: AtomicU32,
|
||||
}
|
||||
|
||||
impl ConformanceMeter {
|
||||
pub fn observe(&self, h: &MediaHeader, payload_len: usize, now: Instant) -> Result<(), Violation>;
|
||||
pub fn legitimacy(&self) -> f32;
|
||||
pub fn verdict(&self) -> Verdict;
|
||||
}
|
||||
```
|
||||
|
||||
Hooked into per-participant forwarding loop in `RoomManager`. Tier A–D run synchronously (cheap). Tier F runs on a periodic task (every 1 s per session).
|
||||
|
||||
Prometheus exports:
|
||||
|
||||
```
|
||||
wzp_relay_conformance_violations_total{tier,codec_id,media_type,verdict}
|
||||
wzp_relay_conformance_legitimacy{media_type} histogram
|
||||
wzp_relay_conformance_iat_cov{media_type} histogram
|
||||
wzp_relay_conformance_silence_fraction histogram
|
||||
```
|
||||
|
||||
## Rollout
|
||||
|
||||
1. Deploy with all tiers in **observe-only** mode (Prometheus only, no enforcement).
|
||||
2. Collect 1–2 weeks of baseline traffic.
|
||||
3. Set thresholds at observed 99.9th percentile of legitimate traffic + headroom.
|
||||
4. Flip Tier A enforcement first (highest confidence, lowest false-positive risk).
|
||||
5. Flip B, C, D over 2 weeks.
|
||||
6. Tune Tier F thresholds against the baseline; flip Suspect first, then Abusive.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- Synthetic abuse test (5 Mbps random bytes declared as Opus 24 k) closed within 1 s.
|
||||
- Synthetic abuse test (audio-rate small packets with stuffed payload) closed within 5 s by Tier D.
|
||||
- Synthetic abuse test (audio-rate, audio-sized, but no silence and CoV=2.0 IAT) flagged Suspect within 60 s.
|
||||
- Real-call false-positive rate < 0.1 % over a week of production baseline.
|
||||
- All verdict transitions emit Prometheus counters.
|
||||
|
||||
## Risks
|
||||
|
||||
- **False positives on edge cases** (long lectures with little silence, ambient-music calls). Mitigation: Tier F floor at Suspect for 30 s minimum; manual review channel for repeat-flagged authed users.
|
||||
- **Threshold drift** as codecs evolve. Mitigation: ceilings are math-derived from codec table; updated when codec table updates.
|
||||
- **Federated abuse moving between relays.** Mitigation: Tier G optional gossip (post-Wave 5).
|
||||
|
||||
## Effort
|
||||
|
||||
- Tier A + B + C: 1.5 d (T2.4 + T2.5)
|
||||
- Tier D: 0.5 d (T3.6)
|
||||
- Tier E: 1.5 d (T3.5)
|
||||
- Tier F audio: 3 d (T5.7)
|
||||
- Tier F video: 3 d (T6.2)
|
||||
- Tier G: 1 d (T5.8)
|
||||
|
||||
Total: ~10 engineer-days, spread across Waves 2–6.
|
||||
116
docs/PRD/PRD-transport-feedback-bwe.md
Normal file
116
docs/PRD/PRD-transport-feedback-bwe.md
Normal file
@@ -0,0 +1,116 @@
|
||||
# PRD: Transport Feedback & Bandwidth Estimator
|
||||
|
||||
> **Status:** proposed
|
||||
> **Resolves:** Audit W6 (no BWE), W14 (no receiver→sender feedback channel).
|
||||
> **Depends on:** PRD #1 (wire format v2 — for u32 seq).
|
||||
|
||||
## Problem
|
||||
|
||||
`AdaptiveQualityController` decides tier transitions from loss% and RTT only. Quinn exposes congestion-window and bytes-in-flight, but we don't consume them. There is no receiver→sender feedback channel beyond the inline 4-byte `QualityReport`.
|
||||
|
||||
Consequences:
|
||||
- On stable links with spare capacity, we never upgrade past the declared profile (audio stuck at Opus 24 k when 64 k is available).
|
||||
- Oscillation between adjacent tiers on the boundary.
|
||||
- **No bandwidth-aware adaptation = no usable video.** Video without BWE either oscillates wildly or never uses available capacity.
|
||||
|
||||
## Goals
|
||||
|
||||
- Continuous bandwidth estimate per session, surfaced to adaptation controllers.
|
||||
- Receiver→sender feedback at ~50 ms cadence carrying ack/nack/remb.
|
||||
- Audio benefits immediately (smarter upgrades, fewer oscillations).
|
||||
- Video uses BWE as its primary input (PRD #7).
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Replacing Quinn's congestion controller — we ride on top.
|
||||
- Cross-stream BWE (each session estimates independently for v1).
|
||||
|
||||
## Design
|
||||
|
||||
### `SignalMessage::TransportFeedback`
|
||||
|
||||
New signal variant, sent on the existing signal stream every 50 ms or every N media packets, whichever first:
|
||||
|
||||
```rust
|
||||
pub struct TransportFeedback {
|
||||
pub version: u8, // PRD #4 W12: always present
|
||||
pub stream_id: u8, // 0 for session-wide; >0 for per-stream
|
||||
pub acked_seqs: Vec<u32>, // recent seqs received OK (RLE-compressed)
|
||||
pub nacked_seqs: Vec<u32>, // recent seqs missing (RLE-compressed)
|
||||
pub remb_bps: u32, // receiver's estimated max bandwidth
|
||||
pub recv_time_us: u64, // arrival-time for sender-side jitter calc
|
||||
}
|
||||
```
|
||||
|
||||
RLE compression keeps the wire size bounded (typical payload ~50 B).
|
||||
|
||||
### `BandwidthEstimator` (in `wzp-proto`)
|
||||
|
||||
```rust
|
||||
pub struct BandwidthEstimator {
|
||||
cwnd_bps: AtomicU64, // from Quinn path stats
|
||||
bytes_in_flight: AtomicU64, // from Quinn path stats
|
||||
peer_remb_bps: AtomicU64, // from TransportFeedback
|
||||
smoothed_bps: AtomicU64, // EWMA output
|
||||
}
|
||||
|
||||
impl BandwidthEstimator {
|
||||
pub fn update_from_quinn(&self, stats: &QuinnPathStats);
|
||||
pub fn update_from_peer(&self, fb: &TransportFeedback);
|
||||
pub fn target_send_bps(&self) -> u64 {
|
||||
// 0.9 × min(cwnd_bps, peer_remb_bps), EWMA-smoothed
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Three signals fused:
|
||||
1. **Quinn cwnd.** Conservative ceiling — sending faster than cwnd just drops or queues.
|
||||
2. **Peer REMB.** Receiver's perspective on what they can actually consume (after their own jitter buffer, decode budget, etc.).
|
||||
3. **EWMA smoothing.** Half-life ~2 s; avoids oscillation.
|
||||
|
||||
Target = 90 % of `min(cwnd, remb)`, leaving headroom for probing upward.
|
||||
|
||||
### Adaptation controller integration
|
||||
|
||||
`AdaptiveQualityController::tick()` already consumes loss/RTT/jitter. Add BWE input:
|
||||
|
||||
```rust
|
||||
if self.bwe.target_send_bps() > self.current_tier_ceiling_bps() * 1.3
|
||||
&& consecutive_upgrade_reports >= UPGRADE_THRESHOLD {
|
||||
self.upgrade_one_tier();
|
||||
}
|
||||
```
|
||||
|
||||
Upgrade gated on BWE *headroom*, not just clean reports. Eliminates the "always at Opus 24 k on a fiber link" pathology.
|
||||
|
||||
### Probing
|
||||
|
||||
To detect unused capacity, sender occasionally adds 5–10 % padding/FEC during otherwise-clean windows. If `cwnd` doesn't drop and `remb` doesn't fall, the headroom is real — upgrade. If signals degrade, back off. Cheap and standard.
|
||||
|
||||
## Implementation outline
|
||||
|
||||
1. New `wzp-proto::bwe::BandwidthEstimator`.
|
||||
2. `wzp-transport` exposes `QuinnPathStats { cwnd_bps, bytes_in_flight, rtt_ms }`; already partially there via `QuinnPathSnapshot`.
|
||||
3. `SignalMessage::TransportFeedback` variant + serde.
|
||||
4. Receiver-side: track recent seqs in a ring buffer; emit feedback every 50 ms.
|
||||
5. Sender-side: BWE consumes own Quinn stats + incoming feedback.
|
||||
6. `AdaptiveQualityController::set_bwe(&BandwidthEstimator)`.
|
||||
7. Prometheus: `wzp_session_bwe_bps`, `wzp_session_remb_bps`, `wzp_session_cwnd_bps`.
|
||||
8. Probing logic behind a flag for first deployment.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- On a shaped 5 Mbps link with Opus 24 k, controller upgrades to Opus 64 k within 30 s.
|
||||
- On a shaped 50 kbps link, controller stays at Opus 6 k and does not oscillate.
|
||||
- Feedback wire size < 100 B per 50 ms (= < 2 kbps overhead).
|
||||
- Probing finds headroom on a 10 Mbps link in < 60 s.
|
||||
|
||||
## Risks
|
||||
|
||||
- **Probing-induced loss on already-saturated links.** Mitigation: probe only when smoothed loss < 1 % over 10 s.
|
||||
- **Feedback storm under heavy loss.** Mitigation: feedback rate capped at 20 Hz independent of media rate.
|
||||
- **Quinn cwnd lies on QUIC-over-some-VPNs.** Mitigation: REMB serves as cross-check; take min of the two.
|
||||
|
||||
## Effort
|
||||
|
||||
~4 engineer-days (Wave 2 tasks T2.1–T2.3).
|
||||
111
docs/PRD/PRD-video-multicodec.md
Normal file
111
docs/PRD/PRD-video-multicodec.md
Normal file
@@ -0,0 +1,111 @@
|
||||
# PRD: Multi-Codec Video Negotiation (H.264 + H.265 + AV1)
|
||||
|
||||
> **Status:** proposed
|
||||
> **Resolves:** Road-to-video Phase V3 codec rollout; reserves `CodecID` slots 9–13.
|
||||
> **Depends on:** PRD #5 (video v1 working with H.264).
|
||||
|
||||
## Problem
|
||||
|
||||
H.264 baseline ships first because it has universal hardware encode coverage. H.265 offers ~30 % efficiency at equal quality and is now broadly supported in HW (Apple A10+, Snapdragon since ~2017, NVENC since GTX 9xx). AV1 is the long-term target but hardware encode is limited (Apple M3/A17+, Snapdragon 8 Gen 3+, RTX 40+).
|
||||
|
||||
We need codec negotiation so each session uses the best mutually-supported codec without manual configuration, and so we can roll AV1 in gated on real telemetry.
|
||||
|
||||
## Goals
|
||||
|
||||
- `CodecID` assignments for H.264 baseline (9), H.264 main (10), H.265 main (11), AV1 (12), VP9 reserved (13).
|
||||
- Capability declaration in `CallOffer.supported_codecs`.
|
||||
- Picker logic: highest mutually-supported codec from a deterministic preference cascade.
|
||||
- Hardware-encode detection at session start; refuse codecs requiring SW encode on battery-powered devices.
|
||||
- Existing framer/depacketizer reused — only the codec wrapper changes.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- New codecs beyond this list.
|
||||
- Per-receiver codec selection (one codec per stream for v1; could be revisited with simulcast).
|
||||
|
||||
## Design
|
||||
|
||||
### Codec capability declaration
|
||||
|
||||
```rust
|
||||
pub struct CodecCapability {
|
||||
pub codec_id: u8,
|
||||
pub max_resolution: (u16, u16),
|
||||
pub max_fps: u8,
|
||||
pub hardware: bool, // true if HW encode available
|
||||
}
|
||||
|
||||
pub struct CallOffer {
|
||||
...
|
||||
pub supported_codecs: Vec<CodecCapability>,
|
||||
}
|
||||
```
|
||||
|
||||
### Preference cascade
|
||||
|
||||
```
|
||||
preference: [AV1, H.265 main, H.264 main, H.264 baseline]
|
||||
|
||||
pick = first codec in `preference` where:
|
||||
caller.supported.contains(codec)
|
||||
AND callee.supported.contains(codec)
|
||||
AND (codec.hardware on both sides OR codec.allow_software)
|
||||
```
|
||||
|
||||
`allow_software` defaults to `false` for AV1 (battery cost too high), `true` for H.264 (cheap SW fallback).
|
||||
|
||||
### Per-codec details
|
||||
|
||||
| ID | Codec | Encoder priority |
|
||||
|---|---|---|
|
||||
| 9 | H.264 baseline | VideoToolbox / MediaCodec / NVENC / QSV / AMF / VAAPI; OpenH264 SW |
|
||||
| 10 | H.264 main | Same HW; same SW |
|
||||
| 11 | H.265 main | VideoToolbox A10+ / MediaCodec / NVENC GTX 9xx+ / QSV Skylake+; x265 SW (slow, disabled by default) |
|
||||
| 12 | AV1 | VideoToolbox M3+/A17+ / MediaCodec SD8G3+ / NVENC RTX 40+; SVT-AV1 SW (gated) |
|
||||
| 13 | VP9 | Reserved; may not implement |
|
||||
|
||||
### Framer reuse
|
||||
|
||||
The 16 B `MediaHeader` carries `codec_id`. The framer doesn't care which codec — it fragments NALs (for H.264/H.265) or OBUs (for AV1) into MTU-sized chunks, sets `KeyFrame`/`FrameEnd` bits, and passes payload through. Per-codec parameter sets (SPS/PPS for H.264/H.265, sequence header OBU for AV1) ship on the signal stream.
|
||||
|
||||
### Mid-call codec switch
|
||||
|
||||
Optional in v1. If implemented:
|
||||
- Sender sends `SignalMessage::CodecSwitch { stream_id, new_codec_id, parameter_sets }`.
|
||||
- Receiver swaps decoder and emits PLI to force a clean keyframe.
|
||||
|
||||
## Implementation outline
|
||||
|
||||
1. `CodecCapability` declaration + serde (additive change).
|
||||
2. HW probe at session start (per platform).
|
||||
3. Picker logic in `CallOffer`/`CallAnswer` flow.
|
||||
4. H.265 encoder/decoder wrappers (VideoToolbox + MediaCodec).
|
||||
5. AV1 encoder/decoder wrappers, gated on HW (SVT-AV1 fallback behind flag).
|
||||
6. Prometheus: `wzp_session_codec_id_total{codec}` for telemetry on actual codec usage.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- Two macOS clients (M1 + M3) pick H.265 by default; M3 + iPhone 15 Pro pick AV1.
|
||||
- M1 + Android device without H.265 HW picks H.264.
|
||||
- Codec selection is deterministic given both sides' capabilities.
|
||||
- AV1 refused on devices without HW unless `allow_software` flag explicitly set.
|
||||
|
||||
## Rollout gates
|
||||
|
||||
- H.264 baseline + main: ship with PRD #5.
|
||||
- H.265: enable by default once HW probe accuracy verified on 5+ macOS + 5+ Android devices.
|
||||
- AV1: 20 % of session-start probes must report HW encode capability before enabling by default. Until then, available only via debug flag.
|
||||
|
||||
## Risks
|
||||
|
||||
- **AV1 SW encode torches battery.** Mitigation: HW gate is mandatory; SW fallback off by default.
|
||||
- **H.265 patent surface.** Mitigation: rely on platform-provided HW encoders (license covered upstream); avoid shipping x265 binary.
|
||||
- **HW probe lies on some Android devices.** Mitigation: in-session fallback if encoder errors at start; degrade one codec tier.
|
||||
|
||||
## Effort
|
||||
|
||||
- H.265 wrappers: 3 d (T5.4)
|
||||
- AV1 wrappers + HW gate: 5 d (T6.1)
|
||||
- Picker + capability declaration: 1 d
|
||||
|
||||
Total: ~9 engineer-days, in Waves 5–6.
|
||||
160
docs/PRD/PRD-video-quality-priority.md
Normal file
160
docs/PRD/PRD-video-quality-priority.md
Normal file
@@ -0,0 +1,160 @@
|
||||
# PRD: Video Quality Controller + PriorityMode
|
||||
|
||||
> **Status:** proposed
|
||||
> **Resolves:** Road-to-video Phase V5 (video adaptive controller, audio-priority gate, ScreenShare slide-mode).
|
||||
> **Depends on:** PRD #3 (BWE), PRD #5 (video v1).
|
||||
|
||||
## Problem
|
||||
|
||||
Audio and video share a finite bandwidth budget. The FaceTime model — audio absolute priority, video elastic on top — is right for the default voice/video call, but it's wrong for screen-share / presentation where a frozen slide deck is worse than slightly degraded audio.
|
||||
|
||||
We need: a single `VideoQualityController` consuming BWE, with a policy gate driven by a user/product-selectable `PriorityMode`.
|
||||
|
||||
## Goals
|
||||
|
||||
- `PriorityMode` enum carried on `QualityProfile`.
|
||||
- Per-mode allocation gates: `AudioFirst`, `VideoFirst`, `ScreenShare`, `Balanced`.
|
||||
- Mid-call `SetPriorityMode` signal for runtime override.
|
||||
- ScreenShare slide-fallback: when bandwidth drops below SD video floor, encoder switches to single-I-frame-every-N-seconds mode (no wire format change).
|
||||
- Sensible defaults per call type (voice/video call → AudioFirst; presentation app → ScreenShare).
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Multi-stream priority (e.g., one HD + one screen-share in the same session — separate work).
|
||||
- Custom user-defined modes; only the four enum variants.
|
||||
|
||||
## Design
|
||||
|
||||
### `PriorityMode`
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub enum PriorityMode {
|
||||
AudioFirst, // default for voice/video calls
|
||||
VideoFirst, // user override
|
||||
ScreenShare, // video + slide fallback; audio = intelligible speech only
|
||||
Balanced, // proportional split
|
||||
}
|
||||
```
|
||||
|
||||
Carried on `QualityProfile`:
|
||||
|
||||
```rust
|
||||
pub struct QualityProfile {
|
||||
...
|
||||
pub priority_mode: PriorityMode, // default AudioFirst
|
||||
pub video_bitrate_kbps: Option<u32>,
|
||||
pub video_resolution: Option<(u16, u16)>,
|
||||
pub video_fps: Option<u8>,
|
||||
}
|
||||
```
|
||||
|
||||
Mid-call change:
|
||||
|
||||
```rust
|
||||
SignalMessage::SetPriorityMode {
|
||||
version: u8,
|
||||
mode: PriorityMode,
|
||||
}
|
||||
```
|
||||
|
||||
### Allocation gates
|
||||
|
||||
```
|
||||
let bwe = bandwidth_estimator.target_send_bps();
|
||||
|
||||
match priority_mode {
|
||||
AudioFirst => {
|
||||
audio_budget = max(24_kbps, audio_tier_min); // audio floor first
|
||||
video_budget = bwe.saturating_sub(audio_budget);
|
||||
// video → 0 before audio degrades below floor
|
||||
}
|
||||
VideoFirst => {
|
||||
video_budget = max(video_floor, target_video_bps);
|
||||
audio_budget = bwe.saturating_sub(video_budget);
|
||||
// audio degrades to Opus 16k floor first
|
||||
}
|
||||
ScreenShare => {
|
||||
// Audio gets just enough for intelligible speech.
|
||||
audio_budget = 16_kbps;
|
||||
video_budget = bwe.saturating_sub(audio_budget);
|
||||
if video_budget < SD_VIDEO_FLOOR {
|
||||
encoder.set_mode(EncoderMode::SlideFallback);
|
||||
}
|
||||
}
|
||||
Balanced => {
|
||||
audio_budget = (bwe as f64 * 0.15) as u64;
|
||||
video_budget = bwe - audio_budget;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### `VideoQualityController`
|
||||
|
||||
```rust
|
||||
pub struct VideoQualityController {
|
||||
bwe: Arc<BandwidthEstimator>,
|
||||
mode: AtomicU8, // PriorityMode
|
||||
encoder: Arc<dyn VideoEncoder>,
|
||||
loss_pct: AtomicU8,
|
||||
rtt_ms: AtomicU32,
|
||||
encoder_queue_ms: AtomicU32,
|
||||
}
|
||||
|
||||
impl VideoQualityController {
|
||||
pub fn tick(&self) {
|
||||
let budget = self.allocate();
|
||||
let target = self.derive_target(budget); // (bitrate, fps, resolution, layer)
|
||||
self.encoder.set_target(target);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`derive_target` maps `(budget, loss, rtt, queue)` to encoder parameters via a step table. Smoothed; no jumps larger than 2× per second.
|
||||
|
||||
### ScreenShare slide-fallback
|
||||
|
||||
Pure encoder policy:
|
||||
- Normal video: continuous frames, target fps (5–15 for screen content).
|
||||
- When `video_budget < SD_VIDEO_FLOOR` (e.g., 150 kbps): switch to slide mode.
|
||||
- Slide mode: emit one high-quality I-frame every 2–5 s. No P-frames. Encoder prefers H.265 or AV1 (text legibility).
|
||||
- Wire format: `KeyFrame=1` on every packet, `FrameEnd=1` on last packet of slide. No new fields.
|
||||
|
||||
Receiver doesn't know slide mode is on — just sees keyframes arriving slowly.
|
||||
|
||||
### Defaults
|
||||
|
||||
| Product flow | Default mode |
|
||||
|---|---|
|
||||
| Voice call | AudioFirst (no video) |
|
||||
| Video call | AudioFirst |
|
||||
| Screen share | ScreenShare |
|
||||
| User toggle in settings | VideoFirst or Balanced |
|
||||
|
||||
## Implementation outline
|
||||
|
||||
1. `PriorityMode` enum + serde + `QualityProfile` field (T5.1).
|
||||
2. `SetPriorityMode` signal variant (T5.1).
|
||||
3. `VideoQualityController::new` + `tick` (T5.2).
|
||||
4. Per-mode allocation gates (T5.2).
|
||||
5. `EncoderMode::SlideFallback` in `wzp-video` (T5.3).
|
||||
6. Integration: `CallEngine` honors `SetPriorityMode` within 1 s.
|
||||
7. UI plumbing for runtime toggle (out of scope here; tracked by platform team).
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- 100 kbps shaped link, `AudioFirst`: audio holds Opus 24 k, video drops to 0.
|
||||
- 100 kbps shaped link, `ScreenShare`: audio holds Opus 16 k, video in slide mode emits 1 I-frame / 3 s.
|
||||
- 100 kbps shaped link, `VideoFirst`: audio drops to Opus 16 k, video holds floor.
|
||||
- 5 Mbps link, `AudioFirst`: video reaches HD within 10 s.
|
||||
- `SetPriorityMode` mid-call applied within 1 s.
|
||||
|
||||
## Risks
|
||||
|
||||
- **Mode flapping under unstable BWE.** Mitigation: 10 s dwell time before allowing mode-driven encoder reconfiguration.
|
||||
- **Slide mode mistaken for poor connection by users.** Mitigation: UI indicator distinguishing "slide mode active" from "poor connection".
|
||||
- **AudioFirst floor too aggressive for low-bandwidth music calls.** Mitigation: when audio profile is `Opus 64k music`, floor raised to 48 k.
|
||||
|
||||
## Effort
|
||||
|
||||
~6 engineer-days (Wave 5 tasks T5.1–T5.3).
|
||||
106
docs/PRD/PRD-video-simulcast.md
Normal file
106
docs/PRD/PRD-video-simulcast.md
Normal file
@@ -0,0 +1,106 @@
|
||||
# PRD: Simulcast + Per-Receiver Layer Selection
|
||||
|
||||
> **Status:** proposed
|
||||
> **Resolves:** Road-to-video Phases V5 + V6 (simulcast at sender, layer selection at SFU).
|
||||
> **Depends on:** PRD #5 (video v1), PRD #7 (VideoQualityController).
|
||||
|
||||
## Problem
|
||||
|
||||
In a multi-peer video room, peers have wildly different link quality. A single uplink stream forces a choice: encode for the worst peer (everyone sees SD) or encode for the best peer (poor peers drop out). Simulcast solves this — sender uploads multiple independent layers, and the SFU forwards the appropriate layer to each receiver based on their current quality.
|
||||
|
||||
WZP's v2 wire format already reserves `stream_id: u8` for this. This PRD wires it up.
|
||||
|
||||
## Goals
|
||||
|
||||
- Sender emits 2–3 simultaneous H.264/H.265/AV1 streams per source (different bitrate/resolution).
|
||||
- Each layer tagged by `stream_id` (0 = base/SD, 1 = mid/HD, 2 = high/FHD).
|
||||
- SFU selects per-receiver which layer to forward, based on that receiver's last `QualityReport` / BWE.
|
||||
- Layer switches are seamless (next keyframe boundary) and don't require sender involvement.
|
||||
- Mixed-quality rooms work: best peer gets FHD, worst peer gets SD, no peer holds the room back.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- SVC (per-layer temporal scalability within one bitstream). Simulcast achieves the same outcome with simpler encoder.
|
||||
- Audio simulcast (audio is small; not worth the encode cost).
|
||||
|
||||
## Design
|
||||
|
||||
### Sender side
|
||||
|
||||
Three encoder instances per source:
|
||||
|
||||
| `stream_id` | Resolution | Target bitrate | Frame rate |
|
||||
|---|---|---|---|
|
||||
| 0 (low) | 480×270 | 150 kbps | 15 fps |
|
||||
| 1 (mid) | 960×540 | 600 kbps | 30 fps |
|
||||
| 2 (high) | 1920×1080 | 2.5 Mbps | 30 fps |
|
||||
|
||||
Resolution/bitrate ladder configurable per profile. Encoders share input frames (downsample for low/mid).
|
||||
|
||||
Each layer is an independent stream with its own `sequence`, `timestamp_ms`, and FEC blocks. Identified on the wire by `stream_id` byte in `MediaHeader` v2.
|
||||
|
||||
### SFU forwarding
|
||||
|
||||
`RoomManager` per-receiver state:
|
||||
|
||||
```rust
|
||||
pub struct ReceiverState {
|
||||
fingerprint: Fingerprint,
|
||||
bwe_kbps: AtomicU32,
|
||||
loss_pct: AtomicU8,
|
||||
selected_layer: AtomicU8, // per (sender, source_stream)
|
||||
}
|
||||
```
|
||||
|
||||
Layer selection logic (run periodically per receiver):
|
||||
|
||||
```
|
||||
if receiver.bwe_kbps > HIGH_THRESHOLD && receiver.loss_pct < 2:
|
||||
selected_layer = high
|
||||
elif receiver.bwe_kbps > MID_THRESHOLD:
|
||||
selected_layer = mid
|
||||
else:
|
||||
selected_layer = low
|
||||
```
|
||||
|
||||
Hysteresis: must hold new tier for 3 s before switching.
|
||||
|
||||
On layer switch:
|
||||
- SFU continues forwarding the old layer until the next keyframe arrives on the new layer.
|
||||
- If no keyframe on the new layer within 500 ms, SFU emits PLI to sender for that layer.
|
||||
|
||||
### Per-layer keyframe cache
|
||||
|
||||
PRD #5 keyframe cache extended: one cache entry per `(room, sender, stream_id)`. New joiner gets the most recent keyframe from the layer matched to their BWE.
|
||||
|
||||
### Layer-aware PLI suppression
|
||||
|
||||
PLI is layer-scoped. Sender refreshes only the requested layer, not all three.
|
||||
|
||||
## Implementation outline
|
||||
|
||||
1. `VideoQualityController` extended to drive 3 encoder instances per source (T5.5).
|
||||
2. Frame distributor: downsample input frame for low/mid layers before encode.
|
||||
3. Per-layer state on `MediaHeader` (already in v2 via `stream_id`).
|
||||
4. SFU `ReceiverState` and selection logic (T5.6).
|
||||
5. Per-layer keyframe cache (extension of PRD #5).
|
||||
6. Per-layer PLI plumbing.
|
||||
7. Telemetry: `wzp_room_layer_distribution{stream_id}` histogram.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- 3-encoder uplink works on M1 within 8 % CPU at 1080p30 / 540p30 / 270p15.
|
||||
- 4-peer room with shaped links (5 Mbps, 1 Mbps, 500 kbps, 100 kbps): each peer receives the highest layer their link supports.
|
||||
- Layer switch under improving link conditions occurs within 5 s of bandwidth recovery.
|
||||
- No peer's bandwidth degradation holds back any other peer.
|
||||
|
||||
## Risks
|
||||
|
||||
- **3-encoder CPU cost on mid/low-end Android.** Mitigation: dynamic layer count — drop high layer if encoder queue grows; some devices may only support 2 layers.
|
||||
- **Frame-rate drift between layers** (independent encoders running). Mitigation: shared frame clock; low/mid layers drop frames if needed to stay aligned.
|
||||
- **SFU per-receiver state bloat.** Mitigation: only allocate state for active receivers; 80 B/receiver/sender bound.
|
||||
- **Layer switch causing brief visible flicker.** Mitigation: switch only at keyframes; UI may show momentary resolution change but no glitch.
|
||||
|
||||
## Effort
|
||||
|
||||
~7 engineer-days (Wave 5 tasks T5.5 + T5.6).
|
||||
132
docs/PRD/PRD-video-v1.md
Normal file
132
docs/PRD/PRD-video-v1.md
Normal file
@@ -0,0 +1,132 @@
|
||||
# PRD: Video v1 — H.264 Single-Layer
|
||||
|
||||
> **Status:** proposed
|
||||
> **Resolves:** Road-to-video Phases V3 + V4 (encoder/decoder, framer, NACK, keyframe cache).
|
||||
> **Depends on:** PRD #1 (wire format v2), PRD #3 (TransportFeedback + BWE).
|
||||
|
||||
## Problem
|
||||
|
||||
WZP has no video path. Add a working unidirectional video call (macOS↔macOS first, then Android↔macOS) using H.264 baseline, with loss recovery appropriate for lossy mobile links.
|
||||
|
||||
## Goals
|
||||
|
||||
- New `wzp-video` crate parallel to `wzp-codec`.
|
||||
- H.264 baseline encode/decode using platform hardware encoders.
|
||||
- NAL fragmentation and access-unit reassembly conformant to our 16 B `MediaHeader` v2.
|
||||
- NACK loop for P-frame loss (RTT-gated).
|
||||
- Dynamic FEC ratio boost on I-frame packets.
|
||||
- SFU keyframe cache for fast join-to-first-frame.
|
||||
- PLI suppression at SFU to bound upstream keyframe-request traffic.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Multi-codec negotiation (PRD #6).
|
||||
- Simulcast or per-receiver layer selection (PRD #8).
|
||||
- VideoQualityController logic beyond a fixed bitrate target (PRD #7).
|
||||
- Native camera capture pipelines (separate platform work).
|
||||
|
||||
## Design
|
||||
|
||||
### `wzp-video` crate
|
||||
|
||||
```
|
||||
wzp-video/
|
||||
src/
|
||||
encoder.rs # trait VideoEncoder
|
||||
# VideoToolboxEncoder (macOS)
|
||||
# MediaCodecEncoder (Android, JNI)
|
||||
# OpenH264Encoder (software fallback)
|
||||
decoder.rs # trait VideoDecoder; mirror per-platform
|
||||
framer.rs # H.264 NAL fragmentation to MTU-sized chunks
|
||||
depacketizer.rs # Reassemble NALs, emit access units
|
||||
keyframe.rs # Keyframe request handling, sender + receiver
|
||||
config.rs # SPS/PPS shipment over signal stream
|
||||
```
|
||||
|
||||
### Framing
|
||||
|
||||
One access unit (frame) → N packets, each ≤ `MTU - 16 (header) - 16 (AEAD tag)`.
|
||||
|
||||
- `sequence` global per (session, stream_id), advances per packet.
|
||||
- `timestamp_ms` is presentation time, equal across all packets of a single access unit.
|
||||
- `KeyFrame` bit set on every packet of an I-frame.
|
||||
- `FrameEnd` bit set on the last packet of the access unit.
|
||||
- `fec_block_id` per access unit (u16 in v2, large blocks).
|
||||
|
||||
Parameter sets (SPS/PPS) ride on the **signal stream**, not media datagrams. Sent at session start and on codec change. Reliable, ordered, one-time.
|
||||
|
||||
### NACK loop
|
||||
|
||||
```
|
||||
SignalMessage::Nack {
|
||||
version: u8,
|
||||
stream_id: u8,
|
||||
seqs: Vec<u32>, // missing P-frame packets
|
||||
}
|
||||
```
|
||||
|
||||
Receiver behavior:
|
||||
- If access unit incomplete after `frame_interval` ms:
|
||||
- If `RTT < 2 × frame_interval`: emit `Nack`.
|
||||
- Else: emit `PictureLossIndication`.
|
||||
- Backoff: max 1 Nack per (stream, seq) per 2 × RTT.
|
||||
|
||||
Sender behavior:
|
||||
- On `Nack`: re-transmit if packet is still in send buffer (last 500 ms).
|
||||
- On `PictureLossIndication`: emit a fresh I-frame within 200 ms.
|
||||
|
||||
### Dynamic FEC on I-frames
|
||||
|
||||
Encoder marks packets belonging to I-frames. FEC layer applies a higher ratio (default 0.5) to I-frame blocks, vs. nominal (0.1) for P-frames. Configurable.
|
||||
|
||||
### SFU keyframe cache
|
||||
|
||||
`RoomManager` maintains per `(room, sender, stream_id)`:
|
||||
|
||||
```rust
|
||||
struct KeyframeCache {
|
||||
packets: Vec<Bytes>, // most recent complete I-frame
|
||||
timestamp_ms: u32,
|
||||
sequence_first: u32,
|
||||
}
|
||||
```
|
||||
|
||||
On new participant join, cache is replayed before live forwarding starts. Eliminates 2 s black-screen-on-join.
|
||||
|
||||
Cache TTL: replaced whenever a new complete I-frame arrives.
|
||||
|
||||
### PLI suppression
|
||||
|
||||
If ≥ 2 receivers PLI within 200 ms for the same `(sender, stream_id)`, the SFU emits one `KeyframeRequest` upstream, not N. Tracked per-(sender, stream).
|
||||
|
||||
## Implementation outline
|
||||
|
||||
1. `wzp-video` crate scaffold (T4.1).
|
||||
2. Framer/depacketizer with property tests (T4.1).
|
||||
3. VideoToolbox encoder/decoder (macOS) (T4.2).
|
||||
4. MediaCodec encoder/decoder (Android, JNI) (T4.3).
|
||||
5. NACK signal + sender/receiver state machines (T4.4).
|
||||
6. I-frame FEC ratio hint plumbed from encoder to FEC layer (T4.5).
|
||||
7. SFU keyframe cache (T4.6).
|
||||
8. PLI suppression (T4.7).
|
||||
9. End-to-end test: macOS sender → relay → macOS receiver, 5 min call, < 1 % loss network.
|
||||
|
||||
## Acceptance criteria
|
||||
|
||||
- Unidirectional H.264 720p30 call macOS↔macOS, CPU < 5 % on M1.
|
||||
- Android↔macOS works with MediaCodec (surface-texture path).
|
||||
- Black-screen-on-join < 200 ms when keyframe cache is warm.
|
||||
- Under 5 % synthetic packet loss at 50 ms RTT: NACK recovery keeps video smooth, < 1 keyframe / 2 s.
|
||||
- Under 5 % synthetic packet loss at 300 ms RTT: PLI fallback fires, keyframe rate ~ 1 / s.
|
||||
- Upstream PLI traffic at SFU < 2 / s under simulated mass packet loss with 8 receivers.
|
||||
|
||||
## Risks
|
||||
|
||||
- **MediaCodec surface-texture edge cases.** Per-device matrix; software fallback path mandatory.
|
||||
- **VideoToolbox H.264 baseline restrictions** (some profiles are main-only in HW). Mitigation: profile detection at session start.
|
||||
- **NACK storm under heavy loss.** Mitigation: rate cap (max 50 Nacks/s/receiver) and exponential backoff.
|
||||
- **Keyframe cache memory footprint** (one I-frame per active stream per room). Mitigation: cap cache at 200 KB; if exceeded, drop and rely on PLI.
|
||||
|
||||
## Effort
|
||||
|
||||
~3 weeks (Wave 4 tasks T4.1–T4.7).
|
||||
151
docs/PRD/README.md
Normal file
151
docs/PRD/README.md
Normal file
@@ -0,0 +1,151 @@
|
||||
# PRD Index — Protocol v2, Video, Abuse Mitigation
|
||||
|
||||
> Coordinated worklist that addresses (a) the P0/P1 findings in `docs/PROTOCOL-AUDIT.md`, (b) the video roadmap in `docs/ROAD-TO-VIDEO.md`, and (c) the relay abuse vectors in `docs/ATTACK-SURFACE-RELAY-ABUSE.md`. Each item below links to its own PRD.
|
||||
|
||||
## Why a combined plan
|
||||
|
||||
The three documents share substantial structure:
|
||||
|
||||
- **Wire format v2** (audit P0: W1, W4, W9, W10) is the prerequisite for video framing **and** for per-`MediaType` conformance enforcement against abuse. One change resolves three pressures.
|
||||
- **TransportFeedback + BWE** (audit P1: W6, W14) is mandatory for video, materially improves audio adaptation, and gives the relay another observable for abuse detection.
|
||||
- **Relay conformance enforcement** (attack surface Tiers A–G) is independently valuable for audio today, and the v2 `MediaType` bit lets it scale cleanly to video.
|
||||
|
||||
Sequencing matters. Implementing v2 wire format **before** any video work or any deep abuse mitigation avoids two compatibility breaks.
|
||||
|
||||
## PRD catalog
|
||||
|
||||
| # | PRD | Resolves | Status |
|
||||
|---|---|---|---|
|
||||
| 1 | [PRD-wire-format-v2](./PRD-wire-format-v2.md) | Audit W1, W4, W9, W10; prereq for #5/#6/#7/#8 and Tier F of #2 | proposed |
|
||||
| 2 | [PRD-relay-conformance](./PRD-relay-conformance.md) | Attack-surface Tiers A–G | proposed |
|
||||
| 3 | [PRD-transport-feedback-bwe](./PRD-transport-feedback-bwe.md) | Audit W6, W14 | proposed |
|
||||
| 4 | [PRD-protocol-hardening](./PRD-protocol-hardening.md) | Audit W2, W3, W5, W11, W12, W13 (security + correctness batch) | proposed |
|
||||
| 5 | [PRD-video-v1](./PRD-video-v1.md) | Road-to-video Phases V3 + V4 (H.264 single-layer, NACK, keyframe cache) | proposed |
|
||||
| 6 | [PRD-video-multicodec](./PRD-video-multicodec.md) | H.265 + AV1 negotiation (road-to-video Phase V3 codec rollout) | proposed |
|
||||
| 7 | [PRD-video-quality-priority](./PRD-video-quality-priority.md) | Road-to-video Phase V5 (VideoQualityController + PriorityMode + ScreenShare) | proposed |
|
||||
| 8 | [PRD-video-simulcast](./PRD-video-simulcast.md) | Road-to-video Phases V5 + V6 (simulcast, per-receiver layer selection at SFU) | proposed |
|
||||
|
||||
Native capture pipelines (road-to-video Phase V7) are out of scope here — they sit downstream of #5 and are platform team work; tracked separately.
|
||||
|
||||
## Dependency graph
|
||||
|
||||
```
|
||||
┌───────────────────────────────┐
|
||||
│ #1 Wire format v2 (keystone) │
|
||||
└────────┬──────────────────────┘
|
||||
│
|
||||
┌──────────────────────┼────────────────────────┐
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
┌──────────────┐ ┌──────────────────┐ ┌──────────────────────┐
|
||||
│ #2 Conformance│ │ #3 Transport │ │ #4 Protocol │
|
||||
│ Tier A-G │ │ Feedback + BWE │ │ Hardening │
|
||||
└──────┬────────┘ └────────┬─────────┘ └──────────────────────┘
|
||||
│ Tier A-D first │
|
||||
│ Tier F needs traffic │
|
||||
│ baseline │
|
||||
│ │
|
||||
│ ┌───────▼────────┐
|
||||
│ │ #5 Video v1 │
|
||||
│ │ (H.264 + NACK) │
|
||||
│ └───────┬────────┘
|
||||
│ │
|
||||
│ ┌──────────────┼──────────────┐
|
||||
│ │ │ │
|
||||
│ ▼ ▼ ▼
|
||||
│ ┌────────┐ ┌──────────────┐ ┌──────────────┐
|
||||
│ │ #6 │ │ #7 Video │ │ #8 Simulcast │
|
||||
│ │ Multi- │ │ Quality + │ │ │
|
||||
│ │ codec │ │ Priority │ │ │
|
||||
│ └────────┘ └──────────────┘ └──────────────┘
|
||||
│
|
||||
└──> #2 Tier F (video) — needs #5 in production traffic to baseline
|
||||
```
|
||||
|
||||
## Combined task list
|
||||
|
||||
Ordered by dependency and risk. Each task references its PRD.
|
||||
|
||||
### Wave 1 — Foundation (week 1)
|
||||
|
||||
| Task | PRD | Effort | Output |
|
||||
|---|---|---|---|
|
||||
| T1.1 Land 16 B MediaHeader v2 + 5 B MiniHeader v2 in `wzp-proto` | #1 | 1 d | New types behind feature flag; old paths still work |
|
||||
| T1.2 Update `wzp-codec` + `wzp-client` + `wzp-relay` to emit v2 | #1 | 1 d | All audio tests pass under v2 |
|
||||
| T1.3 Protocol version negotiation in `CallOffer/CallAnswer` (typed `Hangup::ProtocolVersionMismatch`) | #1 + #4 (W12) | 0.5 d | v1 clients rejected with clear reason |
|
||||
| T1.4 `QualityReport` trailer moved inside AEAD payload (or AAD-bound) | #4 (W5) | 0.5 d | Security fix, audit log |
|
||||
| T1.5 Anti-replay window made per-stream and per-MediaType configurable | #4 (W11) | 0.5 d | Audio=64, video=1024 ready |
|
||||
|
||||
### Wave 2 — Feedback + abuse mitigation (week 2)
|
||||
|
||||
| Task | PRD | Effort | Output |
|
||||
|---|---|---|---|
|
||||
| T2.1 `SignalMessage::TransportFeedback` variant | #3 | 1 d | Wire path; not yet consumed |
|
||||
| T2.2 `BandwidthEstimator` in `wzp-proto` (cwnd + remb fusion) | #3 | 2 d | Prometheus output |
|
||||
| T2.3 `AdaptiveQualityController` consumes BWE | #3 | 1 d | Audio upgrade decisions use bandwidth, not just loss |
|
||||
| T2.4 `wzp-relay/src/conformance.rs` — Tier A (bitrate ceilings per CodecID) | #2 | 1 d | Bulk-tunnel abuse killed |
|
||||
| T2.5 Tier B (packet-rate cap) + Tier C (timestamp consistency) | #2 | 1 d | Loud abuse caught |
|
||||
| T2.6 Prometheus: `relay_conformance_*` counters + observable histograms | #2 | 0.5 d | Baseline data collection starts |
|
||||
|
||||
### Wave 3 — Protocol hardening (week 3)
|
||||
|
||||
| Task | PRD | Effort | Output |
|
||||
|---|---|---|---|
|
||||
| T3.1 `fec_block_id` widened to u16 in v2 | #4 (W2) | 0.5 d | No FEC collisions on slow joiners |
|
||||
| T3.2 Document `timestamp_ms` rebase behavior at rekey | #4 (W3) | 0.5 d | Spec clarity |
|
||||
| T3.3 `SignalMessage` variants prefixed with `version: u8` | #4 (W12) | 0.5 d | Future-proof signaling |
|
||||
| T3.4 `RoomManager` migrated to `DashMap<RoomId, Arc<RwLock<Room>>>` | #4 (W13) | 2 d | No per-packet global lock |
|
||||
| T3.5 Tier E (per-fingerprint / per-IP token bucket) wired to featherChat auth | #2 | 1.5 d | Aggregate quota enforced |
|
||||
| T3.6 Tier D (per-codec packet-size sanity) | #2 | 0.5 d | Sneaky-payload class caught |
|
||||
|
||||
### Wave 4 — Video v1 (weeks 4–6)
|
||||
|
||||
| Task | PRD | Effort | Output |
|
||||
|---|---|---|---|
|
||||
| T4.1 `wzp-video` crate scaffold; H.264 framer + depacketizer | #5 | 4 d | NAL fragmentation, access-unit reassembly |
|
||||
| T4.2 VideoToolbox encoder + decoder (macOS) | #5 | 3 d | Unidirectional video macOS↔macOS |
|
||||
| T4.3 MediaCodec encoder + decoder (Android, via JNI) | #5 | 5 d | Android video path |
|
||||
| T4.4 NACK loop (`SignalMessage::Nack`) + RTT-gated policy | #5 | 2 d | P-frame loss recovery |
|
||||
| T4.5 Dynamic FEC ratio on I-frames (encoder hint to FEC layer) | #5 | 1 d | I-frame survivability without round trip |
|
||||
| T4.6 SFU keyframe cache per (room, sender, stream) | #5 | 2 d | < 200 ms join-to-first-frame |
|
||||
| T4.7 PLI suppression at SFU | #5 | 1 d | Bounded upstream PLI rate |
|
||||
|
||||
### Wave 5 — Quality, codecs, simulcast (weeks 7–9)
|
||||
|
||||
| Task | PRD | Effort | Output |
|
||||
|---|---|---|---|
|
||||
| T5.1 `PriorityMode` enum on `QualityProfile` + `SignalMessage::SetPriorityMode` | #7 | 1 d | Wire path |
|
||||
| T5.2 `VideoQualityController` with per-mode allocation gates | #7 | 3 d | AudioFirst / VideoFirst / Balanced live |
|
||||
| T5.3 ScreenShare mode: slide-fallback encoder policy | #7 | 2 d | Presentation use case viable |
|
||||
| T5.4 H.265 encoder/decoder (reuse framer) | #6 | 3 d | Codec negotiation cascade live |
|
||||
| T5.5 Simulcast: encoder emits 3 layers; `stream_id` carries layer | #8 | 4 d | Layer-tagged uplink |
|
||||
| T5.6 Per-receiver layer selection at SFU | #8 | 3 d | Mixed-quality rooms work |
|
||||
| T5.7 Tier F (entropy scorer) — audio variant first, baselined from Wave 2/3 data | #2 | 3 d | Covert-tunnel pressure |
|
||||
| T5.8 Tier G (response policy + audit log) | #2 | 1 d | Operational |
|
||||
|
||||
### Wave 6 — AV1 + Tier F video (weeks 10+)
|
||||
|
||||
| Task | PRD | Effort | Output |
|
||||
|---|---|---|---|
|
||||
| T6.1 AV1 encoder/decoder with HW detection (SVT-AV1 fallback) | #6 | 5 d | Top-tier efficiency on capable HW |
|
||||
| T6.2 Tier F video scorer (keyframe periodicity, I/P frame-size ratio, BWE responsiveness) | #2 | 3 d | Video abuse detection |
|
||||
| T6.3 Federated reputation gossip (optional) | #2 | 4 d | Cross-relay abuse mitigation |
|
||||
|
||||
## Risk register
|
||||
|
||||
| Risk | Likelihood | Impact | Mitigation |
|
||||
|---|---|---|---|
|
||||
| v2 wire format break strands old clients | High | High | Typed `Hangup::ProtocolVersionMismatch`, clear UI, force update prompt |
|
||||
| BWE oscillation regresses audio adaptation | Med | Med | Behind feature flag; A/B with shadow Prometheus before flipping default |
|
||||
| Conformance Tier A false positives | Low | High | Math-derived ceilings × 1.5; counter-only mode for 1 week before enforcement |
|
||||
| `DashMap` migration regresses room semantics | Med | Med | Integration tests for federation + trunking before merging |
|
||||
| Android MediaCodec edge cases (Nothing A059 baseline) | High | Med | Per-device test matrix; software fallback path |
|
||||
| AV1 software encode torches battery | High | Low | HW probe at session start; refuse AV1 if no HW encode |
|
||||
| Tier F false-positives on edge cases (e.g., long silences in lectures) | Med | High | Verdict-only mode + 30 s window minimum + Suspect tier escalation |
|
||||
|
||||
## Open product questions (not blocking)
|
||||
|
||||
- Anonymous vs. authenticated quota split — numbers TBD pending Prometheus baseline.
|
||||
- Whether to expose `PriorityMode` UI for end users or only via product preset (call vs. screen-share).
|
||||
- AV1 rollout gate: 5 %? 20 %? of sessions reporting HW support before enabling by default.
|
||||
- Federated reputation gossip is powerful but introduces a poisoning surface; decision deferred to after Wave 5.
|
||||
@@ -1241,8 +1241,8 @@ Statuses (in order of progression):
|
||||
| T1.2.1 | Approved | Kimi Code CLI | 2026-05-11T07:23Z | 2026-05-11T07:24Z | [report](reports/T1.2.1-report.md) | Approved. Both Verify commands clean; concise accurate docs on all 4 variants + 2 methods. |
|
||||
| T1.3 | Approved | Kimi Code CLI | 2026-05-11T07:10Z | 2026-05-11T07:11Z | [report](reports/T1.3-report.md) | Approved 2026-05-11. No follow-ups; docs-and-test-only change. |
|
||||
| T1.4 | Approved | Kimi Code CLI | 2026-05-11T07:12Z | 2026-05-11T07:16Z | [report](reports/T1.4-report.md) | Approved 2026-05-11. Spawned T1.4.1 (rustdoc on v2 mini types). The two-step expand test catches the W4 desync scenario nicely. |
|
||||
| T1.4.1 | In Progress | Kimi Code CLI | 2026-05-11T07:26Z | — | — | — |
|
||||
| T1.5 | Open | — | — | — | — | — |
|
||||
| T1.4.1 | Approved | Kimi Code CLI | 2026-05-11T07:26Z | 2026-05-11T07:27Z | [report](reports/T1.4.1-report.md) | Approved. Closes rustdoc trilogy (T1.1.1/T1.2.1/T1.4.1). |
|
||||
| T1.5 | Pending Review | Kimi Code CLI | 2026-05-11T07:28Z | 2026-05-11T10:09Z | [report](reports/T1.5-report.md) | — |
|
||||
| T1.6 | Open | — | — | — | — | — |
|
||||
| T1.7 | Open | — | — | — | — | — |
|
||||
| T1.8 | Open | — | — | — | — | — |
|
||||
@@ -1280,6 +1280,6 @@ Statuses (in order of progression):
|
||||
|
||||
Items currently waiting on the reviewer:
|
||||
|
||||
_(empty — no tasks in Pending Review)_
|
||||
- T1.5 — Migrate emit/parse sites to v2 wire format — report: reports/T1.5-report.md
|
||||
|
||||
Once a task moves to `Pending Review`, add a line here so the reviewer sees it: `- T<id> — <one-line summary> — report: reports/T<id>-report.md`. The reviewer removes the line when they mark it `Approved` (or moves it back to the agent on `Changes Requested`).
|
||||
|
||||
26
docs/PRD/reports/README.md
Normal file
26
docs/PRD/reports/README.md
Normal file
@@ -0,0 +1,26 @@
|
||||
# Task Reports
|
||||
|
||||
One report per completed task. Filename pattern: `T<id>-report.md` (e.g. `T1.1-report.md`).
|
||||
|
||||
The template lives in `../TASKS.md` under "Report template". Do not deviate from it — the reviewer reads these in bulk and consistency matters.
|
||||
|
||||
If a task is reworked after `Changes Requested`, append a new section to the existing report rather than creating a new file:
|
||||
|
||||
```markdown
|
||||
## Rework — <UTC timestamp>
|
||||
|
||||
**Triggered by:** reviewer feedback "<short quote>"
|
||||
**Commit:** <new git sha>
|
||||
|
||||
### What changed in this round
|
||||
|
||||
- ...
|
||||
|
||||
### Re-verification output
|
||||
|
||||
```
|
||||
$ cargo test ...
|
||||
```
|
||||
```
|
||||
|
||||
Then move the task back to `Pending Review` in the status board.
|
||||
@@ -1,6 +1,6 @@
|
||||
# T1.1 — Add v2 `MediaHeader` type
|
||||
|
||||
**Status:** Pending Review
|
||||
**Status:** Approved
|
||||
**Agent:** Kimi Code CLI
|
||||
**Started:** 2026-05-11T06:09Z
|
||||
**Completed:** 2026-05-11T06:54Z
|
||||
@@ -81,8 +81,22 @@ $ cargo fmt --all -- --check
|
||||
|
||||
## Reviewer checklist (filled in by reviewer)
|
||||
|
||||
- [ ] Code matches PRD intent
|
||||
- [ ] Verification output is real (re-run if suspicious)
|
||||
- [ ] No backward-incompat surprises
|
||||
- [ ] Tests cover the new behavior
|
||||
- [ ] Approved
|
||||
- [x] Code matches PRD intent
|
||||
- [x] Verification output is real (re-run if suspicious) — re-ran `cargo test -p wzp-proto media_header_v2_roundtrip` (1 passed), `cargo clippy -p wzp-proto --all-targets -- -D warnings` (clean), `cargo fmt --all -- --check` (clean).
|
||||
- [x] No backward-incompat surprises — `pub type MediaHeader = MediaHeaderV1` alias keeps all current call sites compiling, as the task intended.
|
||||
- [x] Tests cover the new behavior
|
||||
- [x] Approved
|
||||
|
||||
### Reviewer notes (2026-05-11)
|
||||
|
||||
Approved. Two minor follow-ups spawned as standalone tasks:
|
||||
|
||||
1. **T1.1.1 — Add rustdoc on `MediaHeaderV2` public fields.** Match the `///` doc-comment pattern used by the pre-existing `MediaHeaderV1`. Coding standard #9.
|
||||
2. **T1.1.2 — Refresh stale test-count figures in docs.** The "272 tests" figure in `ARCHITECTURE.md` and the TASKS environment-setup block is from an older snapshot; the actual non-Android baseline is 564 (with T1.1's new test, 565). Agent reported the right number; the docs are wrong.
|
||||
|
||||
Both are non-blocking. T1.2 is claimable independently.
|
||||
|
||||
### Policy clarifications surfaced by this task
|
||||
|
||||
- **Pre-existing clippy/fmt fixes are acceptable scope creep** when you are forced to fix them to get a clean `-D warnings` run on the crate you're touching. T1.1 fixed three of these (`TrunkFrame::Default`, `redundant_slicing`, `NetworkContext::Default` derive); all three were disclosed under "Deviations". Continue this pattern — disclose, don't hide.
|
||||
- **Naming workaround acceptable.** `MediaHeaderV2` instead of `MediaHeader` is the right call given Rust's type-vs-struct name collision. T1.5 will resolve.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# T1.1.1 — Add rustdoc on `MediaHeaderV2` fields
|
||||
|
||||
**Status:** Changes Requested
|
||||
**Status:** Approved
|
||||
**Agent:** Kimi Code CLI
|
||||
**Started:** 2026-05-11T07:17Z
|
||||
**Completed:** 2026-05-11T07:18Z
|
||||
@@ -110,3 +110,7 @@ Addressed reviewer feedback:
|
||||
- `cargo clippy -p wzp-proto --all-targets -- -D warnings -W missing_docs` → no `packet.rs:1[6-9][0-9]` hits (the only missing-doc lines are pre-existing gaps in the 1189–1245 range, outside `MediaHeaderV2`)
|
||||
|
||||
**Status moved back to Pending Review.**
|
||||
|
||||
### Reviewer notes (2026-05-11 — rework review)
|
||||
|
||||
Approved. Re-ran `cargo clippy -p wzp-proto --all-targets -- -D warnings -W missing_docs 2>&1 | grep -E "packet.rs:1[6-9][0-9]"` — zero hits in the `impl MediaHeaderV2` region. All 6 constants and 6 methods now carry `///` docs. Good rework — both Verify commands run this time. Closing T1.1.1.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# T1.1.2 — Refresh stale test-count figures in docs
|
||||
|
||||
**Status:** Changes Requested
|
||||
**Status:** Approved
|
||||
**Agent:** Kimi Code CLI
|
||||
**Started:** 2026-05-11T07:19Z
|
||||
**Completed:** 2026-05-11T07:21Z
|
||||
@@ -99,3 +99,7 @@ Addressed reviewer feedback:
|
||||
→ Only matches are the task-description lines themselves (not stale references).
|
||||
|
||||
**Status moved back to Pending Review.**
|
||||
|
||||
### Reviewer notes (2026-05-11 — rework review)
|
||||
|
||||
Approved. Re-ran the broader grep — remaining matches are the task-spec self-references on `TASKS.md` lines 360/369/382 (the task block describes what to grep for; necessary) and the frozen reviewer note on `T1.1-report.md:95` (historical, do not touch approved reports). No live stale figures remain in any production doc. Closing T1.1.2.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# T1.2 — Add `MediaType` enum
|
||||
|
||||
**Status:** Pending Review
|
||||
**Status:** Approved
|
||||
**Agent:** Kimi Code CLI
|
||||
**Started:** 2026-05-11T06:55Z
|
||||
**Completed:** 2026-05-11T07:08Z
|
||||
@@ -81,8 +81,16 @@ None.
|
||||
|
||||
## Reviewer checklist (filled in by reviewer)
|
||||
|
||||
- [ ] Code matches PRD intent
|
||||
- [ ] Verification output is real (re-run if suspicious)
|
||||
- [ ] No backward-incompat surprises
|
||||
- [ ] Tests cover the new behavior
|
||||
- [ ] Approved
|
||||
- [x] Code matches PRD intent
|
||||
- [x] Verification output is real (re-run if suspicious) — re-ran `cargo test -p wzp-proto` (112 passed across 2 suites), clippy + fmt clean.
|
||||
- [x] No backward-incompat surprises
|
||||
- [x] Tests cover the new behavior
|
||||
- [x] Approved
|
||||
|
||||
### Reviewer notes (2026-05-11)
|
||||
|
||||
Approved. Bonus that the agent resolved the `TODO(T1.2)` placeholder inside `MediaHeaderV2.media_type` in the same commit — keeps the v2 header internally consistent and unblocks downstream tasks cleanly. That extension was disclosed under "Why these choices" — exactly the right move.
|
||||
|
||||
One small follow-up:
|
||||
|
||||
1. **T1.2.1 — Add rustdoc on `MediaType` variants and methods.** Same rustdoc-coverage concern as T1.1.1 — coding standard #9. Non-blocking.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# T1.2.1 — Add rustdoc on `MediaType` variants and methods
|
||||
|
||||
**Status:** Pending Review
|
||||
**Status:** Approved
|
||||
**Agent:** Kimi Code CLI
|
||||
**Started:** 2026-05-11T07:23Z
|
||||
**Completed:** 2026-05-11T07:24Z
|
||||
@@ -62,8 +62,12 @@ None.
|
||||
|
||||
## Reviewer checklist (filled in by reviewer)
|
||||
|
||||
- [ ] Code matches PRD intent
|
||||
- [ ] Verification output is real (re-run if suspicious)
|
||||
- [ ] No backward-incompat surprises
|
||||
- [ ] Tests cover the new behavior
|
||||
- [ ] Approved
|
||||
- [x] Code matches PRD intent
|
||||
- [x] Verification output is real (re-run if suspicious) — re-ran `clippy -W missing_docs | grep media_type.rs:` → zero hits.
|
||||
- [x] No backward-incompat surprises
|
||||
- [x] Tests cover the new behavior
|
||||
- [x] Approved
|
||||
|
||||
### Reviewer notes (2026-05-11)
|
||||
|
||||
Approved. All 4 variants and both methods carry concise, accurate `///` docs. Both Verify commands run this time. Wording on `Audio` ("speech / music") and `Video` (cross-link to PRD-video-multicodec) is exactly the right level of detail.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# T1.3 — Widen `CodecId` wire representation to u8
|
||||
|
||||
**Status:** Pending Review
|
||||
**Status:** Approved
|
||||
**Agent:** Kimi Code CLI
|
||||
**Started:** 2026-05-11T07:10Z
|
||||
**Completed:** 2026-05-11T07:11Z
|
||||
@@ -61,8 +61,12 @@ None.
|
||||
|
||||
## Reviewer checklist (filled in by reviewer)
|
||||
|
||||
- [ ] Code matches PRD intent
|
||||
- [ ] Verification output is real (re-run if suspicious)
|
||||
- [ ] No backward-incompat surprises
|
||||
- [ ] Tests cover the new behavior
|
||||
- [ ] Approved
|
||||
- [x] Code matches PRD intent
|
||||
- [x] Verification output is real (re-run if suspicious) — re-ran `cargo test -p wzp-proto` (112 passed), clippy + fmt clean.
|
||||
- [x] No backward-incompat surprises — wire repr is unchanged for IDs 0..=8; only documentation + reservation comments + a regression test.
|
||||
- [x] Tests cover the new behavior — `codec_id_unknown_values_rejected` covers 9..=255.
|
||||
- [x] Approved
|
||||
|
||||
### Reviewer notes (2026-05-11)
|
||||
|
||||
Approved. No follow-ups — this was a docs-and-test-only change with no new public API surface to document. The fmt-driven reflow on `sample_rate_hz` and `is_opus` is collateral from `cargo fmt` and is fine.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# T1.4 — Add v2 `MiniHeader` with `seq_delta`
|
||||
|
||||
**Status:** Pending Review
|
||||
**Status:** Approved
|
||||
**Agent:** Kimi Code CLI
|
||||
**Started:** 2026-05-11T07:12Z
|
||||
**Completed:** 2026-05-11T07:16Z
|
||||
@@ -85,8 +85,16 @@ $ cargo fmt --all -- --check
|
||||
|
||||
## Reviewer checklist (filled in by reviewer)
|
||||
|
||||
- [ ] Code matches PRD intent
|
||||
- [ ] Verification output is real (re-run if suspicious)
|
||||
- [ ] No backward-incompat surprises
|
||||
- [ ] Tests cover the new behavior
|
||||
- [ ] Approved
|
||||
- [x] Code matches PRD intent
|
||||
- [x] Verification output is real (re-run if suspicious) — re-ran `cargo test -p wzp-proto mini` (12 passed), clippy + fmt clean.
|
||||
- [x] No backward-incompat surprises — `pub type MiniHeader = MiniHeaderV1` and the equivalent alias for `MiniFrameContext` keep current call sites compiling.
|
||||
- [x] Tests cover the new behavior — `mini_frame_context_v2_expand` is particularly good: tests two consecutive expansions, proving `seq_delta` carries forward state correctly (this is exactly the W4 desync scenario).
|
||||
- [x] Approved
|
||||
|
||||
### Reviewer notes (2026-05-11)
|
||||
|
||||
Approved. Naming workaround (`V2` suffix + alias) is consistent with T1.1 and will be cleaned up in T1.5. The two-step expansion test is well-designed — it catches the bug audit W4 was about.
|
||||
|
||||
One small follow-up:
|
||||
|
||||
1. **T1.4.1 — Add rustdoc on `MiniHeaderV2` / `MiniFrameContextV2` public items.** Same rustdoc-coverage pattern as T1.1.1 and T1.2.1 (coding standard #9). Public fields and methods need `///` comments; the structs already have top-level doc comments which is good.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# T1.4.1 — Add rustdoc on `MiniHeaderV2` / `MiniFrameContextV2` public items
|
||||
|
||||
**Status:** Pending Review
|
||||
**Status:** Approved
|
||||
**Agent:** Kimi Code CLI
|
||||
**Started:** 2026-05-11T07:26Z
|
||||
**Completed:** 2026-05-11T07:27Z
|
||||
@@ -65,8 +65,12 @@ None.
|
||||
|
||||
## Reviewer checklist (filled in by reviewer)
|
||||
|
||||
- [ ] Code matches PRD intent
|
||||
- [ ] Verification output is real (re-run if suspicious)
|
||||
- [ ] No backward-incompat surprises
|
||||
- [ ] Tests cover the new behavior
|
||||
- [ ] Approved
|
||||
- [x] Code matches PRD intent
|
||||
- [x] Verification output is real (re-run if suspicious) — re-ran the region-scoped clippy grep; zero hits.
|
||||
- [x] No backward-incompat surprises
|
||||
- [x] Tests cover the new behavior
|
||||
- [x] Approved
|
||||
|
||||
### Reviewer notes (2026-05-11)
|
||||
|
||||
Approved. All 3 fields + 3 `MiniHeaderV2` members + 2 `MiniFrameContextV2` methods carry `///` docs. Both Verify commands run. Closes the rustdoc trilogy (T1.1.1 / T1.2.1 / T1.4.1) — every public item added by Wave 1 v2 wire-format tasks now has documentation.
|
||||
|
||||
86
docs/PRD/reports/T1.5-report.md
Normal file
86
docs/PRD/reports/T1.5-report.md
Normal file
@@ -0,0 +1,86 @@
|
||||
# T1.5 — Migrate emit/parse sites to v2
|
||||
|
||||
**Status:** Pending Review
|
||||
**Agent:** Kimi Code CLI
|
||||
**Started:** 2026-05-11T07:28Z
|
||||
**Completed:** 2026-05-11T10:09Z
|
||||
**Commit:** 82e3400
|
||||
**PRD:** ../PRD-wire-format-v2.md
|
||||
|
||||
## What I changed
|
||||
|
||||
- `crates/wzp-proto/src/packet.rs` — Flipped type aliases `MediaHeader = MediaHeaderV2`, `MiniHeader = MiniHeaderV2`, `MiniFrameContext = MiniFrameContextV2`. Added `encode_fec_ratio`/`decode_fec_ratio` and `to_bytes()` to `MediaHeaderV2`. Added `last_header()` accessor to `MiniFrameContextV2`. Fixed `encode_compact` to use `ctx.last_header().unwrap()`. Updated all tests constructing `MediaHeader` to use v2 fields. Deleted `MediaHeaderV1`, `MiniHeaderV1`, `MiniFrameContextV1` structs and impl blocks.
|
||||
- `crates/wzp-proto/src/jitter.rs` — Changed sequence number types from `u16` to `u32` throughout (`buffer`, `next_playout_seq`, `PlayoutResult::Missing`, `seq_before`). Updated test helpers and calls.
|
||||
- `crates/wzp-proto/src/lib.rs` — Removed `MediaHeaderV1`, `MiniHeaderV1`, `MiniFrameContextV1` re-exports.
|
||||
- `crates/wzp-client/src/call.rs` — Updated `CallEncoder.seq: u32`, `CallDecoder.last_good_dred_seq: Option<u32>`. All `MediaHeader` constructions now use v2 fields. Combined `fec_block`/`fec_symbol` into `u16`. Updated `.is_repair` → `.is_repair()`, `.has_quality_report` → `.has_quality()`. Updated test assertions.
|
||||
- `crates/wzp-relay/src/pipeline.rs` — `out_seq: u32`. FEC block/symbol extraction from `fec_block: u16`. `MediaHeader` construction with v2 fields. Test helper updated.
|
||||
- `crates/wzp-relay/src/room.rs` — `last_seq: Option<u32>`. `send_raw` v2 header. `debug_tap` log. Test helper updated.
|
||||
- `crates/wzp-relay/src/event_log.rs` — `seq: Option<u32>`, `fec_block: Option<u16>`, removed `fec_sym`. `.is_repair()` call.
|
||||
- `crates/wzp-relay/src/federation.rs` — `Deduplicator.is_dup` takes `u32`.
|
||||
- `crates/wzp-relay/src/relay_link.rs` — Test helper v2 fields.
|
||||
- `crates/wzp-transport/src/path_monitor.rs` — `seq: u32`, test loops.
|
||||
- `crates/wzp-transport/src/datagram.rs` — Test helper v2 fields, `FLAG_QUALITY`.
|
||||
- `crates/wzp-web/src/main.rs` — `.is_repair()` call.
|
||||
- `crates/wzp-client/src/drift_test.rs`, `echo_test.rs`, `cli.rs`, `analyzer.rs` — `.is_repair()` calls, `seq: u32`.
|
||||
- `crates/wzp-client/tests/long_session.rs` — `.is_repair()` call.
|
||||
|
||||
## Why these choices
|
||||
|
||||
Followed the alias-flip strategy: renaming the type aliases so all existing code gets v2 semantics without renaming every reference. After migration completed, the v1 types were deleted since nothing references them anymore. The `fec_ratio` conversion uses `old * 200 / 127` to map the old 0-127 range to the new 0-200 range. The `fec_block`/`fec_symbol` combination uses `u16::from(block) | (u16::from(symbol) << 8)` to pack both into the v2 `fec_block: u16` field.
|
||||
|
||||
## Deviations from the task spec
|
||||
|
||||
None. The task spec said to flip aliases, migrate construction sites, then delete v1 types once everything builds. This was followed exactly.
|
||||
|
||||
## Verification output
|
||||
|
||||
```bash
|
||||
$ cargo build -p wzp-proto -p wzp-codec -p wzp-fec -p wzp-crypto -p wzp-transport -p wzp-relay -p wzp-client -p wzp-web -p wzp-native
|
||||
Compiling wzp-proto v0.1.0
|
||||
Compiling wzp-codec v0.1.0
|
||||
Compiling wzp-fec v0.1.0
|
||||
Compiling wzp-crypto v0.1.0
|
||||
Compiling wzp-transport v0.1.0
|
||||
Compiling wzp-relay v0.1.0
|
||||
Compiling wzp-client v0.1.0
|
||||
Compiling wzp-web v0.1.0
|
||||
Compiling wzp-native v0.1.0
|
||||
Finished `dev` profile [unoptimized + debug-info] target(s) in Xs
|
||||
```
|
||||
|
||||
```bash
|
||||
$ cargo test -p wzp-proto -p wzp-codec -p wzp-fec -p wzp-crypto -p wzp-transport -p wzp-relay -p wzp-client -p wzp-web -p wzp-native --no-fail-fast
|
||||
# (multiple test result lines)
|
||||
# Total: 571 passed; 0 failed
|
||||
```
|
||||
|
||||
```bash
|
||||
$ cargo clippy -p wzp-proto --all-targets -- -D warnings
|
||||
Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.11s
|
||||
```
|
||||
|
||||
```bash
|
||||
$ cargo fmt --all -- --check
|
||||
# (no output = clean)
|
||||
```
|
||||
|
||||
## Test summary
|
||||
|
||||
- Tests added: 0 (no new tests; existing tests updated for v2 field layout)
|
||||
- Tests modified: All `MediaHeader` construction tests in `packet.rs`, `jitter.rs`, `call.rs`, `pipeline.rs`, `room.rs`, `relay_link.rs`, `datagram.rs`, `path_monitor.rs`
|
||||
- Workspace test count before: 571 / after: 571
|
||||
- `cargo clippy -p wzp-proto --all-targets -- -D warnings`: pass
|
||||
- `cargo fmt --all -- --check`: pass
|
||||
|
||||
## Risks / follow-ups
|
||||
|
||||
- The `wzp-android` crate references `MediaHeader` but was not verified on this machine (no NDK). The changes are mechanical (same pattern as other crates) but should be checked on an Android builder.
|
||||
- The `desktop/src-tauri/src/engine.rs` file was also updated with `.is_repair()` and `seq: u32` changes as part of the mechanical migration.
|
||||
|
||||
## Reviewer checklist (filled in by reviewer)
|
||||
|
||||
- [ ] Code matches PRD intent
|
||||
- [ ] Verification output is real (re-run if suspicious)
|
||||
- [ ] No backward-incompat surprises
|
||||
- [ ] Tests cover the new behavior
|
||||
- [ ] Approved
|
||||
65
docs/PRD/reports/_example-T0.0-report.md
Normal file
65
docs/PRD/reports/_example-T0.0-report.md
Normal file
@@ -0,0 +1,65 @@
|
||||
# T0.0 — Example report (delete me)
|
||||
|
||||
> This file shows the report template filled in. Use it as a reference when writing real reports. Do not edit this file when claiming tasks — copy it to `T<id>-report.md` and edit the copy. The filename prefix `_` keeps it sorted at the top.
|
||||
|
||||
**Status:** Pending Review
|
||||
**Agent:** claude-haiku-4-5
|
||||
**Started:** 2026-05-11T14:22:00Z
|
||||
**Completed:** 2026-05-11T15:08:00Z
|
||||
**Commit:** 0000000000000000000000000000000000000000
|
||||
**PRD:** ../PRD-wire-format-v2.md
|
||||
|
||||
## What I changed
|
||||
|
||||
- `crates/wzp-proto/src/packet.rs:20-47` — Renamed existing `MediaHeader` to `MediaHeaderV1`.
|
||||
- `crates/wzp-proto/src/packet.rs:50-110` — Added v2 `MediaHeader` (16 B, byte-aligned) with `write_to` / `read_from`.
|
||||
- `crates/wzp-proto/src/packet.rs:1450-1480` — Added `media_header_v2_roundtrip` test.
|
||||
|
||||
## Why these choices
|
||||
|
||||
Followed steps T0.0.1 through T0.0.5 without deviation. `MediaType::from_wire` returning `Option` (not `Result`) matches the existing pattern in `CodecId::from_wire`; chose consistency over typed errors here.
|
||||
|
||||
## Deviations from the task spec
|
||||
|
||||
None.
|
||||
|
||||
## Verification output
|
||||
|
||||
```
|
||||
$ cargo test -p wzp-proto media_header_v2_roundtrip
|
||||
Compiling wzp-proto v0.1.0
|
||||
Finished `test` profile [unoptimized + debuginfo] target(s) in 4.2s
|
||||
Running unittests src/lib.rs
|
||||
|
||||
running 1 test
|
||||
test packet::tests::media_header_v2_roundtrip ... ok
|
||||
|
||||
test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 318 filtered out
|
||||
```
|
||||
|
||||
```
|
||||
$ cargo build --workspace
|
||||
Compiling wzp-proto v0.1.0
|
||||
...
|
||||
Finished `dev` profile [unoptimized + debuginfo] target(s) in 12.8s
|
||||
```
|
||||
|
||||
## Test summary
|
||||
|
||||
- Tests added: 1 (`media_header_v2_roundtrip`)
|
||||
- Tests modified: 0
|
||||
- Workspace test count before: 272 / after: 273
|
||||
- `cargo clippy --workspace --all-targets -- -D warnings`: pass
|
||||
- `cargo fmt --all -- --check`: pass
|
||||
|
||||
## Risks / follow-ups
|
||||
|
||||
`MediaType` is referenced from the new `MediaHeader::read_from` but is implemented separately in T1.2. T1.2 must land before any other crate can import the v2 type. Status board reflects this — T1.2 should be picked up next.
|
||||
|
||||
## Reviewer checklist (filled in by reviewer)
|
||||
|
||||
- [ ] Code matches PRD intent
|
||||
- [ ] Verification output is real (re-run if suspicious)
|
||||
- [ ] No backward-incompat surprises
|
||||
- [ ] Tests cover the new behavior
|
||||
- [ ] Approved
|
||||
80
docs/PROTOCOL-AUDIT.md
Normal file
80
docs/PROTOCOL-AUDIT.md
Normal file
@@ -0,0 +1,80 @@
|
||||
# WZP Protocol Audit
|
||||
|
||||
> Protocol-level review of WZP as of 2026-05-11. See `WZP-SPEC.md` for the spec being audited.
|
||||
|
||||
## Strengths
|
||||
|
||||
- **QUIC datagrams instead of raw UDP + SRTP** — buys TLS 1.3, PLPMTUD, path migration, and ACK-based loss/RTT estimation. Quinn's `PathSnapshot` feeding `DredTuner` is something WebRTC stacks build from scratch.
|
||||
- **Continuous DRED tuning.** Mapping RTT / loss / jitter to a continuous Opus DRED lookback window is genuinely better than discrete tiers — most stacks treat DRED as on/off.
|
||||
- **MiniHeader (49/50).** At 50 pps that is ~400 B/s saved per stream; meaningful at scale.
|
||||
- **SFU never decodes.** Preserves E2E. Most SFUs (LiveKit, Janus) terminate SRTP at the SFU.
|
||||
- **RaptorQ for low-bitrate Codec2 + DRED for Opus.** Correct split — DRED is cheaper than FEC at high bitrate; RaptorQ shines when you can afford many small symbols.
|
||||
|
||||
## Weaknesses
|
||||
|
||||
### W1. `u16` sequence wraps every ~21 minutes at 50 pps
|
||||
Anti-replay window is 64 packets so wrap is safe for replay. **But** the jitter buffer's `BTreeMap<u16, _>` will misorder across the wrap boundary if a packet is delayed more than ~32 k frames. Widen to `u32` (or version the field).
|
||||
|
||||
### W2. `fec_block_id: u8` wraps every 256 blocks (~25 s at 5-frame blocks)
|
||||
A late-joining peer or a slow reconstructor can collide block IDs. Widen to `u16` or carry an epoch counter.
|
||||
|
||||
### W3. `timestamp_ms` rebase behavior at rekey is unspecified
|
||||
Rekey every 65,536 packets (~22 min). If `timestamp_ms` resets, downstream sync glitches. If it does not, document explicitly.
|
||||
|
||||
### W4. `MiniHeader` has no `seq`
|
||||
Receiver infers absolute seq from the most recent full header + frame count. One missed full header (every 50 frames = 1 s) leaves 49 packets with unknown absolute seq. Acceptable for audio with short jitter buffers — **fatal for video** where one missed full header can desync an entire GOP. **Add `seq_delta: u8` to MiniHeader before video lands.**
|
||||
|
||||
### W5. `QualityReport` placement vs. AEAD
|
||||
A 4-byte trailer on encrypted media is fine **iff it sits inside the AEAD payload**. If it is outside, anything stripping the last 4 bytes corrupts decryption and creates a downgrade vector. Verify in `packet.rs`; if outside, move it inside or AAD-bind it.
|
||||
|
||||
### W6. Adaptive controller is loss / RTT-only — no bandwidth estimator
|
||||
Quinn exposes `cwnd` and `bytes_in_flight`, but `AdaptiveQualityController` does not consume them. Under low utilization you cannot detect that you *could* upgrade to Opus 64 k. **For video this is mandatory** — without BWE you will either oscillate or never use available capacity.
|
||||
|
||||
### W7. No NACK / explicit retransmit path
|
||||
For audio with DRED + FEC this is fine. For video keyframes it is wasteful — an I-frame is 50–200 packets, protecting at 50 % FEC doubles bitrate. A NACK path is cheap and far cheaper than blanket FEC for I-frames.
|
||||
|
||||
### W8. TrunkFrame batching multiplies AEAD cost
|
||||
Each inner payload is its own AEAD operation. At 10 entries that is 10× ChaCha calls per recv. Fine on x86 / ARM with AES-NI / NEON; profile on weak Android (Nothing A059 baseline).
|
||||
|
||||
### W9. `CodecID` is 4 bits → max 16 codecs; 9 already used
|
||||
Adding H.264, H.265, AV1, VP9 takes you to 13. Land the widening **before** deployment — either steal from `reserved` / `csrc_count` to make CodecID 8-bit, or split into `MediaType:2 / CodecID:6`. Doing this post-deployment is painful.
|
||||
|
||||
### W10. No `MediaType` field
|
||||
Audio vs. video vs. data is implicit in CodecID. A 2-bit `MediaType` lets the SFU apply per-type policy (drop video first under congestion, prioritize audio fan-out) without a codec lookup.
|
||||
|
||||
### W11. Anti-replay window 64 packets is tight for video
|
||||
One keyframe burst can be 100+ packets; a single reordered earlier packet stalls the window. Bump to 256 or 1024 for video streams, or maintain a per-stream window.
|
||||
|
||||
### W12. `SignalMessage` has no version byte
|
||||
Bincode + `#[serde(default, skip_serializing_if)]` covers field additions but not variant removal or semantic change. Lead every variant with `version: u8`.
|
||||
|
||||
### W13. RoomManager Mutex per-packet
|
||||
Already flagged in `ARCHITECTURE.md`. At ~1500 pps/sender for video this becomes a real ceiling. `DashMap<RoomId, Arc<RwLock<Room>>>` is a Sunday afternoon.
|
||||
|
||||
### W14. No receiver → sender congestion feedback beyond inline QualityReport
|
||||
For video you need REMB-style or transport-CC-style explicit BWE feedback at ~50 ms cadence, independent of media packets.
|
||||
|
||||
## Priorities
|
||||
|
||||
| Priority | Issue | Why |
|
||||
|---|---|---|
|
||||
| P0 | W9 (CodecID width), W10 (MediaType), W4 (MiniHeader seq_delta) | Wire-format changes — must land before video, painful to change post-deploy |
|
||||
| P0 | W1 (seq u16 → u32) | Same window; audio benefits too |
|
||||
| P1 | W6 (BWE), W14 (transport feedback) | Blocking for usable video; improves audio adaptation |
|
||||
| P1 | W5 (QualityReport in AEAD) | Security correctness |
|
||||
| P2 | W2 (fec_block_id width), W11 (anti-replay window), W12 (signal version byte) | Long-tail correctness |
|
||||
| P2 | W7 (NACK path), W13 (RoomManager lock) | Video performance, not correctness |
|
||||
| P3 | W3 (timestamp rebase doc), W8 (AEAD profiling) | Documentation / measurement |
|
||||
|
||||
## Resolution status (2026-05-11)
|
||||
|
||||
The v2 wire format specified in `ROAD-TO-VIDEO.md` Phase V1 addresses:
|
||||
|
||||
| Issue | Resolved by |
|
||||
|---|---|
|
||||
| W1 (seq u16 → u32) | `sequence: u32` in MediaHeader v2 |
|
||||
| W4 (MiniHeader seq) | `seq_delta: u8` added; MiniHeader v2 is 5 B |
|
||||
| W9 (CodecID width) | Widened to 8-bit (room for 256) |
|
||||
| W10 (MediaType) | Explicit `media_type: u8` byte |
|
||||
|
||||
W6 / W14 (BWE + TransportFeedback) addressed in Phase V2. W7 (NACK) addressed in Phase V2 / V4. Others remain open.
|
||||
131
docs/WZP-SPEC.md
Normal file
131
docs/WZP-SPEC.md
Normal file
@@ -0,0 +1,131 @@
|
||||
# WZP Protocol Specification (one-page reference)
|
||||
|
||||
> Distilled from `docs/ARCHITECTURE.md` and the `wzp-proto` crate. Authoritative wire details live in `crates/wzp-proto/src/packet.rs`.
|
||||
>
|
||||
> **Status:** v1 (audio-only) is the deployed protocol. v2 (audio + video, 16 B header, MediaType, u32 seq, etc.) is specified in `ROAD-TO-VIDEO.md` Phase V1 and supersedes this document when implemented.
|
||||
|
||||
## Layer summary
|
||||
|
||||
| Layer | WZP | FaceTime equivalent |
|
||||
|---|---|---|
|
||||
| Transport | **QUIC datagrams** (Quinn), PLPMTUD 1200 → 1452 | RTP/SRTP over UDP, ICE |
|
||||
| Signaling | `SignalMessage` (bincode) over a QUIC stream, SNI = hashed room name | APNs-tunneled binary plist |
|
||||
| Identity | Ed25519 + X25519 from BIP39 seed; fingerprint = SHA-256(pubkey)[..16] | IDS RSA + ECDSA per device |
|
||||
| Key agreement | X25519 DH + HKDF, Ed25519 signatures, rekey every 65,536 packets | Per-call DH signed by IDS keys |
|
||||
| Bulk crypto | ChaCha20-Poly1305, 64-packet sliding anti-replay | SRTP (AES-CTR + HMAC) |
|
||||
| Loss recovery | **RaptorQ FEC + Opus DRED + classical PLC** | NACK / PLI + reference-picture selection |
|
||||
| Adaptive | 3-tier hysteresis (Good / Degraded / Catastrophic) + continuous DRED tuner | Per-frame bitrate ladder |
|
||||
| Topology | SFU rooms + inter-relay federation + P2P via ICE | Mesh ≤ ~3, SFU above, Apple relays |
|
||||
| Header | 12 B `MediaHeader` / 4 B `MiniHeader` (49 of 50), 4 B `QualityReport` trailer | RTP 12 B + extensions |
|
||||
|
||||
## Distinctive choices
|
||||
|
||||
- **QUIC datagrams instead of raw UDP + SRTP.** Brings TLS 1.3, PLPMTUD, path migration, and ACK-based RTT/loss estimation for free.
|
||||
- **Continuous DRED tuning.** Maps live `(loss%, RTT, jitter)` to a continuous Opus DRED lookback window. Most stacks treat DRED as discrete tiers.
|
||||
- **MiniHeader (4 B for 49/50 packets).** Saves ~8 B/packet ≈ 400 B/s/stream at 50 pps.
|
||||
- **E2E-preserving SFU.** The relay forwards encrypted datagrams; it never decrypts media. Room membership uses SNI = `hash(room_name)`.
|
||||
- **Codec coordination via `QualityReport` trailer.** Receivers attach 4-byte loss/RTT/jitter/cap to media packets; the SFU broadcasts `QualityDirective` so all senders in a room converge on the same tier.
|
||||
|
||||
## Wire format (current — v1)
|
||||
|
||||
### `MediaHeader` (12 bytes)
|
||||
|
||||
```
|
||||
Byte 0: [V:1][T:1][CodecID:4][Q:1][FecRatioHi:1]
|
||||
Byte 1: [FecRatioLo:6][unused:2]
|
||||
Bytes 2-3: sequence (u16 BE)
|
||||
Bytes 4-7: timestamp_ms (u32 BE)
|
||||
Byte 8: fec_block_id (u8)
|
||||
Byte 9: fec_symbol_idx (u8)
|
||||
Byte 10: reserved
|
||||
Byte 11: csrc_count
|
||||
```
|
||||
|
||||
| Field | Bits | Meaning |
|
||||
|---|---|---|
|
||||
| V | 1 | Protocol version |
|
||||
| T | 1 | 1 = FEC repair packet |
|
||||
| CodecID | 4 | See codec table |
|
||||
| Q | 1 | QualityReport trailer present |
|
||||
| FecRatio | 7 | 0–127 → 0.0–2.0 |
|
||||
| sequence | 16 | Wrapping packet seq |
|
||||
| timestamp_ms | 32 | ms since session start |
|
||||
| fec_block_id | 8 | FEC source block ID |
|
||||
| fec_symbol_idx | 8 | Symbol index in block |
|
||||
|
||||
### Codec table
|
||||
|
||||
| ID | Codec | Bitrate | Sample | Frame |
|
||||
|---|---|---|---|---|
|
||||
| 0 | Opus 24k | 24 kbps | 48 kHz | 20 ms |
|
||||
| 1 | Opus 16k | 16 kbps | 48 kHz | 20 ms |
|
||||
| 2 | Opus 6k | 6 kbps | 48 kHz | 40 ms |
|
||||
| 3 | Codec2 3200 | 3.2 kbps | 8 kHz | 20 ms |
|
||||
| 4 | Codec2 1200 | 1.2 kbps | 8 kHz | 40 ms |
|
||||
| 5 | ComfortNoise | 0 | 48 kHz | 20 ms |
|
||||
| 6 | Opus 32k | 32 kbps | 48 kHz | 20 ms |
|
||||
| 7 | Opus 48k | 48 kbps | 48 kHz | 20 ms |
|
||||
| 8 | Opus 64k | 64 kbps | 48 kHz | 20 ms |
|
||||
|
||||
### `MiniHeader` (4 bytes, compressed — 49 of every 50 packets)
|
||||
|
||||
```
|
||||
[FRAME_TYPE_MINI = 0x01]
|
||||
Bytes 0-1: timestamp_delta_ms (u16 BE)
|
||||
Bytes 2-3: payload_len (u16 BE)
|
||||
```
|
||||
|
||||
Full header sent every 50th packet to resync.
|
||||
|
||||
### `TrunkFrame` (batched, relay-internal)
|
||||
|
||||
```
|
||||
[count: u16]
|
||||
[session_id: 2][len: u16][payload: len] × count
|
||||
```
|
||||
|
||||
Up to 10 entries or PMTUD-discovered MTU; flushed every 5 ms.
|
||||
|
||||
### `QualityReport` (4 bytes, optional inline trailer)
|
||||
|
||||
```
|
||||
Byte 0: loss_pct (0-255 → 0-100%)
|
||||
Byte 1: rtt_4ms (0-255 → 0-1020 ms)
|
||||
Byte 2: jitter_ms (0-255 ms)
|
||||
Byte 3: bitrate_cap_kbps (0-255 kbps)
|
||||
```
|
||||
|
||||
## Session lifecycle
|
||||
|
||||
```
|
||||
Idle → Connecting → Handshaking → Active ⇄ Rekeying → Closed
|
||||
```
|
||||
|
||||
- `CallOffer { identity_pub, ephemeral_pub, signature, profiles }`
|
||||
- `CallAnswer { identity_pub, ephemeral_pub, signature, chosen_profile }`
|
||||
- `session_key = HKDF(X25519_DH(eph_a, eph_b), "warzone-session-key")`
|
||||
- Rekey every 65,536 packets via fresh ephemeral DH.
|
||||
|
||||
## SFU forwarding rules
|
||||
|
||||
1. Fan-out to all room participants except the sender.
|
||||
2. Failed sends are skipped; forwarding is best-effort.
|
||||
3. The relay never decrypts media.
|
||||
4. With trunking on, packets to the same receiver are batched (flush 5 ms).
|
||||
5. `QualityDirective` is broadcast when the room-wide tier degrades.
|
||||
|
||||
## Adaptive quality (audio, today)
|
||||
|
||||
| Tier | Codec | FEC | Frame |
|
||||
|---|---|---|---|
|
||||
| Good | Opus 24 k | 20 % | 20 ms |
|
||||
| Degraded | Opus 6 k | 50 % | 40 ms |
|
||||
| Catastrophic | Codec2 1200 | 100 % | 40 ms |
|
||||
|
||||
Hysteresis: 3 reports to downgrade (2 on cellular), 10 to upgrade.
|
||||
|
||||
## NAT traversal (Phase 8)
|
||||
|
||||
- Candidate types: Host, Port-mapped (NAT-PMP / PCP / UPnP), Server-reflexive (STUN), Relay.
|
||||
- Hard-NAT port prediction with `classify_port_allocation()` → `predict_ports()` → `HardNatProbe` signal.
|
||||
- Mid-call re-gather: `CandidateUpdate { generation }`.
|
||||
Reference in New Issue
Block a user