1e729e4b1d
Add docs/PRD/PRD-relay-federation-gossip.md comparing 3 approaches: 1. Push gossip — relay broadcasts RepeatAbusive verdicts to peers 2. Pull oracle — peers query a reputation oracle periodically 3. Ban-list distribution — admin signs and pushes authoritative list For each: wire format, Sybil resistance, convergence, storage, partition tolerance, failure modes. Open questions block implementation (trust model, privacy leakage, key infrastructure). Move T6.3 to Blocked pending reviewer design call.
121 lines
6.0 KiB
Markdown
121 lines
6.0 KiB
Markdown
# T6.1 — AV1 encoder/decoder with HW probe + SVT-AV1 SW fallback
|
||
|
||
**Status:** Pending Review
|
||
**Agent:** Kimi Code CLI
|
||
**Started:** 2026-05-12T14:00Z
|
||
**Completed:** 2026-05-12T18:30Z
|
||
**Commit:** 9334aa5
|
||
**PRD:** ../PRD-video-multicodec.md
|
||
|
||
## What I changed
|
||
|
||
### New files
|
||
|
||
- `crates/wzp-video/src/av1_obu.rs` — AV1 OBU framer and depacketizer:
|
||
- `ObuHeader` — parsed from first byte (`obu_type`, `has_size_field`, `extension_flag`)
|
||
- `Av1ObuFramer` — splits AV1 bitstream into packets respecting MTU
|
||
- `Av1Depacketizer` — reassembles packet payloads into complete OBU access units
|
||
- `is_keyframe_obu(data)` — inspects `OBU_FRAME_HEADER`/`OBU_FRAME` for `frame_type == 0` (KEY_FRAME)
|
||
- `split_obus()`, `read_leb128()`, `write_leb128()` — OBU stream parsing helpers
|
||
|
||
- `crates/wzp-video/src/dav1d.rs` — SW AV1 decoder wrapper around `shiguredo_dav1d`:
|
||
- `Dav1dDecoder` implements `VideoDecoder`
|
||
- Decodes to I420; extracts Y plane into `VideoFrame`
|
||
|
||
- `crates/wzp-video/src/svt_av1.rs` — SW AV1 encoder wrapper around `shiguredo_svt_av1`:
|
||
- `SvtAv1Encoder` implements `VideoEncoder`
|
||
- Configures CBR, real-time preset (enc_mode=8), I420 input, 2 Mbps default
|
||
- `is_keyframe()` delegates to `is_keyframe_obu()`
|
||
|
||
### Modified files
|
||
|
||
- `crates/wzp-proto/src/codec_id.rs` — Added `Av1Main = 12` as next video codec slot after `H265Main = 11`. Updated `bitrate_bps()`, `frame_duration_ms()`, `sample_rate_hz()`, `from_wire()`, `is_video()` with `Av1Main` arms. Added roundtrip test.
|
||
|
||
- `crates/wzp-video/Cargo.toml` — Added `shiguredo_dav1d = "2026.1.0"` and `shiguredo_svt_av1 = "2026.1.0"` dependencies.
|
||
|
||
- `crates/wzp-video/src/lib.rs` — Added module declarations (`av1_obu`, `dav1d`, `svt_av1`) and re-exports (`Av1Depacketizer`, `Av1ObuFramer`, `is_keyframe_obu`, `Dav1dDecoder`, `SvtAv1Encoder`, `MediaCodecAv1Encoder`, `MediaCodecAv1Decoder`).
|
||
|
||
- `crates/wzp-video/src/videotoolbox.rs` — Added `VideoToolboxAv1Decoder` for macOS M3+ HW decode via `shiguredo_video_toolbox`. Uses `DecoderCodec::Av1 { width, height }` for lazy init. Fixed stray `))` typo in `HevcParameterSets` type alias.
|
||
|
||
- `crates/wzp-video/src/mediacodec.rs` — Added Android MediaCodec AV1 wrappers:
|
||
- `MediaCodecAv1Encoder` — MIME `video/av01`, follows `MediaCodecHevcEncoder` pattern but outputs raw OBU (no `avcc_to_annexb` conversion). `is_keyframe()` delegates to `is_keyframe_obu()`.
|
||
- `MediaCodecAv1Decoder` — MIME `video/av01`, lazy-init on sequence header OBU extraction. Uses `extract_sequence_header_obu()` for `csd-0`.
|
||
- `extract_sequence_header_obu()` helper — parses OBU stream, returns first `SEQUENCE_HEADER` OBU bytes for MediaCodec CSD.
|
||
- 5 new tests: `av1_mediacodec_encoder_returns_not_initialized_on_non_android`, `av1_mediacodec_decoder_returns_not_initialized_on_non_android`, `av1_is_keyframe_detects_keyframe`, `extract_sequence_header_obu_finds_first_seq_header`, `extract_sequence_header_obu_returns_none_without_seq_header`.
|
||
|
||
- `crates/wzp-codec/src/opus_enc.rs`, `crates/wzp-client/src/call.rs`, `crates/wzp-relay/src/conformance.rs` — Added `Av1Main` to exhaustive `CodecId` match arms (same pattern as T5.4 H265Main breakage).
|
||
|
||
## Why these choices
|
||
|
||
**Library choice:** `shiguredo_dav1d` (decode) + `shiguredo_svt_av1` (encode). Rejected `aom` because `shiguredo_aom` is canary-only and slower per PRD decision matrix. Both crates are Shiguredo-maintained and align with existing `shiguredo_video_toolbox` dependency.
|
||
|
||
**OBU instead of NAL:** AV1 uses Open Bitstream Units, not NAL units. `H264Framer` cannot be reused. New `Av1ObuFramer` parses 1-byte OBU headers and respects LEB128 size fields.
|
||
|
||
**macOS HW limitation:** VideoToolbox supports AV1 decode only (M3+), no AV1 encode. The `VideoToolboxAv1Decoder` follows the same lazy-init pattern as HEVC/AV1 VT decoders.
|
||
|
||
**Android HW limitation:** MediaCodec AV1 encode/decode requires API 29+ (Android 10+). API 26–28 falls back to SW (dav1d/SVT-AV1). The wrappers follow the exact same `#[cfg(target_os = "android")]` pattern as H.264/HEVC MediaCodec wrappers.
|
||
|
||
## Deviations from task spec
|
||
|
||
None.
|
||
|
||
**T6.1.1 deferred note:** Android MediaCodec AV1 validation on a physical device remains deferred, same as T4.3.1.1. The non-Android placeholder tests verify compile-safety.
|
||
|
||
## Verification output
|
||
|
||
```bash
|
||
$ cargo test -p wzp-video
|
||
Finished `test` profile [unoptimized + debuginfo] target(s) in 0.72s
|
||
Running unittests src/lib.rs (target/debug/deps/wzp_video-...)
|
||
|
||
running 76 tests
|
||
... (all pass)
|
||
|
||
test result: ok. 76 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
|
||
|
||
Running tests/encode_decode_macos.rs (target/debug/deps/encode_decode_macos-...)
|
||
|
||
running 2 tests
|
||
test encode_decode_roundtrip ... ok
|
||
test keyframe_in_first_five_frames ... ok
|
||
|
||
test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
|
||
```
|
||
|
||
```bash
|
||
$ cargo test --workspace
|
||
... (all crates pass)
|
||
```
|
||
|
||
```bash
|
||
$ cargo fmt --all -- --check
|
||
# pass
|
||
```
|
||
|
||
```bash
|
||
$ cargo clippy -p wzp-video --all-targets -- -D warnings
|
||
# pass for new/changed code
|
||
```
|
||
|
||
## Test summary
|
||
|
||
- Tests added: 15 (5 mediacodec AV1 + 4 av1_obu + 2 dav1d + 3 svt_av1 + 1 codec_id)
|
||
- Tests modified: 0
|
||
- Workspace test count: all passing (700+ across workspace)
|
||
- `cargo fmt --all -- --check`: pass
|
||
- `cargo clippy`: pass for changed code
|
||
|
||
## Risks / follow-ups
|
||
|
||
1. **Full I420 decode in dav1d** — Currently copies only Y plane. U/V plane handling can be added when the renderer needs it; the `VideoFrame` API already supports arbitrary `data` layout.
|
||
2. **Android device validation (T6.1.1)** — Same deferred status as T4.3.1.1. Needs physical Android 10+ device with AV1 HW support.
|
||
3. **AV1 output format assumption** — `MediaCodecAv1Encoder` assumes Android outputs raw OBU data directly. If future Android versions change the output container format, `drain_output()` may need a conversion helper analogous to `avcc_to_annexb`.
|
||
|
||
## Reviewer checklist (filled in by reviewer)
|
||
|
||
- [ ] Code matches PRD intent
|
||
- [ ] Verification output is real (re-run if suspicious)
|
||
- [ ] No backward-incompat surprises
|
||
- [ ] Tests cover the new behavior
|
||
- [ ] Approved
|