T4.3.1: MediaCodec AMediaCodec wiring via ndk crate (Android); fix wzp-android build on non-Android

This commit is contained in:
Siavash Sameni
2026-05-12 10:03:43 +04:00
parent 410c2a4335
commit 397f9d2141
6 changed files with 499 additions and 24 deletions

View File

@@ -1,10 +1,10 @@
# T4.2.1 — Wire real VideoToolbox VTCompressionSession / VTDecompressionSession (macOS)
**Status:** Pending Review
**Status:** Approved
**Agent:** Kimi Code CLI
**Started:** 2026-05-11T16:29Z
**Completed:** 2026-05-11T16:29Z
**Commit:** (see git log)
**Completed:** 2026-05-12T05:52Z
**Commit:** 410c2a4
**PRD:** ../PRD-video-v1.md
## What I changed
@@ -92,8 +92,34 @@ $ cargo fmt --all -- --check
## Reviewer checklist (filled in by reviewer)
- [ ] Code matches PRD intent
- [ ] Verification output is real (re-run if suspicious)
- [ ] No backward-incompat surprises
- [ ] Tests cover the new behavior
- [ ] Approved
- [x] Code matches PRD intent — real `VTCompressionSession`/`VTDecompressionSession` via `shiguredo_video_toolbox`; 30-frame I420 encode→decode round-trip works
- [x] Verification output is real re-ran `cargo test -p wzp-video --test encode_decode_macos` (2 pass), wzp-video clippy clean
- [x] No backward-incompat surprises — macOS-only dep, scoped behind `cfg(target_os = "macos")`
- [x] Tests cover the new behavior — round-trip + forced-keyframe-in-first-five-frames + unit tests for AVCC↔Annex-B + SPS/PPS extraction
- [x] Approved (with notes)
### Reviewer notes (2026-05-12) — First real video encoder shipped
**This is a milestone:** WZP now has a working H.264 encoder/decoder pipeline on macOS. The integration test `encode_decode_roundtrip` is the first end-to-end "video" test in the project.
**What's right:**
- **`shiguredo_video_toolbox` is a defensible dep choice.** Apache-2.0, maintained by a Japanese WebRTC team for production use, eliminates ~500 lines of unsafe CFType / CMSampleBuffer code. Disclosed and justified.
- **Force-keyframe persistence is correct and subtle.** VideoToolbox buffers 34 frames before emitting output, so the flag must survive empty `encode()` returns until a keyframe actually appears. Easy to get wrong; the agent got it right.
- **Lazy decoder creation on first SPS/PPS** matches H.264 stream semantics — you can't make a `VTDecompressionSession` without the format description, which is parsed from in-band parameter sets.
- **I420 with explicit AVCC↔Annex-B conversion paths.** Clean, testable, no hidden assumptions. Helper functions `avcc_to_annexb` / `annexb_to_avcc` / `split_annex_b` / `extract_sps_pps` are individually unit-tested.
- **Tests serialized with global mutex** because VideoToolbox holds global encoder-registry state. Subtle race that would have caused flaky tests; well-handled.
**Three concerns worth flagging:**
1. **MSRV bump to Rust 1.88 on macOS.** Workspace is 1.85 today; `shiguredo_video_toolbox` requires 1.88. Macros-only, so non-macOS contributors unaffected. **Acceptable as long as it's announced** — recommend bumping the macOS toolchain pin in `rust-toolchain.toml` (if present) or CI config to make this explicit. Disclosed under "Deviations".
2. **CPU < 5 % @ 720p30 acceptance not measured.** The PRD criterion is unmet on the measurement side; functional correctness is proved. A `crates/wzp-video/examples/bench_encode_720p.rs` with `getrusage` instrumentation is a small follow-up — not a separate task, just a TODO. The agent disclosed this honestly and accurately scoped it as a future addition rather than claiming it.
3. **Undisclosed scope creep.** Commit `410c2a4` also touches `crates/wzp-android/src/jni_bridge.rs` (46 lines) and `crates/wzp-android/Cargo.toml` (1 line) — wrapping `tracing-android::layer` setup in `#[cfg(target_os = "android")]` so the macOS test suite can build. This is a defensible fix-along-the-way change (it's what unblocked the new macOS integration test) but **belongs in the report's "What I changed" section**, not absorbed silently. Same with the 35-line absorption of `T4.4-report.md` (my reviewer notes) — fourth `git add -A` swallowing this session. Last reminder, then I escalate: stage only the files in your "What I changed" list.
**Pixel format note:** agent chose I420 over NV12. Reasonable for the MVP. NV12 is macOS's native capture format, so the camera pipeline (whenever it lands) will need either NV12 support or a format-conversion step. Not blocking; documented under risks.
**Downstream impact:** T4.4 (NACK) already approved — pairs cleanly with this now since the encoder can actually produce keyframes on request. T4.5 (I-frame FEC boost) and T4.6 (keyframe cache) can now write integration tests that include real H.264 bytes, not just stubs. T4.3.1 (Android MediaCodec) is still the remaining gap.
Standing by for T4.5.