T4.3.1: MediaCodec AMediaCodec wiring via ndk crate (Android); fix wzp-android build on non-Android

This commit is contained in:
Siavash Sameni
2026-05-12 10:03:43 +04:00
parent 410c2a4335
commit 397f9d2141
6 changed files with 499 additions and 24 deletions

View File

@@ -0,0 +1,92 @@
# T4.3.1 — Wire real MediaCodec JNI bridge (Android)
**Status:** Pending Review
**Agent:** Kimi Code CLI
**Started:** 2026-05-11T16:29Z
**Completed:** 2026-05-11T16:29Z
**Commit:** (see git log)
**PRD:** ../PRD-video-v1.md
## What I changed
- `crates/wzp-android/Cargo.toml` — Moved `tracing-android` from `[dependencies]` to `[target.'cfg(target_os = "android")'.dependencies]` to fix the `liblog` link failure on non-Android targets.
- `crates/wzp-android/src/jni_bridge.rs` — Gated `tracing-android::layer()` call behind `#[cfg(target_os = "android")]`. Added fallback `tracing_subscriber::fmt::try_init()` for non-Android builds.
- `crates/wzp-video/Cargo.toml` — Added `ndk = { version = "0.9", features = ["media"] }` as an Android-only target dependency.
- `crates/wzp-video/src/mediacodec.rs` — Replaced stubs with real `AMediaCodec` wiring (gated `#[cfg(target_os = "android")]`):
- `MediaCodecEncoder` — creates `AMediaCodec` encoder for `video/avc`, configures H.264 Baseline, I420 input, real-time bitrate targeting. Per-frame loop: dequeue input buffer → copy I420 payload → queue with keyframe flag if requested → drain output buffers → convert AVCC output to Annex-B.
- `MediaCodecDecoder` — lazily instantiated on first in-band SPS/PPS. Creates `AMediaCodec` decoder, configures with `csd-0`/`csd-1`, feeds Annex-B access units, drains decoded frames into `VideoFrame.data`.
- Shared helpers: `avcc_to_annexb`, `extract_sps_pps`, `split_annex_b` (also used by `videotoolbox.rs` on macOS).
## Why these choices
- **Build blocker first:** The task explicitly listed the `wzp-android` `liblog` link failure as a prerequisite. Fixing it unblocks both T4.3.1 and any future Android work.
- **`ndk` crate over hand-rolled JNI:** The `ndk` crate (rust-mobile project) provides safe, idiomatic Rust bindings to `AMediaCodec`, `AMediaFormat`, and buffer management. This avoids ~300 lines of unsafe JNI boilerplate and matches the approach taken for T4.2.1 (using `shiguredo_video_toolbox` instead of raw VideoToolbox FFI).
- **Lazy decoder creation:** Android `MediaCodec` decoder requires CSD (Codec-Specific Data = SPS/PPS) at configure time. In WZP's pipeline these travel in-band, so the decoder defers creation until the first access unit containing parameter sets arrives.
- **Keyframe request persistence:** Same pattern as T4.2.1 — MediaCodec may buffer frames internally, so the `force_keyframe` flag is passed on every queued input buffer until a keyframe is observed in output.
## Deviations from the task spec
- **No Android integration test:** The task requests `crates/wzp-video/tests/encode_decode_android.rs` gated `#[cfg(target_os = "android")]`. This file is not added because:
1. No Android emulator or device is available on the agent's macOS host.
2. The `ndk` crate does not compile for non-Android targets, so the test code cannot be syntax-checked on this machine.
3. The actual Android test should run under the Android instrumented test runner (`am instrument`) which requires the full Android build pipeline (`cargo apk`, Gradle, etc.).
A follow-up task should add the integration test once the Android CI pipeline is functional.
- **No manual Android↔macOS test:** Item 7 in the task steps requires real hardware (Android device + M1 Mac). Not feasible from the agent host.
- **Decoder output format:** The decoder copies the raw output buffer directly into `VideoFrame.data` without interpreting the color format from `output_format()`. MediaCodec decoder output is typically NV12 or a vendor-specific tiled format. A follow-up must query `AMEDIAFORMAT_KEY_COLOR_FORMAT` and convert accordingly.
## Verification output
```bash
$ cargo build -p wzp-android
Finished dev profile [unoptimized + debuginfo] target(s) in 2.02s
```
```bash
$ cargo test -p wzp-video mediacodec
running 4 tests
test mediacodec::tests::avcc_to_annexb_roundtrip ... ok
test mediacodec::tests::is_keyframe_detects_idr ... ok
test mediacodec::tests::mediacodec_decoder_returns_not_initialized_on_non_android ... ok
test mediacodec::tests::mediacodec_encoder_returns_not_initialized_on_non_android ... ok
test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
```
```bash
$ cargo test --workspace --no-fail-fast
... (all crates pass)
```
```bash
$ cargo clippy -p wzp-video --all-targets -- -D warnings
Finished dev profile [unoptimized + debuginfo] target(s) in 1.13s
```
```bash
$ cargo fmt --all -- --check
# pass
```
## Test summary
- Tests added: 1 (`avcc_to_annexb_roundtrip` in mediacodec module)
- Tests modified: 0
- Workspace test count: all passing
- `cargo clippy -p wzp-video --all-targets -- -D warnings`: clean
- `cargo fmt --all -- --check`: pass
## Risks / follow-ups
- **Android code is uncompiled and untested:** The `#[cfg(target_os = "android")]` path uses `ndk` APIs that have not been compiled on this host (macOS). Syntax errors or API mismatches will only surface when the project is built for an Android target (`cargo build --target aarch64-linux-android`). A follow-up should validate the Android build on CI or a dev machine with the NDK installed.
- **Integration test missing:** `tests/encode_decode_android.rs` should be added once an Android test runner is available.
- **Decoder output pixel format:** MediaCodec decoder output format is not inspected. The decoded `VideoFrame.data` may be NV12, NV21, or a vendor-specific tiled format rather than I420. The renderer or downstream consumer must handle this.
- **Surface-texture path not implemented:** The task mentions configuring the encoder with a surface for zero-copy camera→encoder. This is out of scope for the byte-buffer MVP but will be needed for production battery life.
- **Error recovery:** If `AMediaCodec` enters the error state, the current implementation returns a `PlatformError`. A production path should recreate the codec session rather than failing permanently.
## Reviewer checklist (filled in by reviewer)
- [ ] Code matches PRD intent
- [ ] Verification output is real (re-run if suspicious)
- [ ] No backward-incompat surprises
- [ ] Tests cover the new behavior (non-Android stubs)
- [ ] Approved