31 Commits

Author SHA1 Message Date
Siavash Sameni
12020b019c fix(video): normalize VideoToolbox plane strides to tight I420
Some checks failed
Build Release Binaries / build-amd64 (push) Failing after 3m20s
Mirror to GitHub / mirror (push) Failing after 28s
Android-encoded H.264 decoded cleanly with ffmpeg but showed diagonal
green/magenta banding on macOS. Root cause: shiguredo_video_toolbox's
I420Frame exposes y/u/v planes as bytes_per_row * height, including
CoreVideo's stride padding. VideoToolboxDecoder concatenated those
slices verbatim, then downstream code indexed the buffer as tight I420,
producing per-row drift that wrapped one full row every 16 chroma rows
(32 luma rows) at 960x540.

Add i420_frame_to_tight() helper that copies each plane row-by-row at
width / chroma_width using the plane's actual stride. All three macOS
decoders (H.264, HEVC, AV1) now call it. On first decode each logs the
real plane dimensions and strides at target wzp_video::videotoolbox so
future stride bugs are diagnosable from logs.

Verified mathematically against the corrupted dump:
  band period = u_stride / (u_stride - chroma_width)
              = 512 / (512 - 480) = 16 chroma rows = 32 luma rows
which matches the measured spacing exactly. 640x360 was unaffected
because chroma_width 320 is already 64-aligned.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-26 15:22:40 +04:00
Siavash Sameni
3ea25a0656 fix(android): use MediaCodec input layout for video encode
Some checks failed
Mirror to GitHub / mirror (push) Failing after 29s
Build Release Binaries / build-amd64 (push) Failing after 4m1s
2026-05-26 11:35:24 +04:00
Siavash Sameni
112472609e fix(video): add frame metadata and Android encode diagnostics
Some checks failed
Mirror to GitHub / mirror (push) Failing after 41s
Build Release Binaries / build-amd64 (push) Failing after 4m7s
2026-05-26 11:28:17 +04:00
Siavash Sameni
9a7745978b feat(video): add codec and resolution controls
Some checks failed
Build Release Binaries / build-amd64 (push) Failing after 3m38s
Mirror to GitHub / mirror (push) Failing after 38s
2026-05-26 10:05:20 +04:00
Siavash Sameni
f85efb9576 fix(video): improve android stream smoothness
Some checks failed
Mirror to GitHub / mirror (push) Failing after 27s
Build Release Binaries / build-amd64 (push) Failing after 3m35s
2026-05-26 09:57:10 +04:00
Siavash Sameni
47baa1a765 fix(video): reassemble out-of-order fragments
Some checks failed
Mirror to GitHub / mirror (push) Failing after 27s
Build Release Binaries / build-amd64 (push) Failing after 3m13s
2026-05-26 06:16:53 +04:00
Siavash Sameni
d2046060b5 fix(video): request android sync frames via mediacodec
Some checks failed
Mirror to GitHub / mirror (push) Failing after 27s
Build Release Binaries / build-amd64 (push) Failing after 3m5s
2026-05-25 21:28:59 +04:00
Siavash Sameni
0b7bf1b385 fix(video): feed android h264 encoder nv12
Some checks failed
Mirror to GitHub / mirror (push) Failing after 30s
Build Release Binaries / build-amd64 (push) Failing after 3m16s
2026-05-25 21:20:01 +04:00
Siavash Sameni
e8f139588a chore(video): sample decoded frames periodically
Some checks failed
Mirror to GitHub / mirror (push) Failing after 26s
Build Release Binaries / build-amd64 (push) Failing after 3m30s
2026-05-25 21:14:32 +04:00
Siavash Sameni
fa812a17d9 fix(video): normalize mediacodec buffers
Some checks failed
Mirror to GitHub / mirror (push) Failing after 28s
Build Release Binaries / build-amd64 (push) Failing after 3m13s
2026-05-25 21:02:41 +04:00
Siavash Sameni
e1ca6ca6e6 fix(video): use relay-default stream for room video
Some checks failed
Mirror to GitHub / mirror (push) Failing after 31s
Build Release Binaries / build-amd64 (push) Has been cancelled
2026-05-25 20:39:25 +04:00
Siavash Sameni
06d28a9280 fix(video): preserve annex-b mediacodec output
Some checks failed
Mirror to GitHub / mirror (push) Failing after 31s
Build Release Binaries / build-amd64 (push) Failing after 3m35s
2026-05-25 20:20:22 +04:00
Siavash Sameni
d57ebe3d2c fix(video): force h264 and trace frame pipeline
Some checks failed
Build Release Binaries / build-amd64 (push) Failing after 3m32s
Mirror to GitHub / mirror (push) Failing after 28s
2026-05-25 20:03:11 +04:00
Siavash Sameni
06253fdeeb feat(video+desktop): camera capture, video UI, E2E AEAD wiring, test fixes
Blockers 4 & 5: browser getUserMedia → JPEG IPC → Rust I420 pipeline;
remote video strip renders decoded frames via canvas; EncryptingTransport
wraps QuinnTransport so WZP AEAD is applied to all media (C2 fix).

Test fixes: HandshakeResult.session destructuring across relay/client/crypto
integration tests; video_codecs field added to all CallOffer/CallAnswer
structs; wzp-video pipeline_roundtrip integration tests added.

PRD docs: five Kimi-ready specs for E2E encryption, Android NDK 0.9 migration,
quality upgrade flow, wire-format hardening, and clippy debt.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-25 15:30:26 +04:00
Siavash Sameni
5d05b021aa fix(wzp-video): gate shiguredo AV1 crates to macOS only; fix Linux relay build
- Cargo.toml: merge duplicate [target.macos.deps] sections; move
  shiguredo_dav1d/svt_av1/video_toolbox into single block
- lib.rs: dav1d + svt_av1 modules and re-exports guarded by
  cfg(target_os = "macos") instead of cfg(not(android))
- factory.rs: AV1 encoder/decoder paths split into macos (svt-av1/dav1d)
  and linux fallback (NotInitialized); update doc comments and tests
- build-linux-docker.sh: build only wzp-relay + wzp-web (drops
  wzp-client which pulled in shiguredo crates); fix Docker copy step;
  add --deploy flag + deploy_relay(); fix branch auto-detection
- build-tauri-android.sh: default to release build, arm64 only

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-25 06:33:35 +04:00
Siavash Sameni
15af58a95d fix(wzp-video): fix ndk 0.9 MediaCodec API + missing constants for Android build
- Replace buffer.index() with buffer.buffer_mut()/buffer.buffer() (ndk 0.9 RAII API)
- Replace queue_input_buffer_by_index/release_output_buffer_by_index with
  queue_input_buffer/release_output_buffer taking buffer objects
- Fix MaybeUninit<u8> copy using .write() instead of copy_from_slice
- Add BITRATE_MODE_CBR and AMEDIACODEC_BUFFER_FLAG_KEY_FRAME local constants
  (removes ndk_sys dependency for these values)
- Add unsafe impl Send for all six MediaCodec wrapper structs
- Pin @tauri-apps/api to ^2.11 to match Cargo.lock tauri 2.11.1

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-25 06:05:58 +04:00
Siavash Sameni
f3e3ee5ed0 fix(wzp-video): cfg-gate dav1d + svt-av1 off Android target
shiguredo_dav1d and shiguredo_svt_av1 build scripts panic with
'unsupported target: os=android, arch=aarch64'. The AV1 SW fallback
is only needed on macOS / Linux desktop — Android uses MediaCodec
for AV1 anyway.

- Cargo.toml: AV1 SW deps moved under cfg(not(target_os = "android"))
- lib.rs: cfg-gate the dav1d and svt_av1 modules and re-exports
- factory.rs: on Android, Av1Main paths return NotInitialized when
  HW MediaCodec is also unavailable (only path on Android)
- factory tests: assert NotInitialized on Android, Ok elsewhere

Unblocks T4.3.1.1 (Android target-compile of wzp-video / mediacodec).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 19:58:37 +04:00
Siavash Sameni
086d0a4845 T6.1.2: Wire AV1 into call engine (factory + step tables)
- New: factory.rs — create_video_encoder/decoder dispatch by CodecId with
  platform-aware HW→SW fallback. AV1 encoder: SvtAv1Encoder (universal SW).
  AV1 decoder: VideoToolboxAv1Decoder (macOS M3+) → MediaCodecAv1Decoder
  (Android) → Dav1dDecoder (all platforms fallback).
- controller.rs: codec-specific step tables (H.264/H.265/AV1). AV1 ~30%
  lower thresholds than H.264; H.265 ~20% lower. VideoQualityController
  gains codec field with with_codec()/set_codec()/codec() accessors.
- lib.rs: export factory fns and VideoToolboxAv1Decoder
- wzp-client/Cargo.toml: add wzp-video dependency
- 11 new tests (7 factory + 4 controller); 77→88 wzp-video tests; fmt +
  clippy clean; all workspace tests pass
2026-05-12 19:05:45 +04:00
Siavash Sameni
9334aa5ccd T6.1: AV1 encoder/decoder with HW probe + SVT-AV1 SW fallback
- New: av1_obu.rs — OBU framer, depacketizer, keyframe detection, LEB128 helpers
- New: dav1d.rs — SW AV1 decoder wrapper (shiguredo_dav1d)
- New: svt_av1.rs — SW AV1 encoder wrapper (shiguredo_svt_av1)
- Add CodecId::Av1Main = 12 with match-arm fixes in downstream crates
- Add VideoToolboxAv1Decoder for macOS M3+ HW decode
- Add MediaCodecAv1Encoder/Decoder for Android (video/av01)
- Add extract_sequence_header_obu() helper for AV1 decoder CSD
- Add 10-frame encode-decode roundtrip test (svt_av1 + dav1d)
- Fix clippy unused import in dav1d.rs
- 15 tests; all workspace tests pass; cargo fmt clean
2026-05-12 18:44:44 +04:00
Siavash Sameni
283edd38eb clippy: fix very_complex_type in wzp-video (HevcParameterSets alias) 2026-05-12 15:40:19 +04:00
Siavash Sameni
fdfaed5390 fmt: cargo fmt --all 2026-05-12 15:40:02 +04:00
Siavash Sameni
2f1a9f74d5 T5.5: 3-layer simulcast at sender — SimulcastEncoder + tick_simulcast() + 10 tests 2026-05-12 14:56:48 +04:00
Siavash Sameni
b197651557 T5.4: H.265 encoder/decoder wrappers — VideoToolbox + MediaCodec, CodecId::H265Main 2026-05-12 14:50:20 +04:00
Siavash Sameni
c48cb6fbcb T5.3: EncoderMode::SlideFallback — SD-floor detection + VideoEncoder::set_mode() trait hook 2026-05-12 12:40:53 +04:00
Siavash Sameni
2e0bdc5904 T5.2: VideoQualityController with per-mode allocation gates + 8-step target table 2026-05-12 12:34:32 +04:00
Siavash Sameni
397f9d2141 T4.3.1: MediaCodec AMediaCodec wiring via ndk crate (Android); fix wzp-android build on non-Android 2026-05-12 10:03:43 +04:00
Siavash Sameni
410c2a4335 T4.2.1: Real VideoToolbox VTCompressionSession / VTDecompressionSession wiring (macOS) 2026-05-12 09:51:34 +04:00
Siavash Sameni
81042ac190 T4.4: SignalMessage::Nack + PictureLossIndication; NACK sender/receiver state machines 2026-05-12 09:25:29 +04:00
Siavash Sameni
e177e63843 T4.3: MediaCodec H.264 encoder/decoder stub (Android) 2026-05-12 09:15:06 +04:00
Siavash Sameni
3356ba94c6 T4.2: VideoToolbox H.264 encoder/decoder traits (macOS, MVP) 2026-05-12 09:09:57 +04:00
Siavash Sameni
490d2d31c6 T4.1: wzp-video crate scaffold + H.264 NAL framer + depacketizer 2026-05-12 07:22:54 +04:00