Commit Graph

293 Commits

Author SHA1 Message Date
Siavash Sameni
06253fdeeb feat(video+desktop): camera capture, video UI, E2E AEAD wiring, test fixes
Blockers 4 & 5: browser getUserMedia → JPEG IPC → Rust I420 pipeline;
remote video strip renders decoded frames via canvas; EncryptingTransport
wraps QuinnTransport so WZP AEAD is applied to all media (C2 fix).

Test fixes: HandshakeResult.session destructuring across relay/client/crypto
integration tests; video_codecs field added to all CallOffer/CallAnswer
structs; wzp-video pipeline_roundtrip integration tests added.

PRD docs: five Kimi-ready specs for E2E encryption, Android NDK 0.9 migration,
quality upgrade flow, wire-format hardening, and clippy debt.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-25 15:30:26 +04:00
Siavash Sameni
ca987d547c fix(android): return -6 on Oboe start timeout; fix error toast; add bug report
- oboe_bridge.cpp: return -6 (instead of silent 0) when streams do not
  reach Started within the 2s poll deadline; also clean up streams on
  that path so a retry can succeed
- main.ts: shared connectWithTimeout() so room-join and direct-call
  auto-connect both get the 15s JS timeout; shared errorMessage() so
  Tauri error objects don't show as [object Object] in toasts
- docs/bugs/001-android-join-voice-hang.md: comprehensive bug report
  with root cause chain, evidence, return code table, and next steps

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-25 07:31:55 +04:00
Siavash Sameni
b0a3b1f18e fix: 10s timeout on handshake CallAnswer; button stays visible during connect
- handshake.rs: add 10s timeout on recv_signal() waiting for CallAnswer —
  previously hung forever if relay didn't respond, making join button
  disappear with no feedback
- main.ts: keep join button visible + show "Connecting…" state instead of
  hiding it before the await; button restores correctly on error

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-25 06:59:57 +04:00
Siavash Sameni
5d05b021aa fix(wzp-video): gate shiguredo AV1 crates to macOS only; fix Linux relay build
- Cargo.toml: merge duplicate [target.macos.deps] sections; move
  shiguredo_dav1d/svt_av1/video_toolbox into single block
- lib.rs: dav1d + svt_av1 modules and re-exports guarded by
  cfg(target_os = "macos") instead of cfg(not(android))
- factory.rs: AV1 encoder/decoder paths split into macos (svt-av1/dav1d)
  and linux fallback (NotInitialized); update doc comments and tests
- build-linux-docker.sh: build only wzp-relay + wzp-web (drops
  wzp-client which pulled in shiguredo crates); fix Docker copy step;
  add --deploy flag + deploy_relay(); fix branch auto-detection
- build-tauri-android.sh: default to release build, arm64 only

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-25 06:33:35 +04:00
Siavash Sameni
4ac62d99e0 fix(audit): M1 — add version: u8 to all SignalMessage variants
Convert Hold/Unhold/Mute/Unmute/TransferAck from unit variants to struct
variants with `version: u8` (serde default = 2). Every SignalMessage
variant now carries a version field, enabling future semantic versioning
and clean rejection of deprecated variants during federation routing.

305 tests passing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-25 06:27:23 +04:00
Siavash Sameni
52a6f5e048 fix(audit): address C2, C3, M4, M5 from 2026-05-25 audit
C2: Add EncryptingTransport wrapper — all media I/O now goes through
ChaChaSession encrypt/decrypt before hitting the QUIC datagram path.
cli.rs run_live/run_silence/run_file_mode accept Arc<dyn MediaTransport>
and receive a wrapped transport after the handshake.

C3: Wire VideoScorer::observe() into both plain and trunked forwarding
loops in room.rs. Packets from participants with Abusive verdict are
dropped before forwarding. last_bwe_kbps tracked from quality reports.

M4: Widen FEC repair symbol index from u8 to u16 throughout
(FecEncoder::generate_repair, FecDecoder::add_symbol, all call sites in
call.rs, bench.rs, pipeline.rs, wzp-android). Eliminates theoretical
wrapping when num_source + repair_count > 255.

M5: Track last_encrypt_timestamp in ChaChaSession. debug_assert in
encrypt() that timestamp is non-decreasing across calls (including post-
rekey). complete_rekey() explicitly preserves last_encrypt_timestamp to
prevent accidental timestamp reset regressions.

583 tests passing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-25 06:20:05 +04:00
Siavash Sameni
15af58a95d fix(wzp-video): fix ndk 0.9 MediaCodec API + missing constants for Android build
- Replace buffer.index() with buffer.buffer_mut()/buffer.buffer() (ndk 0.9 RAII API)
- Replace queue_input_buffer_by_index/release_output_buffer_by_index with
  queue_input_buffer/release_output_buffer taking buffer objects
- Fix MaybeUninit<u8> copy using .write() instead of copy_from_slice
- Add BITRATE_MODE_CBR and AMEDIACODEC_BUFFER_FLAG_KEY_FRAME local constants
  (removes ndk_sys dependency for these values)
- Add unsafe impl Send for all six MediaCodec wrapper structs
- Pin @tauri-apps/api to ^2.11 to match Cargo.lock tauri 2.11.1

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-25 06:05:58 +04:00
Siavash Sameni
12b0d9738f fix(wzp-crypto): derive AEAD nonces from MediaHeader.seq, not recv_seq
The previous scheme built ChaCha20-Poly1305 nonces from an internal
recv_seq counter that incremented once per decrypt() call. Under
in-order delivery recv_seq stayed in sync with the sender's send_seq,
but any out-of-order or lost packet caused them to diverge permanently —
every subsequent packet then used the wrong nonce and AEAD decryption
failed for the rest of the session.

Fix: parse the MediaHeader at the top of both encrypt() and decrypt()
and use header.seq as the nonce input. Both sides now derive the nonce
from the same wire field, surviving reordering by construction.

send_seq / recv_seq are kept as pure packet counters for the rekey
interval trigger; they no longer affect nonce derivation.

All tests updated to pass valid v2 MediaHeader bytes instead of raw
byte literals (the new code requires a parseable header for nonce
derivation). New test decrypt_survives_out_of_order_delivery encrypts
5 packets and delivers them out of order (indices 0,2,1,4,3); this
test would have failed under the old counter-based scheme.

Fixes audit finding C1 from AUDIT-2026-05-25.md.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-25 06:00:01 +04:00
Siavash Sameni
f3e3ee5ed0 fix(wzp-video): cfg-gate dav1d + svt-av1 off Android target
shiguredo_dav1d and shiguredo_svt_av1 build scripts panic with
'unsupported target: os=android, arch=aarch64'. The AV1 SW fallback
is only needed on macOS / Linux desktop — Android uses MediaCodec
for AV1 anyway.

- Cargo.toml: AV1 SW deps moved under cfg(not(target_os = "android"))
- lib.rs: cfg-gate the dav1d and svt_av1 modules and re-exports
- factory.rs: on Android, Av1Main paths return NotInitialized when
  HW MediaCodec is also unavailable (only path on Android)
- factory tests: assert NotInitialized on Android, Ok elsewhere

Unblocks T4.3.1.1 (Android target-compile of wzp-video / mediacodec).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-12 19:58:37 +04:00
Siavash Sameni
086d0a4845 T6.1.2: Wire AV1 into call engine (factory + step tables)
- New: factory.rs — create_video_encoder/decoder dispatch by CodecId with
  platform-aware HW→SW fallback. AV1 encoder: SvtAv1Encoder (universal SW).
  AV1 decoder: VideoToolboxAv1Decoder (macOS M3+) → MediaCodecAv1Decoder
  (Android) → Dav1dDecoder (all platforms fallback).
- controller.rs: codec-specific step tables (H.264/H.265/AV1). AV1 ~30%
  lower thresholds than H.264; H.265 ~20% lower. VideoQualityController
  gains codec field with with_codec()/set_codec()/codec() accessors.
- lib.rs: export factory fns and VideoToolboxAv1Decoder
- wzp-client/Cargo.toml: add wzp-video dependency
- 11 new tests (7 factory + 4 controller); 77→88 wzp-video tests; fmt +
  clippy clean; all workspace tests pass
2026-05-12 19:05:45 +04:00
Siavash Sameni
9334aa5ccd T6.1: AV1 encoder/decoder with HW probe + SVT-AV1 SW fallback
- New: av1_obu.rs — OBU framer, depacketizer, keyframe detection, LEB128 helpers
- New: dav1d.rs — SW AV1 decoder wrapper (shiguredo_dav1d)
- New: svt_av1.rs — SW AV1 encoder wrapper (shiguredo_svt_av1)
- Add CodecId::Av1Main = 12 with match-arm fixes in downstream crates
- Add VideoToolboxAv1Decoder for macOS M3+ HW decode
- Add MediaCodecAv1Encoder/Decoder for Android (video/av01)
- Add extract_sequence_header_obu() helper for AV1 decoder CSD
- Add 10-frame encode-decode roundtrip test (svt_av1 + dav1d)
- Fix clippy unused import in dav1d.rs
- 15 tests; all workspace tests pass; cargo fmt clean
2026-05-12 18:44:44 +04:00
Siavash Sameni
f16d650721 T6.2: Tier F video scorer — keyframe periodicity, I/P ratio, BWE responsiveness + 10 tests 2026-05-12 17:42:39 +04:00
Siavash Sameni
517d0ebfe0 T5.7.1: Unify Verdict enum into wzp_relay::verdict, drop RepeatAbusive variant 2026-05-12 16:49:04 +04:00
Siavash Sameni
ffded2a913 clippy: fix wzp-relay lint issues (empty doc, unused var, TokenExhausted, Default, dead field) 2026-05-12 15:40:55 +04:00
Siavash Sameni
283edd38eb clippy: fix very_complex_type in wzp-video (HevcParameterSets alias) 2026-05-12 15:40:19 +04:00
Siavash Sameni
fdfaed5390 fmt: cargo fmt --all 2026-05-12 15:40:02 +04:00
Siavash Sameni
dbbab0decf T5.8: Tier G response policy — Verdict enum + ResponsePolicy + typed Hangup::PolicyViolation + 9 tests 2026-05-12 15:13:20 +04:00
Siavash Sameni
5fda5ecc52 T5.7: Tier F audio scorer — IAT CoV + silence fraction + bitrate + Q-flag + bimodality + 11 tests 2026-05-12 15:09:28 +04:00
Siavash Sameni
2bbb664df4 T5.6: Per-receiver layer selection at SFU — ReceiverState + hysteresis + forwarding filter 2026-05-12 15:05:32 +04:00
Siavash Sameni
2f1a9f74d5 T5.5: 3-layer simulcast at sender — SimulcastEncoder + tick_simulcast() + 10 tests 2026-05-12 14:56:48 +04:00
Siavash Sameni
b197651557 T5.4: H.265 encoder/decoder wrappers — VideoToolbox + MediaCodec, CodecId::H265Main 2026-05-12 14:50:20 +04:00
Siavash Sameni
e34c40dc0f T5.1.1: PriorityMode default = AudioFirst, QualityProfile backward-compat JSON, SetPriorityMode roundtrip 2026-05-12 14:50:06 +04:00
Siavash Sameni
c48cb6fbcb T5.3: EncoderMode::SlideFallback — SD-floor detection + VideoEncoder::set_mode() trait hook 2026-05-12 12:40:53 +04:00
Siavash Sameni
2e0bdc5904 T5.2: VideoQualityController with per-mode allocation gates + 8-step target table 2026-05-12 12:34:32 +04:00
Siavash Sameni
276ecc660e T5.1: PriorityMode enum + SetPriorityMode signal; extend QualityProfile with video fields 2026-05-12 12:21:40 +04:00
Siavash Sameni
001d94f9ae T4.7 rework: make should_forward_pli take now: Instant + 6 unit tests
- Refactor should_forward_pli(room, stream_id) -> should_forward_pli(room, stream_id, now: Instant)
  so the 200 ms dedup window is deterministically testable.
- Update the one caller in run_participant_signals to pass Instant::now().
- Add 6 PLI unit tests covering:
  * first PLI forwards
  * duplicate within 200 ms suppressed
  * after 200 ms forwards again
  * different streams independent
  * different rooms independent
  * no stream owner returns None

Addresses reviewer CR on T4.7 (line drawn at T4.6 — stateful relay features must
have state-transition tests).

wzp-relay tests: 93 -> 99 pass.
2026-05-12 11:39:35 +04:00
Siavash Sameni
36b0421d68 T4.7: PLI suppression at SFU — 200 ms dedup window per (room, stream_id) 2026-05-12 11:25:25 +04:00
Siavash Sameni
828fbea2ea T4.6: SFU keyframe cache — per-(room,sender,stream) I-frame replay on join 2026-05-12 10:54:04 +04:00
Siavash Sameni
cc5aef2534 T4.5: I-frame FEC ratio boost — keyframe-aware repair ratio in RaptorQFecEncoder
- Add add_source_symbol_with_keyframe() default method to FecEncoder trait
- RaptorQFecEncoder tracks has_keyframe per block, uses keyframe_ratio
  when generating repair symbols for keyframe blocks
- AdaptiveFec gains keyframe_repair_ratio (default 0.5) and wires it
  through build_encoder()
- 3 new tests: keyframe boost, non-keyframe nominal ratio, finalize clears flag
- Update status board T4.5 -> Pending Review
2026-05-12 10:36:18 +04:00
Siavash Sameni
397f9d2141 T4.3.1: MediaCodec AMediaCodec wiring via ndk crate (Android); fix wzp-android build on non-Android 2026-05-12 10:03:43 +04:00
Siavash Sameni
410c2a4335 T4.2.1: Real VideoToolbox VTCompressionSession / VTDecompressionSession wiring (macOS) 2026-05-12 09:51:34 +04:00
Siavash Sameni
81042ac190 T4.4: SignalMessage::Nack + PictureLossIndication; NACK sender/receiver state machines 2026-05-12 09:25:29 +04:00
Siavash Sameni
e177e63843 T4.3: MediaCodec H.264 encoder/decoder stub (Android) 2026-05-12 09:15:06 +04:00
Siavash Sameni
3356ba94c6 T4.2: VideoToolbox H.264 encoder/decoder traits (macOS, MVP) 2026-05-12 09:09:57 +04:00
Siavash Sameni
490d2d31c6 T4.1: wzp-video crate scaffold + H.264 NAL framer + depacketizer 2026-05-12 07:22:54 +04:00
Siavash Sameni
f1b86e0fed T3.5: Tier E per-session token bucket 2026-05-12 06:45:56 +04:00
Siavash Sameni
017c371611 T3.4: Tier D per-codec payload size sanity 2026-05-12 06:24:40 +04:00
Siavash Sameni
e73f8a7150 T3.3: SignalMessage version field 2026-05-12 06:11:59 +04:00
Siavash Sameni
1b4f7b0772 T3.2: Document timestamp_ms monotonic across rekey + test 2026-05-11 21:19:03 +04:00
Siavash Sameni
f3398adb95 T3.1: RoomManager concurrency — Arc<RwLock<Room>> per room 2026-05-11 21:12:04 +04:00
Siavash Sameni
54c1a35186 T2.3-T2.6: BWE guard, relay conformance Tier A/B/C, Prometheus metrics 2026-05-11 20:50:22 +04:00
Siavash Sameni
3de56cf1f9 T2.2: BandwidthEstimator with cwnd/REMB target_send_bps 2026-05-11 19:16:25 +04:00
Siavash Sameni
fe1f9484bd T2.1: Add SignalMessage::TransportFeedback 2026-05-11 19:06:45 +04:00
Siavash Sameni
0ef1f574ff T1.8: Per-stream anti-replay window with configurable size 2026-05-11 16:56:09 +04:00
Siavash Sameni
b1c5837495 T1.7: Move QualityReport trailer inside AEAD payload 2026-05-11 16:42:25 +04:00
Siavash Sameni
6f81487778 T1.6: Protocol version negotiation in handshake 2026-05-11 15:53:04 +04:00
Siavash Sameni
30d26fc7f6 T1.5.1: Remove unwrap() from encode_compact 2026-05-11 12:57:35 +04:00
Siavash Sameni
c93d302656 T1.5: Migrate emit/parse sites to v2 wire format 2026-05-11 12:37:32 +04:00
Siavash Sameni
9680b6ff34 T1.4.1: Add rustdoc on MiniHeaderV2 and MiniFrameContextV2 public items 2026-05-11 11:38:04 +04:00
Siavash Sameni
6385b93391 T1.2.1: Add rustdoc on MediaType variants and methods 2026-05-11 11:33:58 +04:00