Commit Graph

99 Commits

Author SHA1 Message Date
Siavash Sameni
06253fdeeb feat(video+desktop): camera capture, video UI, E2E AEAD wiring, test fixes
Blockers 4 & 5: browser getUserMedia → JPEG IPC → Rust I420 pipeline;
remote video strip renders decoded frames via canvas; EncryptingTransport
wraps QuinnTransport so WZP AEAD is applied to all media (C2 fix).

Test fixes: HandshakeResult.session destructuring across relay/client/crypto
integration tests; video_codecs field added to all CallOffer/CallAnswer
structs; wzp-video pipeline_roundtrip integration tests added.

PRD docs: five Kimi-ready specs for E2E encryption, Android NDK 0.9 migration,
quality upgrade flow, wire-format hardening, and clippy debt.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-25 15:30:26 +04:00
Siavash Sameni
2aa6582585 fix(android): call-debug instrumentation for audio startup path
Add emit_call_debug events at every step of the Android connect/audio
path so failures are visible in the Settings debug log without needing
adb logcat:

- connect:handshake_start/done/failed (with timing)
- connect:android_audio_preflight (wzp_native loaded + RECORD_AUDIO
  permission check via new has_record_audio_permission() JNI helper)
- connect:audio_stop_start/done
- connect:audio_mode_start/done/failed
- connect:audio_start_start/failed/panic/done (with oboe error code)
- connect:reuse_endpoint (endpoint reuse diagnostic)

Also adds has_record_audio_permission() to android_audio.rs — used in
the preflight event to confirm the OS has granted mic access before
wzp_oboe_start is called.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-25 07:38:38 +04:00
Siavash Sameni
ca987d547c fix(android): return -6 on Oboe start timeout; fix error toast; add bug report
- oboe_bridge.cpp: return -6 (instead of silent 0) when streams do not
  reach Started within the 2s poll deadline; also clean up streams on
  that path so a retry can succeed
- main.ts: shared connectWithTimeout() so room-join and direct-call
  auto-connect both get the 15s JS timeout; shared errorMessage() so
  Tauri error objects don't show as [object Object] in toasts
- docs/bugs/001-android-join-voice-hang.md: comprehensive bug report
  with root cause chain, evidence, return code table, and next steps

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-25 07:31:55 +04:00
Siavash Sameni
ed8a7ae5aa docs: protocol audit 2026-05-25, update architecture + Obsidian vault
Audit:
- docs/AUDIT-2026-05-25.md: full protocol audit covering 8 findings
  (4 critical, 2 high, 5 medium, 4 low) with code references and fix
  effort estimates
- vault/Audit/Tasks.md: Obsidian Tasks plugin file tracking all audit
  items with priorities, due dates, and per-step checklists

Architecture docs updated for Wire format v2 and Wave 5/6 features:
- ARCHITECTURE.md: adds wzp-video to dependency graph and project
  structure; wire format updated to v2 (16B header, 5B MiniHeader);
  relay concurrency section corrected (DashMap+RwLock is current, not
  a future optimization); test count 571→702; Android note
- PROGRESS.md: Wave 5 and Wave 6 sections appended; test count 372→702;
  current status and open blockers as of 2026-05-25
- ROAD-TO-VIDEO.md: implementation status table inserted (/🟡/🔴/🔲
  per phase); 6-step critical path to first video call
- WZP-SPEC.md: MediaHeader updated to v2 (16B byte-aligned); MiniHeader
  updated to 5B with seq_delta; codec IDs 9-12 added (H.264/H.265/AV1);
  version negotiation section added

Obsidian vault (vault/):
- 114 files across Architecture/, PRDs/, Reports/, Android/,
  Reference/, Audit/ with YAML frontmatter
- 00 - Home.md index note with wiki links
- .obsidian/app.json config

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-25 06:00:17 +04:00
Siavash Sameni
1e729e4b1d T6.3: Design exploration for federated reputation gossip
Add docs/PRD/PRD-relay-federation-gossip.md comparing 3 approaches:
1. Push gossip — relay broadcasts RepeatAbusive verdicts to peers
2. Pull oracle — peers query a reputation oracle periodically
3. Ban-list distribution — admin signs and pushes authoritative list

For each: wire format, Sybil resistance, convergence, storage,
partition tolerance, failure modes. Open questions block implementation
(trust model, privacy leakage, key infrastructure). Move T6.3 to Blocked
pending reviewer design call.
2026-05-12 19:13:31 +04:00
Siavash Sameni
086d0a4845 T6.1.2: Wire AV1 into call engine (factory + step tables)
- New: factory.rs — create_video_encoder/decoder dispatch by CodecId with
  platform-aware HW→SW fallback. AV1 encoder: SvtAv1Encoder (universal SW).
  AV1 decoder: VideoToolboxAv1Decoder (macOS M3+) → MediaCodecAv1Decoder
  (Android) → Dav1dDecoder (all platforms fallback).
- controller.rs: codec-specific step tables (H.264/H.265/AV1). AV1 ~30%
  lower thresholds than H.264; H.265 ~20% lower. VideoQualityController
  gains codec field with with_codec()/set_codec()/codec() accessors.
- lib.rs: export factory fns and VideoToolboxAv1Decoder
- wzp-client/Cargo.toml: add wzp-video dependency
- 11 new tests (7 factory + 4 controller); 77→88 wzp-video tests; fmt +
  clippy clean; all workspace tests pass
2026-05-12 19:05:45 +04:00
Siavash Sameni
9334aa5ccd T6.1: AV1 encoder/decoder with HW probe + SVT-AV1 SW fallback
- New: av1_obu.rs — OBU framer, depacketizer, keyframe detection, LEB128 helpers
- New: dav1d.rs — SW AV1 decoder wrapper (shiguredo_dav1d)
- New: svt_av1.rs — SW AV1 encoder wrapper (shiguredo_svt_av1)
- Add CodecId::Av1Main = 12 with match-arm fixes in downstream crates
- Add VideoToolboxAv1Decoder for macOS M3+ HW decode
- Add MediaCodecAv1Encoder/Decoder for Android (video/av01)
- Add extract_sequence_header_obu() helper for AV1 decoder CSD
- Add 10-frame encode-decode roundtrip test (svt_av1 + dav1d)
- Fix clippy unused import in dav1d.rs
- 15 tests; all workspace tests pass; cargo fmt clean
2026-05-12 18:44:44 +04:00
Siavash Sameni
553c8a4ce1 T6.1 plan: expand skeleton with files/steps/verify/done-when for AV1 encoder/decoder 2026-05-12 18:08:27 +04:00
Siavash Sameni
8d8dddbd35 docs: add T6.2 report and update status board to Pending Review 2026-05-12 17:45:04 +04:00
Siavash Sameni
31f2fdef1e T6.2 plan: expand skeleton with files/steps/verify/done-when for video scorer 2026-05-12 17:14:25 +04:00
Siavash Sameni
fc9908cd4c docs: fix commit SHA in T5.7.1 report 2026-05-12 16:49:16 +04:00
Siavash Sameni
517d0ebfe0 T5.7.1: Unify Verdict enum into wzp_relay::verdict, drop RepeatAbusive variant 2026-05-12 16:49:04 +04:00
Siavash Sameni
cf4940417e docs: add T5.1.1–T5.8 reports and update status board to Pending Review 2026-05-12 15:41:28 +04:00
Siavash Sameni
9c41d1acdd T5.3 status: Approved (reviewer update) 2026-05-12 14:50:12 +04:00
Siavash Sameni
c48cb6fbcb T5.3: EncoderMode::SlideFallback — SD-floor detection + VideoEncoder::set_mode() trait hook 2026-05-12 12:40:53 +04:00
Siavash Sameni
2e0bdc5904 T5.2: VideoQualityController with per-mode allocation gates + 8-step target table 2026-05-12 12:34:32 +04:00
Siavash Sameni
276ecc660e T5.1: PriorityMode enum + SetPriorityMode signal; extend QualityProfile with video fields 2026-05-12 12:21:40 +04:00
Siavash Sameni
36b0421d68 T4.7: PLI suppression at SFU — 200 ms dedup window per (room, stream_id) 2026-05-12 11:25:25 +04:00
Siavash Sameni
828fbea2ea T4.6: SFU keyframe cache — per-(room,sender,stream) I-frame replay on join 2026-05-12 10:54:04 +04:00
Siavash Sameni
cc5aef2534 T4.5: I-frame FEC ratio boost — keyframe-aware repair ratio in RaptorQFecEncoder
- Add add_source_symbol_with_keyframe() default method to FecEncoder trait
- RaptorQFecEncoder tracks has_keyframe per block, uses keyframe_ratio
  when generating repair symbols for keyframe blocks
- AdaptiveFec gains keyframe_repair_ratio (default 0.5) and wires it
  through build_encoder()
- 3 new tests: keyframe boost, non-keyframe nominal ratio, finalize clears flag
- Update status board T4.5 -> Pending Review
2026-05-12 10:36:18 +04:00
Siavash Sameni
397f9d2141 T4.3.1: MediaCodec AMediaCodec wiring via ndk crate (Android); fix wzp-android build on non-Android 2026-05-12 10:03:43 +04:00
Siavash Sameni
410c2a4335 T4.2.1: Real VideoToolbox VTCompressionSession / VTDecompressionSession wiring (macOS) 2026-05-12 09:51:34 +04:00
Siavash Sameni
81042ac190 T4.4: SignalMessage::Nack + PictureLossIndication; NACK sender/receiver state machines 2026-05-12 09:25:29 +04:00
Siavash Sameni
e177e63843 T4.3: MediaCodec H.264 encoder/decoder stub (Android) 2026-05-12 09:15:06 +04:00
Siavash Sameni
1f7d130de9 fix: T4.2 status board → Pending Review 2026-05-12 09:10:50 +04:00
Siavash Sameni
3356ba94c6 T4.2: VideoToolbox H.264 encoder/decoder traits (macOS, MVP) 2026-05-12 09:09:57 +04:00
Siavash Sameni
bb153a331d fix: T4.1 status board → Pending Review 2026-05-12 07:23:15 +04:00
Siavash Sameni
490d2d31c6 T4.1: wzp-video crate scaffold + H.264 NAL framer + depacketizer 2026-05-12 07:22:54 +04:00
Siavash Sameni
db69f7e9d1 fix: T3.5 status board → Pending Review 2026-05-12 06:46:28 +04:00
Siavash Sameni
f1b86e0fed T3.5: Tier E per-session token bucket 2026-05-12 06:45:56 +04:00
Siavash Sameni
8454835c18 fix: T3.4 status board → Pending Review 2026-05-12 06:25:17 +04:00
Siavash Sameni
017c371611 T3.4: Tier D per-codec payload size sanity 2026-05-12 06:24:40 +04:00
Siavash Sameni
3220bd6151 fix: T3.2 status board — Committed → Pending Review 2026-05-12 06:14:07 +04:00
Siavash Sameni
e73f8a7150 T3.3: SignalMessage version field 2026-05-12 06:11:59 +04:00
Siavash Sameni
1b4f7b0772 T3.2: Document timestamp_ms monotonic across rekey + test 2026-05-11 21:19:03 +04:00
Siavash Sameni
f3398adb95 T3.1: RoomManager concurrency — Arc<RwLock<Room>> per room 2026-05-11 21:12:04 +04:00
Siavash Sameni
54c1a35186 T2.3-T2.6: BWE guard, relay conformance Tier A/B/C, Prometheus metrics 2026-05-11 20:50:22 +04:00
Siavash Sameni
3de56cf1f9 T2.2: BandwidthEstimator with cwnd/REMB target_send_bps 2026-05-11 19:16:25 +04:00
Siavash Sameni
fe1f9484bd T2.1: Add SignalMessage::TransportFeedback 2026-05-11 19:06:45 +04:00
Siavash Sameni
0ef1f574ff T1.8: Per-stream anti-replay window with configurable size 2026-05-11 16:56:09 +04:00
Siavash Sameni
b1c5837495 T1.7: Move QualityReport trailer inside AEAD payload 2026-05-11 16:42:25 +04:00
Siavash Sameni
6f81487778 T1.6: Protocol version negotiation in handshake 2026-05-11 15:53:04 +04:00
Siavash Sameni
5cdb50160a T1.5.2: Workspace clippy hygiene + document pre-existing debt 2026-05-11 12:59:14 +04:00
Siavash Sameni
30d26fc7f6 T1.5.1: Remove unwrap() from encode_compact 2026-05-11 12:57:35 +04:00
Siavash Sameni
c93d302656 T1.5: Migrate emit/parse sites to v2 wire format 2026-05-11 12:37:32 +04:00
Siavash Sameni
9680b6ff34 T1.4.1: Add rustdoc on MiniHeaderV2 and MiniFrameContextV2 public items 2026-05-11 11:38:04 +04:00
Siavash Sameni
6b15b8f97c T1.1.2: Address review — fix remaining stale 272 audio tests references 2026-05-11 11:35:15 +04:00
Siavash Sameni
6385b93391 T1.2.1: Add rustdoc on MediaType variants and methods 2026-05-11 11:33:58 +04:00
Siavash Sameni
6eb94f079d T1.1.1: Address review — add rustdoc on impl MediaHeaderV2 constants and methods 2026-05-11 11:32:00 +04:00
Siavash Sameni
5580b794a4 T1.1.2: Refresh stale test-count figures in docs 2026-05-11 11:29:18 +04:00