a7b2f850f11b78e7d6b0d605ce5abb89f340c5ef
177 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Siavash Sameni
|
7515417202 |
feat(telemetry): Phase 4 — LossRecoveryUpdate protocol + relay metrics + DebugReporter
Phase 4 lays the telemetry foundation for distinguishing DRED recoveries
from classical PLC in production: a new SignalMessage variant, two new
per-session Prometheus counters on the relay side, and a highlighted
loss-recovery section in the Android DebugReporter.
The periodic emitter (client → relay) and Grafana panel are deferred to
Phase 4b — this commit ships the protocol surface, the relay sink, and
the immediate user-visible debug output. Once 4b lands the full path
(emitter → relay → Prometheus → Grafana), the metrics here will
automatically start receiving data.
Scope decision — why not extend QualityReport instead:
The existing wire-format QualityReport is a fixed 4-byte media packet
trailer. Adding counter fields to it would shift the binary layout and
break backward compatibility (old receivers would parse the last 4
bytes of the extended trailer as QR, corrupting audio). Using a
new SignalMessage variant on the reliable QUIC signal stream sidesteps
the wire-format problem entirely — serde JSON enums tolerate unknown
variants gracefully on old receivers, and the signal channel is the
right layer for periodic telemetry aggregates.
Changes:
wzp-proto/src/packet.rs:
- New SignalMessage::LossRecoveryUpdate variant carrying:
* dred_reconstructions: u64 (monotonic since call start)
* classical_plc_invocations: u64 (monotonic)
* frames_decoded: u64 (for rate calculation)
- All three fields tagged #[serde(default)] for forward compat.
wzp-client/src/featherchat.rs:
- Added a match arm so signal_to_call_type() handles the new
variant (treat as Offer for featherChat bridging purposes).
wzp-relay/src/metrics.rs:
- Two new IntCounterVec metrics on the relay, labeled by session_id:
* wzp_relay_session_dred_reconstructions_total
* wzp_relay_session_classical_plc_total
- New method update_session_loss_recovery(session_id, dred, plc)
applies monotonic deltas: if the incoming totals exceed the
current counter, the difference is inc_by'd. If the incoming
totals are LOWER (client restart or counter reset), the
Prometheus counter holds steady until the client catches up.
This matches the existing update_session_buffer delta pattern.
- remove_session_metrics() now cleans up the two new labels.
- New test session_loss_recovery_monotonic_delta exercises:
* initial population (10 DRED, 2 PLC)
* forward advance (25, 5 → delta +15, +3)
* lower values ignored (client reset → counters unchanged)
* client catches up (30, 8 → advances to new max)
- Existing session_metrics_cleanup test extended to cover the
new counters.
android/app/src/main/java/com/wzp/debug/DebugReporter.kt:
- Phase 4 users — and incident responders — need to quickly see
whether DRED is actually firing during a call. The stats JSON
already carries the counters (after Phase 3c), but they were
buried in the trailing JSON dump. Added a dedicated
"=== Loss Recovery ===" section to the meta preamble that
extracts dred_reconstructions, classical_plc_invocations,
frames_decoded, and fec_recovered from the JSON and displays
them plainly, plus computed percentages when frames_decoded > 0.
- New extractLongField helper: tiny hand-rolled JSON integer
extractor. We don't want to pull in a full JSON parser for this
single use case and CallStats has a flat, well-known schema.
Verification:
- cargo check --workspace: zero errors
- cargo test -p wzp-proto --lib: 63 passing
- cargo test -p wzp-codec --lib: 68 passing
- cargo test -p wzp-client --lib: 35 passing (+1 ignored probe)
- cargo test -p wzp-relay --lib: 68 passing (+1 new Phase 4 test)
- cargo check -p wzp-android --lib: zero errors
- Android APK build verified earlier today (unridden-alfonso.apk
via the remote Docker builder) — Phase 0–3c confirmed to compile
end-to-end on the NDK target.
Phase 4b remaining (not blocking this commit):
- Periodic LossRecoveryUpdate emitter in wzp-client/src/call.rs and
wzp-android/src/engine.rs (every ~5 s)
- Relay-side handler in main.rs that matches the new variant and
calls metrics.update_session_loss_recovery
- Grafana "Loss recovery breakdown" panel in docs/grafana-dashboard.json
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Siavash Sameni
|
505a834c5b |
feat(codec): Phase 3c — Android engine.rs DRED reconstruction on packet loss
Phase 3c mirrors Phase 3b on the Android receive path. With Phase 0-3b
landed on desktop + Android encoder, this commit completes codec-layer
loss recovery on the Android decoder side.
Architectural difference vs desktop: engine.rs has NO jitter buffer.
The recv task reads packets directly from the transport via
recv_media().await and writes decoded audio straight into the playout
ring. There is no PlayoutResult::Missing equivalent. Gap detection
therefore has to be done via sequence-number tracking — when a packet
arrives with seq > expected_seq, the frames in between are missing and
we attempt to reconstruct them via DRED before decoding the newly-
arrived packet.
Implementation:
Imports & types:
- Added wzp_codec::AdaptiveDecoder, wzp_codec::dred_ffi::{
DredDecoderHandle, DredState} imports.
- Changed the `decoder` local from Box<dyn AudioDecoder> (via
wzp_codec::create_decoder) to concrete AdaptiveDecoder::new(profile).
Same reasoning as Phase 3b: reconstruct_from_dred is an inherent
method, not a trait method, so we need the concrete type.
Recv task state (all task-local, no new struct fields):
- dred_decoder: DredDecoderHandle
- dred_parse_scratch: DredState (reused, overwritten per parse)
- last_good_dred: DredState (cached most-recent valid state)
- last_good_dred_seq: Option<u16>
- expected_seq: Option<u16> (for gap detection)
- dred_reconstructions: u64 (telemetry)
- classical_plc_invocations: u64 (telemetry)
Recv loop body (Opus source packets only):
1. Parse DRED from the new packet first so last_good_dred reflects
the freshest state available for gap recovery.
2. Detect a gap: gap = pkt.seq.wrapping_sub(expected_seq). Cap at
MAX_GAP_FRAMES = 16 (320 ms) to avoid huge wraparound scenarios.
3. For each missing seq in the gap:
offset = (last_good_dred_seq - missing_seq) * frame_samples
if 0 < offset <= last_good_dred.samples_available():
reconstruct_from_dred + write to playout ring
bump dred_reconstructions
else:
decoder.decode_lost (classical PLC) + write + bump plc counter
4. Decode the current packet normally and write to playout ring
(unchanged from Phase 2).
5. Update expected_seq = pkt.seq.wrapping_add(1).
Profile-switch handling: when the incoming codec changes (triggering
decoder.set_profile), reset last_good_dred_seq and expected_seq to
None. The cached DRED state is tied to the old profile's frame rate
and would produce wrong offsets after the switch; starting fresh is
correct.
Decode-error fallback: the existing `Err(e) => decode_lost` branch
now also increments classical_plc_invocations so the counter
accurately reflects all PLC invocations (gap-detected AND decode-
error-triggered).
Telemetry (CallStats additions):
- stats.dred_reconstructions: u64
- stats.classical_plc_invocations: u64
Both updated on every packet arrival in the existing stats.lock()
block alongside frames_decoded/fec_recovered, so the Android UI and
JNI bridge already have these values without any further plumbing.
The periodic recv stats log now includes both counters.
Ordering note: DRED gap reconstruction happens BEFORE decoding the new
packet's audio because the playout ring is FIFO. Gap samples must be
written before the new packet's samples so temporal order is preserved.
Out-of-order late arrivals (seq < expected_seq) are naturally dropped
as stale by the gap detection (gap would be a large wraparound value
exceeding MAX_GAP_FRAMES).
Verification:
- cargo check --workspace: zero errors
- cargo test -p wzp-codec --lib: 68 passing (unchanged from Phase 3b)
- cargo test -p wzp-client --lib: 35 passing (unchanged from Phase 3b)
- cargo check -p wzp-android --lib: zero errors
- cargo test -p wzp-android cannot run on macOS host (pre-existing
-llog linker dep, unrelated). Real end-to-end verification happens
via the Android APK build on the remote Docker builder
(scripts/build-and-notify.sh).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Siavash Sameni
|
27bc264738 |
feat(codec): Phase 3b — CallDecoder DRED reconstruction on packet loss
Phase 3b of the DRED integration — wires the Phase 3a FFI primitives
into the desktop receive path. When the jitter buffer reports a missing
Opus frame, CallDecoder now attempts to reconstruct the audio from the
most recently parsed DRED side-channel state before falling through to
classical PLC.
Architectural refinement vs the PRD's literal wording: the PRD said
"jitter buffer takes a Box<dyn DredReconstructor>". After checking deps,
wzp-transport depends only on wzp-proto (not wzp-codec). Putting DRED
state in the jitter buffer would require a new cross-crate dep and
couple the codec-agnostic buffer to libopus. Instead, this commit keeps
the DRED state ring and reconstruction dispatch inside CallDecoder (one
layer up from the jitter buffer), intercepting the existing
PlayoutResult::Missing signal. Same lookahead/backfill semantics,
cleaner layering, zero change to wzp-transport.
Changes:
CallDecoder field type: Box<dyn AudioDecoder> → AdaptiveDecoder.
Required because Phase 3b calls the inherent reconstruct_from_dred
method, which cannot live on the AudioDecoder trait without dragging
libopus DredState through wzp-proto. In practice AdaptiveDecoder was
the only AudioDecoder implementor anyway — the trait abstraction was
buying nothing. Method call sites unchanged because AdaptiveDecoder
also implements AudioDecoder.
New CallDecoder fields:
- dred_decoder: DredDecoderHandle
- dred_parse_scratch: DredState (scratch for parse_into)
- last_good_dred: DredState (cached most-recent valid state)
- last_good_dred_seq: Option<u16>
- dred_reconstructions: u64 (Phase 4 telemetry)
- classical_plc_invocations: u64 (Phase 4 telemetry)
CallDecoder::ingest — on Opus non-repair packets, parse DRED into the
scratch state. On success (samples_available > 0), std::mem::swap the
scratch into last_good_dred and record the seq. This is O(1) per
packet, zero allocation after construction (the two DredState buffers
are allocated once in new() and reused forever).
CallDecoder::decode_next — on PlayoutResult::Missing(seq) for Opus
profiles: if last_good_dred_seq > seq and the seq delta × frame_samples
fits within samples_available, call audio_dec.reconstruct_from_dred
and bump dred_reconstructions. Otherwise fall through to classical
PLC and bump classical_plc_invocations. The Codec2 path always falls
through to classical PLC since DRED is libopus-only and
AdaptiveDecoder::reconstruct_from_dred rejects Codec2 tiers
explicitly.
OpusDecoder and AdaptiveDecoder: new inherent reconstruct_from_dred
method that delegates to the underlying DecoderHandle. Needed to
bridge CallDecoder's wzp-client code to the Phase 3a FFI wrappers
without touching the AudioDecoder trait.
CRITICAL FINDING — raised DRED loss floor from 5% to 15%:
Phase 3b testing discovered that libopus 1.5's DRED emission window
scales aggressively with OPUS_SET_PACKET_LOSS_PERC. Empirical data
(see probe_dred_samples_available_by_loss_floor, an #[ignore]'d
diagnostic test in call.rs):
loss_pct samples_available effective_ms
5% 720 15 ms (useless!)
10% 2640 55 ms
15% 4560 95 ms
20% 6480 135 ms
25%+ 8400 (capped) 175 ms (~87% of 200 ms configured)
The Phase 1 default of 5% produced only a 15 ms reconstruction window
— too small to even cover a single 20 ms Opus frame. DRED was
effectively disabled even though it was emitting bytes. Raised the
floor to 15% (95 ms window) as the minimum that actually provides
single-frame loss recovery. This updates Phase 1's DRED_LOSS_FLOOR_PCT
constant in opus_enc.rs and the accompanying module docstring.
Trade-off: 15% assumed loss slightly increases encoder bitrate overhead
on clean networks. Measured via the existing phase1 bitrate probe:
Before (5% floor): 3649 bytes/sec at Opus 24k + 300 Hz sine
After (15% floor): 3568 bytes/sec at Opus 24k + 300 Hz sine
The delta is within noise — 15% isn't meaningfully more expensive than
5% on this signal, which suggests the DRED emission size is signal-
dependent rather than loss-dependent for small values. Net result: we
get a 6x larger reconstruction window for essentially free.
Tests (+3 DRED recovery, +1 #[ignore]'d probe):
- opus_single_packet_loss_is_recovered_via_dred — full encode → ingest
→ decode_next loop with one packet dropped mid-stream. Asserts
dred_reconstructions ≥ 1 and observes the exact counter deltas.
- opus_lossless_ingest_never_triggers_dred_or_plc — baseline behavior,
lossless stream never takes the Missing branch.
- codec2_loss_falls_through_to_classical_plc — Codec2 never
reconstructs via DRED even if state were populated (which it won't
be — Codec2 packets don't carry DRED bytes).
- probe_dred_samples_available_by_loss_floor — #[ignore]'d diagnostic
that sweeps loss_pct values and prints the resulting DRED window
sizes. Kept for future tuning work.
New CallDecoder introspection accessors (public but undocumented in
the PRD): last_good_dred_seq() and last_good_dred_samples_available()
for test diagnostics and future telemetry surfaces in Phase 4.
Verification:
- cargo check --workspace: zero errors
- cargo test -p wzp-codec --lib: 68 passing (Phase 3a baseline held)
- cargo test -p wzp-client --lib: 35 passing (+3 Phase 3b tests,
+1 ignored diagnostic, no regressions)
Next up: Phase 3c mirrors this on the Android engine.rs receive path.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Siavash Sameni
|
c27b39d553 |
feat(codec): Phase 3a — DRED FFI primitives (DredDecoderHandle + DredState)
Phase 3a of the DRED integration — the foundation for codec-layer loss recovery. Adds three new safe wrappers to crates/wzp-codec/src/dred_ffi.rs over the raw opusic-sys FFI, plus the reconstruction method on the existing DecoderHandle. No call-site integration yet — that lands in Phase 3b (desktop) and Phase 3c (Android). New types: - `DredDecoderHandle`: owns *mut OpusDREDDecoder from opus_dred_decoder_create. Used for parsing DRED side-channel data out of arriving Opus packets. This is a SEPARATE libopus object from OpusDecoder — it has its own internal state. Freed via opus_dred_decoder_destroy on Drop. - `DredState`: owns *mut OpusDRED from opus_dred_alloc (a fixed ~10.6 KB buffer per libopus 1.5). Holds parsed DRED data between the parse and reconstruct steps. Reusable — parse_into overwrites contents. Tracks samples_available as a cached u32 so callers don't thread the value separately. Freed via opus_dred_free on Drop. New methods: - `DredDecoderHandle::parse_into(&mut self, state: &mut DredState, packet)` wraps opus_dred_parse with max_dred_samples=48000 (1s max), sampling_rate =48000, defer_processing=0. Returns the positive sample offset of the first decodable DRED sample, 0 if no DRED is present, or an error. Populates state.samples_available so subsequent reconstruct calls know the valid offset range. - `DecoderHandle::reconstruct_from_dred(&mut self, state, offset_samples, output)` wraps opus_decoder_dred_decode. Reconstructs audio at a specific sample position (positive, measured backward from the DRED anchor packet) into a caller-provided output buffer. Validates that 0 < offset_samples <= state.samples_available() before calling the FFI to catch range bugs. Tests (+7, wzp-codec total: 68 passing): - dred_decoder_handle_creates_and_drops - dred_state_creates_and_drops - dred_state_reset_zeroes_counter - dred_parse_and_reconstruct_roundtrip — end-to-end validation. Encodes 60 frames of a 300 Hz sine wave through a DRED-enabled Opus 24k encoder, parses DRED state out of each arriving packet, asserts that at least one packet carries non-zero samples_available (DRED warm-up completes within the first second), then reconstructs 20 ms of audio from inside the window and asserts non-zero total energy. This is the hard signal that the full libopus 1.5 DRED FFI chain is correctly wired on our side. - reconstruct_with_out_of_range_offset_errors — offset > samples_available is rejected at the Rust layer before the FFI call. - reconstruct_with_zero_offset_errors — offset <= 0 rejected. - dred_parse_empty_packet_returns_zero — graceful handling of empty input. Architectural note (divergence from PRD's literal wording): The PRD said "jitter buffer takes a Box<dyn DredReconstructor>". After checking Cargo.toml for wzp-transport, it does NOT depend on wzp-codec — only wzp-proto. Adding a DRED state ring inside the jitter buffer would require a new cross-crate dependency and couple the codec-agnostic jitter buffer to libopus internals. Instead, Phase 3b will put the DRED state ring and reconstruction dispatch in CallDecoder (one layer up from the jitter buffer), intercepting the existing PlayoutResult::Missing signal and attempting reconstruction before falling through to classical PLC. The jitter buffer itself stays unchanged. Same lookahead/backfill semantics, cleaner layering. PRD's intent preserved, implementation refined. Verification: - cargo check --workspace: zero errors - cargo test -p wzp-codec --lib: 68 passing (61 Phase 2 baseline + 7 new) - The roundtrip test is the acceptance gate — it proves that opus_dred_decoder_create, opus_dred_alloc, opus_dred_parse, and opus_decoder_dred_decode all work correctly through our wrappers on real libopus 1.5.2 output. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
6db5c25b54 |
feat(codec): Phase 2 — remove RaptorQ from Opus tiers, Codec2 unchanged
Phase 2 of the DRED integration (docs/PRD-dred-integration.md). With
Phase 1 having enabled DRED on every Opus profile, the app-level RaptorQ
layer is now redundant overhead on those tiers: +20% bitrate, +40–100 ms
receive-side latency (block wait), +CPU for stats we never used. This
phase removes RaptorQ from the Opus encode and decode paths on both the
desktop (wzp-client/call.rs) and Android (wzp-android/engine.rs) sides.
Codec2 tiers keep RaptorQ with their current ratios unchanged — DRED is
libopus-only and Codec2 has no neural equivalent.
Encoder changes (the real bandwidth / CPU win):
- CallEncoder::encode_frame and engine.rs encode loop now gate the
RaptorQ path on !codec.is_opus():
- Opus source packets emit fec_block=0, fec_symbol=0,
fec_ratio_encoded=0 in the MediaHeader
- fec_enc.add_source_symbol is skipped on Opus
- generate_repair + repair packet emission is skipped on Opus
- block_id and frame_in_block counters stay frozen at 0 for Opus
- Codec2 path is byte-for-byte identical to pre-Phase-2 behavior.
Decoder changes (mostly cleanup, since both live decoder paths were
already reading audio directly from source packets and only using the
RaptorQ decoder output for stats):
- CallDecoder::ingest skips fec_dec.add_symbol on Opus packets. Source
packets still flow to the jitter buffer; Opus repair packets from old
senders are dropped cleanly (repair packets never hit the jitter
buffer either).
- engine.rs recv loop skips fec_dec.add_symbol, fec_dec.try_decode, and
fec_dec.expire_before on Opus packets. The `fec_recovered` stat
counter becomes Codec2-only (a separate DRED reconstruction counter
lands in Phase 4).
Wire-format backward compat verified at pre-flight:
- Old receiver + new sender: engine.rs pipeline.rs path gates on
non-zero fec_block/fec_symbol which now never fire for Opus, so the
RaptorQ decoder simply isn't fed. Audio flows normally. Desktop
CallDecoder's old path accumulated packets into the stale-eviction
HashMap, which cleans up after 2s — harmless.
- New receiver + old sender: new receiver skips RaptorQ on Opus so
old-sender repair packets are ignored entirely (no crash, no double-
decode). Loses the (previously vestigial) RaptorQ recovery benefit,
which was never actually active in the audio path. Source packets
still decode normally.
- No wire format version bump required. MediaHeader is unchanged; we
just zero the FEC fields on Opus packets.
Test changes:
- Removed `encoder_generates_repair_on_full_block` — asserted the old
(pre-Phase-2) RaptorQ-on-Opus behavior and is now incorrect. Replaced
with two symmetric tests:
- `opus_source_packets_have_zero_fec_header_fields` — verifies
Phase 2 invariants on Opus packets
- `opus_encoder_never_emits_repair_packets` — runs 20 frames of
non-silent sine wave through a GOOD-profile encoder, asserts
exactly 20 output packets, zero repair
- `codec2_encoder_generates_repair_on_full_block` — same shape as
the old test but on CATASTROPHIC profile (Codec2 1200, 8
frames/block, ratio 1.0) to verify Codec2 path still emits
repairs as before
Verification:
- cargo check --workspace: zero errors
- cargo test -p wzp-codec --lib: 61 passing (Phase 1 baseline held)
- cargo test -p wzp-client --lib: 32 passing (+3 new Phase 2 tests,
-1 old test removed)
- cargo check -p wzp-android --lib: zero errors (host link of
wzp-android tests fails on -llog per pre-existing Android-only
build.rs, unrelated to this work; integration build via
build-and-notify.sh will validate Android end-to-end)
- Pre-existing broken integration test in
crates/wzp-client/tests/handshake_integration.rs (SignalMessage
schema drift) is NOT caused by this commit — baseline had the same
3 compile errors before Phase 2. Flagged as a separate cleanup task.
Expected observable effects on a real call:
- Opus 24k outgoing bitrate drops from ~28.8 kbps (ratio 0.2 RaptorQ)
to ~25 kbps (base 24 kbps + DRED ~1–10 kbps signal-dependent)
- Opus receive-side latency drops ~40 ms on clean network (no more
block wait — jitter buffer emits as soon as a source packet arrives)
- Codec2 calls show no latency or bitrate change
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Siavash Sameni
|
54cbebd34e |
feat(codec): Phase 1 — enable DRED on all Opus profiles, disable inband FEC
Phase 1 of the DRED integration (docs/PRD-dred-integration.md). The Opus encoder now emits DRED (Deep REDundancy) bytes in every packet, carrying a neural-coded history of recent audio that the decoder can use to reconstruct loss bursts up to the configured window. Opus inband FEC (LBRR) is disabled because DRED does the same job better and running both wastes bitrate on overlapping protection. Tiered DRED duration policy per PRD: Studio (Opus 32k/48k/64k): 10 frames = 100 ms Normal (Opus 16k/24k): 20 frames = 200 ms Degraded (Opus 6k): 50 frames = 500 ms Each profile switch (via adaptive quality) updates the DRED duration to match the new tier. A 5% packet_loss floor is applied whenever DRED is active, because libopus 1.5 gates DRED emission on non-zero packet_loss. Real loss measurements from the quality adapter override upward. Escape hatch: AUDIO_USE_LEGACY_FEC=1 reverts the encoder to Phase 0 behavior (inband FEC Mode1, DRED off, no loss floor). Read once at OpusEncoder::new; call-scoped, not re-read mid-call. Trait-level set_inband_fec becomes a no-op in DRED mode to preserve the invariant even if external callers forget. Observations from the bitrate probe test (dred_mode_roundtrip_voice_pattern): DRED mode: 3649 bytes/sec (~29.2 kbps) on Opus 24k + 300 Hz sine Legacy mode: 2383 bytes/sec (~19.1 kbps) Delta: +10.1 kbps The delta is considerably larger than the "+1 kbps flat" figure I carried into the PRD from hazy memory of published DRED benchmarks. Likely because the input (300 Hz sine) is very compressible so the base Opus rate in legacy mode is well below the 24 kbps target, making the delta look disproportionate. Signal-dependent — real speech would probably show a different ratio. If production telemetry shows the overhead is excessive, we can cut DRED duration on the normal tier from 200 ms to 100 ms as a first tuning lever. Not blocking Phase 1 since the test still passes within the reasonable 2000–8000 bytes/sec bounds. Test changes (+8 tests, total wzp-codec: 61 passing): - dred_duration_for_studio_tiers_is_100ms (per-profile policy) - dred_duration_for_normal_tiers_is_200ms - dred_duration_for_degraded_tier_is_500ms - dred_duration_for_codec2_is_zero - default_mode_is_dred_not_legacy (sanity check on fresh construction) - dred_mode_roundtrip_voice_pattern (observes DRED bitrate, asserts bounds) - profile_switch_refreshes_dred_duration (verifies set_profile updates DRED) - set_inband_fec_noop_in_dred_mode (trait-level inband FEC no-op) Verification: - cargo check --workspace: zero errors, no new warnings - cargo test -p wzp-codec: 61/61 passing (53 pre-Phase-1 baseline + 8 new) - Empirical DRED bitrate observed via `rtk proxy cargo test dred_mode_roundtrip_voice_pattern -- --nocapture` Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
86526a7ad4 |
feat(codec): Phase 0 — swap audiopus → opusic-c + opusic-sys (libopus 1.5.2)
Phase 0 of the DRED integration (docs/PRD-dred-integration.md). No behavior change: inband FEC stays ON, no DRED, same bitrate, same quality. This commit unblocks Phase 1+ by getting us onto libopus 1.5.2 where DRED lives. Rationale for going straight to a custom DecoderHandle: opusic-c::Decoder's inner *mut OpusDecoder pointer is pub(crate), so we cannot reach it for the Phase 3 DRED reconstruction path. Running two parallel decoders (one for audio, one for DRED) would drift because the DRED decoder wouldn't see normal decode calls. Single unified DecoderHandle over raw opusic-sys is the only correct architecture, so we build it in Phase 0 rather than rewriting opus_dec.rs twice. Changes: - Cargo.toml (workspace + wzp-codec): remove audiopus 0.3.0-rc.0, add opusic-c 1.5.5 (bundled + dred features), opusic-sys 0.6.0 (bundled), bytemuck 1. Pinned exactly for reproducible libopus 1.5.2. - opus_enc.rs: rewritten against opusic_c::Encoder. Argument order for Encoder::new swapped (Channels first). set_inband_fec(bool) now maps to InbandFec::Mode1 (the libopus 1.5 equivalent of 1.3's LBRR). encode uses bytemuck::cast_slice<i16,u16> at the &[u16] boundary. - dred_ffi.rs (new): DecoderHandle wrapping *mut OpusDecoder directly via opusic-sys. Owns the allocation, frees on Drop. Exposes decode, decode_lost, and a pub(crate) as_raw_ptr() for the future Phase 3 DRED reconstruction. Send+Sync justified via &mut self access discipline. - opus_dec.rs: rewritten as a thin AudioDecoder impl over DecoderHandle. Behavior identical to pre-swap. Verification (Phase 0 acceptance gates): - cargo check --workspace: clean (30 pre-existing warnings in jni_bridge.rs unrelated to this work; zero in changed files). - cargo test -p wzp-codec: 53 tests pass (50 pre-swap + 6 new: 3 in dred_ffi.rs for DecoderHandle lifecycle, 3 in opus_enc.rs for version check and roundtrip). - linked_libopus_is_1_5 test asserts opusic_c::version() contains "1.5" — hard signal that the swap landed correctly. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
07873ea598 |
fix(linux-aec): fall back to 0.3 crate + apt lib (2.x bundled is broken)
Switch the webrtc-audio-processing dep from the 2.x git source (bundled mode) back to crates.io 0.3, and link against Debian's apt package libwebrtc-audio-processing-dev (0.3-1+b1 on Bookworm). The 2.x path fails because both the crates.io tarball and the upstream git main branch of webrtc-audio-processing-sys 2.0.3 have a build.rs bug where \`meson setup --reconfigure\` is passed unconditionally, panicking on first-run empty build dirs with "Directory does not contain a valid build tree". The 0.x line sidesteps bundled mode entirely by linking the apt-provided library. Trade-off: we get AEC2 (the older generation) instead of AEC3, but it's the same algorithm family and is what PulseAudio's module-echo-cancel and PipeWire's filter-chain use on current Debian-family distros. Fine for shipping — we can revisit AEC3 once the 2.x bundled build is fixed upstream. API changes: - 0.3's Processor::process_capture_frame and process_render_frame take &mut self, so wrap the module-level processor in a Mutex. Capture and playback threads each lock briefly (sub-ms per 10 ms frame); contention is minimal. - Import NUM_SAMPLES_PER_FRAME from the crate directly instead of hardcoding 480, so the code tracks whatever sample rate the upstream C++ lib exposes (currently 48 kHz hardcoded -> 480). - Helper fns drain_frames_through_apm / tee_render_samples / etc. take &Mutex<Processor> instead of &Processor. - Use explicit EchoCancellationSuppressionLevel and NoiseSuppressionLevel imports rather than fully-qualified paths. Dockerfile: - Drop meson / ninja-build / python3 (only needed for bundled build). - Add libwebrtc-audio-processing-dev for the system link path. - Keep clang (may be needed by the bindgen step in some versions). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
cc00f7cace |
fix(linux-aec): try main branch of webrtc-audio-processing
v2.0.3 bundled build hits 'Directory does not contain a valid build tree' because the crate's build.rs uses `meson setup --reconfigure` unconditionally, which fails on first run when the build dir doesn't yet contain prior meson state. Try the main branch in case it's been fixed post-release. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
eb9de988d6 |
fix(linux-aec): use git dep for webrtc-audio-processing
The crates.io tarball of webrtc-audio-processing-sys 2.0.3 is missing the vendored C++ submodule — the bundled build fails with 'Directory does not contain a valid build tree' when meson tries to configure the ./webrtc-audio-processing subdirectory. Cargo clones git deps with submodules auto-initialized since ~1.27, so pulling from the upstream git repo (pinned to tag v2.0.3) gives us the full source tree. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
4ba77c8c0e |
feat(linux): WebRTC AEC3 capture/playback backend with render-side tee
Adds gold-standard Linux echo cancellation: in-app WebRTC AEC3 (Audio Processing Module) via the webrtc-audio-processing crate, using the same algorithm as Chrome WebRTC, Zoom, Teams, and Jitsi. Runs entirely in-process, so it works identically on ALSA / PulseAudio / PipeWire systems — no dependency on user-configured echo-cancel modules. Architecture: - New crates/wzp-client/src/audio_linux_aec.rs module (~470 lines). Contains LinuxAecCapture and LinuxAecPlayback, both using CPAL under the hood but routing samples through a shared Arc<webrtc_audio_processing::Processor>. The playback path tees each 20 ms frame into APM.process_render_frame as the echo reference BEFORE handing the samples to CPAL's output callback. The capture path runs APM.process_capture_frame on each mic frame in place before pushing to the audio ring buffer. This is the "tee the playback ring" approach that Zoom/Teams/Jitsi use. - New `linux-aec` feature in wzp-client pulling in the webrtc-audio-processing crate at v2.x with the `bundled` sub-feature. Bundled means the vendored PulseAudio WebRTC C++ sources are statically compiled via meson+ninja at cargo build time — no runtime .so dependency, avoids Debian Bookworm's stale libwebrtc-audio-processing-dev 0.3 package (which predates AEC3). Dep is target-gated to Linux, so enabling the feature on non-Linux is a no-op. - lib.rs re-exports LinuxAecCapture/LinuxAecPlayback as AudioCapture/AudioPlayback when `linux-aec` is on, otherwise falls back to the CPAL audio_io path. Shared public API (start/ring/stop/Drop) means downstream code is unchanged. - New `linux-aec` feature in wzp-desktop forwards to wzp-client/linux-aec so `cargo tauri build -- --features wzp-desktop/linux-aec` builds the AEC variant. APM configuration: - EchoCancellation: High suppression, delay-agnostic mode on, extended filter on, stream_delay_ms=60 initial hint - NoiseSuppression: High - HighPassFilter: on - AGC: off (can fight Opus encoder's own gain staging + adaptive quality controller; add later if users report low mic level) Frame size handling: - Pipeline uses 20 ms frames (960 samples @ 48 kHz mono) - APM requires strict 10 ms (480 samples) per call - Each 20 ms frame is split into two 480-sample halves, APM called twice, halves stitched back - Same pattern for render and capture sides - Carry-buffer logic handles the case where CPAL delivers samples in arbitrary chunk sizes that don't divide 960 Build infrastructure: - scripts/Dockerfile.linux-desktop-builder adds meson, ninja-build, python3, clang for the webrtc-audio-processing bundled build - scripts/build-linux-desktop-docker.sh takes a new --aec flag that enables the linux-aec feature and renames the output artifacts with an `-aec` suffix so noAEC and AEC variants can coexist on disk Task #30. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
4e9244eb00 |
fix(windows): add Win32_Security feature + 2024 edition unsafe wrappers
- CreateEventW is gated behind Win32_Security in the windows crate
because its signature takes SECURITY_ATTRIBUTES; add to features.
- Remove unused HANDLE import.
- Wrap GetId() and PWSTR::to_string() in explicit unsafe { ... }
blocks for Rust 2024 edition's unsafe_op_in_unsafe_fn lint.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Siavash Sameni
|
03a80a3196 |
feat(windows): WASAPI capture backend with OS-level AEC
Adds a direct WASAPI microphone capture path for the Windows desktop build that opens the default communications endpoint via IMMDeviceEnumerator -> IAudioClient2 -> SetClientProperties with AudioCategory_Communications, turning on Windows's communications audio processing chain (AEC, noise suppression, automatic gain control). The communications AEC operates at the OS level and uses the system render mix as the reference signal, so echo from our existing CPAL playback stream is cancelled automatically with no per-process reference plumbing. Architecture: - New crates/wzp-client/src/audio_wasapi.rs module (~280 lines). Event-driven capture loop on a dedicated thread; pushes PCM into the same lock-free AudioRing used by the CPAL path. Same public API as audio_io::AudioCapture so downstream code is unchanged. - New `windows-aec` feature in wzp-client that pulls in the `windows` crate (Microsoft's official Rust COM bindings) gated to target_os = "windows" only. Enabling the feature on non-Windows targets is a no-op since both the module and the dep are cfg(target_os = "windows"). - lib.rs re-exports WasapiAudioCapture as AudioCapture when the feature is on, otherwise falls back to the CPAL AudioCapture. AudioPlayback is always the CPAL one — no reason to swap it. - desktop/src-tauri/Cargo.toml Windows target enables the new feature: `features = ["audio", "windows-aec"]`. Implementation notes: - Uses eCommunications role (not eConsole) for GetDefaultAudioEndpoint — the user-configured "communications" device that Teams/Zoom pick up, and the one Windows's AEC is tuned for. - Requests 48 kHz mono i16 with AUDCLNT_STREAMFLAGS_AUTOCONVERTPCM + SRC_DEFAULT_QUALITY so Windows handles any format conversion in the audio engine instead of rejecting our format. - Event-driven with SetEventHandle / WaitForSingleObject — no polling, minimal CPU cost between packets. - 200 ms wait timeout so the capture thread polls `running` often enough for Drop to stop cleanly even if the audio engine stalls (e.g. device unplug). Task #24. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
da09fdb6e9 |
windows(desktop): gate coreaudio / VoiceProcessingIO to macOS-only targets
First step of the Windows x86_64 desktop build: stop pulling
coreaudio-rs into the Windows dependency graph so the project can at
least run `cargo check --target x86_64-pc-windows-msvc`. Software AEC
is already disabled in engine.rs so there's nothing else to stub — the
macOS-specific VPIO path is skipped via #[cfg(target_os = "macos")] on
both sides and Windows falls through to the plain CPAL
AudioCapture/AudioPlayback branch that already existed.
crates/wzp-client/Cargo.toml
- coreaudio-rs optional dep moved under [target.'cfg(target_os = "macos")']
- `vpio` feature now uses `dep:coreaudio-rs` syntax and the gated dep
- Enabling `vpio` on Windows/Linux is a no-op at resolution time
crates/wzp-client/src/lib.rs
- `pub mod audio_vpio` is now #[cfg(all(feature = "vpio", target_os = "macos"))]
- Previously `vpio` alone was enough to try to compile the Core Audio
bindings, which would fail on non-Apple targets the moment the
feature flag was flipped on
desktop/src-tauri/Cargo.toml
- [target.'cfg(not(target_os = "android"))'] removed — was leaking
vpio into Windows/Linux via the catch-all.
- macOS: wzp-client with features = ["audio", "vpio"]
- Windows: wzp-client with features = ["audio"]
- Linux: wzp-client with features = ["audio"]
- Android: wzp-client with default-features = false (unchanged)
- Dropped the unused direct coreaudio-rs = "0.11" dep on macOS —
wzp-desktop's own sources never call Core Audio directly.
Verified via `cargo tree --target x86_64-pc-windows-msvc -p wzp-desktop`
that the Windows target now resolves wzp-client with cpal but without
coreaudio-rs. macOS target still resolves with coreaudio (direct via
vpio feature and transitively via cpal). macOS `cargo check` still
builds cleanly.
Cross-compile from macOS hit a cargo-xwin + llvm-lib setup issue in
ring's build.rs, so the actual `cargo check --target
x86_64-pc-windows-msvc` did not complete locally. Build verification
belongs on the user's Windows x86_64 host where MSVC is present
natively.
See tasks #23 (this one), #24 (Voice Capture DSP / WASAPI Communications
for OS-level AEC on Windows), and #25 (aarch64-pc-windows-msvc support).
|
||
|
Siavash Sameni
|
9e37201198 |
android(audio): Usage::VoiceCommunication + MODE_IN_COMMUNICATION, default handset
With |
||
|
Siavash Sameni
|
da106bd939 |
fix(android-audio): revert to 96be740's Oboe config — VoiceCommunication broke callback drain
Build |
||
|
Siavash Sameni
|
8c36fb5651 |
fix(wzp-native): Oboe ResultWithValue has no value_or, unfold explicitly
cc-rs build of oboe_bridge.cpp failed at
|
||
|
Siavash Sameni
|
cfa9ff67cf |
fix(android-audio): VoIP mode + speakerphone + debug PCM recorder
Build
|
||
|
Siavash Sameni
|
96be740fd9 |
diag(android-audio): aggressive logging across the whole Oboe pipeline
User confirmed: mac hears android, android does not hear mac. So Oboe
capture works end-to-end but Oboe playout on Android silently drops
audio even though QUIC forwards the packets. Archaeology on the legacy
wzp-android crate also revealed that the "last known good" Android audio
path NEVER used Oboe in production — it used Kotlin AudioRecord +
AudioTrack via JNI, and cpp/oboe_bridge.cpp was dead code. So every time
we've "tested" Oboe end-to-end this week was the first production use,
and any of its config knobs could be the bug.
Instrumenting every stage of the pipeline so one smoke-test log dump can
isolate the layer at fault:
C++ (oboe_bridge.cpp)
- Log the ACTUAL stream parameters after openStream for both capture
and playout (sample rate, channels, format, framesPerBurst,
framesPerDataCallback, bufferCapacityInFrames, sharing, perf mode).
Oboe may silently override values we requested — e.g. if we ask for
48kHz mono but the device gives us 44.1kHz stereo our 960-sample
frames are the wrong duration and the pipeline drifts.
- Capture callback: on cb#0 log sample range+RMS of the first frame
to prove we get real mic data (not zeros). Every 50 callbacks
(~1s at 20ms burst) log calls, numFrames, ring available_write,
bytes actually written, ring_full_drops, total_written.
- Playout callback: on cb#0 log numFrames + ring state. On the FIRST
non-empty read log sample range+RMS so we can tell if the samples
coming out of the ring are real audio or zeros. Every 50 callbacks
log calls, nonempty count, numFrames, ring available_read,
underrun_frames, total_played_real.
Rust wzp-native (src/lib.rs)
- wzp_native_audio_write_playout now logs the first 3 writes and then
every 50th: in_len, written, sample range, RMS, ring write/read
cursors before, available_read and available_write after. Reveals
ring-overflow and whether the engine is actually handing us audio.
- Minimal android logcat shim via __android_log_write extern — no
new crate dependency.
- AudioBackend grows a `playout_write_log_count` AtomicU64 to gate
the write-side log throttle.
Rust engine.rs (android branch)
- Recv task: log sample range + RMS for the first 3 decoded PCM
frames and then every 100th. Reveals whether decoder.decode is
producing real audio or silent buffers.
- Recv task: if audio_write_playout returns fewer samples than we
handed it (partial write → ring nearly full) warn about it in the
first 10 frames.
- Recv heartbeat every 2s: recv_fr, decoded_frames, last_decode_n,
last_written, written_samples, decode_errs, codec.
Expected flow in a healthy log:
capture cb#0: numFrames=960 range=[-1200..900] rms=180 ← mic OK
capture stream opened: actualSR=48000 Ch=1 ... ← no override
playout stream opened: actualSR=48000 Ch=1 ...
CallEngine::start invoked ... → connected → audio started
recv: first media packet received ...
recv: decoded PCM sample range decoded_frames=1 range=[-300..250] rms=92
playout WRITE #0: in_len=960 written=960 range=[-300..250] rms=92
playout FIRST nonempty read: to_read=960 range=[-300..250] rms=92
playout heartbeat: calls=50 nonempty=50 underrun=0 ...
recv heartbeat: decoded_frames=100 last_written=960 ...
If any of those are missing/zero we know the exact stage to fix.
|
||
|
Siavash Sameni
|
8c4d640f89 |
fix(android): playout Usage::Media + relay CallSetup advertises real IP
Three real bugs, one smoke-test session's worth of progress. 1. RELAY: wrong advertised addr in CallSetup The direct-call CallSetup computed `relay_addr = addr.ip()` where `addr = connection.remote_address()` — i.e. the CLIENT'S IP, not the relay's. So the relay was telling both parties "the call room is at the answerer's IP:4433", which meant each client dialed either the other client (no server listening) or themselves. Both endpoint.connect calls hung forever and the call never happened. Fix: compute the relay's own advertised IP once at startup. If the listen addr is 0.0.0.0, probe the primary outbound interface via the classic UDP-bind-and-connect(8.8.8.8:80) trick to discover the LAN IP the OS would use to reach external hosts. Thread the resulting advertised_addr_str into the CallSetup sender for both parties. 2. RELAY: accept loop serialized QUIC handshakes Previously the main accept loop called `wzp_transport::accept` which did both `endpoint.accept().await` AND `incoming.await` (the server- side QUIC handshake). A single slow handshake therefore blocked every subsequent client from being accepted. Unroll the helper here and move `incoming.await` into the per-connection spawned task, so every handshake runs in parallel. Also log "accept queue: new Incoming", "QUIC handshake complete", and "QUIC handshake failed" so we can tell immediately whether a client's packets are reaching the relay at all. 3. ANDROID: playout was routed to the silent in-call stream The Oboe playout stream was configured with Usage::VoiceCommunication, which routes to the Android in-call earpiece stream. That stream is silent unless the Activity has called AudioManager.setMode( IN_COMMUNICATION) and, even then, only the earpiece/BT headset get audio (not the loud speaker). Result: android→mac calls worked because mac had a normal media output, but mac→android calls were silent even though packets flowed through the relay just fine. Switch to Usage::Media + ContentType::Speech so Oboe routes to the loud speaker and uses the media volume slider. A later polish step will wire setMode + setSpeakerphoneOn from MainActivity.kt so we can go back to VoiceCommunication for AEC and proximity-sensor routing. Plus: heartbeat tracing every 2s in the send/recv tasks — frames_sent, last_rms, last_pkt_bytes, short_reads on the send side; decoded_frames, last_decode_n, last_written, decode_errs on the recv side. Will make the next "no sound" regression trivial to localize. |
||
|
Siavash Sameni
|
49f101d785 |
fix(android): reuse signal endpoint for direct-call media connection
Direct-call accept hangs forever at the QUIC handshake on Android. Logs
from
|
||
|
Siavash Sameni
|
b35a6b7d92 |
fix(wzp-native): copy WzpOboeRings by value, not by pointer
PlayoutCallback::onAudioReady crashed with SIGSEGV(SEGV_ACCERR) on the
first AAudio callback because g_rings was a `const WzpOboeRings*` pointing
at the caller's stack frame. wzp_native_audio_start() constructs the
rings struct as a stack local in Rust, passes &rings to wzp_oboe_start
(which stored the raw pointer), and returns — at which point the stack
frame unwinds and g_rings becomes a dangling reference. The first audio
callback then read from freed memory and died.
- g_rings is now a static WzpOboeRings value (was `const WzpOboeRings*`).
The raw int16 buffer + atomic index pointers inside the struct still
point into the Rust-owned AudioBackend singleton, which is leaked for
the lifetime of the process, so deep-copying the struct by value is
safe and keeps the inner pointers valid forever.
- g_rings_valid atomic bool gates the audio-callback reads: set to true
after the value copy in wzp_oboe_start, cleared in wzp_oboe_stop BEFORE
the streams are torn down so any in-flight callback sees "no backend"
and returns Stop instead of racing on g_rings.
- All g_rings->x accesses in the capture + playout callbacks switched to
g_rings.x (member-of-value).
Reproduced on Pixel 6 / Android 15 with build
|
||
|
Siavash Sameni
|
c769a476a2 |
phase 2(android): port Oboe C++ bridge + audio FFI into wzp-native
Now that Phase 1 proved the split-cdylib pipeline (build #37 launched cleanly with 'wzp-native dlopen OK: version=42 msg=...' in logcat), this commit brings the real audio code into wzp-native without ever touching the Tauri crate: - cpp/oboe_bridge.{h,cpp}, oboe_stub.cpp, getauxval_fix.c copied verbatim from crates/wzp-android/cpp/ (same files that work in the legacy wzp-android .so on this phone) - build.rs near-identical to crates/wzp-android/build.rs: clones google/oboe@1.8.1 into OUT_DIR, compiles oboe_bridge.cpp + all oboe source files as a single static lib with c++_shared linkage, emits -llog + -lOpenSLES. On non-android hosts it compiles just oboe_stub.cpp so `cargo check` works locally without an NDK. - Cargo.toml gets cc = "1" in [build-dependencies]. This is SAFE because wzp-native is a single-cdylib crate — crate-type is only ["cdylib"], no staticlib, so rust-lang/rust#104707 does not apply. - src/lib.rs extends the FFI surface with the real audio API: wzp_native_audio_start() -> i32 wzp_native_audio_stop() wzp_native_audio_read_capture(*mut i16, usize) -> usize wzp_native_audio_write_playout(*const i16, usize) -> usize wzp_native_audio_capture_latency_ms() -> f32 wzp_native_audio_playout_latency_ms() -> f32 wzp_native_audio_is_running() -> i32 Plus a static AudioBackend singleton holding the two SPSC ring buffers (capture + playout) that are shared with the C++ Oboe callbacks via AtomicI32 cursors. The wzp_native_version() and wzp_native_hello() smoke tests from Phase 1 are preserved. Compiles cleanly on macOS host with the stub oboe .cpp. Next build will exercise the full cargo-ndk path inside docker to verify the whole Oboe compile still works standalone. Phase 3 (next commit): wzp-desktop engine.rs on Android calls wzp-native's audio FFI via the already-wired libloading handle, and the real CallEngine::start() is implemented for Android using the same codec/handshake/send/recv pipeline as desktop but with Oboe rings instead of CPAL rings. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
7cc53aedc7 |
refactor(android): split C++ into wzp-native cdylib, loaded at runtime
Phase 1 of the big refactor. Escape the Tauri Android __init_tcb+4 symbol leak (rust-lang/rust#104707) by making wzp-desktop's Android .so pure Rust — ZERO cc::Build, no cpp/ files, no C++ in the rustc link step. All future C++ (Oboe audio bridge) lives in a new standalone cdylib crate `wzp-native` which is built with cargo-ndk (the same path the legacy wzp-android crate uses successfully on the same phone + same NDK), copied into Tauri's gen/android/app/src/main/jniLibs at build time, and dlopened by wzp-desktop at runtime via libloading. Changes in this commit: - NEW crate crates/wzp-native/ with crate-type = ["cdylib"] only (no staticlib, no rlib — rust#104707 shows mixing staticlib with cdylib leaks non-exported symbols, which is the original bug source). Phase 1 scaffold has TWO extern "C" functions: wzp_native_version() -> i32 (returns 42) wzp_native_hello(buf, cap) -> usize (writes a string) So we can verify dlopen + dlsym + cross-.so FFI end-to-end before adding any real C++. - desktop/src-tauri/cpp/ directory DELETED (7 files gone). - desktop/src-tauri/build.rs reduced to just the git hash capture + tauri_build::build(). No more cc::Build of any kind. - desktop/src-tauri/Cargo.toml: drop cc from build-dependencies, add libloading = "0.8" as an Android-only runtime dep. - desktop/src-tauri/src/lib.rs Builder::setup() now (on Android only) dlopens libwzp_native.so, calls wzp_native_version() and wzp_native_hello(), and logs the result: "wzp-native dlopen OK: version=42 msg=\"hello from wzp-native\"" If this log appears in logcat when the app launches and the home screen still renders, the split-cdylib pipeline is validated and Phase 2 (port the Oboe bridge into wzp-native) can proceed. - scripts/build-tauri-android.sh: insert a `cargo ndk -t arm64-v8a build --release -p wzp-native` step before `cargo tauri android build`, with `-o desktop/src-tauri/gen/android/app/src/main/jniLibs` so the resulting libwzp_native.so lands in the place gradle will package into the final APK. - Workspace Cargo.toml: add crates/wzp-native to [workspace] members. Phase 2 (separate commit, only if Phase 1 works): - Copy cpp/oboe_bridge.{h,cpp} + getauxval_fix.c from the legacy wzp-android crate into crates/wzp-native/cpp/. - Add cc = "1" as a build-dependency on wzp-native (safe: it's a single-cdylib crate with no staticlib, so no symbol leak). - Add build.rs that compiles the Oboe C++ and the wzp-native Rust FFI exposes the audio start/stop/read/write functions. - wzp-desktop::engine.rs dlopens wzp-native at CallEngine::start, uses its audio functions instead of CPAL on Android. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
2288c1ae07 |
feat: direct calling UI for desktop Tauri app + merge android branch
Tauri backend: - register_signal: persistent _signal connection, presence registration - place_call: send DirectCallOffer by fingerprint - answer_call: accept/reject incoming calls - get_signal_status: poll signal state Frontend: - Mode toggle: "Room" vs "Direct Call" - Register button → registers on relay signal channel - Incoming call panel with Accept/Reject - Fingerprint input + Call button - Auto-connect to media room on CallSetup event Also merges feat/android-voip-client into desktop branch: - Federation fixes, time-based dedup, FEC stale blocks - Direct calling protocol types - ACL + SAS verification Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
5d8e743cbf |
feat: Android engine + Kotlin API for direct 1:1 calling
Rust engine: - start_signaling(): persistent _signal connection, presence registration - Signal recv loop: handles DirectCallOffer, CallRinging, CallSetup, Hangup - New CallState variants: Registered, Ringing, IncomingCall - Stats expose incoming_call_id, incoming_caller_fp, incoming_caller_alias, sas_code - New EngineCommands: PlaceCall, AnswerCall, RejectCall JNI bridge: - nativeStartSignaling(relay, seed, token, alias) - nativePlaceCall(targetFp) - nativeAnswerCall(callId, mode) Kotlin API (WzpEngine.kt): - startSignaling(relay, seed, token, alias) - placeCall(targetFingerprint) - answerCall(callId, mode) — 0=Reject, 1=AcceptTrusted, 2=AcceptGeneric Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
6694aebfd9 |
fix: resolve 0.0.0.0 to connectable address in CallSetup relay_addr
When relay listens on 0.0.0.0, derive the actual IP from the client's connection address for the CallSetup message. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
d27e85ecf2 |
feat: SAS (Short Authentication String) for call identity verification
Derive a 4-digit code from the shared DH secret via HKDF with label "warzone-sas-code". Both peers compute the same code; a MITM relay produces a different one. Users compare verbally during the call. - CryptoSession::sas_code() -> Option<u32> on the trait - ChaChaSession stores and returns the SAS - HKDF derivation in WarzoneKeyExchange::derive_session() - Tests: both peers match, MITM produces different code Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
39ac181d63 |
feat: ACL + capacity limit on call rooms, unified fingerprint format
- Call rooms (call-*) restricted to the two authorized participants only - Room capacity enforced at 2 for call rooms - Unauthorized clients get immediate connection close - Unified fingerprint format: SHA-256(Ed25519 pub)[:16] as xxxx:xxxx:... Used consistently in signal registration, handshake, and ACL checks Tested: Alice+Bob authorized, attacker rejected with "not authorized" Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
3351cb6473 |
feat: direct 1:1 calling via relay signaling (Phase 1)
New feature: call someone directly by fingerprint through the relay.
- Client connects with SNI "_signal" for persistent signaling
- RegisterPresence/RegisterPresenceAck for relay registration
- DirectCallOffer routed to target by fingerprint
- DirectCallAnswer with AcceptGeneric/AcceptTrusted/Reject modes
- Relay creates private room (call-{id}), sends CallSetup to both
- Both clients connect to private room for media (existing SFU path)
- Hangup forwarding + cleanup on disconnect
- Desktop CLI: --signal + --call <fingerprint> for testing
- CallRegistry tracks call state (Pending/Ringing/Active/Ended)
- SignalHub manages persistent signaling connections
Tested: Alice calls Bob by fingerprint, relay routes offer, Bob
auto-accepts, both join private room, media flows bidirectionally.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Siavash Sameni
|
1118eac752 |
fix: re-enable FEC + time-based dedup for federation
Restore fec_ratio=0.2 on GOOD profile. Time-based dedup (2s TTL) with payload hash prevents consecutive sender collisions while still catching multi-path duplicates. Verified: 6 consecutive senders across 2 relays, 0 decode errors, 0 drops, FEC active. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
f935bd69cd |
fix: rewrite seq/fec for federation-delivered packets
- Time-based dedup (2s TTL) replaces fixed-window dedup — consecutive senders with same seq numbers no longer collide - Raw byte forwarding for federation local delivery (no re-serialization) - Jitter buffer resets on large backward seq jumps (>100) - recv_media skips malformed datagrams instead of returning connection-closed - SIGTERM handler for clean QUIC shutdown on wzp-client - JSONL event log infrastructure (--event-log flag) for protocol analysis - FEC disabled on GOOD profile for federation debugging (fec_ratio=0.0) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
1c684f6b47 |
fix: rewrite seq/fec for federation-delivered packets
Federation media from different senders had conflicting seq numbers, FEC block IDs, and Opus decoder state. The relay now assigns fresh monotonic seq/fec_block/fec_symbol to all federation-delivered packets, ensuring clients see a clean continuous stream regardless of sender changes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
c92db7e9b7 |
fix: preserve original relay label through multi-hop presence propagation
When propagating GlobalRoomActive to other peers, use tagged participants (with relay_label set to the originating relay) instead of the raw untagged participants. This shows "Relay C" instead of "Relay B" when C's participants are forwarded through hub B to A. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
c3bd657224 |
fix: FEC decoder resets stale blocks — fixes consecutive federation connects
When a new sender reuses the same block_id values as a previous sender, the FEC decoder was silently dropping all data because blocks were marked as "already decoded". Now blocks older than 2 seconds are automatically reset when new data arrives for them. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
8b79cdc6fc |
fix: dedup filter collision between different senders + build scripts default --pull
- Dedup key now includes source peer fingerprint hash, preventing packets from different senders with same room+seq from being dropped as duplicates (was silently killing all multi-hop audio) - Build scripts default to --pull (use --no-pull to skip) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
2eab56beec |
fix: federation presence dedup, stale cleanup, and Android SIGSEGV crash
- Deduplicate remote participants by fingerprint in all merge sites (canonical == raw room name caused double-lookup, doubling every remote participant) - GlobalRoomInactive now propagates updated participant list to other peers (hub relay B was not informing A when C's participants left) - Add 15-second stale presence sweeper that purges remote participants from peers that stop sending data (safety net for QUIC timeout delays) - Add @Synchronized to WzpEngine.getStats/stopCall/destroy to prevent TOCTOU race between stats polling coroutine and engine teardown (SIGSEGV) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
7dadc1ddd6 |
fix: default room 'general', cap auto codec at 24k
- Android default room changed from 'android' to 'general' - Relay choose_profile capped at GOOD (Opus 24k) — studio tiers (32k/48k/64k) cause high packet loss on federation paths due to larger datagrams exceeding path MTU. Will re-enable after MTU discovery is implemented. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
28f4a0fb6f |
fix: multi-hop presence — propagate remote rooms on new peer connect
When a new federation link is established, announce not only LOCAL global rooms but also rooms from OTHER peers (remote_participants). This fixes multi-hop: when R2 connects to R3, R2 tells R3 about R1's rooms that R2 learned about earlier. Previously, only local rooms were announced on link setup. If R1 had a client but R2 had no clients, R2 wouldn't tell R3 about R1. Also added diagnostic logging for room announcements on link setup. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
3d76acf528 |
fix: multi-hop federation — hub relay forwards without local participants
Three fixes for 3-relay chain (R1→R2→R3): 1. Room lookup in handle_datagram: hub relay (R2) has no local participants, so active_rooms() was empty and datagrams were silently dropped. Now also checks global_rooms config directly, allowing hub relays to forward without local clients. 2. Multi-hop forwarding: removed active_rooms filter — forward to ALL connected peers except source. The receiving peer decides whether to deliver or forward further. 3. Android relay_label: native RoomMember now includes relay_label from RoomUpdate signal. Kotlin UI reads it for relay grouping. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
fc721c4217 |
fix: clear stale federated presence on GlobalRoomInactive
When a remote relay's room goes inactive (all participants left), the receiving relay now: 1. Clears remote_participants for that peer+room 2. Broadcasts updated RoomUpdate to local clients with the remote participant removed 3. Updates federation_active_rooms metric Previously, remote participants lingered in the participant list after disconnect, causing ghost entries and stale media forwarding. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
5c24adf1c1 |
feat: remote version query — wzp-client --version-check <relay>
Connects to a relay over QUIC with SNI "version", reads build hash
from a unidirectional stream, prints "<relay> <git-hash>" and exits.
Usage: wzp-client --version-check 172.16.81.175:4434
Output: 172.16.81.175:4434
|
||
|
Siavash Sameni
|
8dbda3e052 |
feat: --version flag with git hash + test script kill fix
wzp-relay --version prints "wzp-relay <short-git-hash>". Build hash also logged on startup: version=abc1234. Enables verifying deployed relay matches expected build. Also fixed federation-test.sh: use kill -INT (not SIGTERM) so clients save recordings before exit. Added save delay. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
54cb6c3b71 |
feat: relay_label in RoomParticipant + tagged remote participants
RoomParticipant.relay_label identifies which relay a participant is connected to. Local participants have None, federated participants get tagged with the peer relay's label when storing remote_participants. This enables clients to group participants by relay in the UI. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
da593f9510 |
feat: relay-grouped participant rendering + relay_label in protocol
RoomParticipant now has optional relay_label field. Desktop client groups participants by relay: "This Relay" (green dot) for local, peer label (blue dot) for federated. Shows all relays in the chain including intermediate ones. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
a3ebf5616f |
fix: unified raw room names + merged presence on join
1. CLI client now sends raw room names (no hash), matching Android JNI and Desktop Tauri. All three clients are now consistent. 2. When a client joins a global room, the relay merges federated remote participants into the initial RoomUpdate. Previously, clients that joined after the GlobalRoomActive signal only saw local participants. Now they see everyone immediately. 3. Added get_remote_participants() to FederationManager for querying cached remote participants from all peer links. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
ff6d0444c0 |
feat: federation Prometheus metrics — peer status, packets, active rooms
Wires up the existing RelayMetrics federation fields:
- wzp_federation_peer_status{peer} — 1=connected, 0=disconnected
- wzp_federation_packets_forwarded_total{peer,direction} — in/out counts
- wzp_federation_active_rooms — number of active federated rooms
These are critical for monitoring federation health and will feed into
the adaptive codec selection system (PRD-coordinated-codec.md).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Siavash Sameni
|
8080713098 |
feat: federated presence — RoomUpdate includes remote participants
GlobalRoomActive signal now carries participant list from the announcing relay. When received, the relay: 1. Stores remote participants per peer link 2. Broadcasts merged RoomUpdate to local clients (local + all remote) This means clients on different relays can now SEE each other in the participant list. Also fixes build: removed non-existent metric field references that were added by linter. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
e813362395 |
feat: federation metrics + dedup + rate limiting
Add Prometheus metrics for federation links (per-peer RTT, packet counters, active rooms gauge, dedup/rate-limit drop counters). Add dedup filter (4096-entry ring buffer) to drop duplicate packets arriving via multiple federation paths. Add per-room token bucket rate limiter (500 pps) to prevent amplification. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
d52b8befd6 |
fix: canonical room hash for federation — handles hashed vs raw room names
Different clients send different room names:
- Android: raw "general" as SNI
- Desktop: hash_room_name("general") = "f09ae11d..." as SNI
Federation datagrams are tagged with an 8-byte room hash. Previously,
each relay computed the hash from the client-provided room name,
causing mismatches between relays with different client types.
Fix: resolve_global_room() maps any room name (raw or hashed) to the
canonical [[global_rooms]] name. global_room_hash() always uses the
canonical name for federation hashing. handle_datagram uses both raw
and canonical hash matching to find the local room.
Also: run_participant now receives the pre-computed federation_room_hash
so the egress uses the canonical hash, not the client-specific name.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|