Phase 3c mirrors Phase 3b on the Android receive path. With Phase 0-3b
landed on desktop + Android encoder, this commit completes codec-layer
loss recovery on the Android decoder side.
Architectural difference vs desktop: engine.rs has NO jitter buffer.
The recv task reads packets directly from the transport via
recv_media().await and writes decoded audio straight into the playout
ring. There is no PlayoutResult::Missing equivalent. Gap detection
therefore has to be done via sequence-number tracking — when a packet
arrives with seq > expected_seq, the frames in between are missing and
we attempt to reconstruct them via DRED before decoding the newly-
arrived packet.
Implementation:
Imports & types:
- Added wzp_codec::AdaptiveDecoder, wzp_codec::dred_ffi::{
DredDecoderHandle, DredState} imports.
- Changed the `decoder` local from Box<dyn AudioDecoder> (via
wzp_codec::create_decoder) to concrete AdaptiveDecoder::new(profile).
Same reasoning as Phase 3b: reconstruct_from_dred is an inherent
method, not a trait method, so we need the concrete type.
Recv task state (all task-local, no new struct fields):
- dred_decoder: DredDecoderHandle
- dred_parse_scratch: DredState (reused, overwritten per parse)
- last_good_dred: DredState (cached most-recent valid state)
- last_good_dred_seq: Option<u16>
- expected_seq: Option<u16> (for gap detection)
- dred_reconstructions: u64 (telemetry)
- classical_plc_invocations: u64 (telemetry)
Recv loop body (Opus source packets only):
1. Parse DRED from the new packet first so last_good_dred reflects
the freshest state available for gap recovery.
2. Detect a gap: gap = pkt.seq.wrapping_sub(expected_seq). Cap at
MAX_GAP_FRAMES = 16 (320 ms) to avoid huge wraparound scenarios.
3. For each missing seq in the gap:
offset = (last_good_dred_seq - missing_seq) * frame_samples
if 0 < offset <= last_good_dred.samples_available():
reconstruct_from_dred + write to playout ring
bump dred_reconstructions
else:
decoder.decode_lost (classical PLC) + write + bump plc counter
4. Decode the current packet normally and write to playout ring
(unchanged from Phase 2).
5. Update expected_seq = pkt.seq.wrapping_add(1).
Profile-switch handling: when the incoming codec changes (triggering
decoder.set_profile), reset last_good_dred_seq and expected_seq to
None. The cached DRED state is tied to the old profile's frame rate
and would produce wrong offsets after the switch; starting fresh is
correct.
Decode-error fallback: the existing `Err(e) => decode_lost` branch
now also increments classical_plc_invocations so the counter
accurately reflects all PLC invocations (gap-detected AND decode-
error-triggered).
Telemetry (CallStats additions):
- stats.dred_reconstructions: u64
- stats.classical_plc_invocations: u64
Both updated on every packet arrival in the existing stats.lock()
block alongside frames_decoded/fec_recovered, so the Android UI and
JNI bridge already have these values without any further plumbing.
The periodic recv stats log now includes both counters.
Ordering note: DRED gap reconstruction happens BEFORE decoding the new
packet's audio because the playout ring is FIFO. Gap samples must be
written before the new packet's samples so temporal order is preserved.
Out-of-order late arrivals (seq < expected_seq) are naturally dropped
as stale by the gap detection (gap would be a large wraparound value
exceeding MAX_GAP_FRAMES).
Verification:
- cargo check --workspace: zero errors
- cargo test -p wzp-codec --lib: 68 passing (unchanged from Phase 3b)
- cargo test -p wzp-client --lib: 35 passing (unchanged from Phase 3b)
- cargo check -p wzp-android --lib: zero errors
- cargo test -p wzp-android cannot run on macOS host (pre-existing
-llog linker dep, unrelated). Real end-to-end verification happens
via the Android APK build on the remote Docker builder
(scripts/build-and-notify.sh).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Phase 3b of the DRED integration — wires the Phase 3a FFI primitives
into the desktop receive path. When the jitter buffer reports a missing
Opus frame, CallDecoder now attempts to reconstruct the audio from the
most recently parsed DRED side-channel state before falling through to
classical PLC.
Architectural refinement vs the PRD's literal wording: the PRD said
"jitter buffer takes a Box<dyn DredReconstructor>". After checking deps,
wzp-transport depends only on wzp-proto (not wzp-codec). Putting DRED
state in the jitter buffer would require a new cross-crate dep and
couple the codec-agnostic buffer to libopus. Instead, this commit keeps
the DRED state ring and reconstruction dispatch inside CallDecoder (one
layer up from the jitter buffer), intercepting the existing
PlayoutResult::Missing signal. Same lookahead/backfill semantics,
cleaner layering, zero change to wzp-transport.
Changes:
CallDecoder field type: Box<dyn AudioDecoder> → AdaptiveDecoder.
Required because Phase 3b calls the inherent reconstruct_from_dred
method, which cannot live on the AudioDecoder trait without dragging
libopus DredState through wzp-proto. In practice AdaptiveDecoder was
the only AudioDecoder implementor anyway — the trait abstraction was
buying nothing. Method call sites unchanged because AdaptiveDecoder
also implements AudioDecoder.
New CallDecoder fields:
- dred_decoder: DredDecoderHandle
- dred_parse_scratch: DredState (scratch for parse_into)
- last_good_dred: DredState (cached most-recent valid state)
- last_good_dred_seq: Option<u16>
- dred_reconstructions: u64 (Phase 4 telemetry)
- classical_plc_invocations: u64 (Phase 4 telemetry)
CallDecoder::ingest — on Opus non-repair packets, parse DRED into the
scratch state. On success (samples_available > 0), std::mem::swap the
scratch into last_good_dred and record the seq. This is O(1) per
packet, zero allocation after construction (the two DredState buffers
are allocated once in new() and reused forever).
CallDecoder::decode_next — on PlayoutResult::Missing(seq) for Opus
profiles: if last_good_dred_seq > seq and the seq delta × frame_samples
fits within samples_available, call audio_dec.reconstruct_from_dred
and bump dred_reconstructions. Otherwise fall through to classical
PLC and bump classical_plc_invocations. The Codec2 path always falls
through to classical PLC since DRED is libopus-only and
AdaptiveDecoder::reconstruct_from_dred rejects Codec2 tiers
explicitly.
OpusDecoder and AdaptiveDecoder: new inherent reconstruct_from_dred
method that delegates to the underlying DecoderHandle. Needed to
bridge CallDecoder's wzp-client code to the Phase 3a FFI wrappers
without touching the AudioDecoder trait.
CRITICAL FINDING — raised DRED loss floor from 5% to 15%:
Phase 3b testing discovered that libopus 1.5's DRED emission window
scales aggressively with OPUS_SET_PACKET_LOSS_PERC. Empirical data
(see probe_dred_samples_available_by_loss_floor, an #[ignore]'d
diagnostic test in call.rs):
loss_pct samples_available effective_ms
5% 720 15 ms (useless!)
10% 2640 55 ms
15% 4560 95 ms
20% 6480 135 ms
25%+ 8400 (capped) 175 ms (~87% of 200 ms configured)
The Phase 1 default of 5% produced only a 15 ms reconstruction window
— too small to even cover a single 20 ms Opus frame. DRED was
effectively disabled even though it was emitting bytes. Raised the
floor to 15% (95 ms window) as the minimum that actually provides
single-frame loss recovery. This updates Phase 1's DRED_LOSS_FLOOR_PCT
constant in opus_enc.rs and the accompanying module docstring.
Trade-off: 15% assumed loss slightly increases encoder bitrate overhead
on clean networks. Measured via the existing phase1 bitrate probe:
Before (5% floor): 3649 bytes/sec at Opus 24k + 300 Hz sine
After (15% floor): 3568 bytes/sec at Opus 24k + 300 Hz sine
The delta is within noise — 15% isn't meaningfully more expensive than
5% on this signal, which suggests the DRED emission size is signal-
dependent rather than loss-dependent for small values. Net result: we
get a 6x larger reconstruction window for essentially free.
Tests (+3 DRED recovery, +1 #[ignore]'d probe):
- opus_single_packet_loss_is_recovered_via_dred — full encode → ingest
→ decode_next loop with one packet dropped mid-stream. Asserts
dred_reconstructions ≥ 1 and observes the exact counter deltas.
- opus_lossless_ingest_never_triggers_dred_or_plc — baseline behavior,
lossless stream never takes the Missing branch.
- codec2_loss_falls_through_to_classical_plc — Codec2 never
reconstructs via DRED even if state were populated (which it won't
be — Codec2 packets don't carry DRED bytes).
- probe_dred_samples_available_by_loss_floor — #[ignore]'d diagnostic
that sweeps loss_pct values and prints the resulting DRED window
sizes. Kept for future tuning work.
New CallDecoder introspection accessors (public but undocumented in
the PRD): last_good_dred_seq() and last_good_dred_samples_available()
for test diagnostics and future telemetry surfaces in Phase 4.
Verification:
- cargo check --workspace: zero errors
- cargo test -p wzp-codec --lib: 68 passing (Phase 3a baseline held)
- cargo test -p wzp-client --lib: 35 passing (+3 Phase 3b tests,
+1 ignored diagnostic, no regressions)
Next up: Phase 3c mirrors this on the Android engine.rs receive path.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Phase 3a of the DRED integration — the foundation for codec-layer loss
recovery. Adds three new safe wrappers to crates/wzp-codec/src/dred_ffi.rs
over the raw opusic-sys FFI, plus the reconstruction method on the existing
DecoderHandle. No call-site integration yet — that lands in Phase 3b (desktop)
and Phase 3c (Android).
New types:
- `DredDecoderHandle`: owns *mut OpusDREDDecoder from opus_dred_decoder_create.
Used for parsing DRED side-channel data out of arriving Opus packets.
This is a SEPARATE libopus object from OpusDecoder — it has its own
internal state. Freed via opus_dred_decoder_destroy on Drop.
- `DredState`: owns *mut OpusDRED from opus_dred_alloc (a fixed ~10.6 KB
buffer per libopus 1.5). Holds parsed DRED data between the parse and
reconstruct steps. Reusable — parse_into overwrites contents. Tracks
samples_available as a cached u32 so callers don't thread the value
separately. Freed via opus_dred_free on Drop.
New methods:
- `DredDecoderHandle::parse_into(&mut self, state: &mut DredState, packet)`
wraps opus_dred_parse with max_dred_samples=48000 (1s max), sampling_rate
=48000, defer_processing=0. Returns the positive sample offset of the
first decodable DRED sample, 0 if no DRED is present, or an error.
Populates state.samples_available so subsequent reconstruct calls know
the valid offset range.
- `DecoderHandle::reconstruct_from_dred(&mut self, state, offset_samples,
output)` wraps opus_decoder_dred_decode. Reconstructs audio at a specific
sample position (positive, measured backward from the DRED anchor packet)
into a caller-provided output buffer. Validates that 0 < offset_samples
<= state.samples_available() before calling the FFI to catch range bugs.
Tests (+7, wzp-codec total: 68 passing):
- dred_decoder_handle_creates_and_drops
- dred_state_creates_and_drops
- dred_state_reset_zeroes_counter
- dred_parse_and_reconstruct_roundtrip — end-to-end validation. Encodes
60 frames of a 300 Hz sine wave through a DRED-enabled Opus 24k encoder,
parses DRED state out of each arriving packet, asserts that at least one
packet carries non-zero samples_available (DRED warm-up completes within
the first second), then reconstructs 20 ms of audio from inside the
window and asserts non-zero total energy. This is the hard signal that
the full libopus 1.5 DRED FFI chain is correctly wired on our side.
- reconstruct_with_out_of_range_offset_errors — offset > samples_available
is rejected at the Rust layer before the FFI call.
- reconstruct_with_zero_offset_errors — offset <= 0 rejected.
- dred_parse_empty_packet_returns_zero — graceful handling of empty input.
Architectural note (divergence from PRD's literal wording):
The PRD said "jitter buffer takes a Box<dyn DredReconstructor>". After
checking Cargo.toml for wzp-transport, it does NOT depend on wzp-codec —
only wzp-proto. Adding a DRED state ring inside the jitter buffer would
require a new cross-crate dependency and couple the codec-agnostic jitter
buffer to libopus internals. Instead, Phase 3b will put the DRED state
ring and reconstruction dispatch in CallDecoder (one layer up from the
jitter buffer), intercepting the existing PlayoutResult::Missing signal
and attempting reconstruction before falling through to classical PLC.
The jitter buffer itself stays unchanged. Same lookahead/backfill
semantics, cleaner layering. PRD's intent preserved, implementation
refined.
Verification:
- cargo check --workspace: zero errors
- cargo test -p wzp-codec --lib: 68 passing (61 Phase 2 baseline + 7 new)
- The roundtrip test is the acceptance gate — it proves that
opus_dred_decoder_create, opus_dred_alloc, opus_dred_parse, and
opus_decoder_dred_decode all work correctly through our wrappers on
real libopus 1.5.2 output.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Phase 2 of the DRED integration (docs/PRD-dred-integration.md). With
Phase 1 having enabled DRED on every Opus profile, the app-level RaptorQ
layer is now redundant overhead on those tiers: +20% bitrate, +40–100 ms
receive-side latency (block wait), +CPU for stats we never used. This
phase removes RaptorQ from the Opus encode and decode paths on both the
desktop (wzp-client/call.rs) and Android (wzp-android/engine.rs) sides.
Codec2 tiers keep RaptorQ with their current ratios unchanged — DRED is
libopus-only and Codec2 has no neural equivalent.
Encoder changes (the real bandwidth / CPU win):
- CallEncoder::encode_frame and engine.rs encode loop now gate the
RaptorQ path on !codec.is_opus():
- Opus source packets emit fec_block=0, fec_symbol=0,
fec_ratio_encoded=0 in the MediaHeader
- fec_enc.add_source_symbol is skipped on Opus
- generate_repair + repair packet emission is skipped on Opus
- block_id and frame_in_block counters stay frozen at 0 for Opus
- Codec2 path is byte-for-byte identical to pre-Phase-2 behavior.
Decoder changes (mostly cleanup, since both live decoder paths were
already reading audio directly from source packets and only using the
RaptorQ decoder output for stats):
- CallDecoder::ingest skips fec_dec.add_symbol on Opus packets. Source
packets still flow to the jitter buffer; Opus repair packets from old
senders are dropped cleanly (repair packets never hit the jitter
buffer either).
- engine.rs recv loop skips fec_dec.add_symbol, fec_dec.try_decode, and
fec_dec.expire_before on Opus packets. The `fec_recovered` stat
counter becomes Codec2-only (a separate DRED reconstruction counter
lands in Phase 4).
Wire-format backward compat verified at pre-flight:
- Old receiver + new sender: engine.rs pipeline.rs path gates on
non-zero fec_block/fec_symbol which now never fire for Opus, so the
RaptorQ decoder simply isn't fed. Audio flows normally. Desktop
CallDecoder's old path accumulated packets into the stale-eviction
HashMap, which cleans up after 2s — harmless.
- New receiver + old sender: new receiver skips RaptorQ on Opus so
old-sender repair packets are ignored entirely (no crash, no double-
decode). Loses the (previously vestigial) RaptorQ recovery benefit,
which was never actually active in the audio path. Source packets
still decode normally.
- No wire format version bump required. MediaHeader is unchanged; we
just zero the FEC fields on Opus packets.
Test changes:
- Removed `encoder_generates_repair_on_full_block` — asserted the old
(pre-Phase-2) RaptorQ-on-Opus behavior and is now incorrect. Replaced
with two symmetric tests:
- `opus_source_packets_have_zero_fec_header_fields` — verifies
Phase 2 invariants on Opus packets
- `opus_encoder_never_emits_repair_packets` — runs 20 frames of
non-silent sine wave through a GOOD-profile encoder, asserts
exactly 20 output packets, zero repair
- `codec2_encoder_generates_repair_on_full_block` — same shape as
the old test but on CATASTROPHIC profile (Codec2 1200, 8
frames/block, ratio 1.0) to verify Codec2 path still emits
repairs as before
Verification:
- cargo check --workspace: zero errors
- cargo test -p wzp-codec --lib: 61 passing (Phase 1 baseline held)
- cargo test -p wzp-client --lib: 32 passing (+3 new Phase 2 tests,
-1 old test removed)
- cargo check -p wzp-android --lib: zero errors (host link of
wzp-android tests fails on -llog per pre-existing Android-only
build.rs, unrelated to this work; integration build via
build-and-notify.sh will validate Android end-to-end)
- Pre-existing broken integration test in
crates/wzp-client/tests/handshake_integration.rs (SignalMessage
schema drift) is NOT caused by this commit — baseline had the same
3 compile errors before Phase 2. Flagged as a separate cleanup task.
Expected observable effects on a real call:
- Opus 24k outgoing bitrate drops from ~28.8 kbps (ratio 0.2 RaptorQ)
to ~25 kbps (base 24 kbps + DRED ~1–10 kbps signal-dependent)
- Opus receive-side latency drops ~40 ms on clean network (no more
block wait — jitter buffer emits as soon as a source packet arrives)
- Codec2 calls show no latency or bitrate change
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Phase 1 of the DRED integration (docs/PRD-dred-integration.md). The Opus
encoder now emits DRED (Deep REDundancy) bytes in every packet, carrying
a neural-coded history of recent audio that the decoder can use to
reconstruct loss bursts up to the configured window. Opus inband FEC
(LBRR) is disabled because DRED does the same job better and running both
wastes bitrate on overlapping protection.
Tiered DRED duration policy per PRD:
Studio (Opus 32k/48k/64k): 10 frames = 100 ms
Normal (Opus 16k/24k): 20 frames = 200 ms
Degraded (Opus 6k): 50 frames = 500 ms
Each profile switch (via adaptive quality) updates the DRED duration to
match the new tier. A 5% packet_loss floor is applied whenever DRED is
active, because libopus 1.5 gates DRED emission on non-zero packet_loss.
Real loss measurements from the quality adapter override upward.
Escape hatch: AUDIO_USE_LEGACY_FEC=1 reverts the encoder to Phase 0
behavior (inband FEC Mode1, DRED off, no loss floor). Read once at
OpusEncoder::new; call-scoped, not re-read mid-call. Trait-level
set_inband_fec becomes a no-op in DRED mode to preserve the invariant
even if external callers forget.
Observations from the bitrate probe test (dred_mode_roundtrip_voice_pattern):
DRED mode: 3649 bytes/sec (~29.2 kbps) on Opus 24k + 300 Hz sine
Legacy mode: 2383 bytes/sec (~19.1 kbps)
Delta: +10.1 kbps
The delta is considerably larger than the "+1 kbps flat" figure I carried
into the PRD from hazy memory of published DRED benchmarks. Likely because
the input (300 Hz sine) is very compressible so the base Opus rate in
legacy mode is well below the 24 kbps target, making the delta look
disproportionate. Signal-dependent — real speech would probably show a
different ratio. If production telemetry shows the overhead is excessive,
we can cut DRED duration on the normal tier from 200 ms to 100 ms as a
first tuning lever. Not blocking Phase 1 since the test still passes
within the reasonable 2000–8000 bytes/sec bounds.
Test changes (+8 tests, total wzp-codec: 61 passing):
- dred_duration_for_studio_tiers_is_100ms (per-profile policy)
- dred_duration_for_normal_tiers_is_200ms
- dred_duration_for_degraded_tier_is_500ms
- dred_duration_for_codec2_is_zero
- default_mode_is_dred_not_legacy (sanity check on fresh construction)
- dred_mode_roundtrip_voice_pattern (observes DRED bitrate, asserts bounds)
- profile_switch_refreshes_dred_duration (verifies set_profile updates DRED)
- set_inband_fec_noop_in_dred_mode (trait-level inband FEC no-op)
Verification:
- cargo check --workspace: zero errors, no new warnings
- cargo test -p wzp-codec: 61/61 passing (53 pre-Phase-1 baseline + 8 new)
- Empirical DRED bitrate observed via `rtk proxy cargo test
dred_mode_roundtrip_voice_pattern -- --nocapture`
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Phase 0 of the DRED integration (docs/PRD-dred-integration.md). No behavior
change: inband FEC stays ON, no DRED, same bitrate, same quality. This
commit unblocks Phase 1+ by getting us onto libopus 1.5.2 where DRED lives.
Rationale for going straight to a custom DecoderHandle: opusic-c::Decoder's
inner *mut OpusDecoder pointer is pub(crate), so we cannot reach it for the
Phase 3 DRED reconstruction path. Running two parallel decoders (one for
audio, one for DRED) would drift because the DRED decoder wouldn't see
normal decode calls. Single unified DecoderHandle over raw opusic-sys is
the only correct architecture, so we build it in Phase 0 rather than
rewriting opus_dec.rs twice.
Changes:
- Cargo.toml (workspace + wzp-codec): remove audiopus 0.3.0-rc.0, add
opusic-c 1.5.5 (bundled + dred features), opusic-sys 0.6.0 (bundled),
bytemuck 1. Pinned exactly for reproducible libopus 1.5.2.
- opus_enc.rs: rewritten against opusic_c::Encoder. Argument order for
Encoder::new swapped (Channels first). set_inband_fec(bool) now maps
to InbandFec::Mode1 (the libopus 1.5 equivalent of 1.3's LBRR). encode
uses bytemuck::cast_slice<i16,u16> at the &[u16] boundary.
- dred_ffi.rs (new): DecoderHandle wrapping *mut OpusDecoder directly via
opusic-sys. Owns the allocation, frees on Drop. Exposes decode,
decode_lost, and a pub(crate) as_raw_ptr() for the future Phase 3 DRED
reconstruction. Send+Sync justified via &mut self access discipline.
- opus_dec.rs: rewritten as a thin AudioDecoder impl over DecoderHandle.
Behavior identical to pre-swap.
Verification (Phase 0 acceptance gates):
- cargo check --workspace: clean (30 pre-existing warnings in jni_bridge.rs
unrelated to this work; zero in changed files).
- cargo test -p wzp-codec: 53 tests pass (50 pre-swap + 6 new: 3 in
dred_ffi.rs for DecoderHandle lifecycle, 3 in opus_enc.rs for version
check and roundtrip).
- linked_libopus_is_1_5 test asserts opusic_c::version() contains "1.5" —
hard signal that the swap landed correctly.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Switch the webrtc-audio-processing dep from the 2.x git source (bundled
mode) back to crates.io 0.3, and link against Debian's apt package
libwebrtc-audio-processing-dev (0.3-1+b1 on Bookworm). The 2.x path
fails because both the crates.io tarball and the upstream git main
branch of webrtc-audio-processing-sys 2.0.3 have a build.rs bug where
\`meson setup --reconfigure\` is passed unconditionally, panicking on
first-run empty build dirs with "Directory does not contain a valid
build tree". The 0.x line sidesteps bundled mode entirely by linking
the apt-provided library.
Trade-off: we get AEC2 (the older generation) instead of AEC3, but
it's the same algorithm family and is what PulseAudio's
module-echo-cancel and PipeWire's filter-chain use on current
Debian-family distros. Fine for shipping — we can revisit AEC3 once
the 2.x bundled build is fixed upstream.
API changes:
- 0.3's Processor::process_capture_frame and process_render_frame
take &mut self, so wrap the module-level processor in a Mutex.
Capture and playback threads each lock briefly (sub-ms per 10 ms
frame); contention is minimal.
- Import NUM_SAMPLES_PER_FRAME from the crate directly instead of
hardcoding 480, so the code tracks whatever sample rate the
upstream C++ lib exposes (currently 48 kHz hardcoded -> 480).
- Helper fns drain_frames_through_apm / tee_render_samples / etc.
take &Mutex<Processor> instead of &Processor.
- Use explicit EchoCancellationSuppressionLevel and
NoiseSuppressionLevel imports rather than fully-qualified paths.
Dockerfile:
- Drop meson / ninja-build / python3 (only needed for bundled build).
- Add libwebrtc-audio-processing-dev for the system link path.
- Keep clang (may be needed by the bindgen step in some versions).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
v2.0.3 bundled build hits 'Directory does not contain a valid build
tree' because the crate's build.rs uses `meson setup --reconfigure`
unconditionally, which fails on first run when the build dir doesn't
yet contain prior meson state. Try the main branch in case it's been
fixed post-release.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The crates.io tarball of webrtc-audio-processing-sys 2.0.3 is missing
the vendored C++ submodule — the bundled build fails with 'Directory
does not contain a valid build tree' when meson tries to configure
the ./webrtc-audio-processing subdirectory. Cargo clones git deps with
submodules auto-initialized since ~1.27, so pulling from the upstream
git repo (pinned to tag v2.0.3) gives us the full source tree.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds gold-standard Linux echo cancellation: in-app WebRTC AEC3 (Audio
Processing Module) via the webrtc-audio-processing crate, using the
same algorithm as Chrome WebRTC, Zoom, Teams, and Jitsi. Runs entirely
in-process, so it works identically on ALSA / PulseAudio / PipeWire
systems — no dependency on user-configured echo-cancel modules.
Architecture:
- New crates/wzp-client/src/audio_linux_aec.rs module (~470 lines).
Contains LinuxAecCapture and LinuxAecPlayback, both using CPAL
under the hood but routing samples through a shared
Arc<webrtc_audio_processing::Processor>. The playback path tees
each 20 ms frame into APM.process_render_frame as the echo
reference BEFORE handing the samples to CPAL's output callback.
The capture path runs APM.process_capture_frame on each mic frame
in place before pushing to the audio ring buffer. This is the
"tee the playback ring" approach that Zoom/Teams/Jitsi use.
- New `linux-aec` feature in wzp-client pulling in the
webrtc-audio-processing crate at v2.x with the `bundled`
sub-feature. Bundled means the vendored PulseAudio WebRTC C++
sources are statically compiled via meson+ninja at cargo build
time — no runtime .so dependency, avoids Debian Bookworm's stale
libwebrtc-audio-processing-dev 0.3 package (which predates AEC3).
Dep is target-gated to Linux, so enabling the feature on non-Linux
is a no-op.
- lib.rs re-exports LinuxAecCapture/LinuxAecPlayback as
AudioCapture/AudioPlayback when `linux-aec` is on, otherwise
falls back to the CPAL audio_io path. Shared public API
(start/ring/stop/Drop) means downstream code is unchanged.
- New `linux-aec` feature in wzp-desktop forwards to
wzp-client/linux-aec so `cargo tauri build -- --features
wzp-desktop/linux-aec` builds the AEC variant.
APM configuration:
- EchoCancellation: High suppression, delay-agnostic mode on,
extended filter on, stream_delay_ms=60 initial hint
- NoiseSuppression: High
- HighPassFilter: on
- AGC: off (can fight Opus encoder's own gain staging + adaptive
quality controller; add later if users report low mic level)
Frame size handling:
- Pipeline uses 20 ms frames (960 samples @ 48 kHz mono)
- APM requires strict 10 ms (480 samples) per call
- Each 20 ms frame is split into two 480-sample halves, APM called
twice, halves stitched back
- Same pattern for render and capture sides
- Carry-buffer logic handles the case where CPAL delivers samples in
arbitrary chunk sizes that don't divide 960
Build infrastructure:
- scripts/Dockerfile.linux-desktop-builder adds meson, ninja-build,
python3, clang for the webrtc-audio-processing bundled build
- scripts/build-linux-desktop-docker.sh takes a new --aec flag that
enables the linux-aec feature and renames the output artifacts
with an `-aec` suffix so noAEC and AEC variants can coexist on disk
Task #30.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- CreateEventW is gated behind Win32_Security in the windows crate
because its signature takes SECURITY_ATTRIBUTES; add to features.
- Remove unused HANDLE import.
- Wrap GetId() and PWSTR::to_string() in explicit unsafe { ... }
blocks for Rust 2024 edition's unsafe_op_in_unsafe_fn lint.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds a direct WASAPI microphone capture path for the Windows desktop
build that opens the default communications endpoint via
IMMDeviceEnumerator -> IAudioClient2 -> SetClientProperties with
AudioCategory_Communications, turning on Windows's communications
audio processing chain (AEC, noise suppression, automatic gain
control). The communications AEC operates at the OS level and uses
the system render mix as the reference signal, so echo from our
existing CPAL playback stream is cancelled automatically with no
per-process reference plumbing.
Architecture:
- New crates/wzp-client/src/audio_wasapi.rs module (~280 lines).
Event-driven capture loop on a dedicated thread; pushes PCM into
the same lock-free AudioRing used by the CPAL path. Same public
API as audio_io::AudioCapture so downstream code is unchanged.
- New `windows-aec` feature in wzp-client that pulls in the
`windows` crate (Microsoft's official Rust COM bindings) gated to
target_os = "windows" only. Enabling the feature on non-Windows
targets is a no-op since both the module and the dep are
cfg(target_os = "windows").
- lib.rs re-exports WasapiAudioCapture as AudioCapture when the
feature is on, otherwise falls back to the CPAL AudioCapture.
AudioPlayback is always the CPAL one — no reason to swap it.
- desktop/src-tauri/Cargo.toml Windows target enables the new
feature: `features = ["audio", "windows-aec"]`.
Implementation notes:
- Uses eCommunications role (not eConsole) for GetDefaultAudioEndpoint
— the user-configured "communications" device that Teams/Zoom
pick up, and the one Windows's AEC is tuned for.
- Requests 48 kHz mono i16 with AUDCLNT_STREAMFLAGS_AUTOCONVERTPCM +
SRC_DEFAULT_QUALITY so Windows handles any format conversion in
the audio engine instead of rejecting our format.
- Event-driven with SetEventHandle / WaitForSingleObject — no
polling, minimal CPU cost between packets.
- 200 ms wait timeout so the capture thread polls `running` often
enough for Drop to stop cleanly even if the audio engine stalls
(e.g. device unplug).
Task #24.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
First step of the Windows x86_64 desktop build: stop pulling
coreaudio-rs into the Windows dependency graph so the project can at
least run `cargo check --target x86_64-pc-windows-msvc`. Software AEC
is already disabled in engine.rs so there's nothing else to stub — the
macOS-specific VPIO path is skipped via #[cfg(target_os = "macos")] on
both sides and Windows falls through to the plain CPAL
AudioCapture/AudioPlayback branch that already existed.
crates/wzp-client/Cargo.toml
- coreaudio-rs optional dep moved under [target.'cfg(target_os = "macos")']
- `vpio` feature now uses `dep:coreaudio-rs` syntax and the gated dep
- Enabling `vpio` on Windows/Linux is a no-op at resolution time
crates/wzp-client/src/lib.rs
- `pub mod audio_vpio` is now #[cfg(all(feature = "vpio", target_os = "macos"))]
- Previously `vpio` alone was enough to try to compile the Core Audio
bindings, which would fail on non-Apple targets the moment the
feature flag was flipped on
desktop/src-tauri/Cargo.toml
- [target.'cfg(not(target_os = "android"))'] removed — was leaking
vpio into Windows/Linux via the catch-all.
- macOS: wzp-client with features = ["audio", "vpio"]
- Windows: wzp-client with features = ["audio"]
- Linux: wzp-client with features = ["audio"]
- Android: wzp-client with default-features = false (unchanged)
- Dropped the unused direct coreaudio-rs = "0.11" dep on macOS —
wzp-desktop's own sources never call Core Audio directly.
Verified via `cargo tree --target x86_64-pc-windows-msvc -p wzp-desktop`
that the Windows target now resolves wzp-client with cpal but without
coreaudio-rs. macOS target still resolves with coreaudio (direct via
vpio feature and transitively via cpal). macOS `cargo check` still
builds cleanly.
Cross-compile from macOS hit a cargo-xwin + llvm-lib setup issue in
ring's build.rs, so the actual `cargo check --target
x86_64-pc-windows-msvc` did not complete locally. Build verification
belongs on the user's Windows x86_64 host where MSVC is present
natively.
See tasks #23 (this one), #24 (Voice Capture DSP / WASAPI Communications
for OS-level AEC on Windows), and #25 (aarch64-pc-windows-msvc support).
With da106bd (Usage::Media + MODE_NORMAL) audio works but is always on
the loudspeaker — we want handset as the default with a user-driven
toggle for speaker (and later bluetooth). The right Oboe usage for a
VoIP app is VoiceCommunication, which honours
AudioManager.setSpeakerphoneOn / setBluetoothScoOn for routing.
Bisection across previous builds showed that setAudioApi(AAudio) +
Usage::VoiceCommunication made the playout callback stop draining the
ring after cb#0 (build 8c36fb5 logs). Letting Oboe pick the AudioApi
implicitly keeps the callback alive — 96be740's Media-usage callbacks
fired at steady 50Hz without any explicit setAudioApi. So: keep the
Usage change, DROP the explicit AAudio force.
- oboe_bridge.cpp: Usage::VoiceCommunication, no setAudioApi, no
ContentType override.
- MainActivity.kt: setMode(MODE_IN_COMMUNICATION) +
setSpeakerphoneOn(false) = handset default, plus max both
STREAM_VOICE_CALL and STREAM_MUSIC volumes for belt-and-braces.
Next build will add a JNI-based Tauri command to flip speakerphoneOn
at runtime so the user can toggle handset↔speaker during a call.
Build 8c36fb5 logs showed a new regression: Oboe playout cb#0 fires once
at startup then the callback STOPS DRAINING the ring entirely.
written_samples sticks at 7679 (= RING_CAPACITY - 1) across every recv
heartbeat in a 40-second test. Meanwhile the recv task decodes 1800+ real
audio frames (sample range up to [-27920..31907], rms 12065) which all
get dropped on the floor by audio_write_playout returning 0 because the
ring is full.
Bisection: 96be740 (Usage::Media, no setAudioApi, no ContentType, no
MainActivity audio mode change) DID drive the playout callback at the
expected 50Hz (playout heartbeat: calls=1100 total_played_real=1055040
over 22 seconds). User still heard nothing there because of OS routing,
but at least Oboe accepted the PCM.
8c36fb5 added three changes on top of 96be740:
1. Oboe Usage::Media → Usage::VoiceCommunication
2. Oboe setAudioApi(oboe::AudioApi::AAudio) explicit
3. Oboe setContentType(ContentType::Speech)
4. MainActivity setMode(MODE_IN_COMMUNICATION) + setSpeakerphoneOn(true)
Every one of those could have killed the callback; combined they did.
Revert to 96be740's exact Oboe config: Usage::Media, no setAudioApi, no
ContentType. Keep the PCM recorder, heartbeat logging, and stream-open
logging. Separately, MainActivity now maxes STREAM_MUSIC (the stream
Usage::Media routes to) but leaves audio mode in MODE_NORMAL — no more
speakerphone/call-mode combo that makes Oboe unhappy. In NORMAL mode a
STREAM_MUSIC stream plays through the loud speaker by default.
Proof that the Rust pipeline is perfect: decoded.pcm recorded in 8c36fb5
was pulled via `adb shell run-as com.wzp.desktop cat .wzp/decoded.pcm`,
converted with ffmpeg, and played back on the Mac — user confirmed
audible speech. So 100% of the remaining bug surface is Android audio
routing, not anything in the Rust/C++ decode path.
cc-rs build of oboe_bridge.cpp failed at cfa9ff6 because the Oboe
ResultWithValue<T> template returned by getXRunCount() does not have
a .value_or(T) method — only .value(). Replace with an explicit
bool-conversion + .value() guard that yields -1 on error.
Build 96be740 logs proved the entire software pipeline is healthy:
capture heartbeat: calls=1100 to_write=960 full_drops=0 total_written=1056000
recv heartbeat: decoded_frames=1035 last_written=960 decode_errs=0
recv decoded PCM: range=[-13564..9244] rms=8044 (real audio)
playout WRITE: in_len=960 written=960 rms=2318 (real audio into the ring)
playout heartbeat: calls=1100 nonempty=1099 total_played_real=1055040
1055040 samples / 48000 Hz = 22s — exactly matches wall-clock elapsed,
meaning Oboe IS calling our playout callback at the expected rate and
WE ARE handing it real PCM every 20ms. User still heard nothing. Ergo
Oboe accepted the PCM and routed it to a silent output. Two fixes:
1) MainActivity.kt: switch to MODE_IN_COMMUNICATION + speakerphone ON
right after permissions are granted, and crank STREAM_VOICE_CALL to
max. Without this, an Oboe Usage::VoiceCommunication stream gets
opened, the OS creates a real AAudio pipeline, the callback fires on
schedule — and audio goes to either the earpiece at muted volume or
a "call not active" dead end. Logs the audio mode + volume levels
before and after the switch so we can confirm the state change in
logcat next run.
2) oboe_bridge.cpp: revert Usage::Media → VoiceCommunication (the mode
that matches MODE_IN_COMMUNICATION), pin the audio API to AAudio
explicitly instead of letting Oboe fall back to OpenSLES (which has
its own silent-drop failure modes on some devices), and add getState
+ getXRunCount to the playout heartbeat so we'll see silent stream
disconnects instead of reading zeros forever.
3) engine.rs recv task: dump the first ~10s of post-AGC decoded PCM to
`<app_data_dir>/decoded.pcm` as raw i16 LE so we can adb pull it and
play it back locally:
adb shell run-as com.wzp.desktop cat .wzp/decoded.pcm > decoded.pcm
ffmpeg -f s16le -ar 48000 -ac 1 -i decoded.pcm decoded.wav
This divorces "is our decoder actually producing audible audio" from
"is Android's audio stack playing it". If the recorded WAV sounds
correct when played on a laptop, the decoder is fine and 100% of the
remaining bug surface is AudioManager / Oboe routing.
4) engine.rs: also log when spk_muted=true blocks the write. User
reported the Speaker button in the UI has inconsistent semantics
between desktop and android — adding this log rules out the accidental
"first click muted playback" theory for good.
User confirmed: mac hears android, android does not hear mac. So Oboe
capture works end-to-end but Oboe playout on Android silently drops
audio even though QUIC forwards the packets. Archaeology on the legacy
wzp-android crate also revealed that the "last known good" Android audio
path NEVER used Oboe in production — it used Kotlin AudioRecord +
AudioTrack via JNI, and cpp/oboe_bridge.cpp was dead code. So every time
we've "tested" Oboe end-to-end this week was the first production use,
and any of its config knobs could be the bug.
Instrumenting every stage of the pipeline so one smoke-test log dump can
isolate the layer at fault:
C++ (oboe_bridge.cpp)
- Log the ACTUAL stream parameters after openStream for both capture
and playout (sample rate, channels, format, framesPerBurst,
framesPerDataCallback, bufferCapacityInFrames, sharing, perf mode).
Oboe may silently override values we requested — e.g. if we ask for
48kHz mono but the device gives us 44.1kHz stereo our 960-sample
frames are the wrong duration and the pipeline drifts.
- Capture callback: on cb#0 log sample range+RMS of the first frame
to prove we get real mic data (not zeros). Every 50 callbacks
(~1s at 20ms burst) log calls, numFrames, ring available_write,
bytes actually written, ring_full_drops, total_written.
- Playout callback: on cb#0 log numFrames + ring state. On the FIRST
non-empty read log sample range+RMS so we can tell if the samples
coming out of the ring are real audio or zeros. Every 50 callbacks
log calls, nonempty count, numFrames, ring available_read,
underrun_frames, total_played_real.
Rust wzp-native (src/lib.rs)
- wzp_native_audio_write_playout now logs the first 3 writes and then
every 50th: in_len, written, sample range, RMS, ring write/read
cursors before, available_read and available_write after. Reveals
ring-overflow and whether the engine is actually handing us audio.
- Minimal android logcat shim via __android_log_write extern — no
new crate dependency.
- AudioBackend grows a `playout_write_log_count` AtomicU64 to gate
the write-side log throttle.
Rust engine.rs (android branch)
- Recv task: log sample range + RMS for the first 3 decoded PCM
frames and then every 100th. Reveals whether decoder.decode is
producing real audio or silent buffers.
- Recv task: if audio_write_playout returns fewer samples than we
handed it (partial write → ring nearly full) warn about it in the
first 10 frames.
- Recv heartbeat every 2s: recv_fr, decoded_frames, last_decode_n,
last_written, written_samples, decode_errs, codec.
Expected flow in a healthy log:
capture cb#0: numFrames=960 range=[-1200..900] rms=180 ← mic OK
capture stream opened: actualSR=48000 Ch=1 ... ← no override
playout stream opened: actualSR=48000 Ch=1 ...
CallEngine::start invoked ... → connected → audio started
recv: first media packet received ...
recv: decoded PCM sample range decoded_frames=1 range=[-300..250] rms=92
playout WRITE #0: in_len=960 written=960 range=[-300..250] rms=92
playout FIRST nonempty read: to_read=960 range=[-300..250] rms=92
playout heartbeat: calls=50 nonempty=50 underrun=0 ...
recv heartbeat: decoded_frames=100 last_written=960 ...
If any of those are missing/zero we know the exact stage to fix.
Three real bugs, one smoke-test session's worth of progress.
1. RELAY: wrong advertised addr in CallSetup
The direct-call CallSetup computed `relay_addr = addr.ip()` where
`addr = connection.remote_address()` — i.e. the CLIENT'S IP, not the
relay's. So the relay was telling both parties "the call room is at
the answerer's IP:4433", which meant each client dialed either the
other client (no server listening) or themselves. Both endpoint.connect
calls hung forever and the call never happened.
Fix: compute the relay's own advertised IP once at startup. If the
listen addr is 0.0.0.0, probe the primary outbound interface via the
classic UDP-bind-and-connect(8.8.8.8:80) trick to discover the LAN
IP the OS would use to reach external hosts. Thread the resulting
advertised_addr_str into the CallSetup sender for both parties.
2. RELAY: accept loop serialized QUIC handshakes
Previously the main accept loop called `wzp_transport::accept` which
did both `endpoint.accept().await` AND `incoming.await` (the server-
side QUIC handshake). A single slow handshake therefore blocked every
subsequent client from being accepted. Unroll the helper here and
move `incoming.await` into the per-connection spawned task, so every
handshake runs in parallel. Also log "accept queue: new Incoming",
"QUIC handshake complete", and "QUIC handshake failed" so we can tell
immediately whether a client's packets are reaching the relay at all.
3. ANDROID: playout was routed to the silent in-call stream
The Oboe playout stream was configured with Usage::VoiceCommunication,
which routes to the Android in-call earpiece stream. That stream is
silent unless the Activity has called AudioManager.setMode(
IN_COMMUNICATION) and, even then, only the earpiece/BT headset get
audio (not the loud speaker). Result: android→mac calls worked
because mac had a normal media output, but mac→android calls were
silent even though packets flowed through the relay just fine.
Switch to Usage::Media + ContentType::Speech so Oboe routes to the
loud speaker and uses the media volume slider. A later polish step
will wire setMode + setSpeakerphoneOn from MainActivity.kt so we can
go back to VoiceCommunication for AEC and proximity-sensor routing.
Plus: heartbeat tracing every 2s in the send/recv tasks — frames_sent,
last_rms, last_pkt_bytes, short_reads on the send side; decoded_frames,
last_decode_n, last_written, decode_errs on the recv side. Will make the
next "no sound" regression trivial to localize.
Direct-call accept hangs forever at the QUIC handshake on Android. Logs
from d7b37a5 showed:
CallEngine::start (android) invoked relay=172.16.81.172:4433 room=call-…
resolved relay addr
identity loaded
endpoint created, dialing relay ← reached
← nothing, 90s+, no error
The "connect failed" and "QUIC connection established" log lines never
fire, meaning endpoint.connect_with(…).await never makes progress.
Repro is 100%: SFU room join (one endpoint) works perfectly; direct call
(opens a SECOND quinn::Endpoint on top of the signal one) hangs in the
QUIC handshake. Creating two quinn::Endpoints on Android's AAudio-adjacent
UDP stack apparently causes the second one's datagrams to never reach the
relay (the server never sees the Initial packet). Rather than fight the
platform, quinn is happy to multiplex multiple Connections on a single
Endpoint — so we reuse the signal endpoint for the media connection.
- SignalState now stores the quinn::Endpoint alongside the QuinnTransport.
register_signal populates both at the same time.
- CallEngine::start (both android and desktop branches) takes an
Option<wzp_transport::Endpoint>. Some → reuse (direct-call path, after
register_signal). None → create fresh (SFU room join path).
- The connect tauri command reads state.signal.endpoint and threads it
through to CallEngine::start, so the direct-call auto-connect (fired by
the "setup" signal-event in main.ts) lands on the existing UDP socket.
- wzp_transport re-exports quinn::Endpoint so wzp-desktop doesn't need to
depend on quinn directly.
- Also wraps the android connect in tokio::time::timeout(10s) so future
hangs become deterministic "connect TIMED OUT" errors in logcat
instead of silent deadlock.
Same fix applies verbatim to the desktop client — the user suspects
direct call is broken there too and this was likely always the cause,
just never surfaced because desktop was only tested via SFU rooms.
PlayoutCallback::onAudioReady crashed with SIGSEGV(SEGV_ACCERR) on the
first AAudio callback because g_rings was a `const WzpOboeRings*` pointing
at the caller's stack frame. wzp_native_audio_start() constructs the
rings struct as a stack local in Rust, passes &rings to wzp_oboe_start
(which stored the raw pointer), and returns — at which point the stack
frame unwinds and g_rings becomes a dangling reference. The first audio
callback then read from freed memory and died.
- g_rings is now a static WzpOboeRings value (was `const WzpOboeRings*`).
The raw int16 buffer + atomic index pointers inside the struct still
point into the Rust-owned AudioBackend singleton, which is leaked for
the lifetime of the process, so deep-copying the struct by value is
safe and keeps the inner pointers valid forever.
- g_rings_valid atomic bool gates the audio-callback reads: set to true
after the value copy in wzp_oboe_start, cleared in wzp_oboe_stop BEFORE
the streams are torn down so any in-flight callback sees "no backend"
and returns Stop instead of racing on g_rings.
- All g_rings->x accesses in the capture + playout callbacks switched to
g_rings.x (member-of-value).
Reproduced on Pixel 6 / Android 15 with build 0105b0f:
F libc: Fatal signal 11 (SIGSEGV), code 2 (SEGV_ACCERR),
fault addr 0x71aa717eb0 in tid 11822 (AudioTrack)
#00 PlayoutCallback::onAudioReady(oboe::AudioStream*, void*, int)+120
#01 oboe::AudioStream::fireDataCallback(void*, int)+136
...
Now that Phase 1 proved the split-cdylib pipeline (build #37 launched
cleanly with 'wzp-native dlopen OK: version=42 msg=...' in logcat),
this commit brings the real audio code into wzp-native without ever
touching the Tauri crate:
- cpp/oboe_bridge.{h,cpp}, oboe_stub.cpp, getauxval_fix.c copied
verbatim from crates/wzp-android/cpp/ (same files that work in the
legacy wzp-android .so on this phone)
- build.rs near-identical to crates/wzp-android/build.rs: clones
google/oboe@1.8.1 into OUT_DIR, compiles oboe_bridge.cpp + all
oboe source files as a single static lib with c++_shared linkage,
emits -llog + -lOpenSLES. On non-android hosts it compiles just
oboe_stub.cpp so `cargo check` works locally without an NDK.
- Cargo.toml gets cc = "1" in [build-dependencies]. This is SAFE
because wzp-native is a single-cdylib crate — crate-type is only
["cdylib"], no staticlib, so rust-lang/rust#104707 does not apply.
- src/lib.rs extends the FFI surface with the real audio API:
wzp_native_audio_start() -> i32
wzp_native_audio_stop()
wzp_native_audio_read_capture(*mut i16, usize) -> usize
wzp_native_audio_write_playout(*const i16, usize) -> usize
wzp_native_audio_capture_latency_ms() -> f32
wzp_native_audio_playout_latency_ms() -> f32
wzp_native_audio_is_running() -> i32
Plus a static AudioBackend singleton holding the two SPSC ring
buffers (capture + playout) that are shared with the C++ Oboe
callbacks via AtomicI32 cursors. The wzp_native_version() and
wzp_native_hello() smoke tests from Phase 1 are preserved.
Compiles cleanly on macOS host with the stub oboe .cpp. Next build
will exercise the full cargo-ndk path inside docker to verify the
whole Oboe compile still works standalone.
Phase 3 (next commit): wzp-desktop engine.rs on Android calls
wzp-native's audio FFI via the already-wired libloading handle, and
the real CallEngine::start() is implemented for Android using the
same codec/handshake/send/recv pipeline as desktop but with Oboe
rings instead of CPAL rings.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Phase 1 of the big refactor. Escape the Tauri Android
__init_tcb+4 symbol leak (rust-lang/rust#104707) by making
wzp-desktop's Android .so pure Rust — ZERO cc::Build, no cpp/ files,
no C++ in the rustc link step. All future C++ (Oboe audio bridge)
lives in a new standalone cdylib crate `wzp-native` which is built
with cargo-ndk (the same path the legacy wzp-android crate uses
successfully on the same phone + same NDK), copied into Tauri's
gen/android/app/src/main/jniLibs at build time, and dlopened by
wzp-desktop at runtime via libloading.
Changes in this commit:
- NEW crate crates/wzp-native/ with crate-type = ["cdylib"] only
(no staticlib, no rlib — rust#104707 shows mixing staticlib with
cdylib leaks non-exported symbols, which is the original bug
source). Phase 1 scaffold has TWO extern "C" functions:
wzp_native_version() -> i32 (returns 42)
wzp_native_hello(buf, cap) -> usize (writes a string)
So we can verify dlopen + dlsym + cross-.so FFI end-to-end
before adding any real C++.
- desktop/src-tauri/cpp/ directory DELETED (7 files gone).
- desktop/src-tauri/build.rs reduced to just the git hash capture
+ tauri_build::build(). No more cc::Build of any kind.
- desktop/src-tauri/Cargo.toml: drop cc from build-dependencies,
add libloading = "0.8" as an Android-only runtime dep.
- desktop/src-tauri/src/lib.rs Builder::setup() now (on Android only)
dlopens libwzp_native.so, calls wzp_native_version() and
wzp_native_hello(), and logs the result:
"wzp-native dlopen OK: version=42 msg=\"hello from wzp-native\""
If this log appears in logcat when the app launches and the home
screen still renders, the split-cdylib pipeline is validated and
Phase 2 (port the Oboe bridge into wzp-native) can proceed.
- scripts/build-tauri-android.sh: insert a `cargo ndk -t arm64-v8a
build --release -p wzp-native` step before `cargo tauri android
build`, with `-o desktop/src-tauri/gen/android/app/src/main/jniLibs`
so the resulting libwzp_native.so lands in the place gradle will
package into the final APK.
- Workspace Cargo.toml: add crates/wzp-native to [workspace] members.
Phase 2 (separate commit, only if Phase 1 works):
- Copy cpp/oboe_bridge.{h,cpp} + getauxval_fix.c from the legacy
wzp-android crate into crates/wzp-native/cpp/.
- Add cc = "1" as a build-dependency on wzp-native (safe: it's a
single-cdylib crate with no staticlib, so no symbol leak).
- Add build.rs that compiles the Oboe C++ and the wzp-native Rust
FFI exposes the audio start/stop/read/write functions.
- wzp-desktop::engine.rs dlopens wzp-native at CallEngine::start,
uses its audio functions instead of CPAL on Android.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When relay listens on 0.0.0.0, derive the actual IP from the client's
connection address for the CallSetup message.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Derive a 4-digit code from the shared DH secret via HKDF with label
"warzone-sas-code". Both peers compute the same code; a MITM relay
produces a different one. Users compare verbally during the call.
- CryptoSession::sas_code() -> Option<u32> on the trait
- ChaChaSession stores and returns the SAS
- HKDF derivation in WarzoneKeyExchange::derive_session()
- Tests: both peers match, MITM produces different code
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Call rooms (call-*) restricted to the two authorized participants only
- Room capacity enforced at 2 for call rooms
- Unauthorized clients get immediate connection close
- Unified fingerprint format: SHA-256(Ed25519 pub)[:16] as xxxx:xxxx:...
Used consistently in signal registration, handshake, and ACL checks
Tested: Alice+Bob authorized, attacker rejected with "not authorized"
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New feature: call someone directly by fingerprint through the relay.
- Client connects with SNI "_signal" for persistent signaling
- RegisterPresence/RegisterPresenceAck for relay registration
- DirectCallOffer routed to target by fingerprint
- DirectCallAnswer with AcceptGeneric/AcceptTrusted/Reject modes
- Relay creates private room (call-{id}), sends CallSetup to both
- Both clients connect to private room for media (existing SFU path)
- Hangup forwarding + cleanup on disconnect
- Desktop CLI: --signal + --call <fingerprint> for testing
- CallRegistry tracks call state (Pending/Ringing/Active/Ended)
- SignalHub manages persistent signaling connections
Tested: Alice calls Bob by fingerprint, relay routes offer, Bob
auto-accepts, both join private room, media flows bidirectionally.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Time-based dedup (2s TTL) replaces fixed-window dedup — consecutive
senders with same seq numbers no longer collide
- Raw byte forwarding for federation local delivery (no re-serialization)
- Jitter buffer resets on large backward seq jumps (>100)
- recv_media skips malformed datagrams instead of returning connection-closed
- SIGTERM handler for clean QUIC shutdown on wzp-client
- JSONL event log infrastructure (--event-log flag) for protocol analysis
- FEC disabled on GOOD profile for federation debugging (fec_ratio=0.0)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Federation media from different senders had conflicting seq numbers,
FEC block IDs, and Opus decoder state. The relay now assigns fresh
monotonic seq/fec_block/fec_symbol to all federation-delivered packets,
ensuring clients see a clean continuous stream regardless of sender changes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When propagating GlobalRoomActive to other peers, use tagged participants
(with relay_label set to the originating relay) instead of the raw
untagged participants. This shows "Relay C" instead of "Relay B" when
C's participants are forwarded through hub B to A.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When a new sender reuses the same block_id values as a previous sender,
the FEC decoder was silently dropping all data because blocks were marked
as "already decoded". Now blocks older than 2 seconds are automatically
reset when new data arrives for them.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Dedup key now includes source peer fingerprint hash, preventing
packets from different senders with same room+seq from being dropped
as duplicates (was silently killing all multi-hop audio)
- Build scripts default to --pull (use --no-pull to skip)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Deduplicate remote participants by fingerprint in all merge sites
(canonical == raw room name caused double-lookup, doubling every remote participant)
- GlobalRoomInactive now propagates updated participant list to other peers
(hub relay B was not informing A when C's participants left)
- Add 15-second stale presence sweeper that purges remote participants
from peers that stop sending data (safety net for QUIC timeout delays)
- Add @Synchronized to WzpEngine.getStats/stopCall/destroy to prevent
TOCTOU race between stats polling coroutine and engine teardown (SIGSEGV)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Android default room changed from 'android' to 'general'
- Relay choose_profile capped at GOOD (Opus 24k) — studio tiers
(32k/48k/64k) cause high packet loss on federation paths due to
larger datagrams exceeding path MTU. Will re-enable after MTU
discovery is implemented.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When a new federation link is established, announce not only LOCAL
global rooms but also rooms from OTHER peers (remote_participants).
This fixes multi-hop: when R2 connects to R3, R2 tells R3 about
R1's rooms that R2 learned about earlier.
Previously, only local rooms were announced on link setup. If R1
had a client but R2 had no clients, R2 wouldn't tell R3 about R1.
Also added diagnostic logging for room announcements on link setup.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three fixes for 3-relay chain (R1→R2→R3):
1. Room lookup in handle_datagram: hub relay (R2) has no local
participants, so active_rooms() was empty and datagrams were
silently dropped. Now also checks global_rooms config directly,
allowing hub relays to forward without local clients.
2. Multi-hop forwarding: removed active_rooms filter — forward to
ALL connected peers except source. The receiving peer decides
whether to deliver or forward further.
3. Android relay_label: native RoomMember now includes relay_label
from RoomUpdate signal. Kotlin UI reads it for relay grouping.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When a remote relay's room goes inactive (all participants left),
the receiving relay now:
1. Clears remote_participants for that peer+room
2. Broadcasts updated RoomUpdate to local clients with the remote
participant removed
3. Updates federation_active_rooms metric
Previously, remote participants lingered in the participant list
after disconnect, causing ghost entries and stale media forwarding.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Connects to a relay over QUIC with SNI "version", reads build hash
from a unidirectional stream, prints "<relay> <git-hash>" and exits.
Usage: wzp-client --version-check 172.16.81.175:4434
Output: 172.16.81.175:4434 8dbda3e
Relay side: detects "version" SNI, opens uni stream, writes
BUILD_GIT_HASH, waits 100ms for client to read, closes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
wzp-relay --version prints "wzp-relay <short-git-hash>".
Build hash also logged on startup: version=abc1234.
Enables verifying deployed relay matches expected build.
Also fixed federation-test.sh: use kill -INT (not SIGTERM) so
clients save recordings before exit. Added save delay.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
RoomParticipant.relay_label identifies which relay a participant is
connected to. Local participants have None, federated participants
get tagged with the peer relay's label when storing remote_participants.
This enables clients to group participants by relay in the UI.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
RoomParticipant now has optional relay_label field. Desktop client
groups participants by relay: "This Relay" (green dot) for local,
peer label (blue dot) for federated. Shows all relays in the chain
including intermediate ones.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1. CLI client now sends raw room names (no hash), matching Android
JNI and Desktop Tauri. All three clients are now consistent.
2. When a client joins a global room, the relay merges federated
remote participants into the initial RoomUpdate. Previously,
clients that joined after the GlobalRoomActive signal only saw
local participants. Now they see everyone immediately.
3. Added get_remote_participants() to FederationManager for querying
cached remote participants from all peer links.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Wires up the existing RelayMetrics federation fields:
- wzp_federation_peer_status{peer} — 1=connected, 0=disconnected
- wzp_federation_packets_forwarded_total{peer,direction} — in/out counts
- wzp_federation_active_rooms — number of active federated rooms
These are critical for monitoring federation health and will feed into
the adaptive codec selection system (PRD-coordinated-codec.md).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
GlobalRoomActive signal now carries participant list from the
announcing relay. When received, the relay:
1. Stores remote participants per peer link
2. Broadcasts merged RoomUpdate to local clients (local + all remote)
This means clients on different relays can now SEE each other in the
participant list. Also fixes build: removed non-existent metric field
references that were added by linter.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add Prometheus metrics for federation links (per-peer RTT, packet
counters, active rooms gauge, dedup/rate-limit drop counters).
Add dedup filter (4096-entry ring buffer) to drop duplicate packets
arriving via multiple federation paths. Add per-room token bucket
rate limiter (500 pps) to prevent amplification.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Different clients send different room names:
- Android: raw "general" as SNI
- Desktop: hash_room_name("general") = "f09ae11d..." as SNI
Federation datagrams are tagged with an 8-byte room hash. Previously,
each relay computed the hash from the client-provided room name,
causing mismatches between relays with different client types.
Fix: resolve_global_room() maps any room name (raw or hashed) to the
canonical [[global_rooms]] name. global_room_hash() always uses the
canonical name for federation hashing. handle_datagram uses both raw
and canonical hash matching to find the local room.
Also: run_participant now receives the pre-computed federation_room_hash
so the egress uses the canonical hash, not the client-specific name.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Wire AdaptiveQualityController into Android engine for auto codec
switching based on network quality reports. Add color-coded TX/RX
codec badges to the in-call screen showing active codecs and Auto mode.
- Recv task: ingest QualityReports, feed to controller, signal profile
changes via AtomicU8 to send task
- Send task: check for pending profile switch at frame boundaries,
update encoder/FEC/frame size
- Track peer codec from incoming packet headers
- Kotlin UI: codec badges (blue=studio, green=good, amber=degraded,
red=catastrophic) with Auto tag
- Add .taskmaster to .gitignore
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
