6805caae0e5cdbfa450ba2b2f75c77ffd56778bc
441 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Siavash Sameni
|
df1a45a5f5 |
fix(cli): port live mode to ring API (read_frame/write_frame removed)
AudioCapture and AudioPlayback no longer expose the old read_frame() and write_frame() methods — they were replaced with ring() returning &Arc<AudioRing> when the lock-free SPSC ring was introduced. The CLI live-mode loop still referenced the removed methods, which broke every workspace build that touched wzp-client bin (including the remote Linux x86_64 docker build). - Send loop: allocate a 960-sample scratch buffer, fill it in a loop via capture.ring().read() until a full 20 ms frame is available, sleep 2 ms between empty reads to avoid hot-spinning. - Recv loop: write decoded PCM into playback.ring() instead of calling write_frame(). Short writes on full ring drop the tail, which is the correct real-time behavior for CLI live mode. No behavioral change on the wire or in the call pipeline — this is purely a compile fix for cli.rs bitrot that accumulated since the ring API landed. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
dd0c714caa |
Revert "fix(deps): restore Cargo.lock from 8ceb6f4 — minimize dep drift from Phase 0"
This reverts commit
|
||
|
Siavash Sameni
|
a7b2f850f1 |
build(script): parametrize branch via WZP_BRANCH (default opus-DRED-v2)
The Linux build script was hardcoded to feat/android-voip-client, which is an older branch that doesn't have the current DRED work or the relay fixes from |
||
|
Siavash Sameni
|
575a39d07a |
fix(deps): restore Cargo.lock from 8ceb6f4 — minimize dep drift from Phase 0
Phase 0 cherry-pick regenerated the lockfile from scratch via
`cargo generate-lockfile`, which bumped at least tokio (1.50.0 → 1.51.1)
and downgraded the lockfile format from version 4 → version 3. Many
other transitive deps may have shifted silently.
Symptoms that pointed here:
1. Direct-call media QUIC handshake silently stalls for exactly the
client-side 10s timeout, with no errors in the log. Classic tokio
runtime / async waker mismatch — tasks queued from one runtime
never run because the endpoint's I/O driver is on another runtime.
2. Every `place_call` gets an immediate `signal: Hangup reason=Normal`
back from the signal recv loop, as if it's consuming stale state.
3. Eventually hits `FORTIFY: pthread_mutex_lock called on a destroyed
mutex` and the process dies.
All three are consistent with a tokio async primitive being shared
across runtimes in a way that tokio 1.51.1 handles differently than
1.50.0 (which was the version on the user's known-good build). Rather
than chase the specific bisection, restore the exact base lockfile
and let cargo add only the three deps Phase 0 actually needs
(opusic-c, opusic-sys, bytemuck).
Verification:
- `git diff 8ceb6f4..HEAD -- Cargo.lock | grep -c '^[+-]version = '` → 0
(no version-line changes beyond what Cargo auto-pulls for new crates)
- tokio back to 1.50.0
- rustls, quinn, quinn-proto, quinn-udp all unchanged
- Lockfile version restored to 4
- cargo test -p wzp-codec --lib: 69 passing (unchanged)
- cargo test -p wzp-client --lib: 35 passing + 1 ignored (unchanged)
Does not fix the pre-existing relay-side advertised-IP bug
(CallSetup may still contain a relay address that the callee cannot
reach from its network), but that is an orthogonal issue that existed
on
|
||
|
Siavash Sameni
|
d63d50cdc0 |
fix(build): remove apostrophe from libc++_shared comment (broke docker bash -c quoting)
Previous commit
|
||
|
Siavash Sameni
|
d269600aa7 |
fix(build): build-tauri-android.sh — copy libc++_shared.so into jniLibs
Root cause of "wzp-native not loaded" at runtime on opus-DRED-v2 APK:
libwzp_native.so has a NEEDED entry for libc++_shared.so (because
crates/wzp-native/build.rs uses cpp_link_stdlib(Some("c++_shared"))),
but the APK only contained:
lib/arm64-v8a/libwzp_desktop_lib.so (192 MB)
lib/arm64-v8a/libwzp_native.so (683 KB)
No libc++_shared.so → Android's dynamic linker fails the dlopen of
libwzp_native.so at runtime with "library libc++_shared.so not found",
and every audio path that routes through wzp_native (capture, playout,
register, direct call) refuses to start.
Diagnosis:
- readelf -d libwzp_native.so shows NEEDED libc++_shared.so
- python zipfile listing of the APK confirms libc++_shared.so is
absent from lib/arm64-v8a/
- scripts/build-and-notify.sh (the legacy wzp-android build path)
already had this fix at lines 126-134 with an explicit comment:
"cargo-ndk may not copy libc++_shared.so — grab it from the NDK if
missing". That fix was never ported to build-tauri-android.sh when
the Tauri mobile pipeline was set up.
Fix: after `cargo ndk build -p wzp-native --release` produces
libwzp_native.so into jniLibs, copy libc++_shared.so from the NDK
sysroot (same find pattern as build-and-notify.sh) into the same
jniLibs dir. Abort with a clear error if the NDK doesn't have the file.
Also noting the 191 MB vs 359 MB size discrepancy the user saw: that's
almost entirely libwzp_desktop_lib.so being a 192 MB debug build. The
old working APK was probably a release build (smaller main lib) or
included multiple arches (doubling/tripling the .so count). The size
is cosmetic — the crash is the real issue, and libc++_shared.so is
~2 MB so this fix doesn't close the size gap. Can investigate the
size difference separately after register + direct call work again.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Siavash Sameni
|
dfbe21fe6e |
feat(tauri-engine): Phase 3b/3c re-port — DRED reconstruction on the live Tauri mobile engine
The original Phase 3b landed on wzp-client/CallDecoder and Phase 3c
landed on wzp-android/src/engine.rs. Both of those are DEAD CODE on
feat/desktop-audio-rewrite: the legacy Kotlin app in android/app/ is
not built by the Tauri mobile pipeline, and the Tauri engine bypasses
CallDecoder by calling wzp_codec::create_decoder directly.
The live Android call engine lives at desktop/src-tauri/src/engine.rs
with two `pub async fn start<F>` functions — one cfg-gated on Android
(Oboe via wzp-native) and one for desktop (CPAL). Both recv tasks
were using `let mut decoder = wzp_codec::create_decoder(...)` which
returns `Box<dyn AudioDecoder>` and doesn't expose the inherent
`reconstruct_from_dred` method.
Changes:
New helper struct `DredRecvState` at the top of engine.rs, wrapping:
- DredDecoderHandle (libopus DRED side-channel parser)
- DredState scratch (for parse_into)
- DredState last_good (cached valid state, swapped on success)
- last_good_seq: Option<u16> (DRED anchor sequence)
- expected_seq: Option<u16> (for gap detection)
- dred_reconstructions / classical_plc_invocations counters
With three methods:
- ingest_opus(seq, payload): parse DRED, swap on success
- fill_gap_to(decoder, current_seq, frame_samples, scratch, emit):
detect gap back from expected_seq, reconstruct each missing
frame via DRED if state covers it, fall through to classical
decoder.decode_lost() when it doesn't. Calls emit() once per
frame with a slice the caller uses for AGC + playout write.
- reset_on_profile_switch(): invalidate tracking when codec changes
Both recv tasks (Android @ ~line 297 and desktop @ ~line 907):
- Decoder type changed from `Box<dyn AudioDecoder>` via
`wzp_codec::create_decoder` to concrete `AdaptiveDecoder::new(profile)`
so we can call the inherent reconstruct_from_dred method.
- Added `use wzp_proto::traits::AudioDecoder;` at the top of
engine.rs to bring decode/decode_lost/set_profile trait methods
into scope on the concrete type.
- New `current_profile` local alongside `current_codec` (used for
frame_duration lookups that drive the DRED sample offset math).
- On codec/profile switch, call dred_recv.reset_on_profile_switch()
because the cached DRED state is tied to the old profile's
frame rate.
- For each arriving Opus source packet:
1. dred_recv.ingest_opus(seq, payload) — parse DRED
2. dred_recv.fill_gap_to(...) — detect gap and reconstruct
missing frames, each emitted through a closure that does
AGC + playout write (wzp_native on Android, playout_ring
on desktop)
3. Normal decoder.decode() fallthrough for the current packet
(unchanged)
- Codec2 packets skip the DRED path entirely (is_opus() gate) —
libopus can't reconstruct Codec2 audio.
Ordering invariant: gap reconstruction writes to playout BEFORE the
current packet's decoded audio, preserving temporal order since the
playout ring is FIFO. The closure captures the `spk_muted` flag once
before the gap loop to avoid mid-gap-fill state changes.
Kept `crates/wzp-android/src/engine.rs` and `crates/wzp-android/src/
stats.rs` from the earlier Phase 3c commit as-is — they're dead code
on feat/desktop-audio-rewrite but harmless, and deleting them would
diverge this branch from an independently-useful intermediate state.
The old Phase 3c commit (
|
||
|
Siavash Sameni
|
b83c31b5d1 |
fix(android): remove duplicate TextAlign import in InCallScreen.kt
Pre-existing build breakage on feat/desktop-audio-rewrite @ |
||
|
Siavash Sameni
|
1f607281fd |
fix(build): build-and-notify.sh — parameterize branch, fail loud on pull errors
Same fix that landed on the old opus-DRED branch as
|
||
|
Siavash Sameni
|
7515417202 |
feat(telemetry): Phase 4 — LossRecoveryUpdate protocol + relay metrics + DebugReporter
Phase 4 lays the telemetry foundation for distinguishing DRED recoveries
from classical PLC in production: a new SignalMessage variant, two new
per-session Prometheus counters on the relay side, and a highlighted
loss-recovery section in the Android DebugReporter.
The periodic emitter (client → relay) and Grafana panel are deferred to
Phase 4b — this commit ships the protocol surface, the relay sink, and
the immediate user-visible debug output. Once 4b lands the full path
(emitter → relay → Prometheus → Grafana), the metrics here will
automatically start receiving data.
Scope decision — why not extend QualityReport instead:
The existing wire-format QualityReport is a fixed 4-byte media packet
trailer. Adding counter fields to it would shift the binary layout and
break backward compatibility (old receivers would parse the last 4
bytes of the extended trailer as QR, corrupting audio). Using a
new SignalMessage variant on the reliable QUIC signal stream sidesteps
the wire-format problem entirely — serde JSON enums tolerate unknown
variants gracefully on old receivers, and the signal channel is the
right layer for periodic telemetry aggregates.
Changes:
wzp-proto/src/packet.rs:
- New SignalMessage::LossRecoveryUpdate variant carrying:
* dred_reconstructions: u64 (monotonic since call start)
* classical_plc_invocations: u64 (monotonic)
* frames_decoded: u64 (for rate calculation)
- All three fields tagged #[serde(default)] for forward compat.
wzp-client/src/featherchat.rs:
- Added a match arm so signal_to_call_type() handles the new
variant (treat as Offer for featherChat bridging purposes).
wzp-relay/src/metrics.rs:
- Two new IntCounterVec metrics on the relay, labeled by session_id:
* wzp_relay_session_dred_reconstructions_total
* wzp_relay_session_classical_plc_total
- New method update_session_loss_recovery(session_id, dred, plc)
applies monotonic deltas: if the incoming totals exceed the
current counter, the difference is inc_by'd. If the incoming
totals are LOWER (client restart or counter reset), the
Prometheus counter holds steady until the client catches up.
This matches the existing update_session_buffer delta pattern.
- remove_session_metrics() now cleans up the two new labels.
- New test session_loss_recovery_monotonic_delta exercises:
* initial population (10 DRED, 2 PLC)
* forward advance (25, 5 → delta +15, +3)
* lower values ignored (client reset → counters unchanged)
* client catches up (30, 8 → advances to new max)
- Existing session_metrics_cleanup test extended to cover the
new counters.
android/app/src/main/java/com/wzp/debug/DebugReporter.kt:
- Phase 4 users — and incident responders — need to quickly see
whether DRED is actually firing during a call. The stats JSON
already carries the counters (after Phase 3c), but they were
buried in the trailing JSON dump. Added a dedicated
"=== Loss Recovery ===" section to the meta preamble that
extracts dred_reconstructions, classical_plc_invocations,
frames_decoded, and fec_recovered from the JSON and displays
them plainly, plus computed percentages when frames_decoded > 0.
- New extractLongField helper: tiny hand-rolled JSON integer
extractor. We don't want to pull in a full JSON parser for this
single use case and CallStats has a flat, well-known schema.
Verification:
- cargo check --workspace: zero errors
- cargo test -p wzp-proto --lib: 63 passing
- cargo test -p wzp-codec --lib: 68 passing
- cargo test -p wzp-client --lib: 35 passing (+1 ignored probe)
- cargo test -p wzp-relay --lib: 68 passing (+1 new Phase 4 test)
- cargo check -p wzp-android --lib: zero errors
- Android APK build verified earlier today (unridden-alfonso.apk
via the remote Docker builder) — Phase 0–3c confirmed to compile
end-to-end on the NDK target.
Phase 4b remaining (not blocking this commit):
- Periodic LossRecoveryUpdate emitter in wzp-client/src/call.rs and
wzp-android/src/engine.rs (every ~5 s)
- Relay-side handler in main.rs that matches the new variant and
calls metrics.update_session_loss_recovery
- Grafana "Loss recovery breakdown" panel in docs/grafana-dashboard.json
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Siavash Sameni
|
505a834c5b |
feat(codec): Phase 3c — Android engine.rs DRED reconstruction on packet loss
Phase 3c mirrors Phase 3b on the Android receive path. With Phase 0-3b
landed on desktop + Android encoder, this commit completes codec-layer
loss recovery on the Android decoder side.
Architectural difference vs desktop: engine.rs has NO jitter buffer.
The recv task reads packets directly from the transport via
recv_media().await and writes decoded audio straight into the playout
ring. There is no PlayoutResult::Missing equivalent. Gap detection
therefore has to be done via sequence-number tracking — when a packet
arrives with seq > expected_seq, the frames in between are missing and
we attempt to reconstruct them via DRED before decoding the newly-
arrived packet.
Implementation:
Imports & types:
- Added wzp_codec::AdaptiveDecoder, wzp_codec::dred_ffi::{
DredDecoderHandle, DredState} imports.
- Changed the `decoder` local from Box<dyn AudioDecoder> (via
wzp_codec::create_decoder) to concrete AdaptiveDecoder::new(profile).
Same reasoning as Phase 3b: reconstruct_from_dred is an inherent
method, not a trait method, so we need the concrete type.
Recv task state (all task-local, no new struct fields):
- dred_decoder: DredDecoderHandle
- dred_parse_scratch: DredState (reused, overwritten per parse)
- last_good_dred: DredState (cached most-recent valid state)
- last_good_dred_seq: Option<u16>
- expected_seq: Option<u16> (for gap detection)
- dred_reconstructions: u64 (telemetry)
- classical_plc_invocations: u64 (telemetry)
Recv loop body (Opus source packets only):
1. Parse DRED from the new packet first so last_good_dred reflects
the freshest state available for gap recovery.
2. Detect a gap: gap = pkt.seq.wrapping_sub(expected_seq). Cap at
MAX_GAP_FRAMES = 16 (320 ms) to avoid huge wraparound scenarios.
3. For each missing seq in the gap:
offset = (last_good_dred_seq - missing_seq) * frame_samples
if 0 < offset <= last_good_dred.samples_available():
reconstruct_from_dred + write to playout ring
bump dred_reconstructions
else:
decoder.decode_lost (classical PLC) + write + bump plc counter
4. Decode the current packet normally and write to playout ring
(unchanged from Phase 2).
5. Update expected_seq = pkt.seq.wrapping_add(1).
Profile-switch handling: when the incoming codec changes (triggering
decoder.set_profile), reset last_good_dred_seq and expected_seq to
None. The cached DRED state is tied to the old profile's frame rate
and would produce wrong offsets after the switch; starting fresh is
correct.
Decode-error fallback: the existing `Err(e) => decode_lost` branch
now also increments classical_plc_invocations so the counter
accurately reflects all PLC invocations (gap-detected AND decode-
error-triggered).
Telemetry (CallStats additions):
- stats.dred_reconstructions: u64
- stats.classical_plc_invocations: u64
Both updated on every packet arrival in the existing stats.lock()
block alongside frames_decoded/fec_recovered, so the Android UI and
JNI bridge already have these values without any further plumbing.
The periodic recv stats log now includes both counters.
Ordering note: DRED gap reconstruction happens BEFORE decoding the new
packet's audio because the playout ring is FIFO. Gap samples must be
written before the new packet's samples so temporal order is preserved.
Out-of-order late arrivals (seq < expected_seq) are naturally dropped
as stale by the gap detection (gap would be a large wraparound value
exceeding MAX_GAP_FRAMES).
Verification:
- cargo check --workspace: zero errors
- cargo test -p wzp-codec --lib: 68 passing (unchanged from Phase 3b)
- cargo test -p wzp-client --lib: 35 passing (unchanged from Phase 3b)
- cargo check -p wzp-android --lib: zero errors
- cargo test -p wzp-android cannot run on macOS host (pre-existing
-llog linker dep, unrelated). Real end-to-end verification happens
via the Android APK build on the remote Docker builder
(scripts/build-and-notify.sh).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Siavash Sameni
|
27bc264738 |
feat(codec): Phase 3b — CallDecoder DRED reconstruction on packet loss
Phase 3b of the DRED integration — wires the Phase 3a FFI primitives
into the desktop receive path. When the jitter buffer reports a missing
Opus frame, CallDecoder now attempts to reconstruct the audio from the
most recently parsed DRED side-channel state before falling through to
classical PLC.
Architectural refinement vs the PRD's literal wording: the PRD said
"jitter buffer takes a Box<dyn DredReconstructor>". After checking deps,
wzp-transport depends only on wzp-proto (not wzp-codec). Putting DRED
state in the jitter buffer would require a new cross-crate dep and
couple the codec-agnostic buffer to libopus. Instead, this commit keeps
the DRED state ring and reconstruction dispatch inside CallDecoder (one
layer up from the jitter buffer), intercepting the existing
PlayoutResult::Missing signal. Same lookahead/backfill semantics,
cleaner layering, zero change to wzp-transport.
Changes:
CallDecoder field type: Box<dyn AudioDecoder> → AdaptiveDecoder.
Required because Phase 3b calls the inherent reconstruct_from_dred
method, which cannot live on the AudioDecoder trait without dragging
libopus DredState through wzp-proto. In practice AdaptiveDecoder was
the only AudioDecoder implementor anyway — the trait abstraction was
buying nothing. Method call sites unchanged because AdaptiveDecoder
also implements AudioDecoder.
New CallDecoder fields:
- dred_decoder: DredDecoderHandle
- dred_parse_scratch: DredState (scratch for parse_into)
- last_good_dred: DredState (cached most-recent valid state)
- last_good_dred_seq: Option<u16>
- dred_reconstructions: u64 (Phase 4 telemetry)
- classical_plc_invocations: u64 (Phase 4 telemetry)
CallDecoder::ingest — on Opus non-repair packets, parse DRED into the
scratch state. On success (samples_available > 0), std::mem::swap the
scratch into last_good_dred and record the seq. This is O(1) per
packet, zero allocation after construction (the two DredState buffers
are allocated once in new() and reused forever).
CallDecoder::decode_next — on PlayoutResult::Missing(seq) for Opus
profiles: if last_good_dred_seq > seq and the seq delta × frame_samples
fits within samples_available, call audio_dec.reconstruct_from_dred
and bump dred_reconstructions. Otherwise fall through to classical
PLC and bump classical_plc_invocations. The Codec2 path always falls
through to classical PLC since DRED is libopus-only and
AdaptiveDecoder::reconstruct_from_dred rejects Codec2 tiers
explicitly.
OpusDecoder and AdaptiveDecoder: new inherent reconstruct_from_dred
method that delegates to the underlying DecoderHandle. Needed to
bridge CallDecoder's wzp-client code to the Phase 3a FFI wrappers
without touching the AudioDecoder trait.
CRITICAL FINDING — raised DRED loss floor from 5% to 15%:
Phase 3b testing discovered that libopus 1.5's DRED emission window
scales aggressively with OPUS_SET_PACKET_LOSS_PERC. Empirical data
(see probe_dred_samples_available_by_loss_floor, an #[ignore]'d
diagnostic test in call.rs):
loss_pct samples_available effective_ms
5% 720 15 ms (useless!)
10% 2640 55 ms
15% 4560 95 ms
20% 6480 135 ms
25%+ 8400 (capped) 175 ms (~87% of 200 ms configured)
The Phase 1 default of 5% produced only a 15 ms reconstruction window
— too small to even cover a single 20 ms Opus frame. DRED was
effectively disabled even though it was emitting bytes. Raised the
floor to 15% (95 ms window) as the minimum that actually provides
single-frame loss recovery. This updates Phase 1's DRED_LOSS_FLOOR_PCT
constant in opus_enc.rs and the accompanying module docstring.
Trade-off: 15% assumed loss slightly increases encoder bitrate overhead
on clean networks. Measured via the existing phase1 bitrate probe:
Before (5% floor): 3649 bytes/sec at Opus 24k + 300 Hz sine
After (15% floor): 3568 bytes/sec at Opus 24k + 300 Hz sine
The delta is within noise — 15% isn't meaningfully more expensive than
5% on this signal, which suggests the DRED emission size is signal-
dependent rather than loss-dependent for small values. Net result: we
get a 6x larger reconstruction window for essentially free.
Tests (+3 DRED recovery, +1 #[ignore]'d probe):
- opus_single_packet_loss_is_recovered_via_dred — full encode → ingest
→ decode_next loop with one packet dropped mid-stream. Asserts
dred_reconstructions ≥ 1 and observes the exact counter deltas.
- opus_lossless_ingest_never_triggers_dred_or_plc — baseline behavior,
lossless stream never takes the Missing branch.
- codec2_loss_falls_through_to_classical_plc — Codec2 never
reconstructs via DRED even if state were populated (which it won't
be — Codec2 packets don't carry DRED bytes).
- probe_dred_samples_available_by_loss_floor — #[ignore]'d diagnostic
that sweeps loss_pct values and prints the resulting DRED window
sizes. Kept for future tuning work.
New CallDecoder introspection accessors (public but undocumented in
the PRD): last_good_dred_seq() and last_good_dred_samples_available()
for test diagnostics and future telemetry surfaces in Phase 4.
Verification:
- cargo check --workspace: zero errors
- cargo test -p wzp-codec --lib: 68 passing (Phase 3a baseline held)
- cargo test -p wzp-client --lib: 35 passing (+3 Phase 3b tests,
+1 ignored diagnostic, no regressions)
Next up: Phase 3c mirrors this on the Android engine.rs receive path.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Siavash Sameni
|
c27b39d553 |
feat(codec): Phase 3a — DRED FFI primitives (DredDecoderHandle + DredState)
Phase 3a of the DRED integration — the foundation for codec-layer loss recovery. Adds three new safe wrappers to crates/wzp-codec/src/dred_ffi.rs over the raw opusic-sys FFI, plus the reconstruction method on the existing DecoderHandle. No call-site integration yet — that lands in Phase 3b (desktop) and Phase 3c (Android). New types: - `DredDecoderHandle`: owns *mut OpusDREDDecoder from opus_dred_decoder_create. Used for parsing DRED side-channel data out of arriving Opus packets. This is a SEPARATE libopus object from OpusDecoder — it has its own internal state. Freed via opus_dred_decoder_destroy on Drop. - `DredState`: owns *mut OpusDRED from opus_dred_alloc (a fixed ~10.6 KB buffer per libopus 1.5). Holds parsed DRED data between the parse and reconstruct steps. Reusable — parse_into overwrites contents. Tracks samples_available as a cached u32 so callers don't thread the value separately. Freed via opus_dred_free on Drop. New methods: - `DredDecoderHandle::parse_into(&mut self, state: &mut DredState, packet)` wraps opus_dred_parse with max_dred_samples=48000 (1s max), sampling_rate =48000, defer_processing=0. Returns the positive sample offset of the first decodable DRED sample, 0 if no DRED is present, or an error. Populates state.samples_available so subsequent reconstruct calls know the valid offset range. - `DecoderHandle::reconstruct_from_dred(&mut self, state, offset_samples, output)` wraps opus_decoder_dred_decode. Reconstructs audio at a specific sample position (positive, measured backward from the DRED anchor packet) into a caller-provided output buffer. Validates that 0 < offset_samples <= state.samples_available() before calling the FFI to catch range bugs. Tests (+7, wzp-codec total: 68 passing): - dred_decoder_handle_creates_and_drops - dred_state_creates_and_drops - dred_state_reset_zeroes_counter - dred_parse_and_reconstruct_roundtrip — end-to-end validation. Encodes 60 frames of a 300 Hz sine wave through a DRED-enabled Opus 24k encoder, parses DRED state out of each arriving packet, asserts that at least one packet carries non-zero samples_available (DRED warm-up completes within the first second), then reconstructs 20 ms of audio from inside the window and asserts non-zero total energy. This is the hard signal that the full libopus 1.5 DRED FFI chain is correctly wired on our side. - reconstruct_with_out_of_range_offset_errors — offset > samples_available is rejected at the Rust layer before the FFI call. - reconstruct_with_zero_offset_errors — offset <= 0 rejected. - dred_parse_empty_packet_returns_zero — graceful handling of empty input. Architectural note (divergence from PRD's literal wording): The PRD said "jitter buffer takes a Box<dyn DredReconstructor>". After checking Cargo.toml for wzp-transport, it does NOT depend on wzp-codec — only wzp-proto. Adding a DRED state ring inside the jitter buffer would require a new cross-crate dependency and couple the codec-agnostic jitter buffer to libopus internals. Instead, Phase 3b will put the DRED state ring and reconstruction dispatch in CallDecoder (one layer up from the jitter buffer), intercepting the existing PlayoutResult::Missing signal and attempting reconstruction before falling through to classical PLC. The jitter buffer itself stays unchanged. Same lookahead/backfill semantics, cleaner layering. PRD's intent preserved, implementation refined. Verification: - cargo check --workspace: zero errors - cargo test -p wzp-codec --lib: 68 passing (61 Phase 2 baseline + 7 new) - The roundtrip test is the acceptance gate — it proves that opus_dred_decoder_create, opus_dred_alloc, opus_dred_parse, and opus_decoder_dred_decode all work correctly through our wrappers on real libopus 1.5.2 output. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
6db5c25b54 |
feat(codec): Phase 2 — remove RaptorQ from Opus tiers, Codec2 unchanged
Phase 2 of the DRED integration (docs/PRD-dred-integration.md). With
Phase 1 having enabled DRED on every Opus profile, the app-level RaptorQ
layer is now redundant overhead on those tiers: +20% bitrate, +40–100 ms
receive-side latency (block wait), +CPU for stats we never used. This
phase removes RaptorQ from the Opus encode and decode paths on both the
desktop (wzp-client/call.rs) and Android (wzp-android/engine.rs) sides.
Codec2 tiers keep RaptorQ with their current ratios unchanged — DRED is
libopus-only and Codec2 has no neural equivalent.
Encoder changes (the real bandwidth / CPU win):
- CallEncoder::encode_frame and engine.rs encode loop now gate the
RaptorQ path on !codec.is_opus():
- Opus source packets emit fec_block=0, fec_symbol=0,
fec_ratio_encoded=0 in the MediaHeader
- fec_enc.add_source_symbol is skipped on Opus
- generate_repair + repair packet emission is skipped on Opus
- block_id and frame_in_block counters stay frozen at 0 for Opus
- Codec2 path is byte-for-byte identical to pre-Phase-2 behavior.
Decoder changes (mostly cleanup, since both live decoder paths were
already reading audio directly from source packets and only using the
RaptorQ decoder output for stats):
- CallDecoder::ingest skips fec_dec.add_symbol on Opus packets. Source
packets still flow to the jitter buffer; Opus repair packets from old
senders are dropped cleanly (repair packets never hit the jitter
buffer either).
- engine.rs recv loop skips fec_dec.add_symbol, fec_dec.try_decode, and
fec_dec.expire_before on Opus packets. The `fec_recovered` stat
counter becomes Codec2-only (a separate DRED reconstruction counter
lands in Phase 4).
Wire-format backward compat verified at pre-flight:
- Old receiver + new sender: engine.rs pipeline.rs path gates on
non-zero fec_block/fec_symbol which now never fire for Opus, so the
RaptorQ decoder simply isn't fed. Audio flows normally. Desktop
CallDecoder's old path accumulated packets into the stale-eviction
HashMap, which cleans up after 2s — harmless.
- New receiver + old sender: new receiver skips RaptorQ on Opus so
old-sender repair packets are ignored entirely (no crash, no double-
decode). Loses the (previously vestigial) RaptorQ recovery benefit,
which was never actually active in the audio path. Source packets
still decode normally.
- No wire format version bump required. MediaHeader is unchanged; we
just zero the FEC fields on Opus packets.
Test changes:
- Removed `encoder_generates_repair_on_full_block` — asserted the old
(pre-Phase-2) RaptorQ-on-Opus behavior and is now incorrect. Replaced
with two symmetric tests:
- `opus_source_packets_have_zero_fec_header_fields` — verifies
Phase 2 invariants on Opus packets
- `opus_encoder_never_emits_repair_packets` — runs 20 frames of
non-silent sine wave through a GOOD-profile encoder, asserts
exactly 20 output packets, zero repair
- `codec2_encoder_generates_repair_on_full_block` — same shape as
the old test but on CATASTROPHIC profile (Codec2 1200, 8
frames/block, ratio 1.0) to verify Codec2 path still emits
repairs as before
Verification:
- cargo check --workspace: zero errors
- cargo test -p wzp-codec --lib: 61 passing (Phase 1 baseline held)
- cargo test -p wzp-client --lib: 32 passing (+3 new Phase 2 tests,
-1 old test removed)
- cargo check -p wzp-android --lib: zero errors (host link of
wzp-android tests fails on -llog per pre-existing Android-only
build.rs, unrelated to this work; integration build via
build-and-notify.sh will validate Android end-to-end)
- Pre-existing broken integration test in
crates/wzp-client/tests/handshake_integration.rs (SignalMessage
schema drift) is NOT caused by this commit — baseline had the same
3 compile errors before Phase 2. Flagged as a separate cleanup task.
Expected observable effects on a real call:
- Opus 24k outgoing bitrate drops from ~28.8 kbps (ratio 0.2 RaptorQ)
to ~25 kbps (base 24 kbps + DRED ~1–10 kbps signal-dependent)
- Opus receive-side latency drops ~40 ms on clean network (no more
block wait — jitter buffer emits as soon as a source packet arrives)
- Codec2 calls show no latency or bitrate change
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Siavash Sameni
|
54cbebd34e |
feat(codec): Phase 1 — enable DRED on all Opus profiles, disable inband FEC
Phase 1 of the DRED integration (docs/PRD-dred-integration.md). The Opus encoder now emits DRED (Deep REDundancy) bytes in every packet, carrying a neural-coded history of recent audio that the decoder can use to reconstruct loss bursts up to the configured window. Opus inband FEC (LBRR) is disabled because DRED does the same job better and running both wastes bitrate on overlapping protection. Tiered DRED duration policy per PRD: Studio (Opus 32k/48k/64k): 10 frames = 100 ms Normal (Opus 16k/24k): 20 frames = 200 ms Degraded (Opus 6k): 50 frames = 500 ms Each profile switch (via adaptive quality) updates the DRED duration to match the new tier. A 5% packet_loss floor is applied whenever DRED is active, because libopus 1.5 gates DRED emission on non-zero packet_loss. Real loss measurements from the quality adapter override upward. Escape hatch: AUDIO_USE_LEGACY_FEC=1 reverts the encoder to Phase 0 behavior (inband FEC Mode1, DRED off, no loss floor). Read once at OpusEncoder::new; call-scoped, not re-read mid-call. Trait-level set_inband_fec becomes a no-op in DRED mode to preserve the invariant even if external callers forget. Observations from the bitrate probe test (dred_mode_roundtrip_voice_pattern): DRED mode: 3649 bytes/sec (~29.2 kbps) on Opus 24k + 300 Hz sine Legacy mode: 2383 bytes/sec (~19.1 kbps) Delta: +10.1 kbps The delta is considerably larger than the "+1 kbps flat" figure I carried into the PRD from hazy memory of published DRED benchmarks. Likely because the input (300 Hz sine) is very compressible so the base Opus rate in legacy mode is well below the 24 kbps target, making the delta look disproportionate. Signal-dependent — real speech would probably show a different ratio. If production telemetry shows the overhead is excessive, we can cut DRED duration on the normal tier from 200 ms to 100 ms as a first tuning lever. Not blocking Phase 1 since the test still passes within the reasonable 2000–8000 bytes/sec bounds. Test changes (+8 tests, total wzp-codec: 61 passing): - dred_duration_for_studio_tiers_is_100ms (per-profile policy) - dred_duration_for_normal_tiers_is_200ms - dred_duration_for_degraded_tier_is_500ms - dred_duration_for_codec2_is_zero - default_mode_is_dred_not_legacy (sanity check on fresh construction) - dred_mode_roundtrip_voice_pattern (observes DRED bitrate, asserts bounds) - profile_switch_refreshes_dred_duration (verifies set_profile updates DRED) - set_inband_fec_noop_in_dred_mode (trait-level inband FEC no-op) Verification: - cargo check --workspace: zero errors, no new warnings - cargo test -p wzp-codec: 61/61 passing (53 pre-Phase-1 baseline + 8 new) - Empirical DRED bitrate observed via `rtk proxy cargo test dred_mode_roundtrip_voice_pattern -- --nocapture` Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
86526a7ad4 |
feat(codec): Phase 0 — swap audiopus → opusic-c + opusic-sys (libopus 1.5.2)
Phase 0 of the DRED integration (docs/PRD-dred-integration.md). No behavior change: inband FEC stays ON, no DRED, same bitrate, same quality. This commit unblocks Phase 1+ by getting us onto libopus 1.5.2 where DRED lives. Rationale for going straight to a custom DecoderHandle: opusic-c::Decoder's inner *mut OpusDecoder pointer is pub(crate), so we cannot reach it for the Phase 3 DRED reconstruction path. Running two parallel decoders (one for audio, one for DRED) would drift because the DRED decoder wouldn't see normal decode calls. Single unified DecoderHandle over raw opusic-sys is the only correct architecture, so we build it in Phase 0 rather than rewriting opus_dec.rs twice. Changes: - Cargo.toml (workspace + wzp-codec): remove audiopus 0.3.0-rc.0, add opusic-c 1.5.5 (bundled + dred features), opusic-sys 0.6.0 (bundled), bytemuck 1. Pinned exactly for reproducible libopus 1.5.2. - opus_enc.rs: rewritten against opusic_c::Encoder. Argument order for Encoder::new swapped (Channels first). set_inband_fec(bool) now maps to InbandFec::Mode1 (the libopus 1.5 equivalent of 1.3's LBRR). encode uses bytemuck::cast_slice<i16,u16> at the &[u16] boundary. - dred_ffi.rs (new): DecoderHandle wrapping *mut OpusDecoder directly via opusic-sys. Owns the allocation, frees on Drop. Exposes decode, decode_lost, and a pub(crate) as_raw_ptr() for the future Phase 3 DRED reconstruction. Send+Sync justified via &mut self access discipline. - opus_dec.rs: rewritten as a thin AudioDecoder impl over DecoderHandle. Behavior identical to pre-swap. Verification (Phase 0 acceptance gates): - cargo check --workspace: clean (30 pre-existing warnings in jni_bridge.rs unrelated to this work; zero in changed files). - cargo test -p wzp-codec: 53 tests pass (50 pre-swap + 6 new: 3 in dred_ffi.rs for DecoderHandle lifecycle, 3 in opus_enc.rs for version check and roundtrip). - linked_libopus_is_1_5 test asserts opusic_c::version() contains "1.5" — hard signal that the swap landed correctly. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
56e3417063 |
docs: add PRD for DRED integration and Opus-tier FEC simplification
Plans the libopus 1.5.2 upgrade (audiopus → opusic-c/opusic-sys), DRED enablement with tiered durations (100/200/500ms studio/normal/degraded), removal of RaptorQ and Opus inband FEC from the Opus tiers, jitter buffer lookahead/backfill refactor, and runtime escape hatch for rollout safety. RaptorQ + current ratios preserved on Codec2 tiers (no DRED there). Includes pre-flight verification findings: opusic-c Decoder inner pointer is inaccessible (requires unified opusic-sys DecoderHandle), libopus 1.5 DRED API semantics clarified against xiph/opus opus.h, wire-format backward compat verified on both live receive paths. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
8ceb6f45d5 |
fix(build): declare VARIANT in local script half (was remote-only)
The VARIANT variable was set inside the REMOTE_SCRIPT heredoc for naming artifacts during the cargo tauri build, but never declared in the local half of the script where it's used to rename downloaded files. Under `set -u` strict mode this aborted the local downloads with "unbound variable: VARIANT" after a successful remote build. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
07873ea598 |
fix(linux-aec): fall back to 0.3 crate + apt lib (2.x bundled is broken)
Switch the webrtc-audio-processing dep from the 2.x git source (bundled mode) back to crates.io 0.3, and link against Debian's apt package libwebrtc-audio-processing-dev (0.3-1+b1 on Bookworm). The 2.x path fails because both the crates.io tarball and the upstream git main branch of webrtc-audio-processing-sys 2.0.3 have a build.rs bug where \`meson setup --reconfigure\` is passed unconditionally, panicking on first-run empty build dirs with "Directory does not contain a valid build tree". The 0.x line sidesteps bundled mode entirely by linking the apt-provided library. Trade-off: we get AEC2 (the older generation) instead of AEC3, but it's the same algorithm family and is what PulseAudio's module-echo-cancel and PipeWire's filter-chain use on current Debian-family distros. Fine for shipping — we can revisit AEC3 once the 2.x bundled build is fixed upstream. API changes: - 0.3's Processor::process_capture_frame and process_render_frame take &mut self, so wrap the module-level processor in a Mutex. Capture and playback threads each lock briefly (sub-ms per 10 ms frame); contention is minimal. - Import NUM_SAMPLES_PER_FRAME from the crate directly instead of hardcoding 480, so the code tracks whatever sample rate the upstream C++ lib exposes (currently 48 kHz hardcoded -> 480). - Helper fns drain_frames_through_apm / tee_render_samples / etc. take &Mutex<Processor> instead of &Processor. - Use explicit EchoCancellationSuppressionLevel and NoiseSuppressionLevel imports rather than fully-qualified paths. Dockerfile: - Drop meson / ninja-build / python3 (only needed for bundled build). - Add libwebrtc-audio-processing-dev for the system link path. - Keep clang (may be needed by the bindgen step in some versions). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
cc00f7cace |
fix(linux-aec): try main branch of webrtc-audio-processing
v2.0.3 bundled build hits 'Directory does not contain a valid build tree' because the crate's build.rs uses `meson setup --reconfigure` unconditionally, which fails on first run when the build dir doesn't yet contain prior meson state. Try the main branch in case it's been fixed post-release. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
eb9de988d6 |
fix(linux-aec): use git dep for webrtc-audio-processing
The crates.io tarball of webrtc-audio-processing-sys 2.0.3 is missing the vendored C++ submodule — the bundled build fails with 'Directory does not contain a valid build tree' when meson tries to configure the ./webrtc-audio-processing subdirectory. Cargo clones git deps with submodules auto-initialized since ~1.27, so pulling from the upstream git repo (pinned to tag v2.0.3) gives us the full source tree. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
4ba77c8c0e |
feat(linux): WebRTC AEC3 capture/playback backend with render-side tee
Adds gold-standard Linux echo cancellation: in-app WebRTC AEC3 (Audio Processing Module) via the webrtc-audio-processing crate, using the same algorithm as Chrome WebRTC, Zoom, Teams, and Jitsi. Runs entirely in-process, so it works identically on ALSA / PulseAudio / PipeWire systems — no dependency on user-configured echo-cancel modules. Architecture: - New crates/wzp-client/src/audio_linux_aec.rs module (~470 lines). Contains LinuxAecCapture and LinuxAecPlayback, both using CPAL under the hood but routing samples through a shared Arc<webrtc_audio_processing::Processor>. The playback path tees each 20 ms frame into APM.process_render_frame as the echo reference BEFORE handing the samples to CPAL's output callback. The capture path runs APM.process_capture_frame on each mic frame in place before pushing to the audio ring buffer. This is the "tee the playback ring" approach that Zoom/Teams/Jitsi use. - New `linux-aec` feature in wzp-client pulling in the webrtc-audio-processing crate at v2.x with the `bundled` sub-feature. Bundled means the vendored PulseAudio WebRTC C++ sources are statically compiled via meson+ninja at cargo build time — no runtime .so dependency, avoids Debian Bookworm's stale libwebrtc-audio-processing-dev 0.3 package (which predates AEC3). Dep is target-gated to Linux, so enabling the feature on non-Linux is a no-op. - lib.rs re-exports LinuxAecCapture/LinuxAecPlayback as AudioCapture/AudioPlayback when `linux-aec` is on, otherwise falls back to the CPAL audio_io path. Shared public API (start/ring/stop/Drop) means downstream code is unchanged. - New `linux-aec` feature in wzp-desktop forwards to wzp-client/linux-aec so `cargo tauri build -- --features wzp-desktop/linux-aec` builds the AEC variant. APM configuration: - EchoCancellation: High suppression, delay-agnostic mode on, extended filter on, stream_delay_ms=60 initial hint - NoiseSuppression: High - HighPassFilter: on - AGC: off (can fight Opus encoder's own gain staging + adaptive quality controller; add later if users report low mic level) Frame size handling: - Pipeline uses 20 ms frames (960 samples @ 48 kHz mono) - APM requires strict 10 ms (480 samples) per call - Each 20 ms frame is split into two 480-sample halves, APM called twice, halves stitched back - Same pattern for render and capture sides - Carry-buffer logic handles the case where CPAL delivers samples in arbitrary chunk sizes that don't divide 960 Build infrastructure: - scripts/Dockerfile.linux-desktop-builder adds meson, ninja-build, python3, clang for the webrtc-audio-processing bundled build - scripts/build-linux-desktop-docker.sh takes a new --aec flag that enables the linux-aec feature and renames the output artifacts with an `-aec` suffix so noAEC and AEC variants can coexist on disk Task #30. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
7b8a2d0fba |
feat(build): add Linux x86_64 Tauri desktop build pipeline
New Dockerfile and build script for producing wzp-desktop as a Linux
x86_64 binary (plus .deb and .AppImage bundles via tauri-cli).
- scripts/Dockerfile.linux-desktop-builder: thin extension of
wzp-android-builder that adds the Tauri Linux runtime deps
(libwebkit2gtk-4.1-dev, libsoup-3.0-dev, libgtk-3-dev,
libayatana-appindicator3-dev, librsvg2-dev, libglib2.0-dev, patchelf).
Everything else (Rust, Node, cmake, pkg-config, libasound2-dev,
tauri-cli) is inherited from the base image.
- scripts/build-linux-desktop-docker.sh: mirrors the pattern of
build-windows-docker.sh and build-linux-docker.sh. Ships
\`cargo tauri build\` which produces target/release/wzp-desktop
plus bundles under target/release/bundle/{deb,appimage}/. Uploads
the .deb (or raw binary if bundling fails) to rustypaste and
notifies ntfy.sh/wzp on start + completion. Downloads all three
artifact types (raw binary, .deb, .AppImage) to target/linux-desktop/
when they exist.
Image cache volumes are shared with the Android pipeline for cargo
registry + git, but the target dir is in its own cache-linux-desktop/
path to avoid stomping on the Android / Linux-CLI / Windows target
caches.
Branch default is feat/desktop-audio-rewrite (where the actual
wzp-desktop source lives), not feat/android-voip-client.
Task #29.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Siavash Sameni
|
5cd7a20152 |
fix(ui): disable WebView pinch-zoom and desktop right-click menu
Two small WebView hardening tweaks that apply to both Android (Tauri mobile) and desktop (Tauri) since the frontend is shared: - index.html viewport meta now sets maximum-scale=1.0, minimum-scale=1.0, and user-scalable=no. This stops users on Android from pinch-zooming out of the fixed-layout UI. Desktop is unaffected because the Tauri WebView ignores pinch gestures anyway. - main.ts installs global listeners that preventDefault on contextmenu (kills the browser-style right-click menu that exposed Inspect / Reload / Back / Forward entries on desktop), keydown Ctrl+-/+/0 (stops keyboard zoom of the fixed layout), and gesture* + ctrl-wheel events (trackpad pinch on WebKit + Chromium respectively). Dev tools remain accessible via F12 / Cmd-Opt-I keyboard shortcuts — only the right-click entry point is suppressed. Android has no right-click so that part is a no-op there. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
a5c00fe5cb |
docs: add BRANCH-desktop-audio-rewrite.md and update ARCH/ADMIN/USER_GUIDE
Documents the feat/desktop-audio-rewrite branch story end-to-end: - Purpose: shared codebase with android-rewrite via Tauri, platform- specific audio backends via target-dep sections + feature flags - Audio backend matrix: CPAL baseline + macOS VPIO + Windows WASAPI AudioCategory_Communications - Recent work: desktop direct calling feature with history dedup, macOS VPIO integration, Windows cross-compile via cargo-xwin, the libopus/clang-cl vendored audiopus_sys fix, icon.ico generation, and the WASAPI communications capture backend (task #24) - Build pipelines: native cargo on macOS/Linux, Docker on SepehrHomeserverdk for Windows, Hetzner Cloud alternative - Testing procedures for direct calling parity and Windows AEC A/B - Known quirks: vendor path relative, cargo-xwin override.cmake clobber, WebView2 runtime prerequisite, 2024 edition unsafe lint warnings Also appends shared-doc sections (identical on both branches): - ARCHITECTURE.md: "Audio Backend Architecture (Platform Matrix)" - ADMINISTRATION.md: "Build Pipelines" - USER_GUIDE.md: "Direct 1:1 Calling" and "Windows AEC Variants" Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
ec41f179cd |
fix(windows): drop dead override.cmake patch from Dockerfile
The RUN step that baked an OPUS_DISABLE_INTRINSICS patch into cargo-xwin's override.cmake was inert from the start: cargo-xwin rewrites that file from scratch on every \`cargo xwin build\` invocation (src/compiler/clang_cl.rs line ~444 uses include_bytes! to overwrite it), so anything baked at image build time gets wiped at runtime. The libopus SSE4.1/SSSE3 compile failure is now fixed upstream at the source level by the vendored audiopus_sys patch (see vendor/audiopus_sys/opus/CMakeLists.txt and the MSVC_CL distinction for clang-cl). Remove the dead RUN step and leave a breadcrumb comment pointing at the real fix location. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
4e9244eb00 |
fix(windows): add Win32_Security feature + 2024 edition unsafe wrappers
- CreateEventW is gated behind Win32_Security in the windows crate
because its signature takes SECURITY_ATTRIBUTES; add to features.
- Remove unused HANDLE import.
- Wrap GetId() and PWSTR::to_string() in explicit unsafe { ... }
blocks for Rust 2024 edition's unsafe_op_in_unsafe_fn lint.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Siavash Sameni
|
03a80a3196 |
feat(windows): WASAPI capture backend with OS-level AEC
Adds a direct WASAPI microphone capture path for the Windows desktop build that opens the default communications endpoint via IMMDeviceEnumerator -> IAudioClient2 -> SetClientProperties with AudioCategory_Communications, turning on Windows's communications audio processing chain (AEC, noise suppression, automatic gain control). The communications AEC operates at the OS level and uses the system render mix as the reference signal, so echo from our existing CPAL playback stream is cancelled automatically with no per-process reference plumbing. Architecture: - New crates/wzp-client/src/audio_wasapi.rs module (~280 lines). Event-driven capture loop on a dedicated thread; pushes PCM into the same lock-free AudioRing used by the CPAL path. Same public API as audio_io::AudioCapture so downstream code is unchanged. - New `windows-aec` feature in wzp-client that pulls in the `windows` crate (Microsoft's official Rust COM bindings) gated to target_os = "windows" only. Enabling the feature on non-Windows targets is a no-op since both the module and the dep are cfg(target_os = "windows"). - lib.rs re-exports WasapiAudioCapture as AudioCapture when the feature is on, otherwise falls back to the CPAL AudioCapture. AudioPlayback is always the CPAL one — no reason to swap it. - desktop/src-tauri/Cargo.toml Windows target enables the new feature: `features = ["audio", "windows-aec"]`. Implementation notes: - Uses eCommunications role (not eConsole) for GetDefaultAudioEndpoint — the user-configured "communications" device that Teams/Zoom pick up, and the one Windows's AEC is tuned for. - Requests 48 kHz mono i16 with AUDCLNT_STREAMFLAGS_AUTOCONVERTPCM + SRC_DEFAULT_QUALITY so Windows handles any format conversion in the audio engine instead of rejecting our format. - Event-driven with SetEventHandle / WaitForSingleObject — no polling, minimal CPU cost between packets. - 200 ms wait timeout so the capture thread polls `running` often enough for Drop to stop cleanly even if the audio engine stalls (e.g. device unplug). Task #24. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
7fecf285ea |
fix(windows): add icons/icon.ico for tauri-build Windows resource
tauri-build's Windows path unconditionally looks up icons/icon.ico to embed as the PE file resource (taskbar/Explorer icon). We only had icon.png (32x32 placeholder) which is fine on macOS/Linux but blocks the Windows cross-compile with "icons/icon.ico not found; required for generating a Windows Resource file during tauri-build". Generated a multi-size ICO (16/24/32/48/64/128/256) from the existing placeholder icon.png via Pillow. It's ugly at 256 due to upscaling from 32x32 with LANCZOS, but unblocks the build. Real branded icons can replace it later without any build-system changes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
0683dde5d3 |
fix(windows): vendor audiopus_sys + patch libopus for clang-cl SIMD
cargo-xwin drives the Windows MSVC cross-compile via clang-cl, under which CMake sets MSVC=1 — causing libopus 1.3.1's `if(NOT MSVC)` guards to skip the per-file `-msse4.1` / `-mssse3` COMPILE_FLAGS that its x86 SIMD source files need. Clang-cl (unlike real cl.exe) still honors Clang's target-feature system, so those files then fail to compile with "always_inline function '_mm_cvtepi16_epi32' requires target feature 'sse4.1'" errors across silk/NSQ_sse4_1.c, NSQ_del_dec_sse4_1.c, and VQ_WMat_EC_sse4_1.c. Earlier attempts to fix this downstream (cargo-xwin toolchain file, override.cmake CMAKE_C_COMPILE_OBJECT <FLAGS> replace, CFLAGS env vars) all failed because cargo-xwin rewrites override.cmake from scratch on every `cargo xwin build` invocation and cmake-rs's -DCMAKE_C_FLAGS= assembly happens before toolchain FORCE sets propagate. Fixing it upstream at the source: vendor audiopus_sys 0.2.2 into vendor/audiopus_sys, patch its bundled opus/CMakeLists.txt to introduce an MSVC_CL var (true only when CMAKE_C_COMPILER_ID == "MSVC", i.e. real cl.exe), and flip the eight `if(NOT MSVC)` SIMD guards to `if(NOT MSVC_CL)`. Clang-cl then gets the GCC-style per-file flags and the SSE4.1 sources build cleanly. Also flip the `if(MSVC)` global /arch block at line 445 to `if(MSVC_CL)` so only cl.exe applies /arch:AVX and clang-cl relies purely on per-file flags (no global/per-file mixing). Wire via [patch.crates-io] in the workspace root Cargo.toml; the patch is resolved relative to the workspace root as `vendor/audiopus_sys`. Upstream context: xiph/opus#256, xiph/opus PR #257 (both stale). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
53f57eea07 | fix(windows): printf instead of heredoc in Dockerfile RUN (parser hated <<EOF) | ||
|
Siavash Sameni
|
ff3f7e8e4f |
fix(windows): patch override.cmake not toolchain — inject SSE via COMPILE_OBJECT template
The previous 'patch the toolchain file' approach ( |
||
|
Siavash Sameni
|
48d2bd4f65 | fix(windows): bake SSE patch into docker image instead of runtime | ||
|
Siavash Sameni
|
234a798df2 |
fix(windows): append SSE flags as a pure-CMake block to xwin toolchain
The previous sed-based patch didn't stick in the docker-bash-c
heredoc (bash single-quoting made the newline escaping fragile).
Switch to a much simpler approach: just 'cat >>' a pure-CMake block
to the end of the cargo-xwin toolchain file. The block does:
set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} /clang:-msse4.1 ..." CACHE STRING "" FORCE)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} /clang:-msse4.1 ..." CACHE STRING "" FORCE)
Running AFTER the toolchain's own FORCE-set and AFTER cmake-rs's
-DCMAKE_C_FLAGS= command-line override, it unconditionally wins. No
sed, no awk, no python, no newline escaping — just CMake reading the
toolchain file like it normally does.
Idempotent via the WZP_SSE_PATCH sentinel grep in the comment block.
|
||
|
Siavash Sameni
|
fa042b130c |
fix(windows): sed-patch cargo-xwin toolchain to enable SSE4.1/SSSE3
The CFLAGS_x86_64_pc_windows_msvc env-var approach from |
||
|
Siavash Sameni
|
990b6f1ee0 |
fix(windows): set CFLAGS +sse4.1 +ssse3 so audiopus_sys builds under clang-cl
libopus ships per-file SSE4.1 / SSSE3 C sources (opus/silk/x86/NSQ_del_dec_sse4_1.c etc.) that assume the compiler picks up `-msse4.1` / `-mssse3` as per-file CMake COMPILE_FLAGS. With clang-cl those bare -m flags are silently dropped, so _mm_cvtepi16_epi32 + friends fail compile with 'always_inline function requires target feature sse4.1, but would be inlined into a function that is compiled without support for sse4.1'. Workaround: set CFLAGS_x86_64_pc_windows_msvc + CXXFLAGS_x86_64_pc_windows_msvc to `/clang:-msse4.1 /clang:-mssse3 /clang:-msse3 /clang:-msse2` before running cargo xwin build. Every x86_64 Windows CPU shipped since 2008 has these instruction sets so globally enabling them on this target is safe. Also bump the tail -30 on cargo xwin output to tail -50 so the actual compiler errors (not just the cmake wrapper panic) make it into the ntfy / remote log file next time. |
||
|
Siavash Sameni
|
7949266e11 |
windows: docker + hcloud build scripts for cross-compile
Two parallel paths to build wzp-desktop.exe for x86_64-pc-windows-msvc:
scripts/Dockerfile.windows-builder
Debian 12 base, matches scripts/Dockerfile.android-builder's layout:
- apt: build-essential, cmake, ninja-build, llvm, clang, lld, nasm,
libssl-dev, node 20 LTS
- rust stable + x86_64-pc-windows-msvc target
- cargo-xwin pre-installed
- Pre-warmed ~/.cache/cargo-xwin layer: creates a throwaway cargo
project and runs `cargo xwin build` once during image build so the
MSVC CRT + Windows SDK (~1.5 GB) is baked into an image layer.
Saves ~4 minutes off every cold cross-compile run.
- Builder user uid 1000 to match existing bind-mount perms on
SepehrHomeserverdk.
scripts/build-windows-docker.sh
Same pattern as scripts/build-tauri-android.sh but for Windows:
- Fires a remote build on SepehrHomeserverdk via ssh + heredoc
- Mounts the shared cargo-registry + cargo-git cache + a
target-windows dir (separate from the android target cache so
different triples don't stomp each other)
- Runs npm install + npm run build for the frontend dist, then
cargo xwin build --release --target x86_64-pc-windows-msvc
--bin wzp-desktop inside the container
- Uploads the resulting .exe to rustypaste (via the .env token on
the remote, same as android script) and fires ntfy.sh/wzp
notifications at start + completion
- scp's the .exe back to target/windows-exe/wzp-desktop.exe locally
- --image-build flag triggers a fire-and-forget `docker build` of
the Dockerfile.windows-builder on the remote (used once after the
Dockerfile changes). The image is already built at the moment of
this commit — sha256:f3895cb2fde7
scripts/build-windows-cloud.sh
Kept as an alternative cross-compile path using a fresh Hetzner VM
(cx33, 8 vCPU, 8 GB — bumped from cx23 after the smaller size OOM'd
mid-rustc). The docker-on-SepehrHomeserverdk path is now the
preferred fast path because the image has a pre-warmed xwin cache
and a persistent cargo target volume, making warm builds ~3 minutes
vs the cloud path's ~20 minutes cold each run. The cloud script
stays around for when we want a truly isolated environment.
Both scripts notify via ntfy.sh/wzp and upload to paste.dk.manko.yoga
so the user can pick up the artefact + see status without polling.
|
||
|
Siavash Sameni
|
d774f5f8c5 |
feat(history): dedupe by call_id + explicit Incoming/Outgoing/Missed labels
User reported that outgoing direct calls from macOS show up in the history list as "missed" even when the call completes successfully. Adds two changes to fix / diagnose: 1. history::log now dedupes by call_id. If an entry for this call_id already exists in the store, it updates the existing row's direction + timestamp in place instead of appending a duplicate. Protects against double-emit (caller side adding Missed on top of Placed, or any future signal loop that fires twice). One row per call_id, which matches what the user intuitively expects. 2. history::log now logs every write with tracing::info — call_id, peer_fp, direction, alias. Plus an extra line when we replace an existing entry: "history::log replacing existing entry from=Placed to=Missed" etc. Makes it easy to see in the desktop stderr which side is writing what, so we can find the outgoing => missed regression immediately if it recurs. 3. main.ts now renders an explicit text label next to the direction arrow: "Outgoing", "Incoming", or "Missed" instead of just the ↗ ↙ ✗ icons. Removes any ambiguity about what the icon means so future users can't misread a Placed entry as Missed based on icon shape alone. Side fix for scripts/build-windows-cloud.sh: - die() and the do_full ERR trap now respect WZP_KEEP_VM=1 so a failed build doesn't auto-destroy the debug VM (previously the trap fired before the KEEP_VM check and tore down the VM on any error). - Bump default server type cx23 → cx33. 4GB RAM is not enough for a cold tauri + rustls + quinn + wzp-client cross-compile — the cx23 run got "Read from remote host ... Connection reset by peer" partway through rustc, which is the classic signature of an OOM kill on the SSH session. cx33 has 8GB RAM and 8 vCPU which should comfortably fit the build. |
||
|
Siavash Sameni
|
2fd94651e4 |
fix(desktop): direct calls used wrong identity file — mac identity mismatch
The non-Android branch of CallEngine::start loaded the seed from \$HOME/.wzp/identity directly, while register_signal in lib.rs goes through the shared load_or_create_seed() helper which resolves via APP_DATA_DIR → Tauri's app_data_dir(). On macOS those are two completely different files: register_signal → ~/Library/Application Support/com.wzp.desktop/.wzp/identity CallEngine::start (old) → ~/.wzp/identity On a fresh install they end up holding two different random seeds. Register and CallEngine then derive two different fingerprints from those seeds, and when a direct call comes in the relay routes it to "you" under the register_signal fingerprint, but once CallEngine tries to join the call-* room it advertises a DIFFERENT fingerprint — which fails the call_registry ACL check on the relay side (only the two authorised participants of a call can join its room). Silent hang, the call never completes. Android hit this bug earlier in the week and was fixed by switching its CallEngine::start branch to `crate::load_or_create_seed()`. Backport the same single-line change to the desktop branch so both platforms share one identity source of truth. Also bring the desktop branch up to parity with the android branch on diagnostic logging: - log CallEngine::start entry with relay/room/alias/quality/has_reuse - log endpoint.local_addr on reuse / create - log "QUIC connection established, performing handshake" between connect() and perform_handshake() so a hang at either step is immediately localisable - map_err all three potential failure points (create_endpoint, connect, perform_handshake) to an explicit error! trace |
||
|
Siavash Sameni
|
da09fdb6e9 |
windows(desktop): gate coreaudio / VoiceProcessingIO to macOS-only targets
First step of the Windows x86_64 desktop build: stop pulling
coreaudio-rs into the Windows dependency graph so the project can at
least run `cargo check --target x86_64-pc-windows-msvc`. Software AEC
is already disabled in engine.rs so there's nothing else to stub — the
macOS-specific VPIO path is skipped via #[cfg(target_os = "macos")] on
both sides and Windows falls through to the plain CPAL
AudioCapture/AudioPlayback branch that already existed.
crates/wzp-client/Cargo.toml
- coreaudio-rs optional dep moved under [target.'cfg(target_os = "macos")']
- `vpio` feature now uses `dep:coreaudio-rs` syntax and the gated dep
- Enabling `vpio` on Windows/Linux is a no-op at resolution time
crates/wzp-client/src/lib.rs
- `pub mod audio_vpio` is now #[cfg(all(feature = "vpio", target_os = "macos"))]
- Previously `vpio` alone was enough to try to compile the Core Audio
bindings, which would fail on non-Apple targets the moment the
feature flag was flipped on
desktop/src-tauri/Cargo.toml
- [target.'cfg(not(target_os = "android"))'] removed — was leaking
vpio into Windows/Linux via the catch-all.
- macOS: wzp-client with features = ["audio", "vpio"]
- Windows: wzp-client with features = ["audio"]
- Linux: wzp-client with features = ["audio"]
- Android: wzp-client with default-features = false (unchanged)
- Dropped the unused direct coreaudio-rs = "0.11" dep on macOS —
wzp-desktop's own sources never call Core Audio directly.
Verified via `cargo tree --target x86_64-pc-windows-msvc -p wzp-desktop`
that the Windows target now resolves wzp-client with cpal but without
coreaudio-rs. macOS target still resolves with coreaudio (direct via
vpio feature and transitively via cpal). macOS `cargo check` still
builds cleanly.
Cross-compile from macOS hit a cargo-xwin + llvm-lib setup issue in
ring's build.rs, so the actual `cargo check --target
x86_64-pc-windows-msvc` did not complete locally. Build verification
belongs on the user's Windows x86_64 host where MSVC is present
natively.
See tasks #23 (this one), #24 (Voice Capture DSP / WASAPI Communications
for OS-level AEC on Windows), and #25 (aarch64-pc-windows-msvc support).
|
||
|
Siavash Sameni
|
510eae2089 |
feat(direct-call): call history, recent contacts, deregister button
Persistent JSON-backed call history for the direct-call screen so users
can see what they've placed / received / missed and dial back with one
click. Also fixes two small latent UX issues reported alongside.
Backend (Rust)
- new crate/module desktop/src-tauri/src/history.rs: thread-safe in-
process store (OnceLock<RwLock<Vec<CallHistoryEntry>>>) backed by
<APP_DATA_DIR>/call_history.json. Atomic writes via temp+rename. Max
200 entries, FIFO pruning. CallDirection { Placed, Received, Missed }.
- Log hooks in the signal loop + commands:
* place_call → Placed entry (with target fingerprint)
* DirectCallOffer → Missed entry up front; upgraded to Received
inside answer_call when accept_mode != Reject
via history::mark_received_if_pending(call_id).
If user rejects or never answers, it stays Missed.
- New Tauri commands:
* get_call_history() → all entries, newest first
* get_recent_contacts() → unique peers by fp, newest interaction first
* clear_call_history() → wipes JSON + in-memory
* deregister() → tears down signal transport + endpoint
Backend emits `history-changed` events so the UI can live-refresh
without polling.
Frontend (main.ts + index.html + style.css)
- Direct-call panel now has:
* Recent contacts chip row (top 6 unique peers). Click a chip → dial.
* Call history list (up to 50 rows). Direction icon (↗ placed, ↙
received, ✗ missed), peer alias/fp, relative timestamp, callback
button. Both click handlers populate target-fp and fire place_call.
* Deregister button in the "registered" header — calls the new
deregister command, tears down the signal transport, returns the
UI to the pre-register state.
* Clear-history link in the history header.
- Subscribes to `history-changed` events so the list updates the moment
the backend logs a new entry. Also refreshed on register + after a
clear.
- Nothing is rendered until there is data — empty sections stay hidden.
Tasks #20 + #21 (small UX items bundled in)
- Default room "general" for new installations: the html input value
attribute is now "general" and loadSettings() defaults match. Existing
users' localStorage still wins.
- Random alias on desktop: already latent but confirmed working — the
startup IIFE at main.ts:374 calls get_app_info() and prefills the
alias input from derive_alias(seed) when the input is empty. No code
change needed, just verified it flows through the same path as the
Android client.
Known follow-ups (deferred to step 6 polish)
- Call duration tracking (currently all entries have no duration field)
- Hangup signal from an unanswered incoming should emit history-changed
so the missed state is visible even when the user never tapped accept
- Android UI layout fit-check on the smaller Nothing screen
|
||
|
Siavash Sameni
|
76a4c53e21 |
fix(android-audio): spawn_blocking for Oboe restart — unblock tokio executor
Build
|
||
|
Siavash Sameni
|
4c6aac654a |
fix(android-audio): restart Oboe on speakerphone toggle + unbreak button UI
Build
|
||
|
Siavash Sameni
|
4f2ad65418 | fix(android_audio): add explicit pointer types for .cast() — was rejected by rustc E0282 on android target | ||
|
Siavash Sameni
|
0178cbd91d |
android(audio): Speaker button toggles earpiece↔speaker via JNI (WIP, untested)
Build
|
||
|
Siavash Sameni
|
9e37201198 |
android(audio): Usage::VoiceCommunication + MODE_IN_COMMUNICATION, default handset
With |
||
|
Siavash Sameni
|
da106bd939 |
fix(android-audio): revert to 96be740's Oboe config — VoiceCommunication broke callback drain
Build |
||
|
Siavash Sameni
|
8c36fb5651 |
fix(wzp-native): Oboe ResultWithValue has no value_or, unfold explicitly
cc-rs build of oboe_bridge.cpp failed at
|
||
|
Siavash Sameni
|
cfa9ff67cf |
fix(android-audio): VoIP mode + speakerphone + debug PCM recorder
Build
|
||
|
Siavash Sameni
|
96be740fd9 |
diag(android-audio): aggressive logging across the whole Oboe pipeline
User confirmed: mac hears android, android does not hear mac. So Oboe
capture works end-to-end but Oboe playout on Android silently drops
audio even though QUIC forwards the packets. Archaeology on the legacy
wzp-android crate also revealed that the "last known good" Android audio
path NEVER used Oboe in production — it used Kotlin AudioRecord +
AudioTrack via JNI, and cpp/oboe_bridge.cpp was dead code. So every time
we've "tested" Oboe end-to-end this week was the first production use,
and any of its config knobs could be the bug.
Instrumenting every stage of the pipeline so one smoke-test log dump can
isolate the layer at fault:
C++ (oboe_bridge.cpp)
- Log the ACTUAL stream parameters after openStream for both capture
and playout (sample rate, channels, format, framesPerBurst,
framesPerDataCallback, bufferCapacityInFrames, sharing, perf mode).
Oboe may silently override values we requested — e.g. if we ask for
48kHz mono but the device gives us 44.1kHz stereo our 960-sample
frames are the wrong duration and the pipeline drifts.
- Capture callback: on cb#0 log sample range+RMS of the first frame
to prove we get real mic data (not zeros). Every 50 callbacks
(~1s at 20ms burst) log calls, numFrames, ring available_write,
bytes actually written, ring_full_drops, total_written.
- Playout callback: on cb#0 log numFrames + ring state. On the FIRST
non-empty read log sample range+RMS so we can tell if the samples
coming out of the ring are real audio or zeros. Every 50 callbacks
log calls, nonempty count, numFrames, ring available_read,
underrun_frames, total_played_real.
Rust wzp-native (src/lib.rs)
- wzp_native_audio_write_playout now logs the first 3 writes and then
every 50th: in_len, written, sample range, RMS, ring write/read
cursors before, available_read and available_write after. Reveals
ring-overflow and whether the engine is actually handing us audio.
- Minimal android logcat shim via __android_log_write extern — no
new crate dependency.
- AudioBackend grows a `playout_write_log_count` AtomicU64 to gate
the write-side log throttle.
Rust engine.rs (android branch)
- Recv task: log sample range + RMS for the first 3 decoded PCM
frames and then every 100th. Reveals whether decoder.decode is
producing real audio or silent buffers.
- Recv task: if audio_write_playout returns fewer samples than we
handed it (partial write → ring nearly full) warn about it in the
first 10 frames.
- Recv heartbeat every 2s: recv_fr, decoded_frames, last_decode_n,
last_written, written_samples, decode_errs, codec.
Expected flow in a healthy log:
capture cb#0: numFrames=960 range=[-1200..900] rms=180 ← mic OK
capture stream opened: actualSR=48000 Ch=1 ... ← no override
playout stream opened: actualSR=48000 Ch=1 ...
CallEngine::start invoked ... → connected → audio started
recv: first media packet received ...
recv: decoded PCM sample range decoded_frames=1 range=[-300..250] rms=92
playout WRITE #0: in_len=960 written=960 range=[-300..250] rms=92
playout FIRST nonempty read: to_read=960 range=[-300..250] rms=92
playout heartbeat: calls=50 nonempty=50 underrun=0 ...
recv heartbeat: decoded_frames=100 last_written=960 ...
If any of those are missing/zero we know the exact stage to fix.
|