7515417202d7ffb7171c289dc96f6233a8fef5b0
13 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Siavash Sameni
|
4ba77c8c0e |
feat(linux): WebRTC AEC3 capture/playback backend with render-side tee
Adds gold-standard Linux echo cancellation: in-app WebRTC AEC3 (Audio Processing Module) via the webrtc-audio-processing crate, using the same algorithm as Chrome WebRTC, Zoom, Teams, and Jitsi. Runs entirely in-process, so it works identically on ALSA / PulseAudio / PipeWire systems — no dependency on user-configured echo-cancel modules. Architecture: - New crates/wzp-client/src/audio_linux_aec.rs module (~470 lines). Contains LinuxAecCapture and LinuxAecPlayback, both using CPAL under the hood but routing samples through a shared Arc<webrtc_audio_processing::Processor>. The playback path tees each 20 ms frame into APM.process_render_frame as the echo reference BEFORE handing the samples to CPAL's output callback. The capture path runs APM.process_capture_frame on each mic frame in place before pushing to the audio ring buffer. This is the "tee the playback ring" approach that Zoom/Teams/Jitsi use. - New `linux-aec` feature in wzp-client pulling in the webrtc-audio-processing crate at v2.x with the `bundled` sub-feature. Bundled means the vendored PulseAudio WebRTC C++ sources are statically compiled via meson+ninja at cargo build time — no runtime .so dependency, avoids Debian Bookworm's stale libwebrtc-audio-processing-dev 0.3 package (which predates AEC3). Dep is target-gated to Linux, so enabling the feature on non-Linux is a no-op. - lib.rs re-exports LinuxAecCapture/LinuxAecPlayback as AudioCapture/AudioPlayback when `linux-aec` is on, otherwise falls back to the CPAL audio_io path. Shared public API (start/ring/stop/Drop) means downstream code is unchanged. - New `linux-aec` feature in wzp-desktop forwards to wzp-client/linux-aec so `cargo tauri build -- --features wzp-desktop/linux-aec` builds the AEC variant. APM configuration: - EchoCancellation: High suppression, delay-agnostic mode on, extended filter on, stream_delay_ms=60 initial hint - NoiseSuppression: High - HighPassFilter: on - AGC: off (can fight Opus encoder's own gain staging + adaptive quality controller; add later if users report low mic level) Frame size handling: - Pipeline uses 20 ms frames (960 samples @ 48 kHz mono) - APM requires strict 10 ms (480 samples) per call - Each 20 ms frame is split into two 480-sample halves, APM called twice, halves stitched back - Same pattern for render and capture sides - Carry-buffer logic handles the case where CPAL delivers samples in arbitrary chunk sizes that don't divide 960 Build infrastructure: - scripts/Dockerfile.linux-desktop-builder adds meson, ninja-build, python3, clang for the webrtc-audio-processing bundled build - scripts/build-linux-desktop-docker.sh takes a new --aec flag that enables the linux-aec feature and renames the output artifacts with an `-aec` suffix so noAEC and AEC variants can coexist on disk Task #30. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
03a80a3196 |
feat(windows): WASAPI capture backend with OS-level AEC
Adds a direct WASAPI microphone capture path for the Windows desktop build that opens the default communications endpoint via IMMDeviceEnumerator -> IAudioClient2 -> SetClientProperties with AudioCategory_Communications, turning on Windows's communications audio processing chain (AEC, noise suppression, automatic gain control). The communications AEC operates at the OS level and uses the system render mix as the reference signal, so echo from our existing CPAL playback stream is cancelled automatically with no per-process reference plumbing. Architecture: - New crates/wzp-client/src/audio_wasapi.rs module (~280 lines). Event-driven capture loop on a dedicated thread; pushes PCM into the same lock-free AudioRing used by the CPAL path. Same public API as audio_io::AudioCapture so downstream code is unchanged. - New `windows-aec` feature in wzp-client that pulls in the `windows` crate (Microsoft's official Rust COM bindings) gated to target_os = "windows" only. Enabling the feature on non-Windows targets is a no-op since both the module and the dep are cfg(target_os = "windows"). - lib.rs re-exports WasapiAudioCapture as AudioCapture when the feature is on, otherwise falls back to the CPAL AudioCapture. AudioPlayback is always the CPAL one — no reason to swap it. - desktop/src-tauri/Cargo.toml Windows target enables the new feature: `features = ["audio", "windows-aec"]`. Implementation notes: - Uses eCommunications role (not eConsole) for GetDefaultAudioEndpoint — the user-configured "communications" device that Teams/Zoom pick up, and the one Windows's AEC is tuned for. - Requests 48 kHz mono i16 with AUDCLNT_STREAMFLAGS_AUTOCONVERTPCM + SRC_DEFAULT_QUALITY so Windows handles any format conversion in the audio engine instead of rejecting our format. - Event-driven with SetEventHandle / WaitForSingleObject — no polling, minimal CPU cost between packets. - 200 ms wait timeout so the capture thread polls `running` often enough for Drop to stop cleanly even if the audio engine stalls (e.g. device unplug). Task #24. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
da09fdb6e9 |
windows(desktop): gate coreaudio / VoiceProcessingIO to macOS-only targets
First step of the Windows x86_64 desktop build: stop pulling
coreaudio-rs into the Windows dependency graph so the project can at
least run `cargo check --target x86_64-pc-windows-msvc`. Software AEC
is already disabled in engine.rs so there's nothing else to stub — the
macOS-specific VPIO path is skipped via #[cfg(target_os = "macos")] on
both sides and Windows falls through to the plain CPAL
AudioCapture/AudioPlayback branch that already existed.
crates/wzp-client/Cargo.toml
- coreaudio-rs optional dep moved under [target.'cfg(target_os = "macos")']
- `vpio` feature now uses `dep:coreaudio-rs` syntax and the gated dep
- Enabling `vpio` on Windows/Linux is a no-op at resolution time
crates/wzp-client/src/lib.rs
- `pub mod audio_vpio` is now #[cfg(all(feature = "vpio", target_os = "macos"))]
- Previously `vpio` alone was enough to try to compile the Core Audio
bindings, which would fail on non-Apple targets the moment the
feature flag was flipped on
desktop/src-tauri/Cargo.toml
- [target.'cfg(not(target_os = "android"))'] removed — was leaking
vpio into Windows/Linux via the catch-all.
- macOS: wzp-client with features = ["audio", "vpio"]
- Windows: wzp-client with features = ["audio"]
- Linux: wzp-client with features = ["audio"]
- Android: wzp-client with default-features = false (unchanged)
- Dropped the unused direct coreaudio-rs = "0.11" dep on macOS —
wzp-desktop's own sources never call Core Audio directly.
Verified via `cargo tree --target x86_64-pc-windows-msvc -p wzp-desktop`
that the Windows target now resolves wzp-client with cpal but without
coreaudio-rs. macOS target still resolves with coreaudio (direct via
vpio feature and transitively via cpal). macOS `cargo check` still
builds cleanly.
Cross-compile from macOS hit a cargo-xwin + llvm-lib setup issue in
ring's build.rs, so the actual `cargo check --target
x86_64-pc-windows-msvc` did not complete locally. Build verification
belongs on the user's Windows x86_64 host where MSVC is present
natively.
See tasks #23 (this one), #24 (Voice Capture DSP / WASAPI Communications
for OS-level AEC on Windows), and #25 (aarch64-pc-windows-msvc support).
|
||
|
Siavash Sameni
|
0178cbd91d |
android(audio): Speaker button toggles earpiece↔speaker via JNI (WIP, untested)
Build
|
||
|
Siavash Sameni
|
7cc53aedc7 |
refactor(android): split C++ into wzp-native cdylib, loaded at runtime
Phase 1 of the big refactor. Escape the Tauri Android __init_tcb+4 symbol leak (rust-lang/rust#104707) by making wzp-desktop's Android .so pure Rust — ZERO cc::Build, no cpp/ files, no C++ in the rustc link step. All future C++ (Oboe audio bridge) lives in a new standalone cdylib crate `wzp-native` which is built with cargo-ndk (the same path the legacy wzp-android crate uses successfully on the same phone + same NDK), copied into Tauri's gen/android/app/src/main/jniLibs at build time, and dlopened by wzp-desktop at runtime via libloading. Changes in this commit: - NEW crate crates/wzp-native/ with crate-type = ["cdylib"] only (no staticlib, no rlib — rust#104707 shows mixing staticlib with cdylib leaks non-exported symbols, which is the original bug source). Phase 1 scaffold has TWO extern "C" functions: wzp_native_version() -> i32 (returns 42) wzp_native_hello(buf, cap) -> usize (writes a string) So we can verify dlopen + dlsym + cross-.so FFI end-to-end before adding any real C++. - desktop/src-tauri/cpp/ directory DELETED (7 files gone). - desktop/src-tauri/build.rs reduced to just the git hash capture + tauri_build::build(). No more cc::Build of any kind. - desktop/src-tauri/Cargo.toml: drop cc from build-dependencies, add libloading = "0.8" as an Android-only runtime dep. - desktop/src-tauri/src/lib.rs Builder::setup() now (on Android only) dlopens libwzp_native.so, calls wzp_native_version() and wzp_native_hello(), and logs the result: "wzp-native dlopen OK: version=42 msg=\"hello from wzp-native\"" If this log appears in logcat when the app launches and the home screen still renders, the split-cdylib pipeline is validated and Phase 2 (port the Oboe bridge into wzp-native) can proceed. - scripts/build-tauri-android.sh: insert a `cargo ndk -t arm64-v8a build --release -p wzp-native` step before `cargo tauri android build`, with `-o desktop/src-tauri/gen/android/app/src/main/jniLibs` so the resulting libwzp_native.so lands in the place gradle will package into the final APK. - Workspace Cargo.toml: add crates/wzp-native to [workspace] members. Phase 2 (separate commit, only if Phase 1 works): - Copy cpp/oboe_bridge.{h,cpp} + getauxval_fix.c from the legacy wzp-android crate into crates/wzp-native/cpp/. - Add cc = "1" as a build-dependency on wzp-native (safe: it's a single-cdylib crate with no staticlib, so no symbol leak). - Add build.rs that compiles the Oboe C++ and the wzp-native Rust FFI exposes the audio start/stop/read/write functions. - wzp-desktop::engine.rs dlopens wzp-native at CallEngine::start, uses its audio functions instead of CPAL on Android. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
6071eb1b02 |
fix(android): drop staticlib from crate-type — root cause of __init_tcb crash
External research (per rust-lang/rust#104707) pointed at this as the highest-probability cause of our byte-identical __init_tcb+4 / pthread_create SIGSEGVs: > Having 'staticlib' alongside 'cdylib' in crate-type leaks non-exported > symbols from the staticlib into the cdylib's symbol table. For a > Tauri Android cdylib, that means bionic's private pthread_create / > __init_tcb code — which got pulled in statically from libc.a the > moment any cc::Build C++ file added C++-linkage overhead — ends up > bound LOCALLY inside our .so instead of being resolved dynamically > against libc.so at dlopen time. Symptoms that match the theory exactly: - llvm-nm on the crashing .so shows __init_tcb and pthread_create as LOCAL symbols with C++ name mangling (bionic's own pthread_create.cpp) - Adding any cc::Build cpp(true) step reliably triggers the crash, independent of which linker (android24-clang vs android26-clang) or which libc++ linkage (shared/static/none) - The legacy wzp-android crate (["cdylib", "rlib"]) works fine on the same phone with the same NDK + Rust toolchain + Oboe C++ code - tauri.conf.json bundle.android.minSdkVersion=26 propagates to gradle but the .so still crashes byte-identically Drop 'staticlib' from crate-type. If we ever need it for iOS, re-add behind a target.'cfg(target_os = "ios")' gate. The desktop binary still links against the rlib, so the bin target on macOS/Linux/Windows is unaffected. Source: https://github.com/rust-lang/rust/issues/104707 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
ae4f366b05 |
step B(android): depend on wzp-client with default-features=false
Second incremental variable on the path to Oboe. Adds a `[target.'cfg(target_os = "android")'.dependencies]` block that pulls in wzp-client with NO features enabled — no audio (no CPAL), no vpio (no VoiceProcessingIO). This gives the Android build access to wzp-client's platform-independent modules (call, handshake, audio_ring, codec wiring) without any system audio bindings. Deliberately no new imports in lib.rs or engine.rs. The only effect should be: cargo-tauri on Android now has to compile wzp-client and all its transitive crates (wzp-codec, wzp-fec, wzp-proto, wzp-crypto already pulled directly; now also audiopus, raptorq, etc.) and link them into libwzp_desktop_lib.so. Goal: verify that merely expanding the compiled code set to include wzp-client doesn't regress the previous working state. If it does, we know one of wzp-client's transitive deps is the problem — probably a C dep like audiopus or codec2. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
f96d7ce3e1 |
step A(android): add cc=1 build-dep + compile single trivial hello.c
First incremental variable on the path back to Oboe integration. Changes
are deliberately minimal: add cc = "1" to [build-dependencies] (cargo
build-deps resolve against the host so the line is unconditional), and
on the Android target run a single cc::Build step that compiles
cpp/hello.c — a 6-line file that defines one function (`wzp_hello_stub`)
that is never called from Rust.
Goal: verify that merely introducing a C static library into the .so
via cc::Build does not regress the working build (#17, commit
|
||
|
Siavash Sameni
|
530993854f |
revert(android): roll back to build #6 (35642d1) — pre-oboe known-good state
Spent 10+ builds chasing a __init_tcb+4 / pthread_create SIGSEGV after adding the oboe audio backend. Every "fix" made things worse. Reverting all Android-specific files to the state at |
||
|
Siavash Sameni
|
b314138caf |
feat(android): oboe/AAudio audio backend + runtime mic permission (step 3)
This is the big one — the Tauri Android app now has a real audio stack
capable of full-duplex VoIP, reusing the proven C++ Oboe bridge from the
legacy wzp-android crate.
Architecture:
- desktop/src-tauri/cpp/ — copies of oboe_bridge.{h,cpp}, oboe_stub.cpp,
and getauxval_fix.c from crates/wzp-android/cpp/. build.rs clones
google/oboe@1.8.1 into OUT_DIR and compiles the bridge + all Oboe
sources as "oboe_bridge" static lib, linking against shared libc++
(static would pull broken libc stubs that SIGSEGV in .so libraries).
- src/oboe_audio.rs — Rust side: an SPSC ring buffer matching the C++
bridge's AtomicI32 layout, plus OboeHandle::start() which returns
(capture_ring, playout_ring, owning_handle). The ring exposes the same
(available / read / write) methods as wzp_client::audio_ring::AudioRing
so CallEngine treats both backends interchangeably.
- src/engine.rs — compiled on every platform now. A cfg-switched type
alias picks wzp_client::audio_ring::AudioRing on desktop and
crate::oboe_audio::AudioRing on Android. The audio setup block has
three branches: VPIO/CPAL on macOS, CPAL on Linux/Windows, Oboe on
Android. Send/recv tasks are identical across platforms.
- src/lib.rs — removes all the "step 3 not done" Android stubs. The
engine module is no longer cfg-gated; connect / disconnect / toggle_mic
/ toggle_speaker / get_status are single implementations used by both
desktop and Android. Identity path resolves via app.path().app_data_dir()
from the Tauri setup() callback (already wired in step 1).
Runtime mic permission:
- scripts/build-tauri-android.sh now injects RECORD_AUDIO + MODIFY_AUDIO_
SETTINGS into gen/android/app/src/main/AndroidManifest.xml after init,
and overwrites MainActivity.kt with a version that calls
ActivityCompat.requestPermissions in onCreate. This is idempotent:
every build re-applies the patches so tauri re-init can't regress them.
Cargo.toml:
- cc is now an unconditional build-dep (build.rs runs on the host, so
target-gating build-deps doesn't work).
- wzp-client is now a dep on every platform. On Android it gets default
features only (no "audio"/"vpio") so CPAL isn't dragged in — oboe_audio
provides the capture/playout rings instead.
- tracing-android is added on Android so tracing events flow into logcat.
build.rs also gained embedded git hash (WZP_GIT_HASH) capture, which is
shown under the fingerprint on the home screen — already committed in
|
||
|
Siavash Sameni
|
e6f77a78a7 |
feat(desktop): split main.rs into lib.rs for Tauri Mobile (Android/iOS)
Tauri 2.x Mobile links the app as a cdylib loaded from a Java Activity, so all of the Builder/command code has to live in a library crate. Move the existing logic verbatim into src/lib.rs::run() and reduce src/main.rs to a two-line desktop entry point that calls into it. Cargo.toml gets a [lib] section (crate-types: staticlib + cdylib + rlib, named wzp_desktop_lib) and the wzp-client dependency — which pulls CPAL + VoiceProcessingIO — is moved behind cfg(not(target_os = "android")) so the Android cdylib doesn't need an audio backend yet. Engine-backed Tauri commands (connect/disconnect/toggle_mic/toggle_speaker/get_status) get Android stubs that return clear "not yet wired" errors. The signaling commands (register_signal/place_call/answer_call/get_signal_status/ ping_relay/get_identity) are platform-independent and unchanged. Also: get_identity / register_signal now auto-create the seed if missing instead of erroring with "connect to a room first", and the identity dir resolves to /data/data/com.wzp.phone/files/.wzp on Android (proper app-internal storage) vs \$HOME/.wzp on desktop. Side note: src/main.rs was previously untracked — desktop builds were working only because it existed in the local worktree. This commit fixes that too. Step 1 of the Android rewrite plan (tauri-mobile scaffold). No audio yet. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
f726f8cfa4 |
feat: desktop GUI enhancements — audio level, call timer, VPIO, settings
Some checks failed
Build Release Binaries / build-amd64 (push) Failing after 3m47s
- Audio level meter with log-scale RMS visualization - Call duration timer - VPIO (OS AEC) wired through to engine with fallback to CPAL - "You" badge on own participant entry - Recent rooms list (click to reuse) - Enter key to connect from form fields - Improved dark theme with pulse animation on status dot - Settings persistence via localStorage (relay, room, alias, AEC, recent rooms) - Fingerprint display on connect screen - Keyboard shortcuts skip input fields Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Siavash Sameni
|
e468454464 |
feat: Tauri desktop GUI app with call engine
Some checks failed
Build Release Binaries / build-amd64 (push) Failing after 3m27s
- New desktop/ directory with Tauri v2 + Vite + TypeScript - Rust backend: CallEngine wrapping wzp-client audio + transport - Web frontend: connect screen, in-call screen with participants, mic/speaker mute, keyboard shortcuts (m/s/q) - Dark theme UI, settings persistence via localStorage - Platform-aware --os-aec: warns on Windows/Linux (not yet implemented) - Workspace updated to include desktop/src-tauri Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |