- Wire AdaptiveQualityController into desktop engine send/recv tasks
(mirrors Android pattern: AtomicU8 pending_profile, auto-mode check)
- Wire same into Android engine send task (was only in recv before)
- QualityDirective SignalMessage variant for relay-initiated codec switch
- ParticipantQuality tracking in relay RoomManager (per-participant
AdaptiveQualityController, weakest-link tier computation)
- Relay broadcasts QualityDirective to all participants when room-wide
tier degrades (coordinated codec switching)
- Oboe stream state polling: poll getState() for up to 2s after
requestStart() to ensure both streams reach Started before proceeding
(fixes intermittent silent calls on cold start, Nothing Phone A059)
Tasks: #7, #25, #26, #31, #35
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause: Oboe capture at 48kHz with InputPreset::VoiceCommunication
cannot open against a BT SCO device (only supports 8/16kHz). The stream
silently falls back to builtin mic, delivering zeros.
Fix: add bt_active flag to WzpOboeConfig. When set, capture skips
setSampleRate and setInputPreset, letting the system route to BT SCO
at its native rate. Oboe's SampleRateConversionQuality::Best resamples
to 48kHz for our ring buffers. Playout uses Usage::Media in BT mode.
New API: wzp_native_audio_start_bt() for BT mode, called from
set_bluetooth_sco(on=true). Normal audio_start() restores the
standard config when switching back to earpiece/speaker.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Two fixes for BT audio silence:
1. Switch Oboe streams from Exclusive to Shared sharing mode. Exclusive
mode bypasses Oboe's internal resampler, so opening a 48kHz stream
against a BT SCO device (8/16kHz only) fails at the AudioPolicy
level. Shared mode lets Oboe's resampler bridge the gap.
2. Add 500ms post-SCO delay before Oboe restart. The audio policy needs
time to apply the bt-sco route after setCommunicationDevice returns.
Without the delay, Oboe opens against the old device (handset).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
BT SCO devices only support 8kHz or 16kHz but our Oboe streams request
48kHz. Without resampling, AudioPolicyManager rejects the input stream
("getInputProfile could not find profile for... sampling rate 48000").
Fix: add setSampleRateConversionQuality(Best) to both capture and
playout stream builders. Oboe resamples internally so our ring buffers
stay at 48kHz regardless of the hardware sample rate.
Also removes the broken setBluetoothScoOn/isBluetoothScoOn calls from
stop_bluetooth_sco — just call stopBluetoothSco() unconditionally.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
With da106bd (Usage::Media + MODE_NORMAL) audio works but is always on
the loudspeaker — we want handset as the default with a user-driven
toggle for speaker (and later bluetooth). The right Oboe usage for a
VoIP app is VoiceCommunication, which honours
AudioManager.setSpeakerphoneOn / setBluetoothScoOn for routing.
Bisection across previous builds showed that setAudioApi(AAudio) +
Usage::VoiceCommunication made the playout callback stop draining the
ring after cb#0 (build 8c36fb5 logs). Letting Oboe pick the AudioApi
implicitly keeps the callback alive — 96be740's Media-usage callbacks
fired at steady 50Hz without any explicit setAudioApi. So: keep the
Usage change, DROP the explicit AAudio force.
- oboe_bridge.cpp: Usage::VoiceCommunication, no setAudioApi, no
ContentType override.
- MainActivity.kt: setMode(MODE_IN_COMMUNICATION) +
setSpeakerphoneOn(false) = handset default, plus max both
STREAM_VOICE_CALL and STREAM_MUSIC volumes for belt-and-braces.
Next build will add a JNI-based Tauri command to flip speakerphoneOn
at runtime so the user can toggle handset↔speaker during a call.
Build 8c36fb5 logs showed a new regression: Oboe playout cb#0 fires once
at startup then the callback STOPS DRAINING the ring entirely.
written_samples sticks at 7679 (= RING_CAPACITY - 1) across every recv
heartbeat in a 40-second test. Meanwhile the recv task decodes 1800+ real
audio frames (sample range up to [-27920..31907], rms 12065) which all
get dropped on the floor by audio_write_playout returning 0 because the
ring is full.
Bisection: 96be740 (Usage::Media, no setAudioApi, no ContentType, no
MainActivity audio mode change) DID drive the playout callback at the
expected 50Hz (playout heartbeat: calls=1100 total_played_real=1055040
over 22 seconds). User still heard nothing there because of OS routing,
but at least Oboe accepted the PCM.
8c36fb5 added three changes on top of 96be740:
1. Oboe Usage::Media → Usage::VoiceCommunication
2. Oboe setAudioApi(oboe::AudioApi::AAudio) explicit
3. Oboe setContentType(ContentType::Speech)
4. MainActivity setMode(MODE_IN_COMMUNICATION) + setSpeakerphoneOn(true)
Every one of those could have killed the callback; combined they did.
Revert to 96be740's exact Oboe config: Usage::Media, no setAudioApi, no
ContentType. Keep the PCM recorder, heartbeat logging, and stream-open
logging. Separately, MainActivity now maxes STREAM_MUSIC (the stream
Usage::Media routes to) but leaves audio mode in MODE_NORMAL — no more
speakerphone/call-mode combo that makes Oboe unhappy. In NORMAL mode a
STREAM_MUSIC stream plays through the loud speaker by default.
Proof that the Rust pipeline is perfect: decoded.pcm recorded in 8c36fb5
was pulled via `adb shell run-as com.wzp.desktop cat .wzp/decoded.pcm`,
converted with ffmpeg, and played back on the Mac — user confirmed
audible speech. So 100% of the remaining bug surface is Android audio
routing, not anything in the Rust/C++ decode path.
cc-rs build of oboe_bridge.cpp failed at cfa9ff6 because the Oboe
ResultWithValue<T> template returned by getXRunCount() does not have
a .value_or(T) method — only .value(). Replace with an explicit
bool-conversion + .value() guard that yields -1 on error.
Build 96be740 logs proved the entire software pipeline is healthy:
capture heartbeat: calls=1100 to_write=960 full_drops=0 total_written=1056000
recv heartbeat: decoded_frames=1035 last_written=960 decode_errs=0
recv decoded PCM: range=[-13564..9244] rms=8044 (real audio)
playout WRITE: in_len=960 written=960 rms=2318 (real audio into the ring)
playout heartbeat: calls=1100 nonempty=1099 total_played_real=1055040
1055040 samples / 48000 Hz = 22s — exactly matches wall-clock elapsed,
meaning Oboe IS calling our playout callback at the expected rate and
WE ARE handing it real PCM every 20ms. User still heard nothing. Ergo
Oboe accepted the PCM and routed it to a silent output. Two fixes:
1) MainActivity.kt: switch to MODE_IN_COMMUNICATION + speakerphone ON
right after permissions are granted, and crank STREAM_VOICE_CALL to
max. Without this, an Oboe Usage::VoiceCommunication stream gets
opened, the OS creates a real AAudio pipeline, the callback fires on
schedule — and audio goes to either the earpiece at muted volume or
a "call not active" dead end. Logs the audio mode + volume levels
before and after the switch so we can confirm the state change in
logcat next run.
2) oboe_bridge.cpp: revert Usage::Media → VoiceCommunication (the mode
that matches MODE_IN_COMMUNICATION), pin the audio API to AAudio
explicitly instead of letting Oboe fall back to OpenSLES (which has
its own silent-drop failure modes on some devices), and add getState
+ getXRunCount to the playout heartbeat so we'll see silent stream
disconnects instead of reading zeros forever.
3) engine.rs recv task: dump the first ~10s of post-AGC decoded PCM to
`<app_data_dir>/decoded.pcm` as raw i16 LE so we can adb pull it and
play it back locally:
adb shell run-as com.wzp.desktop cat .wzp/decoded.pcm > decoded.pcm
ffmpeg -f s16le -ar 48000 -ac 1 -i decoded.pcm decoded.wav
This divorces "is our decoder actually producing audible audio" from
"is Android's audio stack playing it". If the recorded WAV sounds
correct when played on a laptop, the decoder is fine and 100% of the
remaining bug surface is AudioManager / Oboe routing.
4) engine.rs: also log when spk_muted=true blocks the write. User
reported the Speaker button in the UI has inconsistent semantics
between desktop and android — adding this log rules out the accidental
"first click muted playback" theory for good.
User confirmed: mac hears android, android does not hear mac. So Oboe
capture works end-to-end but Oboe playout on Android silently drops
audio even though QUIC forwards the packets. Archaeology on the legacy
wzp-android crate also revealed that the "last known good" Android audio
path NEVER used Oboe in production — it used Kotlin AudioRecord +
AudioTrack via JNI, and cpp/oboe_bridge.cpp was dead code. So every time
we've "tested" Oboe end-to-end this week was the first production use,
and any of its config knobs could be the bug.
Instrumenting every stage of the pipeline so one smoke-test log dump can
isolate the layer at fault:
C++ (oboe_bridge.cpp)
- Log the ACTUAL stream parameters after openStream for both capture
and playout (sample rate, channels, format, framesPerBurst,
framesPerDataCallback, bufferCapacityInFrames, sharing, perf mode).
Oboe may silently override values we requested — e.g. if we ask for
48kHz mono but the device gives us 44.1kHz stereo our 960-sample
frames are the wrong duration and the pipeline drifts.
- Capture callback: on cb#0 log sample range+RMS of the first frame
to prove we get real mic data (not zeros). Every 50 callbacks
(~1s at 20ms burst) log calls, numFrames, ring available_write,
bytes actually written, ring_full_drops, total_written.
- Playout callback: on cb#0 log numFrames + ring state. On the FIRST
non-empty read log sample range+RMS so we can tell if the samples
coming out of the ring are real audio or zeros. Every 50 callbacks
log calls, nonempty count, numFrames, ring available_read,
underrun_frames, total_played_real.
Rust wzp-native (src/lib.rs)
- wzp_native_audio_write_playout now logs the first 3 writes and then
every 50th: in_len, written, sample range, RMS, ring write/read
cursors before, available_read and available_write after. Reveals
ring-overflow and whether the engine is actually handing us audio.
- Minimal android logcat shim via __android_log_write extern — no
new crate dependency.
- AudioBackend grows a `playout_write_log_count` AtomicU64 to gate
the write-side log throttle.
Rust engine.rs (android branch)
- Recv task: log sample range + RMS for the first 3 decoded PCM
frames and then every 100th. Reveals whether decoder.decode is
producing real audio or silent buffers.
- Recv task: if audio_write_playout returns fewer samples than we
handed it (partial write → ring nearly full) warn about it in the
first 10 frames.
- Recv heartbeat every 2s: recv_fr, decoded_frames, last_decode_n,
last_written, written_samples, decode_errs, codec.
Expected flow in a healthy log:
capture cb#0: numFrames=960 range=[-1200..900] rms=180 ← mic OK
capture stream opened: actualSR=48000 Ch=1 ... ← no override
playout stream opened: actualSR=48000 Ch=1 ...
CallEngine::start invoked ... → connected → audio started
recv: first media packet received ...
recv: decoded PCM sample range decoded_frames=1 range=[-300..250] rms=92
playout WRITE #0: in_len=960 written=960 range=[-300..250] rms=92
playout FIRST nonempty read: to_read=960 range=[-300..250] rms=92
playout heartbeat: calls=50 nonempty=50 underrun=0 ...
recv heartbeat: decoded_frames=100 last_written=960 ...
If any of those are missing/zero we know the exact stage to fix.
Three real bugs, one smoke-test session's worth of progress.
1. RELAY: wrong advertised addr in CallSetup
The direct-call CallSetup computed `relay_addr = addr.ip()` where
`addr = connection.remote_address()` — i.e. the CLIENT'S IP, not the
relay's. So the relay was telling both parties "the call room is at
the answerer's IP:4433", which meant each client dialed either the
other client (no server listening) or themselves. Both endpoint.connect
calls hung forever and the call never happened.
Fix: compute the relay's own advertised IP once at startup. If the
listen addr is 0.0.0.0, probe the primary outbound interface via the
classic UDP-bind-and-connect(8.8.8.8:80) trick to discover the LAN
IP the OS would use to reach external hosts. Thread the resulting
advertised_addr_str into the CallSetup sender for both parties.
2. RELAY: accept loop serialized QUIC handshakes
Previously the main accept loop called `wzp_transport::accept` which
did both `endpoint.accept().await` AND `incoming.await` (the server-
side QUIC handshake). A single slow handshake therefore blocked every
subsequent client from being accepted. Unroll the helper here and
move `incoming.await` into the per-connection spawned task, so every
handshake runs in parallel. Also log "accept queue: new Incoming",
"QUIC handshake complete", and "QUIC handshake failed" so we can tell
immediately whether a client's packets are reaching the relay at all.
3. ANDROID: playout was routed to the silent in-call stream
The Oboe playout stream was configured with Usage::VoiceCommunication,
which routes to the Android in-call earpiece stream. That stream is
silent unless the Activity has called AudioManager.setMode(
IN_COMMUNICATION) and, even then, only the earpiece/BT headset get
audio (not the loud speaker). Result: android→mac calls worked
because mac had a normal media output, but mac→android calls were
silent even though packets flowed through the relay just fine.
Switch to Usage::Media + ContentType::Speech so Oboe routes to the
loud speaker and uses the media volume slider. A later polish step
will wire setMode + setSpeakerphoneOn from MainActivity.kt so we can
go back to VoiceCommunication for AEC and proximity-sensor routing.
Plus: heartbeat tracing every 2s in the send/recv tasks — frames_sent,
last_rms, last_pkt_bytes, short_reads on the send side; decoded_frames,
last_decode_n, last_written, decode_errs on the recv side. Will make the
next "no sound" regression trivial to localize.
PlayoutCallback::onAudioReady crashed with SIGSEGV(SEGV_ACCERR) on the
first AAudio callback because g_rings was a `const WzpOboeRings*` pointing
at the caller's stack frame. wzp_native_audio_start() constructs the
rings struct as a stack local in Rust, passes &rings to wzp_oboe_start
(which stored the raw pointer), and returns — at which point the stack
frame unwinds and g_rings becomes a dangling reference. The first audio
callback then read from freed memory and died.
- g_rings is now a static WzpOboeRings value (was `const WzpOboeRings*`).
The raw int16 buffer + atomic index pointers inside the struct still
point into the Rust-owned AudioBackend singleton, which is leaked for
the lifetime of the process, so deep-copying the struct by value is
safe and keeps the inner pointers valid forever.
- g_rings_valid atomic bool gates the audio-callback reads: set to true
after the value copy in wzp_oboe_start, cleared in wzp_oboe_stop BEFORE
the streams are torn down so any in-flight callback sees "no backend"
and returns Stop instead of racing on g_rings.
- All g_rings->x accesses in the capture + playout callbacks switched to
g_rings.x (member-of-value).
Reproduced on Pixel 6 / Android 15 with build 0105b0f:
F libc: Fatal signal 11 (SIGSEGV), code 2 (SEGV_ACCERR),
fault addr 0x71aa717eb0 in tid 11822 (AudioTrack)
#00 PlayoutCallback::onAudioReady(oboe::AudioStream*, void*, int)+120
#01 oboe::AudioStream::fireDataCallback(void*, int)+136
...
Now that Phase 1 proved the split-cdylib pipeline (build #37 launched
cleanly with 'wzp-native dlopen OK: version=42 msg=...' in logcat),
this commit brings the real audio code into wzp-native without ever
touching the Tauri crate:
- cpp/oboe_bridge.{h,cpp}, oboe_stub.cpp, getauxval_fix.c copied
verbatim from crates/wzp-android/cpp/ (same files that work in the
legacy wzp-android .so on this phone)
- build.rs near-identical to crates/wzp-android/build.rs: clones
google/oboe@1.8.1 into OUT_DIR, compiles oboe_bridge.cpp + all
oboe source files as a single static lib with c++_shared linkage,
emits -llog + -lOpenSLES. On non-android hosts it compiles just
oboe_stub.cpp so `cargo check` works locally without an NDK.
- Cargo.toml gets cc = "1" in [build-dependencies]. This is SAFE
because wzp-native is a single-cdylib crate — crate-type is only
["cdylib"], no staticlib, so rust-lang/rust#104707 does not apply.
- src/lib.rs extends the FFI surface with the real audio API:
wzp_native_audio_start() -> i32
wzp_native_audio_stop()
wzp_native_audio_read_capture(*mut i16, usize) -> usize
wzp_native_audio_write_playout(*const i16, usize) -> usize
wzp_native_audio_capture_latency_ms() -> f32
wzp_native_audio_playout_latency_ms() -> f32
wzp_native_audio_is_running() -> i32
Plus a static AudioBackend singleton holding the two SPSC ring
buffers (capture + playout) that are shared with the C++ Oboe
callbacks via AtomicI32 cursors. The wzp_native_version() and
wzp_native_hello() smoke tests from Phase 1 are preserved.
Compiles cleanly on macOS host with the stub oboe .cpp. Next build
will exercise the full cargo-ndk path inside docker to verify the
whole Oboe compile still works standalone.
Phase 3 (next commit): wzp-desktop engine.rs on Android calls
wzp-native's audio FFI via the already-wired libloading handle, and
the real CallEngine::start() is implemented for Android using the
same codec/handshake/send/recv pipeline as desktop but with Oboe
rings instead of CPAL rings.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
