User confirmed: mac hears android, android does not hear mac. So Oboe
capture works end-to-end but Oboe playout on Android silently drops
audio even though QUIC forwards the packets. Archaeology on the legacy
wzp-android crate also revealed that the "last known good" Android audio
path NEVER used Oboe in production — it used Kotlin AudioRecord +
AudioTrack via JNI, and cpp/oboe_bridge.cpp was dead code. So every time
we've "tested" Oboe end-to-end this week was the first production use,
and any of its config knobs could be the bug.
Instrumenting every stage of the pipeline so one smoke-test log dump can
isolate the layer at fault:
C++ (oboe_bridge.cpp)
- Log the ACTUAL stream parameters after openStream for both capture
and playout (sample rate, channels, format, framesPerBurst,
framesPerDataCallback, bufferCapacityInFrames, sharing, perf mode).
Oboe may silently override values we requested — e.g. if we ask for
48kHz mono but the device gives us 44.1kHz stereo our 960-sample
frames are the wrong duration and the pipeline drifts.
- Capture callback: on cb#0 log sample range+RMS of the first frame
to prove we get real mic data (not zeros). Every 50 callbacks
(~1s at 20ms burst) log calls, numFrames, ring available_write,
bytes actually written, ring_full_drops, total_written.
- Playout callback: on cb#0 log numFrames + ring state. On the FIRST
non-empty read log sample range+RMS so we can tell if the samples
coming out of the ring are real audio or zeros. Every 50 callbacks
log calls, nonempty count, numFrames, ring available_read,
underrun_frames, total_played_real.
Rust wzp-native (src/lib.rs)
- wzp_native_audio_write_playout now logs the first 3 writes and then
every 50th: in_len, written, sample range, RMS, ring write/read
cursors before, available_read and available_write after. Reveals
ring-overflow and whether the engine is actually handing us audio.
- Minimal android logcat shim via __android_log_write extern — no
new crate dependency.
- AudioBackend grows a `playout_write_log_count` AtomicU64 to gate
the write-side log throttle.
Rust engine.rs (android branch)
- Recv task: log sample range + RMS for the first 3 decoded PCM
frames and then every 100th. Reveals whether decoder.decode is
producing real audio or silent buffers.
- Recv task: if audio_write_playout returns fewer samples than we
handed it (partial write → ring nearly full) warn about it in the
first 10 frames.
- Recv heartbeat every 2s: recv_fr, decoded_frames, last_decode_n,
last_written, written_samples, decode_errs, codec.
Expected flow in a healthy log:
capture cb#0: numFrames=960 range=[-1200..900] rms=180 ← mic OK
capture stream opened: actualSR=48000 Ch=1 ... ← no override
playout stream opened: actualSR=48000 Ch=1 ...
CallEngine::start invoked ... → connected → audio started
recv: first media packet received ...
recv: decoded PCM sample range decoded_frames=1 range=[-300..250] rms=92
playout WRITE #0: in_len=960 written=960 range=[-300..250] rms=92
playout FIRST nonempty read: to_read=960 range=[-300..250] rms=92
playout heartbeat: calls=50 nonempty=50 underrun=0 ...
recv heartbeat: decoded_frames=100 last_written=960 ...
If any of those are missing/zero we know the exact stage to fix.
332 lines
11 KiB
Rust
332 lines
11 KiB
Rust
//! wzp-native — standalone Android cdylib for all the C++ audio code.
|
|
//!
|
|
//! Built with `cargo ndk`, NOT `cargo tauri android build`. Loaded at
|
|
//! runtime by the Tauri desktop cdylib (`wzp-desktop`) via libloading.
|
|
//! See `docs/incident-tauri-android-init-tcb.md` for why the split exists.
|
|
//!
|
|
//! Phase 2: real Oboe audio backend.
|
|
//!
|
|
//! Architecture: Oboe runs capture + playout streams on its own high-
|
|
//! priority AAudio callback threads inside the C++ bridge. Two SPSC ring
|
|
//! buffers (capture and playout) are shared between the C++ callbacks
|
|
//! and the Rust side via atomic indices — no locks on the hot path.
|
|
//! `wzp-desktop` drains the capture ring into its Opus encoder and fills
|
|
//! the playout ring with decoded PCM.
|
|
|
|
use std::sync::atomic::{AtomicI32, Ordering};
|
|
|
|
// ─── Phase 1 smoke-test exports (kept for sanity checks) ─────────────────
|
|
|
|
/// Returns 42. Used by wzp-desktop's setup() to verify dlopen + dlsym
|
|
/// work before any audio code runs.
|
|
#[unsafe(no_mangle)]
|
|
pub extern "C" fn wzp_native_version() -> i32 {
|
|
42
|
|
}
|
|
|
|
/// Writes a NUL-terminated string into `out` (capped at `cap`) and
|
|
/// returns bytes written excluding the NUL.
|
|
#[unsafe(no_mangle)]
|
|
pub unsafe extern "C" fn wzp_native_hello(out: *mut u8, cap: usize) -> usize {
|
|
const MSG: &[u8] = b"hello from wzp-native\0";
|
|
if out.is_null() || cap == 0 {
|
|
return 0;
|
|
}
|
|
let n = MSG.len().min(cap);
|
|
unsafe {
|
|
core::ptr::copy_nonoverlapping(MSG.as_ptr(), out, n);
|
|
*out.add(n - 1) = 0;
|
|
}
|
|
n - 1
|
|
}
|
|
|
|
// ─── C++ Oboe bridge FFI ─────────────────────────────────────────────────
|
|
|
|
#[repr(C)]
|
|
struct WzpOboeConfig {
|
|
sample_rate: i32,
|
|
frames_per_burst: i32,
|
|
channel_count: i32,
|
|
}
|
|
|
|
#[repr(C)]
|
|
struct WzpOboeRings {
|
|
capture_buf: *mut i16,
|
|
capture_capacity: i32,
|
|
capture_write_idx: *mut AtomicI32,
|
|
capture_read_idx: *mut AtomicI32,
|
|
playout_buf: *mut i16,
|
|
playout_capacity: i32,
|
|
playout_write_idx: *mut AtomicI32,
|
|
playout_read_idx: *mut AtomicI32,
|
|
}
|
|
|
|
// SAFETY: atomics synchronise producer/consumer; raw pointers are owned
|
|
// by the AudioBackend singleton below whose lifetime covers all calls.
|
|
unsafe impl Send for WzpOboeRings {}
|
|
unsafe impl Sync for WzpOboeRings {}
|
|
|
|
unsafe extern "C" {
|
|
fn wzp_oboe_start(config: *const WzpOboeConfig, rings: *const WzpOboeRings) -> i32;
|
|
fn wzp_oboe_stop();
|
|
fn wzp_oboe_capture_latency_ms() -> f32;
|
|
fn wzp_oboe_playout_latency_ms() -> f32;
|
|
fn wzp_oboe_is_running() -> i32;
|
|
}
|
|
|
|
// ─── SPSC ring buffer (shared with C++ via AtomicI32) ────────────────────
|
|
|
|
/// 20 ms @ 48 kHz mono = 960 samples.
|
|
const FRAME_SAMPLES: usize = 960;
|
|
/// ~160 ms headroom at 48 kHz.
|
|
const RING_CAPACITY: usize = 7680;
|
|
|
|
struct RingBuffer {
|
|
buf: Vec<i16>,
|
|
capacity: usize,
|
|
write_idx: AtomicI32,
|
|
read_idx: AtomicI32,
|
|
}
|
|
|
|
// SAFETY: SPSC with atomic read/write cursors; producer and consumer
|
|
// are always on different threads.
|
|
unsafe impl Send for RingBuffer {}
|
|
unsafe impl Sync for RingBuffer {}
|
|
|
|
impl RingBuffer {
|
|
fn new(capacity: usize) -> Self {
|
|
Self {
|
|
buf: vec![0i16; capacity],
|
|
capacity,
|
|
write_idx: AtomicI32::new(0),
|
|
read_idx: AtomicI32::new(0),
|
|
}
|
|
}
|
|
|
|
fn available_read(&self) -> usize {
|
|
let w = self.write_idx.load(Ordering::Acquire);
|
|
let r = self.read_idx.load(Ordering::Relaxed);
|
|
let avail = w - r;
|
|
if avail < 0 { (avail + self.capacity as i32) as usize } else { avail as usize }
|
|
}
|
|
|
|
fn available_write(&self) -> usize {
|
|
self.capacity - 1 - self.available_read()
|
|
}
|
|
|
|
fn write(&self, data: &[i16]) -> usize {
|
|
let count = data.len().min(self.available_write());
|
|
if count == 0 {
|
|
return 0;
|
|
}
|
|
let mut w = self.write_idx.load(Ordering::Relaxed) as usize;
|
|
let cap = self.capacity;
|
|
let buf_ptr = self.buf.as_ptr() as *mut i16;
|
|
for sample in &data[..count] {
|
|
unsafe { *buf_ptr.add(w) = *sample; }
|
|
w += 1;
|
|
if w >= cap { w = 0; }
|
|
}
|
|
self.write_idx.store(w as i32, Ordering::Release);
|
|
count
|
|
}
|
|
|
|
fn read(&self, out: &mut [i16]) -> usize {
|
|
let count = out.len().min(self.available_read());
|
|
if count == 0 {
|
|
return 0;
|
|
}
|
|
let mut r = self.read_idx.load(Ordering::Relaxed) as usize;
|
|
let cap = self.capacity;
|
|
let buf_ptr = self.buf.as_ptr();
|
|
for slot in &mut out[..count] {
|
|
unsafe { *slot = *buf_ptr.add(r); }
|
|
r += 1;
|
|
if r >= cap { r = 0; }
|
|
}
|
|
self.read_idx.store(r as i32, Ordering::Release);
|
|
count
|
|
}
|
|
|
|
fn buf_ptr(&self) -> *mut i16 {
|
|
self.buf.as_ptr() as *mut i16
|
|
}
|
|
fn write_idx_ptr(&self) -> *mut AtomicI32 {
|
|
&self.write_idx as *const AtomicI32 as *mut AtomicI32
|
|
}
|
|
fn read_idx_ptr(&self) -> *mut AtomicI32 {
|
|
&self.read_idx as *const AtomicI32 as *mut AtomicI32
|
|
}
|
|
}
|
|
|
|
// ─── AudioBackend singleton ──────────────────────────────────────────────
|
|
//
|
|
// There is one global AudioBackend instance because Oboe's C++ side
|
|
// holds its own singleton of the streams. The `Box::leak`'d statics own
|
|
// the ring buffers for the lifetime of the process — dropping them while
|
|
// Oboe is still running would cause use-after-free in the audio callback.
|
|
|
|
use std::sync::OnceLock;
|
|
|
|
struct AudioBackend {
|
|
capture: RingBuffer,
|
|
playout: RingBuffer,
|
|
started: std::sync::Mutex<bool>,
|
|
/// Per-write logging throttle counter for wzp_native_audio_write_playout.
|
|
playout_write_log_count: std::sync::atomic::AtomicU64,
|
|
}
|
|
|
|
static BACKEND: OnceLock<&'static AudioBackend> = OnceLock::new();
|
|
|
|
fn backend() -> &'static AudioBackend {
|
|
BACKEND.get_or_init(|| {
|
|
Box::leak(Box::new(AudioBackend {
|
|
capture: RingBuffer::new(RING_CAPACITY),
|
|
playout: RingBuffer::new(RING_CAPACITY),
|
|
started: std::sync::Mutex::new(false),
|
|
playout_write_log_count: std::sync::atomic::AtomicU64::new(0),
|
|
}))
|
|
})
|
|
}
|
|
|
|
// ─── C FFI for wzp-desktop ───────────────────────────────────────────────
|
|
|
|
/// Start the Oboe audio streams. Returns 0 on success, non-zero on error.
|
|
/// Idempotent — calling while already running is a no-op that returns 0.
|
|
#[unsafe(no_mangle)]
|
|
pub extern "C" fn wzp_native_audio_start() -> i32 {
|
|
let b = backend();
|
|
let mut started = match b.started.lock() {
|
|
Ok(g) => g,
|
|
Err(_) => return -1,
|
|
};
|
|
if *started {
|
|
return 0;
|
|
}
|
|
|
|
let config = WzpOboeConfig {
|
|
sample_rate: 48_000,
|
|
frames_per_burst: FRAME_SAMPLES as i32,
|
|
channel_count: 1,
|
|
};
|
|
let rings = WzpOboeRings {
|
|
capture_buf: b.capture.buf_ptr(),
|
|
capture_capacity: b.capture.capacity as i32,
|
|
capture_write_idx: b.capture.write_idx_ptr(),
|
|
capture_read_idx: b.capture.read_idx_ptr(),
|
|
playout_buf: b.playout.buf_ptr(),
|
|
playout_capacity: b.playout.capacity as i32,
|
|
playout_write_idx: b.playout.write_idx_ptr(),
|
|
playout_read_idx: b.playout.read_idx_ptr(),
|
|
};
|
|
let ret = unsafe { wzp_oboe_start(&config, &rings) };
|
|
if ret != 0 {
|
|
return ret;
|
|
}
|
|
*started = true;
|
|
0
|
|
}
|
|
|
|
/// Stop Oboe. Idempotent. Safe to call from any thread.
|
|
#[unsafe(no_mangle)]
|
|
pub extern "C" fn wzp_native_audio_stop() {
|
|
let b = backend();
|
|
if let Ok(mut started) = b.started.lock() {
|
|
if *started {
|
|
unsafe { wzp_oboe_stop() };
|
|
*started = false;
|
|
}
|
|
}
|
|
}
|
|
|
|
/// Read captured PCM samples from the capture ring. Returns the number
|
|
/// of `i16` samples actually copied into `out` (may be less than
|
|
/// `out_len` if the ring is empty).
|
|
#[unsafe(no_mangle)]
|
|
pub unsafe extern "C" fn wzp_native_audio_read_capture(out: *mut i16, out_len: usize) -> usize {
|
|
if out.is_null() || out_len == 0 {
|
|
return 0;
|
|
}
|
|
let slice = unsafe { std::slice::from_raw_parts_mut(out, out_len) };
|
|
backend().capture.read(slice)
|
|
}
|
|
|
|
/// Write PCM samples into the playout ring. Returns the number of
|
|
/// samples actually enqueued (may be less than `in_len` if the ring
|
|
/// is nearly full — in practice the caller should pace to 20 ms
|
|
/// frames and spin briefly if the ring is full).
|
|
#[unsafe(no_mangle)]
|
|
pub unsafe extern "C" fn wzp_native_audio_write_playout(input: *const i16, in_len: usize) -> usize {
|
|
if input.is_null() || in_len == 0 {
|
|
return 0;
|
|
}
|
|
let slice = unsafe { std::slice::from_raw_parts(input, in_len) };
|
|
let b = backend();
|
|
let before_w = b.playout.write_idx.load(std::sync::atomic::Ordering::Relaxed);
|
|
let before_r = b.playout.read_idx.load(std::sync::atomic::Ordering::Relaxed);
|
|
let written = b.playout.write(slice);
|
|
// First few writes: log ring state + sample range so we can compare what
|
|
// engine.rs hands us to what the C++ playout callback reads.
|
|
let first_writes = b.playout_write_log_count.fetch_add(1, std::sync::atomic::Ordering::Relaxed);
|
|
if first_writes < 3 || first_writes % 50 == 0 {
|
|
let (mut lo, mut hi, mut sumsq) = (i16::MAX, i16::MIN, 0i64);
|
|
for &s in slice.iter() {
|
|
if s < lo { lo = s; }
|
|
if s > hi { hi = s; }
|
|
sumsq += (s as i64) * (s as i64);
|
|
}
|
|
let rms = (sumsq as f64 / slice.len() as f64).sqrt() as i32;
|
|
let avail_w_after = b.playout.available_write();
|
|
let avail_r_after = b.playout.available_read();
|
|
let msg = format!(
|
|
"playout WRITE #{first_writes}: in_len={} written={} range=[{lo}..{hi}] rms={rms} before_w={before_w} before_r={before_r} avail_read_after={avail_r_after} avail_write_after={avail_w_after}",
|
|
slice.len(), written
|
|
);
|
|
unsafe {
|
|
android_log(msg.as_str());
|
|
}
|
|
}
|
|
written
|
|
}
|
|
|
|
// Minimal android logcat shim so we can print from the cdylib without pulling
|
|
// in android_logger crate (which would add another dep that has to build with
|
|
// cargo-ndk). Uses libc's __android_log_print via extern linkage.
|
|
#[cfg(target_os = "android")]
|
|
unsafe extern "C" {
|
|
fn __android_log_write(prio: i32, tag: *const u8, text: *const u8) -> i32;
|
|
}
|
|
|
|
#[cfg(target_os = "android")]
|
|
unsafe fn android_log(msg: &str) {
|
|
// ANDROID_LOG_INFO = 4. Tag and text must be NUL-terminated.
|
|
let tag = b"wzp-native\0";
|
|
let mut buf = Vec::with_capacity(msg.len() + 1);
|
|
buf.extend_from_slice(msg.as_bytes());
|
|
buf.push(0);
|
|
unsafe { __android_log_write(4, tag.as_ptr(), buf.as_ptr()); }
|
|
}
|
|
|
|
#[cfg(not(target_os = "android"))]
|
|
#[allow(dead_code)]
|
|
unsafe fn android_log(_msg: &str) {}
|
|
|
|
/// Current capture latency reported by Oboe, in milliseconds. Returns
|
|
/// NaN / 0.0 if the stream isn't running.
|
|
#[unsafe(no_mangle)]
|
|
pub extern "C" fn wzp_native_audio_capture_latency_ms() -> f32 {
|
|
unsafe { wzp_oboe_capture_latency_ms() }
|
|
}
|
|
|
|
/// Current playout latency reported by Oboe, in milliseconds.
|
|
#[unsafe(no_mangle)]
|
|
pub extern "C" fn wzp_native_audio_playout_latency_ms() -> f32 {
|
|
unsafe { wzp_oboe_playout_latency_ms() }
|
|
}
|
|
|
|
/// Non-zero if both Oboe streams are currently running.
|
|
#[unsafe(no_mangle)]
|
|
pub extern "C" fn wzp_native_audio_is_running() -> i32 {
|
|
unsafe { wzp_oboe_is_running() }
|
|
}
|