Remove Unity resampling#271
Open
MaxHeimbrock wants to merge 2 commits intomax/audiostream-replay-on-device-changefrom
Open
Remove Unity resampling#271MaxHeimbrock wants to merge 2 commits intomax/audiostream-replay-on-device-changefrom
MaxHeimbrock wants to merge 2 commits intomax/audiostream-replay-on-device-changefrom
Conversation
MaxHeimbrock
commented
May 6, 2026
90d0b04 to
3b570fc
Compare
3b570fc to
337c769
Compare
Configuring the native stream from AudioSettings.speakerMode at construction time can disagree with what OnAudioFilterRead actually delivers (per-source mono routing, spatializers, mixer chains, headphone profiles), and the C# RemixAndResample path it covered for is heavy work on the OS-audio-scheduled FFI callback thread. Defer the NewAudioStreamRequest until the first audio callback so we configure with Unity's authoritative (channels, sampleRate) and let Rust's NativeAudioStream do the rate/channel conversion once. OnAudioStreamEvent becomes a memcpy into the ring buffer. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When the system output device switches mid-call (e.g. headphones unplugged), Unity tears down its audio engine and OnAudioFilterRead resumes with a (channels, sampleRate) that may differ from what Rust was configured for. Detect the mismatch in OnAudioRead, post a CreateOrRecreateFfiStream to the main thread, and gate OnAudioStreamEvent on a _pendingFfiRequest flag so we drop in-flight frames at the old format. After the new stream is built, swap _handle, clear the buffer, reset priming, and dispose the old handle so its Rust task exits. The handle-id filter remains as a second line of defense for old-stream stragglers arriving during the brief overlap. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
337c769 to
433ff53
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Remove second resampling in Unity, by configuring the Rust audio stream with correct
(channels, sampleRate)to begin with. The problem is, that in Unity you don't know the channel number of each audio source until it runs, so we need to delay the initialization of the native audio stream until audio is read in Unity. The same lazy-create path is reused to recreate the native audio stream when Unity's delivered format changes mid-stream — for example after a system audio device switch.Changes
RemixAndResampleper-frame path.OnAudioStreamEventnow just memcpys the inboundAudioFrameinto the ring buffer — no FFI roundtrip on the OS-audio-scheduled callback thread. Rust'sNativeAudioStreamdoes the rate/channel conversion internally, once, with its existing pipeline.OnAudioRead.AudioSettings.GetConfiguration().speakerModedisagrees with whatOnAudioFilterReadactually delivers when the AudioSource is routed differently (per-source mono routing, spatializers, mixer chains, headphone profiles) — observed on macOS Editor + Sony headphones (system Stereo, source mono → low-pitched playback). The first callback is the authoritative source for(channels, sampleRate); we post a one-shotCreateOrRecreateFfiStream(...)to the main thread viaFfiClient._context, build theNewAudioStreamRequestwith the observed values, and only then subscribe toAudioStreamEventReceived.OnAudioReadcovers both first-create and runtime mismatch (channels != _ffiNumChannels || sampleRate != _ffiSampleRate). On mismatch we post the sameCreateOrRecreateFfiStreamto the main thread; the new stream is built outside the lock, then under the lock we swap_handle, update the_ffi*trackers, clear the buffer, reset priming, and clear the_pendingFfiRequestflag. Old handle is disposed AFTER the swap so its Rust task exits cleanly.OnAudioStreamEventdrops frames while_pendingFfiRequestis set so old-format bytes don't land in the new buffer; the existing handle-id filter catches any in-flight stragglers from the old stream after the swap. This pairs with Recover AudioStream when system audio output device changes #274 (re-attachAudioProbeand re-Play()the AudioSource on device change) so audio resumes seamlessly when the user plugs/unplugs headphones, switches Bluetooth devices, etc.Files
Runtime/Scripts/AudioStream.cs—_resamplerfield gone;Handleis now_handlebehind a property;CreateOrRecreateFfiStreamadded;_ffiNumChannels/_ffiSampleRate/_pendingFfiRequesttrack FFI-side configuration; constructor only sets up AudioSource/AudioProbe/pause hook.Tests/EditMode/MediaStreamLifetimeTests.cs— assertions updated for the renamed/removed members.Profiler
Before change, after 5 minutes and multiple audio sources: Total Audio CPU reaches ~100%

With change, after 5 minutes and multiple audio sources: Total Audio CPU stays below ~10%
