Question SetActive audio issue

sasamori_tetsuya · May 29, 2023

Hello

SetActive(false) causes MainThread to hang until OnAudioFilterRead() completes

Is there a workaround?

SeventhString · May 29, 2023

I think you could add a "quick exit" check in your OnAudioFilterRead algo's heaviest loop to bail out quickly on demand.

sasamori_tetsuya · May 30, 2023

Dear SeventhString,

Thank you for your reply

with reference to the following

https://blog.unity.com/en/engine-platform/precise-framerates-in-unity

The video is linked with an external clock (GenLock)

Timing is adjusted by processing OnAudioFilterRead() so that the audio does not deviate from the video.

Therefore, this process is interrupted
i don't want to

SeventhString · May 30, 2023

If I understand correctly, OnAudioFilterRead() is the frame-based method you are using to throttle the rendering.

If your goal is to stop the audio while keeping a tight leash on the rendering, I think you will have to find a way to decouple your throttling code from OnAudioFilterRead and sync/signal your audio another way.

But honestly I am not certain to understand what you are trying to achieve. Are you trying to...

...control the game rendering based on the progress of an audio file?

...control a video playback based on the progress of an audio file?

...sync an audio file on game rendering or video playback?

What is the context of your `SetActive(false)` usage?

sasamori_tetsuya · May 31, 2023

I'm developing a plug-in that plays video and audio rendered by Unity on another device.
I'm trying to control audio rendering based on game rendering progress.

I want the audio rendering to progress at a pace of 801 (or 800) samples per frame (Genlocked)

I'm adding a wait inside OnAudioFilterRead() when the audio renders too early

Frame not advancing when SetActive() stops Mainthread

wait in OnAudioFilterRead() does not end until the frame advances

deadlock occurs

My customer uses SetActive() to toggle the display (sound playback) of two objects.

GameObject1 (Image and Audio)

GameObject2 (Image and Audio)

when button 1 is clicked
GameObject1.SetActive(TRUE);
GameObject2.SetActive(FALSE);

when button 2 is clicked
GameObject1.SetActive(FALSE);
GameObject2.SetActive(TRUE);

SeventhString · Jun 1, 2023

I think you're swimming against the current here. I would strongly advise against waiting in OnAudioFilterRead(), since it is called back from the audio mixing thread and could seriously impair other audio processing.

Can I suggest that you remove the waiting and instead, buffer the excess audio samples in a queue and feed them on the next frame?

sasamori_tetsuya · Jun 1, 2023

Yes. I'm buffering audio samples in a queue.
But, If I remove the wait the queue overflows in a long time.

because the genlocked output is slightly slower than 59.94.
(for example 59.93962)

Input is faster than output, so wait is necessary.

Is there any way to wait for audio rendering other than OnAudioFilterRead()?

SeventhString · Jun 2, 2023

I don't think there is. I think your best course of action would be to implement some shutdown code that would be executed prior to the SetActive(false) call.

sasamori_tetsuya · Jun 5, 2023

What do you mean by shutdown code?

Is it to skip waiting for OnAudioFilterRead()?

Is it to stop the audio completely?

sasamori_tetsuya · Jun 8, 2023

Is there a way to swap the order of the components from script?
I want to add a component before AudioSource that notifies SetActive to OnAudioFilterRead().

The below only works in Editor.
I want it to work on exe as well.

UnityEditorInternal.ComponentUtility.MoveComponentUp(component);

SeventhString · Jun 8, 2023

You can use Edit > Project Settings > Script Execution Order to specify in what order Update() are called.
What I meant by "shutdown code" was some code that would allow a graceful exit of your OAFR loop before calling SetActive(false).

sasamori_tetsuya · Jun 8, 2023

I have tried [script execution order].
Update() changed order but OnDisable() did not.
It seems to be in component order.
I am detecting SetActive(false) of GemeObject in OnDisable().

Is there a way to swap the order of the components from script?

sasamori_tetsuya · Jun 15, 2023

I changed it to swap the order of the components at build time.

SeventhString · Jun 21, 2023

sasamori_tetsuya said: ↑

I have tried [script execution order].
Update() changed order but OnDisable() did not.
It seems to be in component order.
Click to expand...

This is super interesting... thank you for bringing this up!

Were you able to fix your problem with this?

sasamori_tetsuya · Jun 22, 2023

For the AudioSource component

I was able to avoid the freeze.

For VideoPlayer component

I couldn't get around the freeze.
Is the timing of waiting for OnAudioFilterRead() to finish different from AudioSource?

SeventhString · Jun 22, 2023

OnAudioFilterRead is executed whenever data is drawn from the AudioSource before it's processed by the audio engine.

sasamori_tetsuya · Jun 23, 2023

Inside Update() I call SetActive(false) on the GameObject.

If the GameObject has an AudioSource component,
SetActive(false) never exits.
Because it keeps waiting for OnAudioFilterRead to finish.

If GameObject has VideoPlayer component,
SetActive(false) exit immediately.
but after 10ms,
It seems to keep waiting for the end of OnAudioFilterRead.

why?
Is the Disable processing always performed after one frame?

sasamori_tetsuya · Jun 27, 2023

I have two more questions.

without changing the sample rate
Is it possible to shorten the OnAudioFilterRead interval?

If I want the audio to work with an external clock (GenLock)
What should I do? Is there a way?

No description below.
https://blog.unity.com/en/engine-platform/precise-framerates-in-unity

sasamori_tetsuya · Jun 27, 2023

My customer asked me the following question.
Please give priority to answering the following.

Isn't it possible to solve the root cause (prevent SetActive from stopping the main thread) instead of a workaround?

SeventhString · Jun 27, 2023

I honestly do not know the answer right now. The best course of action is to submit a bug report and this will be taken to the most relevant team. I'm sorry I cannot answer this right away.

https://unity.com/releases/editor/qa/bug-reporting

sasamori_tetsuya · Jun 29, 2023

Dear SeventhString,

thank you for your advice.

I submitted a bug report

SeventhString · Jun 29, 2023

Sincerely sorry we couldn't figure this out here. Good luck @sasamori_tetsuya !

sasamori_tetsuya · Jul 5, 2023

Problems with AVProVideo

When SetActive (true) to GameObject2 below,
Waiting for OnAudioFilterRead() to finish, MainThread may freeze.

I'd like to cancel the processing of OnAudioFilterRead() before it freezes,
but MyComponent::OnEnable() is called after freezing.

Is there any way?

--------------------------------------------

GameObject1
- RenderHeads.Media.AVProVideo.MediaPlayer
(PlatFormSpecific/Windows/AudioMode=Unity)

GameObject2
- MyComponent
- AudioSource
- RenderHeads.Media.AVProVideo.AudioOutput
(MediaPlayer=GameObject1)

--------------------------------------------

SeventhString · Jul 5, 2023

In your OnAudioFilterRead method, you could add an additional check to see if audioSource.isPlaying is true before executing any further code. If you stop the AudioSource before disabling the GameObject, this could prevent additional processing in OnAudioFilterRead().

sasamori_tetsuya · Jul 6, 2023

I tried audioSource.isPlaying inside AudioFilterRead.
I got the following exception.

-------------------------------------------------------------------------
UnityException: get_isPlaying can only be called from the main thread.
-------------------------------------------------------------------------

SeventhString · Jul 6, 2023

Ok.... then maybe instead of calling SetActive(true) immediately, you could try delaying this call to the next frame using a coroutine or using Invoke("Method", delayTime). This could allow OnAudioFilterRead() enough time to finish its current cycle.

Code (CSharp):

StartCoroutine(ActivateNextFrame());

IEnumerator ActivateNextFrame()

{

yield return null; // Wait for next frame

gameObject2.SetActive(true);

}

sasamori_tetsuya · Jul 7, 2023

thank you

My ideal is to not change my customer's code.
gameObject2.SetActive(true) is my customer's code.

before the freeze occurs,
Is there a way for MyComponent to detect SetActive(true)?

SeventhString · Jul 7, 2023

I'm not aware of a way to setup the equivalent of an OnChange callback on this value.

Search Unity

Unity ID

Useful Searches

Question SetActive audio issue

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies

Unity Technologies