Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice
  3. Dismiss Notice

Audio VideoPlayer and OnAudioFilterRead()

Discussion in 'Audio & Video' started by restreao_iis, Apr 26, 2024.

  1. restreao_iis

    restreao_iis

    Joined:
    Jul 11, 2022
    Posts:
    6
    Hi,

    Is it possible to feed the audio output of the Video Player to the OnAudioFilterRead() method? Or are they not compatible?

    I had already implemented some audio processing via the OnAudioFilterRead() method, which worked when I assigned an audio clip to the audio source and added my processing script.

    Now, I am trying to apply the same processing to the audio output of a VideoPlayer object. The VideoPlayer's audio output mode is set to "Audio Source", "Controlled tracks" = 1, I enabled the toggle button for "track 0", and assigned the audio source from my previous implementation to the VideoPlayer's audio source field. the audio output of the video is routed correctly to the audio source and all the inspector properties are applied correctly (i.e. output mixer routing, volume and spatial blend), but my OnAudioFilterRead() method is receiving an empty buffer.

    I also tried adding the audio processing script directly to the VideoPlayer and setting the output mode to "direct" but the OnAudioFilterRead() method is not even being called.
     
    Last edited: Apr 29, 2024
  2. SeventhString

    SeventhString

    Unity Technologies

    Joined:
    Jan 12, 2023
    Posts:
    454
    What version of Unity are you using? I just tried routing a video's audio through an AudioSource on the latest build and filters and OAFR are working as expected.
    Could you provide code examples and screenshots of you VideoPlayer component setup?
     
  3. restreao_iis

    restreao_iis

    Joined:
    Jul 11, 2022
    Posts:
    6
    I am using Editor 2022.3.12f1

    Here is my Video Player component
    upload_2024-5-2_13-50-46.png
    *A different script sets the video URL on Start()

    And here is the script on my Audio Source
    Code (CSharp):
    1. public class GBRSourceProcessing : MonoBehaviour
    2. {
    3.     [DllImport("audioplugin_gbr")]
    4.     private static extern void ingestCh(float[] inBuffer, int bufferLength, int numchannels, string streamID);
    5.     public string selectedStreamID;
    6.     void OnAudioFilterRead(float[] data, int channels)
    7.     {
    8.         int bufferLength = data.Length / channels;
    9.         ingestCh(data, bufferLength, channels, selectedStreamID);
    10.     }
    11. }
     
  4. restreao_iis

    restreao_iis

    Joined:
    Jul 11, 2022
    Posts:
    6
    Upon closer inspection, this is what I found:
    If I run the project with the GBRSourceProcessing script enabled and insert a break point on line 8 of the code above, data only contains 0s (even after a couple seconds of video playback).

    On the other hand, if I disable the GBRSourceProcessing script, run the project and enable the script after running, data has the right values and my script works as expected o_O
     
  5. SeventhString

    SeventhString

    Unity Technologies

    Joined:
    Jan 12, 2023
    Posts:
    454
    Can you try to do the same kind of test, but without the `DllImport` and just checking the content of `data`?
    1. Without the DLL, component enabled from the start, what's in `data`?
    2. Without the DLL, component enabled later at runtime, what's in `data`?
    If you see the same thing happening (all-zero `data` vs expected data) then could you please submit a bug report with a simplified version of you project isolating this? It could indicate an issue around VideoPlayer and OAFR's early calls from the audio thread.

    In the case where you get the expected behaviour in both cases without the DLL, then we'll dig deeper in that direction. It would not be the first time that something goes wrong in such a scenario.

    P.S.: my colleague from the desk besides suggests you check `channels` for 0 values, just in case!
     
  6. restreao_iis

    restreao_iis

    Joined:
    Jul 11, 2022
    Posts:
    6
    Without the DLL I get the same behaviour and channels = 2 so I have sent a bug report, thanks for your help!
     
    SeventhString likes this.
  7. SeventhString

    SeventhString

    Unity Technologies

    Joined:
    Jan 12, 2023
    Posts:
    454
    Hi @restreao_iis

    Can you send me the id you got for the bug you submitted? Thank you!
     
  8. restreao_iis

    restreao_iis

    Joined:
    Jul 11, 2022
    Posts:
    6
    Case IN-75697
    Internal ID UUM-72367
     
    SeventhString likes this.