Search Unity

Bug WebGL Spatial audio for Videoplayer

Discussion in 'Audio & Video' started by Zebet, Aug 6, 2020.

  1. Zebet

    Zebet

    Joined:
    Mar 29, 2020
    Posts:
    6
    Hi all. I have a webgl project and I'm having a problem. I have a videoplayer which pulls in an external video clip (via URL) - this is working fine, but I set the Audio Output Mode to Audio Source and then set up the audio source so that Spatial Blend is 1 (3D) and custom rolloff with correct max and min distance settings.

    In the editor, it all works like a dream. When I build and run as webgl, the audio for the video player plays loudly and at the same level (volume) throughout the entire scene.

    There are several other audio sources in the scene which are not associated with a video player and they all work as expected.

    I read in another forum from a year or two ago that setting a videoplayer audio output to audio source is not supported in webgl (but that was a known bug a couple of years ago and I presume would have been fixed by now). Does anyone know of a fix for this?
     
  2. DominiqueLrx

    DominiqueLrx

    Unity Technologies

    Joined:
    Dec 14, 2016
    Posts:
    260
    Hi!

    This is not a bug as much as a limitation of the HTML5 <video> tag implementation, which is what Unity's VideoPlayer is using. One cannot take samples out of its audio output and pipe them through arbitrary processing (like would be required in this case): only straight audio output into the system's audio device is supported.

    There exists other code paths in HTML5 media support that may, in some browsers, allow this, but this isn't planned for the near future.

    In the mean time, your best option would be to work with a separate audio clip and play this in sync with the (audio-less) video.

    Hope this helps,

    Dominique Leroux
    A/V developer at Unity
     
    Zebet likes this.
  3. Zebet

    Zebet

    Joined:
    Mar 29, 2020
    Posts:
    6
    Thanks Dominique. Do you have any thoughts on how to synch the audio and video? I can use C# to load the audio from a URL (as the video is also loading from a URL) - is there a way to delay playback on both the videoplayer and the audiosource until both are loaded?
     
  4. Zebet

    Zebet

    Joined:
    Mar 29, 2020
    Posts:
    6
    Sortof Solved but also sortof not lol

    So as far as I can see, there is no way of streaming an external mp3 using webgl as a platform. In our project, loading time and project size are already problematic so we're trying to externalize as much size-intensive assets (eg media) as possible. there *must* be a way to stream mpeg audio in webgl but setting that aside, if I just swallow the bad news and take the size hit, I can easily sync the video and the audio. See error message below lol

    Streaming of 'mpeg' on this platform is not supported

    ps - if there is a way to stream mpeg audio I'd love to hear about it - looked all over the webs and haven't found anything yet.

    pps - if there is not a way to stream mpeg audio, can this be added to the punchlist for future revisions? I'm sure I'm not alone in wanting to keep as much unnecessary weight out of the project size as possible and assets like audio clips and video tracks are huge... Thank you in advance!
     
  5. DominiqueLrx

    DominiqueLrx

    Unity Technologies

    Joined:
    Dec 14, 2016
    Posts:
    260
    Hi again!

    The limitation of audio mp3 streaming seems to be one imposed by Unity, not by the browsers, possibly coming from an era where this wasn't broadly supported (just guessing here...). We'll test it out and remove this restriction if everything works out. It's been pointed out earlier: https://issuetracker.unity3d.com/is...le-in-editor-with-build-platform-set-to-webgl so it's already in the todo.

    Looking at the current code, .wav may work but streaming of uncompressed audio is not a very exciting solution.

    So for the time being having a built-in audio asset (not streamed) and a video stream seems to be the other compromise that's available to you.

    Sorry for all the trouble... Hope you find a way to your goal; let us know how this goes and we'll see if there's anything we can think of that can help.

    Dominique
     
  6. alteruss

    alteruss

    Joined:
    Mar 3, 2020
    Posts:
    4
    I am running into the same issue. I decided I will use built-in audio and stream the video (without sound)
    I am clueless as to how I can sync the video and the audio. Does anyone have some clues or script I can look at?
     
  7. Wasserwecken

    Wasserwecken

    Joined:
    Oct 9, 2017
    Posts:
    6
    I solved that problem for me. But only because I am interested into the audio fallof by distance.

    I just controll the volume manually by the player distance to my videoplayer.
    But I think with a dot-product and mutliple audio tracks you can also emulate the spread.

    I have to say that I'm not an audio guy and just made it work on a ver simple level.

    Code (CSharp):
    1. using UnityEngine;
    2. using UnityEngine.Video;
    3.  
    4. public class VideoWebGL : MonoBehaviour
    5. {
    6.     [Header("References...")]
    7.     public VideoPlayer Video;
    8.     public Transform PlayersView;
    9.     public string VideoLocation;
    10.  
    11.     [Header("Audio...")]
    12.     public AnimationCurve distanceInterpolation = AnimationCurve.Linear(0, 0, 1, 1);
    13.     public float minDistance = 1;
    14.     public float maxDistance = 5;
    15.  
    16.  
    17.     private void Awake()
    18.     {
    19.         Video.url = System.IO.Path.Combine(Application.streamingAssetsPath, VideoLocation);
    20.     }
    21.  
    22.     private void Update()
    23.     {
    24.         var currentDistance = Vector3.Distance(transform.position, PlayersView.position);
    25.         var relativeDistance = Mathf.Clamp01((currentDistance - minDistance) / (maxDistance - minDistance));
    26.         var currentVolume = distanceInterpolation.Evaluate(1 - relativeDistance);
    27.         Video.SetDirectAudioVolume(0, currentVolume);
    28.     }
    29.  
    30.     private void OnDrawGizmosSelected()
    31.     {
    32.         Gizmos.color = new Color(.5f, .5f, 1, 1);
    33.         Gizmos.DrawWireSphere(transform.position, minDistance);
    34.         Gizmos.DrawWireSphere(transform.position, maxDistance);
    35.     }
    36. }
    37.  
     
    BhudevKestone, vuexr and AdrianGTR like this.
  8. AdrianGTR

    AdrianGTR

    Joined:
    Jan 6, 2021
    Posts:
    4
    I love the way you think! Hahaha

    You saved my day.

    Just to save some time to the next users (I'm sure it is an obviousness for many people): You have to put the Video Player "Audio Output Mode" in "Direct" to take effect. ;)
     
    vuexr likes this.
  9. melones

    melones

    Joined:
    Dec 19, 2021
    Posts:
    2
    I have spent 2 days debugging the same issue with 10 different unity versions, 5 browsers, and 3 OSes, dammit.
    I don't understand why this hasn't been fixed yet, sad to lose all this time.
     
    MaxLohMusic likes this.