Search Unity

Question Best video file format to use in Unity? (and how to fix issues)

Discussion in 'Audio & Video' started by Steven-1, Sep 28, 2022.

  1. Steven-1

    Steven-1

    Joined:
    Sep 11, 2010
    Posts:
    471
    I have some prerendered cutscenes in my game, but the videoplayer doesn't always play them correctly.

    Almost always, only the left or right audio channel is playing, even though they are all stereo, and playing them in the preview window in the editor plays the sound correctly.
    The videoplayer also doesn't always knows when the video has ended, making it difficult for me to trigger something after the video has ended.
    The video also lags in the beginning, then speeds up to catch up with it, even though I wait for the video to be prepared.
    Some people even have the game crash after playing a video

    The videos are not transcoded, as I noticed the colors are wrong when enabling it. Could that be the cause of all these problems? (the fact that they are not transcoded)

    The videos are all H.264 MP4, as that always seems to me to be the default format that is supported everywhere.
    Maybe I'm wrong and there is a better format to use?

    And If I should transcode them, how do I keep the colors correct?
     
    Steven-1, Sep 28, 2022
    #1

  2. The_Island

    The_Island

    Unity Technologies

    Joined:
    Jun 1, 2021
    Posts:
    502
    Are you experiencing this on every platform? When you say "doesn't always knows when the video has ended" do you mean there is a delay or you never get the callback?
    Have you try VideoPlayer.waitForFirstFrame enable to true?
    Do you have a log, an error or reported the crash?
    For color related issues, do you mean they are wrong when compared in Unity vs an external software or between Unity build? When you imported your clips did you receive a warning about missing color standards? After that, is it possible the issue is with the material? Can you try with a simple unlit shader to see if you have the same issue?

    MP4 is the most popular format. The drawdown with MP4 is that the backend is different on every platform. Not only that, we expect the user to have the necessary dependency (like MediaFoundation). So it can be challenging to have the same result across platforms. If this is important for you, I would use Webm instead. Compare to MP4, webm has the same backend on every platform and has no dependency which makes it perfect for cross platforms.

    Edit: You can transcode your MP4 to Webm in the Editor by changing the codec from Auto to VP8 if you want to try.
     
    Last edited: Sep 28, 2022
    The_Island, Sep 28, 2022
    #2

  3. Steven-1

    Steven-1

    Joined:
    Sep 11, 2010
    Posts:
    471
    I only build for pc.

    I never get the callback. Atleast, on some computers it seems. I do get the callback on my pc.
    Also, the video stops 1 or 2 frames before the actual end of the video, dunno why that is.

    Yes

    No, it never crashed on my pc. I've just heard it from other people and saw it happen multiple times when streamers played my game.

    They are wrong compared to when they aren't transcoded. When they aren't transcoded they look in Unity exactly like they do outside Unity.
    When the colors are wrong, It kinda looks like when having the wrong colorspace in photoshop or something like that.
    I did not receive a warning about missing color standards.
    It's definitely not a material problem, I play the videos on the camera. Also, it looks correct when not transcoded.

    I make the videos in Adobe After Effects, I'll see if I can just select Webm with VP8 codec, instead of H.264 MP4
    Edit: Webm is not an option, I'll try what you said in Unity

    But will that solve the issues I'm having? Is the videos not being transcoded the reason for these issues?

    Edit:
    Using VP8 does make the colors be closer to the source than when just using auto (still not 100% the same though), but it also reduces the video quality quite a lot in some cases; I have 1 video that was about 30MB, and after transcoding it is only 3.8MB and contains a lot of compression artifacts. Bitrate mode and spatial quality are both set to High.
    Is there any way to prevent this extreme compression being applied?

    And still, is this necessary? Does having it be encoded to VP8 fix any of the mentioned problems?
    As most of the problems don't occur on my pc, but only for some (quite a lot actually) other players, I can't quickly test it.

    Also, VP8 did not fixed the audio issue, the videos still often only play one of the 2 audio channels
     
    Last edited: Sep 29, 2022
    Steven-1, Sep 29, 2022
    #3

  4. The_Island

    The_Island

    Unity Technologies

    Joined:
    Jun 1, 2021
    Posts:
    502
    So I assume you have the same problem on Mac, Windows and Linux, right? Or is it specific to one of them?
    With just this amount of information, it is difficult to help you out. If the VideoPlayer crashes it could potentially be, they are missing a dependency like MediaFoundation, which would not be an issue with Webm. Can you upload and share your clip here or in DM?
    Can you screenshot your settings like this? It should look similar.

    upload_2022-9-29_16-5-18.png

    What Unity version are you on? If you are not on the latest patch, can you try to update it and see if you still have the issue?
     
    The_Island, Sep 29, 2022
    #4

  5. Steven-1

    Steven-1

    Joined:
    Sep 11, 2010
    Posts:
    471
    I meant windows pc. I do intend to built for max and linux as well, but I haven't tested that yet.

    I'll send it to you later

    upload_2022-9-30_12-27-30.png

    This is one that doesn't have sound, for the others with sound, I ofcourse have import audio enabled.

    I'm using 2019.3.6f1
    Is there a known issue with audio channels not working that got fixed in a newer version?
    I don't like to update Unity all the time as that can break stuff and leaves me with a ton of different unity installations, so I only do that for good reasons or when I have the time to check and fix everything.
    I'm preparing for steam next fest next week, and was hoping to get the final demo build up today, so now would not be ideal to update.
     
    Steven-1, Sep 30, 2022
    #5

  6. The_Island

    The_Island

    Unity Technologies

    Joined:
    Jun 1, 2021
    Posts:
    502
    Nothing comes to mind, unfortunately. I know you don't want to upgrade, but I would try 2019.4.40f1. Every change from .3 to .4 should only be bug fixes and pretty stable.

    Otherwise, you can try to convert your mp4 to webm manually using FFmpeg.
    ffmpeg -i input.mp4 -c:v libvpx -pix_fmt yuva420p -b:v 2M -auto-alt-ref 0 output.webm


    At the same time, can you probe your file using FFprobe? (should come with FFmpeg)
    ffprobe input.mp4


    EDIT: Before that, if you are using an AudioSource, can you check if the pan stereo is at zero? https://docs.unity3d.com/ScriptReference/AudioSource-panStereo.html
     
    Last edited: Oct 3, 2022
    The_Island, Oct 3, 2022
    #6