Search Unity

Video How to add Audio to Video within Unity

Discussion in 'Audio & Video' started by apev, Apr 3, 2019.

  1. apev

    apev

    Joined:
    Feb 6, 2017
    Posts:
    4
    Hi everyone,
    I am trying to figure out how to add an audio file (or files) to a video from within Unity. If possible I'd like to avoid using ffmpeg, as it seems more complicated than what I need. For platforms I am aiming at iOS and Android. Any help would be appreciated!

    Context:
    I am working on an app in which at the end of gameplay, the user can choose to save a playback of their finished session. The way the app works, the users are combining a series of short video clips into a longer single sequence. The videos are rendered using material overrides, and each clip is made up of multiple layers with depth between them.

    The first limitation is that I am interposing dynamic objects between the different layers of the individual videos, so in order to get their completed session exactly as it was in game, I am recording the in-game camera view to generate the video.

    The second limitation is caused by the first, and is that I cannot record the in-game audio when I am recording the playback, due to the restriction that Android devices cannot record internal audio.

    Which lead me to recording the video silently, and adding the audio (which is not dynamic,) to the finished video afterwards. Each individual video clip that makes up the larger sequence has its own audio clip associated that plays simultaneously. I imagine it may be easier to combine these into a single audio clip before adding it to the video, but I am not certain.

    This needs to be done within Unity, as I'd like to avoid a server-based solution where I assemble video elsewhere, and the sequences users create are pretty dynamic. The intent is to allow users to share these videos after completion, and without interrupting gameplay by bogging their devices down with rendering.

    Thanks!
     
  2. Ikaro88

    Ikaro88

    Joined:
    Jun 6, 2016
    Posts:
    300
    no one can solve this?
     
  3. unitybru

    unitybru

    Unity Technologies

    Joined:
    Jan 28, 2020
    Posts:
    225
    I don't see how to do this using just Unity code: you want to open a (say, MP4 H.264) video-only file and add an audio track.
    There are standards to respect when doing this, which is exactly what ffmpeg allows you to do. Use ffmpeg because it does this properly.
     
    Ikaro88 likes this.
  4. Ikaro88

    Ikaro88

    Joined:
    Jun 6, 2016
    Posts:
    300
    Thanks a lot for your time i will start study that, but i can do that operation on runtime inside a unity mobile app?
     
  5. unitybru

    unitybru

    Unity Technologies

    Joined:
    Jan 28, 2020
    Posts:
    225
    I found this on Github: https://github.com/tanersener/mobile-ffmpeg
    I don't know how to integrate this into Unity, but it looks like a well maintained package. It will allow you to do that operation at runtime on mobiles for both Android and iOS.
    This is not a Unity endorsement of that package, and make sure to check the licenses (you might need to add the ffmpeg license in your end product, I'm not a lawyer but it's something to think of).