Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Question Recorder + Wwise sync

Discussion in 'Audio & Video' started by nehvaleem, Mar 12, 2022.

  1. nehvaleem

    nehvaleem

    Joined:
    Dec 13, 2012
    Posts:
    421
    I am struggling with recording gameplay using the recorder and capturing audio that is driven by wwise. I am able to trigger capturing the output of wwise when recording starts but my main issue is that the recorder delays rendering of unity to maintain constant framerate while wwise is playing audio in real time. This is causing two different timelines which I think cannot be synced. Imagine ambient sound, for example, rain that is constantly playing and footsteps while the character runs. Those events will be out of sync when unity waits for the rendering of each frame to maintain a constant framerate.

    This makes the workarounds available in the documentation (https://docs.unity3d.com/Packages/com.unity.recorder@3.0/manual/KnownIssues.html) pretty useless in my opinion - or I am missing something really obvious.

    I would really appreciate any help about that. My goal is: record smooth gameplay with extra quality (uncompressed image sequence for example) with audio that is driven by wwise. Im ofc all up for composition in 3rd party software.
     
    highp likes this.
  2. vladala

    vladala

    Unity Technologies

    Joined:
    Mar 3, 2017
    Posts:
    189
    Hey,
    Indeed our audio recording implementation does not work with wise and I am not sure we can do much to support wwise. We could at least improve on the error handling / reporting.

    For the framerate issue: The recorder supports Variable framerate: Playback: Variable
    Screen Shot 2022-03-14 at 8.09.09 AM.png
    And recorder 4.0.0-pre.2 supports VFR with mp4 and webm files (make sure you don't get any later one because the latest minimum unity version jumps to 22.1 starting with 4.0.0-pre.3 )

    Hope this helps.

     
  3. nehvaleem

    nehvaleem

    Joined:
    Dec 13, 2012
    Posts:
    421
    well - I think that there is something like offline rendering in Wwise (and capturing output in Wwise Api) so I believe that recorder working with 3rd party solutions such Wwise is technically possible - but I guess its all down to your approach as a company - is it worth to just have look at it?
     
  4. max_autious

    max_autious

    Joined:
    Dec 29, 2017
    Posts:
    9
    I'm also missing this feature, being able to capture Wwise sound would make my life easier, it appears there's support for FMOD already?

    My current work-around is to record the desktop audio separately using Audacity and the WASAPI Loopback api, then remix the audio in a video-editing software in post.
     
    Last edited: Dec 23, 2022
  5. nehvaleem

    nehvaleem

    Joined:
    Dec 13, 2012
    Posts:
    421
    I've actually managed to get around this limitation, by writing a custom unity recorder that intercepts wwise. The only downside is that it records to separate wav and I didn't get to figure out why there is a slight difference in the duration of the audio track vs the video track. But it is synchronizable manually.
     
  6. Mr-Bigglesworth

    Mr-Bigglesworth

    Joined:
    Oct 28, 2013
    Posts:
    1
    Would you mind giving some insight on how you implemented the custom wwise recorder? Any information would be appreciated!
     
  7. nehvaleem

    nehvaleem

    Joined:
    Dec 13, 2012
    Posts:
    421
    Actually, some of you asked about that, so I am sharing it as a package available here: https://github.com/nehvaleem/com.undefined.wwise-recorder

    Please let me know if something is wrong.
     
  8. Even_taotao1

    Even_taotao1

    Joined:
    Jun 30, 2022
    Posts:
    1
    Hi, I tryed to use your package, but encountered some issues. When recording the GameView, it seems that to maintain a stable frame rate, it plays at nearly 0.5x speed during the recording, although the recorded video plays at the normal speed afterward. This resulted in recorded audio matches the recording speed but doesn't align with the final video. Did I use the package in a wrong way?