Search Unity

  1. Unity 2019.2 is now released.
    Dismiss Notice

Confusion about how audio flows through playable graph

Discussion in 'Timeline' started by 5argon, Nov 6, 2019.

  1. 5argon

    5argon

    Joined:
    Jun 10, 2013
    Posts:
    1,303
    My plain is to have an always playing PlayableGraph with one master AudioMixer. The output then set its source playable to this master mixer. (SetSourcePlayable(mixer)) To live-add an audio to the mix, I planned to dynamically add more AudioClipPlayable to this mixer while the graph is playing.

    However I want some kind of logic before it so I made AudioObjectPlayableBehaviour. (An example is that when it detects the clip is done, then it destroy itself from the graph, or unloads audio clip, or reset time and repeat with a new audio clip, etc.)

    upload_2019-11-6_22-17-50.png

    In both case it works as intended, but I don't understand :

    1. Why the mixer could "mix my script" and understand that it should mix the AudioClipPlayable behind it? The mixer is usually connected to an another AudioMixerPlayable or AudioClipPlayable.
    2. Why when I set my script's `SetTraversalMode(PlayableTraversalMode.Passthrough);` then I cannot hear an audio anymore? From enum's description, it sounds like this should be what allowed me to hear the audio (so it passthrough, not blocking the mixer and the actual clip) but turns out setting it disconnect the "data flow"?
    3. Is the `output.SetSourcePlayable(playable)` overload without port, meant port 0? Or does that means all ports? Why when I set it explicitly to port 1, the case where there are 3 branches I could still hear the mixed result? It is supposed to cause the output to only evaluate a specific sub branch from the documentation. I look at the TimelinePlayable code and see that it creates an output for each track, which each output use SetSourcePlayable(timelinePlayabe,x) to the same root timeline playable, but with explicit port number so that the output only works with a sub branch of that. That goes according to my understanding, but not in my own graph/branches. Why is it different?

    My questions above is required to solve this next problem. Now I don't want to just plug in an audio clip, I want to plug in a playable created from TimelineAsset with audio tracks. So I manually call .CreatePlayable on the TimelineAsset in my own graph (with an output already, so it didn't create output per track anymore) then because I know all tracks are audio tracks, I mimic `ILayerable` by creating one more AudioMixerPlayable to sum them all up with full weight, and so a single output should be able to receive the audio. I understand that I cannot remove TimelinePlayable as it is the brain that activates the subgraph representing clips according to time interval, so that has to stand in-between the output and the audio.


    mixer.gif

    So I use the same reasoning, I want some custom logic in front of Timeline as that blue node (when it ends, remove itself, etc.) However in the graph above I cannot hear any audio. I expect the data to flow through somehow like the first case where my script is blocking the output, but now it didn't happen anymore. I want to know why?

    I notice that the Timeline has Passthrough set in the source code, so I think that maybe what my script is missing and I tried on my blue node. The result didn't change as I still cannot hear anything, plus it make the simpler case above broken as mentioned before. So I think I need some explanation why exactly Passthrough is needed on the TimelinePlayable, and how to solve this data path problem.

    Next I tried to remove just my script and reconnect so that there is only master mixer in the way. Output is still connecting to the same master mixer with no explicit port argument (or 0). I still can't hear anything. Why just the TimelinePlayable could block the audio still? You can see the output still cause an entire graph to be evaluated because TimelinePlayable's time goes forward and its interval logic works.

    licecap.gif

    Finally, this is how I get it to work. So it is just like how Timeline do it to a single track. But I lose my custom script playable and my master mixer (which I planned it to sum up many sub-audio-timeline and simpler audio clips to one output) in the process. So I would like to know how to properly "relay" the data through obstacle to the output. Thank you.

    licecap.gif
     
  2. 5argon

    5argon

    Joined:
    Jun 10, 2013
    Posts:
    1,303
    Hmm a week later I still can't figure this out..
     
  3. seant_unity

    seant_unity

    Unity Technologies

    Joined:
    Aug 25, 2015
    Posts:
    1,054
    The audio traverser knows to bypass scripts. You'll notice the audio object never has ProcessFrame called on it. Like animation, the process frame pass is custom.

    Passthrough changes the traversal. So instead of visiting all inputs, it only visits one input, which is determined by which output triggered the traversal. In your case, I wouldn't think it would make a difference. Maybe it's a bug since everything is 1 input, 1 output.

    Your understanding is correct. It should work the way you expect. Bug, or I'm missing something.


    If I may make a suggestion, after building the timeline set your outputs up this way. (This might work....I'm seen similar setups for animation, I haven't tried for audio).

    (Script)PlayableOutput 0 -> Set source to the timelinePlayable. Then disconnect it's input, so the audio subgraph isn't attached.

    (Audio)PlayableOutput 1 -> Set source to an audio mixer. No passthrough, so ports. Either the mixer timeline created, or a new one.

    When the graph evaluates, output 0 will 'prepare' and set the time for the audio clip playables, as if it were connected.
    output 1 will then play. Timeline will act like it's updating and activate clips, but you will take full control of your audio subgraph.
     
    5argon likes this.
  4. 5argon

    5argon

    Joined:
    Jun 10, 2013
    Posts:
    1,303
    Thank you, I followed this approach and finally got it to work! I didn't realize what is the use of ScriptPlayableOutput until now. While it make sense that the logic should not be in the same chain as the audio, it is a little weird that a logic somewhere else could control a non-child node.

    So for individual play the script playable check for audio clip in an another chain and remove itself together with the clip playable once it ends. Also I have to create an empty playable just so that one output could support multiple script playables, not sure if that is the proper way. (Kind of "ScriptMixerPlayable" but it didn't mix anything)

    selfChoke.gif

    For timeline play it is almost the same but there is a timeline script playable chained at the end so that could on-off the weight according to authored timeline, while an another script before it look for when the timeline's end and cut off itself and the whole timeline audio tree.

    Screenshot 2019-11-17 13.55.08.png

    Though I tried using the created ScriptPlayable<TimelinePlayable>'s GetDuration to predict when it should be considered to end (since mixer has infinite length, and any clips that composed the timeline are not indicative of an ending anymore) but it returns a very large number, so I use .duration of the timeline asset instead which seems to be correct. I suspected that maybe a bug so I submitted 1199034.
     
  5. seant_unity

    seant_unity

    Unity Technologies

    Joined:
    Aug 25, 2015
    Posts:
    1,054
    Looks good. As for the bug about duration, I think that's just because the playable director sets the duration of the playable, instead of the timeline asset itself.