Search Unity

How to create subclasses of AudioTrack and AudioPlayableAsset

Discussion in 'Timeline' started by panta, Jul 25, 2018.

  1. panta

    panta

    Joined:
    Aug 10, 2012
    Posts:
    71
    Hi,

    I'm trying to add support to the AudioTrack in Timeline to be able to play a random AudioClip instead of a hardcoded one. My idea was to have 2 new classes:
    1.) RandomAudioPlayableTrack.cs
    Code (CSharp):
    1. using System.Collections;
    2. using System.Collections.Generic;
    3. using UnityEngine;
    4. using UnityEngine.Playables;
    5. using UnityEngine.Timeline;
    6. [TrackColor(1f, 0.67843137254901960784313725490196f, 0f)]
    7. [TrackBindingType(typeof(AudioSource))]
    8. [TrackClipType(typeof(RandomAudioPlayableAsset))]
    9. public class RandomAudioPlayableTrack : AudioTrack
    10. {
    11.     public override void GatherProperties(PlayableDirector director, IPropertyCollector driver)
    12.     {
    13.         Debug.Log("RandomAudioPlayableTrack.GatherProperties");
    14.         IEnumerable<TimelineClip> clips = this.GetClips();
    15.         foreach (TimelineClip curClip in clips)
    16.         {
    17.             RandomAudioPlayableAsset audioAsset = curClip.asset as RandomAudioPlayableAsset;
    18.             Debug.Assert(audioAsset != null, "There should only be " + typeof(RandomAudioPlayableAsset).ToString() + " clips on the track");
    19.             if (audioAsset.audioSpawner != null)
    20.             {
    21.                 audioAsset.clip = audioAsset.audioSpawner.GetRandomClip();
    22.                 curClip.displayName = audioAsset.clip.name;
    23.                 curClip.duration = audioAsset.clip.length;
    24.                 Debug.Log(audioAsset.clip.name);
    25.             }
    26.         }
    27.         director.Evaluate();
    28.         base.GatherProperties(director, driver);
    29.     }
    30. }
    31.  
    This class inherits from the built-in AudioTrack class, except it uses RandomAudioPlayableAssets for its clips, instead of the built-in AudioPlayableAsset. Whenever the track is updated (e.g., on Timeline instantiation in our game), the audio clip is randomized.

    2.) RandomAudioPlayableAsset.cs
    Code (CSharp):
    1. using System.Collections;
    2. using UnityEngine.Timeline;
    3. using Wonderstorm.Game.Abilities;
    4. public class RandomAudioPlayableAsset : AudioPlayableAsset
    5. {
    6.     public AudioSpawner audioSpawner;
    7. }
    8.  
    This class inherits from the built-in AudioPlayableAsset class, except it also has a reference to AudioSpawner, which is a ScriptableObject we've made that has a list of clips and a function to randomly select a clip.

    I'm able to create RandomAudioPlayableTracks in my Timeline, and I'm even able to create and assign audio clips to the RandomAudioPlayableAssets that I create on the timeline. I can hear them play back, but since this is a child class of AudioPlayableAsset, it seems that it doesn't get the waveform visual treatment on the timeline.

    Do you know why this is the case? Is there a way to enable that visual treatment for my custom subclass?

    Please let me know if you need more information, I can create a post a link to a min-repro if need be.
     
    Last edited: Jul 25, 2018
  2. seant_unity

    seant_unity

    Unity Technologies

    Joined:
    Aug 25, 2015
    Posts:
    1,190
    Oversight on our part. We didn't anticipate the AudioPlayableAsset to be derived from. The waveform drawing is linked explicitly to that class.
     
  3. panta

    panta

    Joined:
    Aug 10, 2012
    Posts:
    71
    @seant_unity is there any chance you can open up the waveform feature to either
    1. Any child clip class of AudioPlayableAsset
    2. As an option for any clip class that has an AudioClip as a property
    please let me know if you have any plans for this on your roadmap
    Thank you!
     
  4. seant_unity

    seant_unity

    Unity Technologies

    Joined:
    Aug 25, 2015
    Posts:
    1,190
    We plan on addressing both Audio, and customization soon, so we will definitely keep it in mind.
     
  5. panta

    panta

    Joined:
    Aug 10, 2012
    Posts:
    71
    @seant_unity thanks that's great to hear!

    I have one last question - I'm unable to get my code that I've placed in RandomAudioPlayableTrack.CreateTrackMixer to be called. I followed the patterns from the Default Playables examples on the Asset Store, but for whatever reason, the code I've put in RandomAudioPlayableTrack.CreateTrackMixer is never called, as well as all of the callbacks I overrode in RandomAudioPlayableMixerBehaviour. The strange part is that RandomAudioPlayableTrack.GatherProperties is called! Is there something special going on since RandomAudioPlayableTrack inherits from AudioTrack that is preventing CreateTrackMixer from being called?

    My goal is basically to call RandomAudioPlayableTrack.RandomizeAudioClips whenever my timeline is instantiated, so that I get random audio clips for all the different audio clips I have on my timeline. Maybe there's a better way to do this?


    Code (CSharp):
    1. using System.Collections.Generic;
    2. using UnityEngine;
    3. using UnityEngine.Playables;
    4. using UnityEngine.Timeline;
    5.  
    6. [TrackColor(1f, 0.67843137254901960784313725490196f, 0f)]
    7. [TrackBindingType(typeof(AudioSource))]
    8. [TrackClipType(typeof(RandomAudioPlayableAsset))]
    9. public class RandomAudioPlayableTrack : AudioTrack
    10. {
    11.     public override void GatherProperties(PlayableDirector director, IPropertyCollector driver)
    12.     {
    13.         Debug.Log("RandomAudioPlayableTrack.GatherProperties");
    14.  
    15.         RandomizeAudioClips();
    16.  
    17.         director.Evaluate();
    18.  
    19.         base.GatherProperties(director, driver);
    20.     }
    21.  
    22.     public override Playable CreateTrackMixer(PlayableGraph graph, GameObject go, int inputCount)
    23.     {
    24.         Debug.Log("RandomAudioPlayableTrack.CreateTrackMixer");
    25.         ScriptPlayable<RandomAudioPlayableMixerBehaviour> result = ScriptPlayable<RandomAudioPlayableMixerBehaviour>.Create(graph, inputCount);
    26.         result.GetBehaviour()._clips = GetClips() as TimelineClip[];
    27.         return result;
    28.     }
    29.  
    30.     public void RandomizeAudioClips()
    31.     {
    32.         IEnumerable<TimelineClip> clips = this.GetClips();
    33.         foreach (TimelineClip curClip in clips)
    34.         {
    35.             RandomAudioPlayableAsset audioAsset = curClip.asset as RandomAudioPlayableAsset;
    36.             Debug.Assert(audioAsset != null, "There should only be " + typeof(RandomAudioPlayableAsset).ToString() + " clips on the track");
    37.  
    38.             if (audioAsset.audioSpawner != null)
    39.             {
    40.                 audioAsset.clip = audioAsset.audioSpawner.GetRandomClip();
    41.                 curClip.displayName = audioAsset.clip.name;
    42.                 curClip.duration = audioAsset.clip.length;
    43.                 Debug.Log(audioAsset.clip.name);
    44.             }
    45.         }
    46.     }
    47. }
    48.  
    49. using UnityEngine;
    50. using UnityEngine.Playables;
    51. using UnityEngine.Timeline;
    52.  
    53. public class RandomAudioPlayableMixerBehaviour : PlayableBehaviour
    54. {
    55.     public TimelineClip[] _clips;
    56.  
    57.     public override void OnPlayableCreate(Playable playable)
    58.     {
    59.         Debug.Log("RandomAudioPlayableMixerBehaviour.OnPlayableCreate");
    60.     }
    61.  
    62.     public override void OnBehaviourPlay(Playable playable, FrameData info)
    63.     {
    64.         Debug.Log("RandomAudioPlayableMixerBehaviour.OnBehaviourPlay");
    65.         base.OnBehaviourPlay(playable, info);
    66.     }
    67.  
    68.     public override void OnPlayableDestroy(Playable playable)
    69.     {
    70.         Debug.Log("RandomAudioPlayableMixerBehaviour: OnPlayableDestroy");
    71.     }
    72. }
    73.  
    74. using System;
    75. using UnityEngine;
    76. using UnityEngine.Playables;
    77. using UnityEngine.Timeline;
    78. using Wonderstorm.Game.Abilities;
    79.  
    80. [Serializable]
    81. public class RandomAudioPlayableAsset : AudioPlayableAsset
    82. {
    83.     public AudioSpawner audioSpawner;
    84.  
    85.     public override Playable CreatePlayable(PlayableGraph graph, GameObject owner)
    86.     {
    87.         Debug.Log("RandomAudioPlayableAsset.CreatePlayable");
    88.         var playable = ScriptPlayable<RandomAudioPlayableBehaviour>.Create(graph);
    89.         return base.CreatePlayable(graph, owner);
    90.     }
    91. }
    92.  
    93. using System;
    94. using UnityEngine;
    95. using UnityEngine.Playables;
    96. using UnityEngine.Timeline;
    97.  
    98. [Serializable]
    99. public class RandomAudioPlayableBehaviour : PlayableBehaviour
    100. {
    101.  
    102. }
    103.  
    EDIT: I am able to get a callback for RandomAudioPlayableAsset.CreatePlayable, which is a start! I'm still confused why my callbacks in RandomAudioPlayableTrack and RandomAudioPlayableMixerBehaviour aren't getting called though...
     
    Last edited: Jul 25, 2018
  6. seant_unity

    seant_unity

    Unity Technologies

    Joined:
    Aug 25, 2015
    Posts:
    1,190
    AudioTrack doesn't use the track mixer method. It is hardcoded to use an AudioMixerPlayable, and that can't be overridden.
     
  7. panta

    panta

    Joined:
    Aug 10, 2012
    Posts:
    71
    @seant_unity ahh that makes much more sense now why I wasn't getting the callback... thank you for the reply
     
unityunity