Timelines with dynamically added outputs on creation.

Hi there, I'm trying to extend the timeline workflow for our artists to be able to specify tracks that affect multiple objects at once.

Basically the use case is:
We have effects and game events that have random number of involved elements, but we still want an artist to be able to specify how those elements (divided in to logical groups) are being animated and handled.

I have to say I got pretty close to having this system running, I only have one issue: being able to specify animation tracks to run with this.

I was able to specify custom tracks that when the TrackAsset.CreateTrackMixer is called, will pre cache all the objects that need to be involved. And when the TimelinePlayable invokes the `outputs` getter I return these objects wrapped in ScripPlayableBinding.

The issue I hit over here was the lack of playerData set on that output when the timeline was executing. I tried to SetGenericBindings, but unfortunately I don't have access to the playable director anywhere from the TrackAsset creation (except from GatherProperties). And I didn't want to additionally add functionality on top of the PlayableDirector to pre execute these bindings as the code creating those groupings is enclosed in a query system on top of the tracks and custom clips.
My solution, was to create a custom mixer that on ProcessFrame would actually call the output.GetReferenceObject() and output.SetPlayerData to that returned object - That worked great!

I was able to run both custom movement scripts nicely, Spine animations and such...

Until I went to integrate Unity's AnimationClips and AnimationOutputs.
The complications came from both what the animations require to properly run and how the timeline playables are executed when generating the graph.
Because the animation graph tree is quite complicated when I generate my custom graph I want to create my "wrapper" graph (for sorting out the output playerData) with a AnimationLayerMixer and AnimationMixer.
Unfortunately the TrackAsset.CreateTrackMixer() only allows me to return 1 target mixer, hence I cannot connect these tree.
I decided to try coming at it the other way, using the AnimationTrack as a base class, unfortunately that approach is also flawed as it overwrites the default creation of playable without giving any way of modifying it, which meant I never got access to the GameObject to fetch my dynamic objects.

I was wondering if there is a flow that I might be missing, I want to keep the logic of creating these dynamic outputs as simple as possible with possibly no changes to the component / asset structure apart from custom tracks and clips with extra functionality.

I already tried custom TimelineAsset extensions overwriting the CreatePlayable but it poses similar problems as when I do create the graph and want to process it later to potentially create the outputs, I'm missing all of the data from the tracks.

I have created custom outputs from the TrackAsset.CreateTrackMixer() and bound them manually, though they're always added before the actual timeline track output and don't get the time control, which I had to manually do, and sync from the main track - this worked, and allowed for animation to be played out, but had at least 1 frame worth of delay, and the last frame of a clip had never executed.

If there is nothing in place for allowing me to do this, would it be possible to consider some of these points:
- Ability to create custom PlayableBinding types with our own Create methods for outputs - currently most of this is internal and I'm unable to customise that flow
- Callback from TimelinePlayable.CreateTrackOutput() on the track, so we would be able to process these if needed
- Ability to overwrite AnimationTrack.CompileTrackPlayable or a callback to process the created tree.

We really want to use the timeline system for our in game animations, and it's proven to be quite nice and flexible - unfortunately our current implementation works around the above issues creating that delay in execution.

Regards,
Jakub

 

Hey @seant_unity thanks for your reply.
This will be long... but I'll try to get a lay down of how we ended up working around the issue

We're able to wrap the functionality in and get the result. I think my question would be if the Timeline team could consider in future feature set extending a bit more the control over what get's created, or extend the creation a bit more.

Just to give you an example what we've ended up doing...
I'll call all of our dynamic objects that the tracks are created for Targets, to make sure it's not mistaken with the Bindings
We have two main track classes:
1) Basic one that modifies the TrackAsset.outputs getter and creates a mixer with multiple outputs.
2) A wrapper track which creates custom outputs for each Target in the CreateTrackMixer method and custom mixers for synchronising the weights with the output and mixer that's attached to the timeline.

1. The implementation overwrites the CreateTrackMixer at which point it creates the list of dynamic Targets.
Creates a mixer that has to extend from an abstract class (will mention later why), returns it, and proceeds.
Next we overwrite the TrackAsset.outputs getter, which instead of returning the default getter will return custom list of
PlayableBindings with those Target objects.

When that track is executed, it hits the custom mixer ProcessFrame and checks if the playerData is null - first time it always is as I'm unable to write the correct user data for the output.
It then writes that userData from PlayableOutputExtensions.GetReferenceObject(info.output); and it proceeds to execute the mixer.

2. The wrapper approach comes from the need of creating a new playable tree per item, when it cannot be just a lot of outputs attached to a single mixer, like when using Unity Animations or Spine Animations.

In the TrackAsset.CreateTrackMixer we not create custom outputs on the graph, which we cache and we create a custom set of mixers. We attach those mixers to the output, and we create a Wrapper mixer which get's the custom mixers cached in - and that's the mixer we return.

When the track creates playables for each of the clips, we attach them to the custom cached mixers as well as the default connection to the Wrapper mixer.

Here comes the problem, we now have let's say custom animation tracks for 6 Targets, but they're not connected to the timeline, they're connected to custom outputs, so the connection between our custom mixers and the clips will never get it weight set.

That's where the Wrapper mixer comes in, having the list of custom mixers it will iterate trough all inputs, and then update the weights on all cached mixers.
Unfortunately, because the custom created outputs are made before the actual timeline output, the execution of the weight update is always after the custom mixer update, causing the objects to update 1 frame to late, or never to execute the last frame if it's the last frame of the timeline.

--

I have tried the post processing of a graph after it's created, but it proves to be more tricky, as all of the logic for finding targets and assigning them needs to be unified in that processing class, that makes further extensions more complicated as the targets have different rules and types of object being affected - keeping that to the track is a much nicer way to work with.

Tried using SetCustomBinding for all of the objects involved, but similarly it makes a big "master class" with a lot of junk and we additionally need to cache these lists for each instance as we're caching all of our effects, the bindings don't just get reset, and than random things start animating.

I've tried creating a new AnimationOutput for each animator and assigning to the same track and as you mentioned, that was definitely not the right way. Got some interesting glitches.

I don't really want to modify the asset dynamically to create new tracks, as I mentioned, each time we spawn that effect, it's coming from a object pool, it than needs to be reset, and keeping track that the asset has no modifications after each play is also a hurdle.

--

So in conclusion, this approach works for us, and we're able to get around it, we've been able to mostly solve the last frame not executing by adding additional OnGraphStop() and some value caches in some of the Mixers.

My question though would be, would the Timeline team consider adding some more flexibility around track creation and or being able to extend some of the internal classes a bit better.

Like:
Creating custom PlayableBinding types with custom CreateOuput methods <- that would be actually great, would solve the ability to assign custom set user data.

TrackAsset.CreateTrackMixer - to have additional flow in which we can specify more than just one playable, basically to return one that get's assigned to the Timeline playable, and one that the clips get assigned to.
That way we could easily create a AnimationLayerMixer and AnimationPlayableMixer combo on a track, now I'm only able to return one playable :/

Ability to add outputs that get assigned to the Timeline playable - as mentioned before, when I create a custom output, I'm unable to set in which order it's created and the playables I assign there don't have a reference to the Timeline playable
As a workaround, I'm ok having the Synchronisation Wrapper Mixer that I mentioned before, but if my custom outputs get executed before that, so an ability to create those outputs after the default ones are done, would be amazing. Even a custom callback.

Thanks! Hope this sheds some more detail on what we're doing with the timelines, and maybe gives you ideas for consideration around dynamic features of the Timelines.

 

Just noticed I posted the messages from different Unity accounts ;P

 

Hello and sorry for unburying this old thread.
I do not at all understand

outputs

and maybe one of you guys could help me out.
My issue is described here: https://forum.unity.com/threads/create-a-track-that-both-plays-an-audioclip-and-animations.1284296/