Feature Request New animation system with graph node

Taken idea and familiarity from VFX and shadergraph. I think unity animation, animator, blendtree, mecanim, rigging, state transition and everythings about animation should be reworked into graph node system like shadergraph

There was many times I would like to have be an animation parameter from the system. Such as camera position. Angle between model facing and target as blendshape value. Calculate direct animation / blend tree position / animation speed or mirror with math formula. Such as I might have only -1 to 1 value for animation parameter. But if it negative it will set animation as mirror and blend with absolute value. Or control direct motion by sine(distance) instead of time

Not to mention reaction with arbitrary object as IK and map one parameter to another value. Change quaternion to euler angle or change angle to trigonometry value. Convert radian to degree and so on

All of this are too tedious to control by monobehaviour. It also need to go back and forth between animation and animator and blendshapes and code so if these system was all become graph node that can control the flow of values and animation motion

I can imagine we could have animationclip node with time as default node's input



And then we might have complex blendtree and input to control animation with any function and logic in between



We might reuse Animation Transition UI and BlendTree UI in here but the concept is the same. And we might have some clip that input into in the blendtree have direct motion based on another input. We might have another kind of time function. And might be making whole animation rigging over this system

Any thought?

 

I've made some similar attempts, but I didn't finish it, there're too many limitations under the current Playable structure:
Puppeteer: A graph based animation controller for Unity.

If you know Chinese, here's the conclusion of the puppeteer plugin.

Expect to see Unity's official graph-based animation tool.

 

@SolarianZ Your efforts look really nice. Good work. I was also able to auto-translate your conclusions and they were quite readable. Thanks for that.

I am wondering if you encountered a specific concern during your project.

I am attempting a similar project (though different somewhat in final objective) and I have been trying to build a GraphView-based Editor Window that can render a PlayableGraph.

PROBLEM:
I need to know what input on node n is connected to what output on node m. Currently, every single API that ultimately called Graph.Connect requires the source output port and destination input port. However, once you already made this connection it doesn't seem possible to ask "what output port is this input coming from" or vice-versa.

Curious to know if you ran into this as well. I see your example images all seem to show nodes that have only one rendered output port.

 

For all I known, there is not PlayableGraph API to check "what input on node n is connected to what output on node m". You can maintain a connection mapping table manually and pass it to your PlayableGraph render window.

In my PlayableGraph Monitor tool, it doesn't support to render node with multi outputs. If there are nodes with multi outputs, the PlayableGraph hierarchy will turn from a tree into a directed-graph and I don't know how to calucate a perfect layout for a directed-graph.

 

@SolarianZ Thanks for the quick response. This confirms my understanding of the situation as well (I cannot believe you cannot do this!!).

I have found a pretty ugly hackaround that I am using currently but it's pretty sad that we are forced into this situation.

 

I found that there is a method PlayableExtensions.GetOutput can return the Playable connected at the given output port index, this might help you!

 

@SolarianZ I appreciate the thought and the message about it. Unfortunately, there are still things you cannot do with the run-time API for a playablegraph. If you have two nodes connected to each other on multiple ports ( for example, a timeline is often connected multiple times to a mixer when there are multiple tracks of the same type ). This means that you have no way to know which output port of the timeline is connected to which input port of the mixer since the API will only tell you what PLAYABLE a given port is connected to but NOT what port that is connection is made to. This is an egregious oversight as it makes it impossible, in general, to render a graph without storing additional information about how the graph is wired somewhere outside the API.

 

Do you mean these kinds of situations:

a) a TimelinePlayable connected to multi PlayableOutputs:


b) Playables with multi outputs:


c) cycle reference:


I found differences between general PlayableGraph and timeline PlayableGraph.
In general PlayableGraph,

PlayableOutput.GetSourceOutputPort()

returns the correct output index;
but in timeline PlayableGraph, all PlayableOutput connect to same TimelinePlayable and

PlayableOutput.GetSourceOutputPort()

returns multi different output index, but in fact the TimelinePlayable only has ONE output port!
I have reported a bug about this situation, if I get any response, I will reply to you here.

 

@SolarianZ First let me say that we have been using your Monitor tool for the past several weeks and we really appreciate it!

It has saved us a lot of time implementing a similar tool and the improvements you have made recently in rendering the nodes without duplication and correctly handling multiple outputs are great!

In our game, we are experimenting with using Timelines to construct character abilities (think attacks or special moves) that we then play on a playable graph at run-time. We have several custom tracks but also make use of the built-in tracks where possible.

We iterate through the tracks in the asset to determine what type of track they are and what kind of output binding they have. We then connect those outputs from the timeline playable to appropriate places in our graph. Often times, we connect outputs to mixers in our graph. A good example of this would be audio tracks: We frequently have multiple audio tracks defined on a single timeline. We connect all these tracks to a mixer in our own playable graph. This means we have multiple output ports on the timeline playable connected to multiple input ports on the mixer. This works although we have encountered bugs that we have had to workaround.

One such bug is that audio coming from the same timeline playable must go through a dummy node (we call it a noop node) before being connected to the mixer. If it is not, only one connected node will play.



This is a sample of a timeline with its outputs being connected to various places in our host graph. Unfortunately, the current layout algorithm in your Monitor spread these out a lot and makes it hard to take a screenshot of the graph as a whole.

Importantly, we have to manually set the output count on the timeline playable to the input count!

playable.SetOutputCount(playable.GetInputCount())

Otherwise, the timeline playable by default only has a single output.

 

Thank you, huh

Yesterday I created a new branch "multi-output-layout" to try to solve this problem, maybe I can complete it after several days.

By the way, how do you do this? @ somebody directly instead of quote a text:

 

@SolarianZ Type @ then start typing my name to see a list of matches.

It should look like this:

Your mulit-output branch sounds great. Good luck on it and we'd be happy to contribute if the need arises.

 

@kanesteven Got it and thank you! And I have released a new version 2.1.1 with new PlayableOutput layout logic, performance improvement, more detailed document(Readme.md) and bug fixes. Just try it! And if you have any good idea to help improve this tool, welcome to create issue or pull request on the repository page!

 

@SolarianZ Awesome!

Great work it's looking good.

I will post a few screenshots from our run-time Timeline system in a few hours.

 

I just found out that NVidia omniverse also have a great graph scripting tools for animation. This could be a great reference for same system in unity