Search Unity

  1. Unity 2019.1 is now released.
    Dismiss Notice

Question regarding Playables

Discussion in 'Scripting' started by PedroGV, May 6, 2018.

  1. PedroGV

    PedroGV

    Joined:
    Nov 1, 2010
    Posts:
    410
  2. PedroGV

    PedroGV

    Joined:
    Nov 1, 2010
    Posts:
    410
    Anyone?
     
  3. RomainFailliot

    RomainFailliot

    Unity Technologies

    Joined:
    Nov 29, 2016
    Posts:
    110
    Hi @PedroGV !

    All the Playable nodes are structs in order to prevent boxing/unboxing when bridging from C# to C++, and vice versa. But when using a PlayableBehaviour, it is based on the interface IPlayableBehaviour, which will induce boxing/unboxing anyway, even if the PlayableBehaviour would have been a struct. So, to make it easier for the user, we made PlayableBehaviour a class.

    Don't hesitate if you need more info ;)

    Cheers!
     
  4. PedroGV

    PedroGV

    Joined:
    Nov 1, 2010
    Posts:
    410
    Hi Romain, thanks for replying.

    Implementing an interface with a struct only implies boxing when you store the struct or cast it as such interface. But it does not box the struct when you use generics, properly.
     
    Last edited: May 8, 2018
  5. PedroGV

    PedroGV

    Joined:
    Nov 1, 2010
    Posts:
    410
    I don't know how you internally handle playables, but I would gladly tell you, if you want to, how to unify criteria with C#, and use only structs that implement IPlayableBehaviour (even custom ones) without boxing at all.
     
    Last edited: May 8, 2018
  6. RomainFailliot

    RomainFailliot

    Unity Technologies

    Joined:
    Nov 29, 2016
    Posts:
    110
    Hi!

    That's actually what we're doing for IPlayable and IPlayableOutput, we have extension methods (resp. in PlayableExtensions and PlayableOutputExtension) that use generics.

    For the PlayableBehaviour, it's a bit different. A custom Playable, by definition, can be very versatile. We think the benefits for using a class are higher than the benefits for using an interface.

    You can also take a look at the ScriptPlayable in our C# ref repository to see an example of how we're using the IPlayableBehaviour (there are other places where we uses it, feel free to fork and browse the repo, but some packages, like Timeline, also uses IPlayableBehaviour).

    That being said, I'm interested in what you have in mind. If we can improve the system more, it's always interesting ;)
     
  7. PedroGV

    PedroGV

    Joined:
    Nov 1, 2010
    Posts:
    410
    Hi, thanks for the links to the repo on github!

    If the decision for using structs when defining built-in playables is based on perf, then the cost of dealing with reference types for custom ones may exceed such benefits you mention, again, in the sake of perf.

    But I can understand why you prefer to use a PlayerBehaviour class with virtual operations with empty methods given that it makes it simpler for devs.

    ---------------------------------------------------

    Having said that, my idea was to remove IPlayableBehaviour and ScriptPlayable<T> and always use IPlayable and IPlayableOutput instead for both, built-in and custom playables. And then, many of the static operations, like Create/Clone/..., could be assigned to extension methods that expect a generic parameter of type struct, IPlayable (ditto for IPlayableOutput). Not to mention that you could use many optimizations when Unity supports Roslyn with C# features in v7.x (avoiding copies of value-type parameters, for example) and also in the upcoming v8 (like default members in interfaces, say, for creation/cloning, and even the virtual operations of PlayableBehaviour)

    You could also add an IPlayable<TPlayable> interface ...:

    Code (CSharp):
    1. public interface IPlayable<TPlayable>: IPlayable, IEquatable<TPlayable>
    2.    where TPlayable: struct, IPlayable { }
    (you could move all the operations of the PlayableBehaviour class to IPlayable or to IPlayable<TPlayable>)

    ... and thus, request others to implement it for custom playables:

    Code (CSharp):
    1. public struct MyCustomPlayable: IPlayable<MyCustomPlayable>
    2. {
    3.     private PlayableHandle m_Handle;
    4.  
    5.     ...
    6. }
    To simplify things, whenever the user wants to create a new custom playable, a menu item option on the editor must be selected that shall create a base template with all needed basic operations and methods implemented (Equals, implicit/explicit operators, etc.).

    So, as long as any operation declared on the builtin/custom playable as well as overridden ones do not call operations defined in the "base" type (ValueType or object), and you call them either directly or by using generics where TPlayable: struct, IPlayable, then no boxing shall occur.

    Since, for what I can see, you are handling loops from the unmanaged side of the engine, no more optimizations could be done here, but in case you need to do some foreach loops on the managed side, no boxing occurs when using generics either (and there are tricks to avoid the use of member fields like List<IPlayable> m_playables, which causes boxing).

    ---------------------------------------------------

    But again, I can understand why you prefer to use a PlayerBehaviour class with virtual operations with empty methods.
     
  8. RomainFailliot

    RomainFailliot

    Unity Technologies

    Joined:
    Nov 29, 2016
    Posts:
    110
    I think I understand your suggestion and there might be some good nuggets to take, thanks ;)

    Correct me if I'm wrong, but I think you're mainly focusing on the ScriptPlayable part of the Playable system. The reason why performance is less important for the ScriptPlayable is that, eventually, it will run during the scripting pass, and on the main thread anyway, which is not really the fastest code path in Unity.

    And Playable is also about Animation, Audio, Texture, Video, etc... These areas implement Playable nodes in C++ and have C# structs equivalents that allow to create and edit PlayableGraphs (see AnimationClipPlayable, AnimationMixerPlayable, AnimationControllerPlayable, AudioClipPlayable, AudioMixerPlayable, etc.).
     
  9. PedroGV

    PedroGV

    Joined:
    Nov 1, 2010
    Posts:
    410
    I concentrate on the "custom" side of playables, and whether they implement nodes on the unmanaged or managed side is of no consequence to the fact that PlayableBehaviours are also playables with graph representations. So, as structs, would it possible to update them on the same thread than those native playables?
     
    Last edited: May 9, 2018
  10. RomainFailliot

    RomainFailliot

    Unity Technologies

    Joined:
    Nov 29, 2016
    Posts:
    110
    Technically, the PlayableBehaviour is not the Playable, but the ScriptPlayable is. The PlayableBehaviour is the ScriptPlayable content, the user code. The ScriptPlayable is a struct, and only PlayableBehaviour is a class. So, in that sense, the ScriptPlayable is treated like any other Playable.

    The ScriptPlayable content, i.e. the PlayableBehaviour, is managed code, and it is its force and weakness: it brings all the Unity API into a Playable. But all the Unity API also implies non-multithreadable stuff as well and that's why PlayableGraph using ScriptPlayable nodes must be run on the main thread.

    The way to go to do multi-threaded, thread-safe, managed code in Unity is the new C# Jobs system that landed in 2018.1. We're leveraging this technology into the Playable with the AnimationScriptPlayable that will land in 2018.2 (it's in beta right now if you want to give it a try). It is limited to animation for now, but depending on the demand we might create other Playable for other data types...
     
    Last edited: May 10, 2018
  11. PedroGV

    PedroGV

    Joined:
    Nov 1, 2010
    Posts:
    410
    Understood.

    My question is: does it make sense to bring such access into a playable? (in this case, through the PlayableBehaviour) I mean, for what I can see, custom playables are mainly used as a mixing nodes for other playables. So I wonder whether it would be better to implement an event system for playables (OnPlayableBegins, OnBeforePlayableUpdate, OnAfterPlayableUpdate, OnPlayableEnds, etc.) and let other game objects react in consequence of such events? No need for ScriptPlayable nor PlayableBehaviour, just a "mixer" playable struct that you (= Unity Team) define.

    What brings me to the next subject: Timeline. The absence of events for playables (and for what I could see, timeline it-self) would likely result in the use of PlayableBehaviours to workaround it. Which means, extra coding, handling more instances of ref-types not as playables per se but as workarounds of such limitations.
     
  12. RomainFailliot

    RomainFailliot

    Unity Technologies

    Joined:
    Nov 29, 2016
    Posts:
    110
    Hi @PedroGV!

    The reality is a bit more complex though. We see three kind of playables: data, mixers, and filters. "Data" are the kind of playable producing a data, like the AnimationClipPlayable. "Mixer" are the kind of playable taking data streams and mixing them together, like AudioMixerPlayable. And "Filter" are the kind of playable transforming a data stream, like the experimental MaterialEffectPlayable.

    The AnimationScriptPlayable can be any of them, because it can produce as well as mixer or filter animation data stream.

    A fourth kind of Playable could sneak in though: the controllers. They could basically be put in the "mixer" bag, the main difference is that, during PrepareFrame, these playable changes the topology of the graph. This is the case of the AnimationControllerPlayable and, most of the time, also what the custom ScriptPlayables do (like the TimelinePlayable).

    But the kind of playable is one thing, the other very important part is the data type they carry. Each data type needs a specific set of playable. That's why there is an AnimationMixerPlayable and an AudioMixerPlayable for instance.

    So technically, the ScriptPlayable can be of any kind, but there is no data type really associated with it. That's why, most of the time, it is used to control the graph beneath it. Worded differently, ScriptPlayable has the user data and the callbacks that allows to take decision on how to edit the graph it controls.

    All that to get to the fact that ScriptPlayable is a custom, versatile Playable. It is but one of the types of Playables in a PlayableGraph. Its strength is its versatility that comes from the fact that it runs in user land and on the main thread, like all the other scripts, which allows to have a more deterministic behavior. Its flaws is that it is less optimized than other, more specific Playables.

    The events you're talking about are actually more callbacks. I agree, callbacks are events, but events are not necessarily callbacks. There is an important distinction that I will get back on later.

    Calling callbacks at anytime in a process can be very useful, but since you can do anything in a callback (load a file, play a sound, make a network request, etc...), including modify the PlayableGraph, it breaks the determinism of the process.

    Events are simpler in that they are mere piece of data floating their way into a system until they are "executed". An event can be gathered at one place before being executed, which is way better for determinism. And eventually, an event can call a callback, but doesn't have to.

    And this kind of event is exactly what the Timeline team is working on right now: a customizable event system for the Playable that can't mess up the PlayableGraph. So, it's in the pipes and it will come in a Unity store near you ;)
     
  13. PedroGV

    PedroGV

    Joined:
    Nov 1, 2010
    Posts:
    410
    Hi there, when I speak of events I refer to the observable pattern (with subscribers), not specifically to callbacks.