RFC: How are you using the transform systems at run-time?

Hi all!

We're interested in gathering some information about how people are using the transform systems at run-time today. What exists now obviously contains a lot of functionality, but we would like some feedback on how or if people are making use of it. We imagine 99% of uses entail cases like:

• Translating/rotating root-level entities around in the world
• Positioning objects/hierarchies after instantiating them
• Attaching objects/hierarchies to specified points on other objects/hierarchies
• Applying scale to static meshes/colliders

We'd love to know who is using or exploiting it to do anything interesting we should know about! We're especially interested in questions such as:

• Are you making use of components other than Translation, Rotation, Scale/NonUniformScale, and Parent?
• Are you making use of NonUniformScale in hierarchies that have rotated children? If so, to what effect?
• Are you dynamically changing hierarchies at run-time? If so, to what effect?
• Are there things you are not using the transform systems for right now, that you expect or hope it will be able to support?

We're also interested in other general feedback:

• How easy/hard do you find it to use?
• Do you find it hard to reason about latency/timing of transform updates during your frame?
• Anything else?

Thanks in advance!

 

My biggest complaint is that transform systems page - just looking at it makes my head spin. After watching the Mike Acton presentation about write groups I have a better idea of how it turned out that way, but I feel like it could use a major overhaul to make it a lot more accessible to people who are not super familiar with ECS. I'd much rather know how to properly use the components rather than get all these weirdly detailed diagrams explaining exactly how it works. And it would be nice to have some examples of common use cases like reparenting at runtime.

Also a common issue I see come up in discord is properly timing entity spawns in a frame against the transform system. A lot of people seem to end up with entities spawning at origin for a single frame since they don't specify update order, or they just use EndSimulationBuffers and their entities happen to spawn after the transform system but before rendering. Setting up your spawning systems to run in the correct order should really be covered in the manual page as well. Or - I'm not sure if it's possible but maybe the transform group could be moved to happen after the SimulationGroup?

 

I sometimes dynamically scale at runtime weapons, projectiles, and moving environment.

Yes. I like to add random variation to prefabs in procedurally-generated worlds.

It would be useful to know whether Scale should even be used since uniform scales get converted to NonUniformScale in the TransformConversion process.

It is harder to use than GameObjects, but that is an understandable tradeoff as it is tough to beat performance-wise. I think you could make it a little more beginner-friendly by adding easier ways to update the transform system more often or have some sort of single-threaded job and burst compatible utility struct that allows a user to interact with entity transforms similar to the GameObject transforms where the hierarchy is kept up to date.

It is easy to have a child with a modified local translation and rotation and then need it's world space translation and rotation to spawn, snap, or manipulate some other entity correctly. The utility I proposed would probably solve this problem for all use cases where this kind of thing matters. I'll probably write such a utility myself at some point if Unity decides to not make changes to the Transform System.

Also please make TransformConversion public so that people can better customize the conversion pipeline.

 

Thanks! Could you elaborate a little more on this? We can of course always add more samples of how to use anything that exists in the run-time today. But I'm specifically interested in knowing about use cases people would like to see illustrated. For instance, are there common use cases more complicated than "make this thing follow this point on this other thing" that are important for you?

 

Thanks this all useful feedback! I'm interested a bit more in this use case.

• When you scale weapons, I assume those are in a hierarchy. Are you just scaling the weapons themselves, or their parents as well?
• If their parents, do you do it non-uniformly?
• I'm also wondering if you're changing hierarchy topologies much at run-time, or just swapping out different weapons

 

A few off the top of my head:

• How to freeze an entity.
• How to get "gameobject-like" behaviour in terms of destroying/hiding a parent and having it apply to all children.
  
• How to apply common local and world operations on transforms using the new math library, like rotations, relative movement, following another entity, looking at another entity, transforming points, etc.

For point two specifically it seems like the conversion system is adding a linkedentitygroup which gives that behaviour by default, but manually reparenting doesnt. It would be good to know exactly how the conversion system decides which transform related components to add.

 

Here's one use case: I need to traverse the hierarchy, i.e. I do get dynamic buffer of children from entities. The problem is that in doing so, even though I only read that information, the entities end up "changed" and their LocalToWorld gets recalculated. I've started a separate thread on that here: https://forum.unity.com/threads/need-read-only-access-to-dynamic-buffers.843934/
I imagine anyone trying to traverse the transforms hierarchy will come across that issue eventually.

 

Mostly adding support to everything @Sarkahn mentioned. One obvious re-parenting use would be picking-up/dropping items.
For an easy example of non-uniform scaled entity as a child of a rotated entity: a non-uniform scaled sword (tweaked at runtime to feel in good proportion to a character) being wielded.

Speaking a bit more broadly, using the above as an example, there's currently a very heavy weighting (at least implied) towards "for that, you should make an asset the right size and use a uniform scale". Which would be the ideal. In fact, in general DOTS is a huge step forward in users being nudged towards doing the more efficient thing. Yet it does somewhat run very counter to fast iteration & experimentation that Unity has traditionally been great for.
In this case are you asking for use-cases to help with usability or out of considering removal of unused features? My reply is anticipation of the latter. As DOTS is so strongly opinionated about the way to do things fast, I'm personally in favour of maintaining flexibility in expression where possible.

 

Animation parts of object hierarchy just to maintain parts as separate meshrenderers and dont make it skinned because skinned is less performant, so we need to have parts of say Tank hierarchy be relatively animated with local rotations and non uniform scaling and skew for cartoon look.

 

One fairly common use case for that is for attaching weapons or equipment in characters (like in third-person shooters).

 

@Adam-Mechtley,
To give some sense, what I am doing, here is one of my projects that I use transformations.
https://forum.unity.com/threads/wip-ecopico-v0-0-12-prototype-dots.773210/


1. Translating/rotating root-level entities around in the world

So I got lifeforms, which have cells. Each cell is child of lifeform, as standard approach.
I instantiate lifeforms, then generate cells and attach them accordingly, by assigning cell entity to parent entity, using Parent component. Children and other required components are auto generated, by DOTS apis.


2. Positioning objects/hierarchies after instantiating them

I use custom system, for positioning and orientation of lifeforms. Currently iterating through children (cells), to set their local scale and position, in respect to parent (lifeform), when it moves and grows. Technically that should be dealt, by DOTS apis, but at current, it is legacy of my older system, which also involved individual cell scale. I will probably change this behaviour later point. That deals automatically, with attaching cells at specific local position of lifeform.


3. Applying scale to static meshes/colliders

While I currently apply scale to cells, I use

Code (CSharp):

1. Unity.Physics.BoxCollider* scPtr   = (Unity.Physics.BoxCollider*) physicsCollider.ColliderPtr ;

for dynamic collider of life forms (not individual cells), and for collider scaling.
It isn't my favorite approach, but it works.


4. Are you making use of components other than Translation, Rotation, Scale/NonUniformScale, and Parent?

I have custm buffer in lifeform entity, with cells entities references and their local positions, rater using children buffer of lifeform parent. This is kind of duplicate, of DOTS child API, but is more for convenience, as I wasn't sure, if I keep using default DOTS child buffer. For now cells mainly contains information about used shaders properties, along material and transformation.

Also, I have custom position, rotation, scale components for lifeform, which stores position as int. At least until float will be deterministic. These are my prime values I work with, when operating in a world.


5. Are you dynamically changing hierarchies at run-time? If so, to what effect?

I add and remove cells (children) at runtime.


7. Are there things you are not using the transform systems for right now, that you expect or hope it will be able to support?

Can you elaborate please?


8. How easy/hard do you find it to use?

Once grasped, is rather intuitive.
But figuring out, what and how things work, require a bit of reading and searching.
For example children parenting. But documentation shows quite clearly, what and how should be done.


9. Do you find it hard to reason about latency/timing of transform updates during your frame?

I haven't noticed. At least it is not an issue for me. I relies on my integers for transforms, as primary values. then assign transformation only for rendering, other than collider itself.


10. Anything else?

Not using DOTS Unity Rendering V2.

I was considering using colliders for cells, but I found too much hassle with it. Haven't figured out, how to remove colliders from compounds colliders for example. I use own solution for interacting with individual cells, using mouse pointer. Otherwise, I need regenerate compund collider, every time number of cells entities is changing in lifeform entity.

 

I typically break up a weapon into two or three pieces and apply different scale values to each piece. I scale them non-uniformly.

For hierarchies, I am usually just swapping stuff. Not just weapons. Character customization gets lots of swaps too. I don't personally use hierarchy rearrangement for gameplay. Instead I swap prefabs and transfer a memento. I do that more for the authoring flexibility as opposed to trying to circumvent a technical roadblock.

 

Can you talk more about what specific problem you are solving by traversing the hierarchy? Is there some data or entity reference that cannot either a) be statically known at conversion time or b) represented using a non-hierarchical grouping concept?

 

Right now we're looking at different systems that need to interact with transforms and each other (e.g., animation and physics), and wondering what options we might have on the table for optimizations/simplifications. Because GameObject hierarchies have a lot of interesting properties (e.g., dynamic reparenting, group placement/activation/deactivation), we're interested in knowing how people are or expect to be able to exploit transform hierarchies to do more than just position individual entities in the world. Having actual use cases lets us reason about whether we want people to solve those types of problems with a transform hierarchy or with some different mechanism.

 

Sure. I'm trying to do something similar to what @supron has shown here: https://forum.unity.com/threads/source-code-dotsui-open-source-ui-framework-for-dots.715880/ (even though the author ceased working on it).
The idea is to convert Unity UI (Canvas, RectTransform, etc.) into ECS. And to implement RectTransform you need to traverse transform hierarchy to properly layout rects. You see, when rects have been calculated I'd very much like not to touch them again unless I have to, thus I need a change check similar to one that is done in LocalToParentSystem. Unfortunately, because my rect calculation system READS children buffers, entities become changed for both my system and LocalToParentSystem (which unnecessarily recalculates LocalToWorld again and again).
I guess the easies way for you to see the problem yourself is simply run your LocalToParentSystem twice back-to-back - it will never stop updating LocalToWorld (even though it now has change checks implemented).

 

I spent a long time trying to get projectiles to spawn with a child entity's translation and rotation. Still didn't get the rotation to work right. Also the projectiles always spawn at the origin on the first frame. Spawning objects on child entity's is fairly common and it would be nice to have an example of how to do it correctly.

 

Actually I want to see and test workflow with groups and may be subgroups.

Just for debugging purposes (like names of entities) add group and subgroup to entities so eve root entities can be seen in some structure for ease of develop and debug but this is about editor.

About runtime:
- I can see that enabling/disabling hierarchies can be achieved by putting all entities into some group (may be on conversion time) and able to easily enable/disable this group may be it can be just component on root entity with buffer component. This approach looks enough in most cases for me. e.g. character can have many weapons on it but visible only one in time and every weapon is compound from many entities for some reason.
- Group placement can be achieved by some sockets approach like in unreal

 

Up until recently, I had never really used the transform system very much other than for simple things, such as parenting, and most everything I did was all handled for me during GameObject conversion, which works great. But recently, I dived into doing some more (of what I thought) was going to be trivially simplistic calculations at runtime that set an entity's rotation, it was in that moment (after fiddling around) I realized that things were going to have to be good deal more complicated than I originally anticipated.

All I wanted to do was make a child entity face its local right towards a point in Worldspace, which seemed pretty simple. But once I realized that I needed to also update the many other components on various other entities (such as the Children) that controlled their own Worldspace matrices, things got crazy, fast. And, as far as I now know, in nearly every interaction of setting an entities Rotation (or Translation, for that matter), you need to also update all potential children's Translations and Rotations, in addition to recalculating all LocalToWorld's and LocalToParent's on applicable entities, which include the original parent entity that had its transforms changed.

Now, that's a lot. And most of the time I believe you will be doing that "a lot" A LOT.
What I would find really nice would be helper methods that would take in the required Translation, Rotation, Scale, LocalToWorld, and LocalToParent, and then recalculate the LocalToWorld and LocalToParent according to the Translation, Rotation, and Scale, for all entities in that parent-child relationship. This would make these kinds of transactions less filled with redundant code that has to be there if you do not want to operate on faulty transformational values for a frame.

Doing this also has the benefit of assisting newbies learn the DOTS transform system, as there are a surprising amount of components that need to be updated that most people entering the land of ECS won't be aware of, and definitely won't think to manually update every time they make a change.


Also, does anybody know how a Quaternion can become NaN (other than through an exactly opposite quaternion.LookRotation)? For some odd reason, (after tracking the values of the entity in question's LocalToWorld), once the entity is processed through the transform system it has NaN Rotation in its LocalToWorld, which, of course, breaks everything. I have no idea why this is happening. As far as I'm aware (and I've done a fair amount of debugging) none of the values on any of the Rotation, LocalToWorld, or LocalToParent components are NaN before the transform systems run at the end of update. So... How might I be getting NaN values? I'm sure I must be doing something wrong here... But what?



EDIT: Added some personal answers to some of the questions raised by the OP.

I make use of LocalToWorld mainly and only ever use Translation and Rotation because I need to update those respective values if I make changes to LocalToWorld. Just out of curiosity... Is there any particular reason Translation and Rotation exist? Couldn't we just use the Worldspace matrix on LocalToWorld and LocalToParent to get the relevant data instead? I mean, we already have getters to get the data from LocalToWorld and LocalToParent, couldn't we also have setters? Or am I missing something?

As of now, I have not used Scale or NonUniformScale at all actually. While scaling support of entities in my game would be nice -- I don't see the use behind it right now. (Plus scaling and pixel art don't mix very well).

So far, I have not reached the limits of the transform system. There's nothing I wish it could support as of this moment.

I find the transform system to be deceptively complicated. In short... Until you really know how it works, you don't really know how it works. There's a lot of moving parts in ECS, and the transform system is no exception. It would be nice to simplify the use of the transform system for those that do no require complex behavior.

I update all of my systems manually in one giant script on a GameObject. As far as I can tell, I have not had any issues with timing. But I do realize creating giant monolithic updating scripts was not exactly the original idea when it came to defining update order, so I can't say if it's fair to acknowledge my two cents. Personally, I find the placing of UpdateBefore and UpdateAfter to be super confusing, and I don't think it would ever work for large games or projects... And even if it did, I could only imagine the debugging nightmares along the way.
Considering the timing of the transform system itself, I wish it was more explicit. It kinda feels "in the background" if that makes any sense. It gives me the jeebies.

Thanks for making this thread and taking community feedback! It's awesome and it's super appreciated. Personally, I think all the devs who work on DOTS (and DOTS related technologies) / the DOTS Forum are especially good at their job. Just don't go shouting down the halls of Unity's physical infrastructure that I said that, though.

 

@GliderGuy
Chances are, that your transform matrix values are all 0 at initialization.
Set them as identity. For example

Code (CSharp):

1. new Rotation () { Value = quaternion.identity }
2. float4x4 f4x4 = float4x4.identity ;
3. new LocalToParent () { Value = f4x4 }
4. new LocalToWorld () { Value = f4x4 }

 

Originally I thought this might be the case as I missed setting the LocalToParent by accident when I created the entity. After adding it in though, (and iterating over all children created with the corresponding entity and setting those entity's values as well), it seems to not have made any effect.

I'm printing out the Rotation, LocalToWorld.Rotation, and LocalToParent.Rotation (I'm calculating it with quaternion.LookRotation) every FixedUpdate and then again every LateUpdate. Everything is fine in FixedUpdate (where all of my code is handled) the Rotation is (0, 0, 0, 1), the LocalToWorld.Rotation is (0, 0, 0, 1), and the LocalToParent.Rotation is (0, 0, 0, 1), but when it reaches LateUpdate the Rotation of LocalToWorld is NaN.
I have no code in Update (or SimulationSystemGroup, but I have moved SimulationSystemGroup to a FixedUpdate frequency). I wish an error would be thrown whenever NaN existed from a math computation so that way I'd know exactly where things were failing.

EDIT: Entities JUST came out with the 0.8 release! (Exciting! I have no idea what the heck's in it because the changlog hasn't been updated yet but YIPPEE).

ALSO: I ran my updated project and (to my suprise) it seems like LocalToParent is now printing out as NaN in my FixedUpdate debug logs! Now I just need to figure out where it's happening and I should be good to go.

EDIT2: I'M A DUNCE... But I found my problem. -.-

 

In your use case is the hierarchy statically known at conversion time, or do you have use cases where the hierarchies can change in unanticipated ways at run-time?

 

Well, we're talking about UI here - it is inherently dynamic, with creation/removal of sub-hierarchies at runtime. When that happens the system responsible for rect transform calculation has to traverse hierarchies to recalculate rects. The same is required when animating parts of the UI. The rect transform system is actually similar in nature to the LocalToParentSystem that works for transform hierarchies.

 

Among other things, this is primarily a data bandwidth question. For example, if all you need to do is update the translation of a bunch of things, you don't want to actually load all those extra bytes of data for the rotation or for the last column

 

Hey @GliderGuy I'm having the same problem as you had, could you tell me what was the cause of your error, please? It's been several hours and I cannot find anything that could cause this.

 

Oh baby.
I wish I wrote it down.

*Spends some time looking through old project files*

Okay, @LuisEGV, I've jogged my memory on what kinda-sorta happened three months ago.

The gist of it is this (and hence why I'm a dunce): While I thought I was properly initializing my LocalToParent and / or LocalToWorld, it turns out that I wasn't.

The actual reason behind it was due to me forgetting to update an overloaded method of mine. The method was for creating hitboxes in my game, so it handled a lot of component creation. When I set the LocalToWorld matrix on my hitbox's LocalToWorld, it turns out I was feeding it a null quaternion accidentally. (NEVER ASSUME ANYTHING).

You see, doing this:

Code (CSharp):

1. public quaternion rotation;

does not initialize the quaternion to a valid value.
This is obvious of course, but in my instance it was super hidden due to multiple methods, abstraction, complications, and assumptions (NEVER ASSUME). (It took me a week to find what I did wrong because it broke stuff waaayyyy down the chain, and I assumed that all my methods were properly doing everything right (but it broke in a tiny spot)).

If you want to initialize a quaternion right, always make sure that you do this:

Code (CSharp):

1. public quaternion rotation = quaternion.identity;

at bare-minimum.
If you are initializing a quaternion with a custom rotation (anything other than quaternion.identity), makes sure to print out the values you used to generate it and make sure they aren't the culprit.

Unfortunately, that information probably won't solve your problem because you're not an idiot like me. So here's some troubleshooting advice:

• quaternions pretty much only ever break when you try to construct them — and usually because you construct them improperly or due to the rare chance your values aren't valid for creating a quaternion (even when you think they are). Poke around those areas with Debug.Log() and see if you can find where things go NaN NaN NaN NaN.
  
  
• A broken quaternion loves to make things more broken, so I always like to go from bottom to top of my grand system-chain to find the issue. Climb the ladder of systems that could possibly be making your quaternion go ballistic and eventually you'll find the point where the quaternion gets corrupted.
  
  
• When in doubt, hardcode. Sometimes just throwing in valid numbers is a great way to find where something is (or is not) breaking.
  
  
• And — most importantly — NEVER ASSUME ANYTHING, because assumptions KILL any debugging process. Always start with educated guesses on what variables and values could be the issue, but always be ready to print out EVERYTHING in order to find the cause of sneaky, difficult problems.

Welp, I hope that helped.
I'm sorry I don't have a "just do this and everything will work" reply (because I know how annoying these kinds of problems can be and that you just want to be over with it). But I'm sure you'll be able to find your issue! If you have any more questions or need assistance, feel free to throw it out there, and I'll do my best to help.

 

Well, this advices work beyond this issue haha. I solved it. Thanks!
I indeed assumed that a quaternion was initialized, because some other entities that work in a similar way, had it's rotations initialized, the difference was that those entities were converted from GameObjects, and the error entity was created from archetype.

Thanks a lot.