Official Exciting developments in upcoming Entities 1.0 releases

Howdy. I'm Joe Valenzuela from the DOTS team. We've been hard at work making changes in both the Unity Editor and various packages as part of our initial Entities 1.0 experimental release. There are all sorts of bug fixes, optimizations, and new APIs that we think will allow Unity developers to make their games more ambitious, more performant and, in general, more awesome.

Thanks to everyone who used these packages in preview and provided feedback in the forums or elsewhere - our experimental 1.0 release has relied on your insights and efforts. As part of that process we’ve taken a careful look at our existing APIs and in some cases decided to deprecate, rename, or remove types and functions. We’ll have a detailed upgrade guide for folks who have been following along with Entities before full release, but in the meantime I wanted to show off some things I think are cool about the experimental release.

Conversion -> Baking
Conversion has been the heart of the Entities data pipeline. In the 1.0 experimental release, we’ve reengineered it to the point where we decided it needed a new name: Baking.

In the old Conversion system, component data is authored in the familiar Unity environment you know and love, and data is converted from this to a fast, memory-ready format either at run-time (via ConvertToEntity) or at Editor time (by virtue of being in a Subscenes).

In Baking, you still author data in the Unity Editor and get fast, memory-ready binary data at runtime, but the mechanism is substantially different. We’ve split the process into a two-pass system in order to do the most expensive processing with ECS, so it’s faster. The dependency tracking is precise, deterministic, and robust, meaning you only update the data that’s changed, which improves live update with open subscenes. And the code itself is simpler and easier to understand, and as a consequence has fewer bugs and edge cases. The API is safer as well, making it harder to make mistakes (and simpler as well, we think, but that’s for you to decide!).

Here’s a small example of the API:

Code (CSharp):

1. public class SelfDestructAuthoring : MonoBehaviour
2. {
3.     public float TimeToLive;
4.  
5.     class Baker : Baker<SelfDestructAuthoring>
6.     {
7.         public override void Bake(SelfDestructAuthoring authoring)
8.         {
9.  
10.             AddComponent(new SelfDestruct {TimeToLive = authoring.TimeToLive});
11.         }
12.     }
13. }

One thing developers should be aware of ahead of time is that we’ll be deprecating runtime conversion - like ConvertToEntity - and removing it before our full 1.0 release. We know this is a commonly used feature, but runtime conversion has been a constant source of bugs (because it requires maintaining a separate pipeline for moving authoring data around) and isn’t possible to maintain with our goal of keeping editor functionality out of the DOTS runtime. Thankfully, runtime conversion is (almost) never necessary, and in the few cases it is, the subset of functionality needed is straightforward enough to support on the client side.


Build Configs
Standalone builds previously required the use of Build Configuration Assets and the separate “build config” menu options.


This will no longer be the way of building standalone projects using DOTS.

Having a completely separate build tool allowed us to rapidly prototype configuration options for DOTS, and it’s necessary for some work we’re doing unrelated to Entities 1.0. But having a different way of making standalone builds was a frequent source of confusion for new users, and consolidating build systems is not possible in any reasonable timeframe. While we think Build Configs may be a great solution someday, they aren’t strictly necessary today, and we want to prioritize the simple solution that works for almost every case. So we’ve made the classic Unity build page work for the vast majority of cases. Developers who need more specialized build configuration can use the Scriptable Build Pipeline, a powerful API for building content.

Build Configs will still operate as usual in the experimental build, and building with the classic Unity build window will be available as a toggle-able option (currently Preferences -> Entities -> Builtin Builds Enabled, but subject to change). By release we plan to make building via the build window the default.

Transforms
While developing Entities, we gradually realized that Transforms - the components and related systems that maintain an object’s position, orientation, and hierarchy - needed revision. Based on our own experience and those of our partners, we’re rolling out a substantially improved API and backend. In addition we’re taking the opportunity to prevent errors by precluding the use of features at runtime that are difficult or impossible to support well, like non-uniform scale (non-uniform scale for static data is handled at Bake time, and we have special case path to allow some common uses of non-uniform scale).

There are some finishing touches we have to put on before fully switching over to the new Transform system, but when we're done I think you'll find...

It’s simpler.

Transforms are now baked from the GameObject Transform down to a wafer thin Entity representation of only 8 floats. The Entity component, in turn, is transformed by fewer systems in a more streamlined fashion, using fewer systems, and operating on less data.

It’s cheaper.

Gameplay code frequently deals with rotation and position. Previously, transforms were based completely on matrices, making extracting rotation expensive (at best). Now we maintain data in its quaternion form, the format in which it’s most often used.

It’s robust.

Our previous Transform data protocol had 15 different component types, some of which were very infrequently used, but all had to be supported. The new data protocol is substantially smaller and easier to support.

It’s more correct.

The previous Transform data protocol didn’t put any real world constraints on what was feasible in game code - literally just offering a matrix - so every system made up its own rules on what it could handle. This can lead to unintuitive errors as systems ignore features of the provided transform (like non-uniform scale).

idiomatic foreach
When Entities.ForEach was introduced, it was the quick and easy alternative to writing full-blown job code. However, the fullness of time revealed some limitations. The biggest problem was that nested loops were impossible, making it cumbersome to model many typical solutions. And after the introduction of IJobEntity, the barrier to jobified code was now no longer as steep, so we went back to the drawing board to express the most common remaining use case for Entities.ForEach - main thread immediate code - using more natural C# syntax. We call the mechanism idiomatic foreach.

Code (CSharp):

1. foreach (var (rotateAspect, speedModifierRef) in
2.                 SystemAPI.Query<RotateAspect, RefRO<SpeedModifier>>())
3.      rotateAspect.Rotate(time, speedModifierRef.ValueRO.Value);

The biggest takeaway is that there are fewer edge-cases where APIs that work outside of a loop don't work inside, and there’s some behind-the-scenes improvements as well. Entities.ForEach remains in 1.0, but we’re strongly encouraging folks to use idiomatic foreach and jobs instead. To that end, we’re not supporting Entities.ForEach in ISystem (it will remain in SystemBase) and plan to phase out support post 1.0.

Aspects
One of the most obvious new API features is the introduction of Aspects. Aspects offer a view on commonly used data (like translation and rotation) along with methods in order to provide a low-cost, fully-burstable interface. This allows developers to write code against an Aspect without coupling themselves to irrelevant implementation details. You can also write your own Aspects to make your code easier to use correctly.

Here’s a simple example to keep something oriented toward another entity’s position. The code is straightforward, but requires us to query both the Translation and Rotation component. If one of these became redundant (unlikely in this case, but stay with me) you’d have to audit every query to make sure you were using the minimal set of parameters or else you’d be introducing superfluous dependencies. (Small note - currently Aspects will pull in dependencies for all contained components whether they are used at the call site or not, so there’s room for performance improvement in future releases).

Code (CSharp):

1. public partial struct LookJob : IJobEntity
2. {
3.     [ReadOnly]
4.     public ComponentLookup<Translation> lookup;
5.  
6.     public void Execute(ref Rotation r, in Translation t, in LookAt_RotationTranslation lookAt)
7.     {
8.         float3 head = lookup[lookAt.Other].Value;
9.         float3 forward = new float3(head.x - t.Value.x, t.Value.y, head.z - t.Value.z);
10.  
11.         r.Value = quaternion.LookRotation(forward, math.up());
12.     }
13. }
14.  
15. protected override void OnUpdate() {
16.     new LookJob{lookup=GetComponentLookup<Translation>(true)}.Schedule();
17. }
18.  

With the TransformAspect, the specific mechanism used to encode position and orientation are kept up to date. Write your API against the aspect and it’s trivial to keep up to date. And best of all, we use the same query mechanisms behind the scenes, so you still get performance by default.

Code (CSharp):

1. public partial struct LookJob : IJobEntity
2. {
3.     [NativeDisableContainerSafetyRestriction]
4.     [ReadOnly]
5.     public ComponentLookup<Translation> lookup;
6.  
7.     public void Execute(ref TransformAspect ta, in LookAt_TransformAspect lookAt)
8.     {
9.         float3 otherPosition = lookup[lookAt.Other].Value;
10.  
11.         ta.LookAt(otherPosition, math.up());
12.     }
13. }
14.  
15. protected override void OnUpdate() {
16.     new LookJob{lookup=GetComponentLookup<Translation>(true)}.Schedule();
17. }

We’re providing two Aspect interfaces to start - TransformAspect and RigidBodyAspect - and are looking forward to hearing your feedback on the feature.

Enableable Components
One of the most far-reaching changes we did was to introduce the concept of Enableable components. Previously, if you wanted to exclude a set of entities in the same Archetype from a query, you’d have to either group them with SharedComponent filters or change their Archetype by adding and removing tag components. Both options require expensive structural changes, and increase memory fragmentation. Furthermore, neither operation can be performed immediately from job code; the desired operations must be recorded into an EntityCommandBuffer and played back later in the frame, leaving the entity in a potentially inconsistent state in the meantime.

Enableable components can be efficiently enabled and disabled at runtime without triggering a structural change (even from job code running on a worker thread). A similar effect can already be achieved with a component containing a one-byte “isEnabled” field, but enableable components are a first-class feature of the Entities package, fully supported by EntityQuery and Entities job types. Disabling a component on an entity prevents that entity from matching a query that requires the component, which means that jobs running against this query will automatically skip the entity as well. SIMD-optimized query-matching and chunk-iteration code keeps the overhead of skipping disabled components to a minimum. Enableable tag components are ideal for high-frequency state changes, without increasing the number of unique archetype permutations. In our tests, toggling enableable components from a parallel job runs 9x faster than adding/removing the same components using structural changes.

Code (CSharp):

1. public struct TargetEnableable : IComponentData, IEnableableComponent
2. {
3.     public float3 target;
4. }
5.  
6. // ...
7. var archetype = m_Manager.CreateArchetype(typeof(EcsTestData), typeof(TargetEnableable));
8. var entities = m_Manager.CreateEntity(archetype, 1024, World.UpdateAllocator.ToAllocator);
9.  
10. EntityQuery query = GetEntityQuery(typeof(TargetEnableable));
11.  
12. // Disable the target on one of the entities.
13. // entities[4] will now be excluded from the query.
14. m_Manager.SetComponentEnabled<TargetEnableable>(entities[4], false);
15.  
16. int fooCount = query.CalculateEntityCount(); // returns 1023

ISystem & IJobEntity
While they aren’t new in 1.0, we’ve added support throughout Entities packages to make ISystem and IJobEntity more useful than ever.

ISystem is an interface implemented by unmanaged component systems. It’s specially designed to keep you on the fast path using Burst and away from C# managed memory, which introduces garbage collection. Support has been added throughout Entities (and other packages) to keep us on the fast path as much as possible.

While it can’t write your code for you, It’s never been more straightforward to stick to the “high performance” subset of C# (HPC#) that unlocks next-gen performance.

Code (CSharp):

1. [BurstCompile]
2. public partial struct TheFriendlyOgreSystem : ISystem
3. {
4.   // Store reference to a friendly ogre entity
5.   Entity friendlyOgreEntity;
6.  
7.   [BurstCompile]
8.   public void OnCreate(ref SystemState state)
9.   {
10.     // Create a temporary ComponentType NativeArray.
11.     using var friendlyOgreComponentTypes =
12.     new FixedList64Bytes<ComponentType>
13.     {
14.       ComponentType.ReadWrite<HealthData>(),
15.       ComponentType.ReadWrite<AliveTag>()
16.     }.ToNativeArray(Allocator.Temp);
17.  
18.     // Create an Entity of Archetype (HealthData, ShieldData)
19.     var friendlyOgreArchetype = state.EntityManager.CreateArchetype(friendlyOgreComponentTypes);
20.     friendlyOgreEntity = state.EntityManager.CreateEntity(friendlyOgreArchetype);
21.   }
22.  
23.   public void OnDestroy(ref SystemState state) {}
24.  
25.   [BurstCompile]
26.   public void OnUpdate(ref SystemState state)
27.   {
28.     // Check if the friendly ogre is alive
29.     if (state.EntityManager.IsComponentEnabled<AliveTag>(friendlyOgreEntity))
30.     {
31.       // Get HealthData of the friendly ogre. and kill it over time. (10HP per second)
32.       var healthData = SystemAPI.GetComponent<HealthData>(friendlyOgreEntity);
33.       healthData.Left -= 10f*SystemAPI.Time.DeltaTime;
34.       if (healthData.Left > 0)
35.         SystemAPI.SetComponent(friendlyOgreEntity, healthData);
36.       else
37.         state.EntityManager.SetComponentEnabled<AliveTag>(friendlyOgreEntity, false);
38.     }
39.   }
40. }

Part of this effort has gone into making it easier to move code from the main thread to a job, or from your update to a utility function, or from a SystemBase system to an ISystem one.

In part because of this expanded support, IJobEntity is now the recommended job type to use when convenience is desired.

Code (CSharp):

1. partial struct UpdateChunkBoundsJob : IJobEntity
2. {
3.     [ReadOnly]
4.     public ComponentTypeHandle<BoundsComponent> ChunkComponentTypeHandle;
5.  
6.     void Execute(ref ChunkBoundsComponent chunkBounds, in ChunkHeader chunkHeader)
7.     {
8.         var curBounds = new ChunkBoundsComponent
9.                         {
10.                           boundsMin = new float3(1000, 1000, 1000),
11.                           boundsMax = new float3(-1000, -1000, -1000)
12.                         };
13.         var boundsChunk = chunkHeader.ArchetypeChunk;
14.         var bounds = boundsChunk.GetNativeArray(ChunkComponentTypeHandle);
15.         for (int j = 0; j < bounds.Length; ++j)
16.         {
17.             curBounds.boundsMin = math.min(curBounds.boundsMin,
18.                                            bounds[j].boundsMin);
19.             curBounds.boundsMax = math.max(curBounds.boundsMax,
20.                                            bounds[j].boundsMax);
21.         }
22.  
23.         chunkBounds = curBounds;
24.     }
25. }

But wait there’s more!
There’s a lot to look forward to in Entities 1.0 (some of these code samples include “sneak previews” of other changes we’ll be discussing in upcoming posts) and we’re all looking forward to finally getting our work in front of y'all. While we’re proud of the work we’re doing, the real payoff is seeing what all the incredible Unity developers do with it. Thanks for reading and we will keep you posted for more detail as we approach the official release of Entities 1.0!

 

> Can you explain more how can we scale entity non-uniformly in new transform system?

There's a post-transformation matrix you can use to apply arbitrary linear multiplies with your final LocalToWorld. They don't propagate hierarchically and won't be used by the Transform system.

> Can we use ComponentLookup outside of Job?

Yop!

 

You may want to think about removing Entities.ForEach in SystemBase before 1.0. The reason I say this is because a lot of old tutorials use Entities.ForEach. Many new users are probably going to be using those old tutorials. If anything is going to force the rewriting of tutorials it will be the 1.0 release. I just think it makes sense to get the API stabilized at least as far as the fundaments are concerned before 1.0.​

Thanks for the suggestion. It's been a hotly debated topic on the team, especially since 1.0 represents the best time to clean up the API. While we've got a lot of confidence in the replacements for Entities.ForEach, removing it in the anticipated timeframe is unlikely.

 

Just put [BurstCompile] on a static function and its declaring type, and when you call it it should be the bursted version; this has worked for several burst versions now.

 

Thank you for flagging this. We're aware of this issue and are currently working on a fix that should be available in a couple of weeks.

 

One strategy might be to convert prefabs or groups of entities at bake time and instantiate them individually (and optionally compose them according to whatever your runtime criteria is). We'll be posting more baking related content soon, keep a lookout for a more comprehensive guide linked in the forums.

Right.

I can't speak to the release schedule of anything unannounced just yet, but if you provide more info on your use case someone might be able to weigh in.
Also, check out the 1.0 related content in the sample repo:

 

We're aware of the issue, have reproduced it locally, and anticipate a fix in an imminent editor release.

 

That the same issue as here? We're investigating.

 

We like it fine too, it just kills iteration time, and we couldn't make it stop.

[Edit: for me EFE looks more like linq than the new foreach thing. Also, linq is really slow, whereas this is very fast ]

 

Since enabled components are optional I read this question as "why isn't the 128 entities chunk limit special cased only to archetypes which contain enablable components?"

We take advantage of the 128 entity limit to employ some optimizations, particularly the use of SIMD, that make both the general case and the enableable case faster. Obviously performance depends a lot on context and YMMV, but it was pretty rare for us to find an archetype that we thought represented a likely user situation that performed differently enough to justify the problems that maintaining a separate code path would introduce.

That said, part of having an experimental release is getting early feedback from folks who may run into problems and situations we didn't anticipate (or with a differing frequency). If the 128 entity limit presents a performance problem for you we'd love to hear more about it.