State of ECS & DOTS

yeah sorry I mean more like: it's gonna cause a high volume of stat changes every frame, and so it could become a bottleneck

 

Thanks for sharing this! This is a hard problem, perhaps even impossible to do correctly.

The reason I say that, is that you haven't specified a rule for resolving circular dependencies. I'm guessing the "order" aspect reflects that rule, but without knowing that rule, it makes it difficult to propose an alternative. I'm going to try anyways though, and see what you think of it.

So assuming some rule exists that can resolve circular dependencies, instead of representing stat modifiers as entities with custom components executed by independent systems, I would represent a modifier as a DAG node as an entity. Each DAG entity would have a reference to its root, a dynamic buffer of inputs, another dynamic buffer of constants, and a function pointer or switch case key evaluator (or perhaps an expression string if a runtime interpreter is cheap enough). A group of interdependent stats (like all the stats for a given character) belong to the same DAG. Whenever a DAG node is added, removed, or modified, the DAG root is dirtied. The job system safety will fight you on this, but fortunately this issue is of the "solve once, use everywhere" type. I can't tell you whether an unsafe container or an asmref Entities extension is easier though.

So for each dirty DAG root, I would find all DAG members, build the actual DAG in temp memory using a DAG builder which considers the circular dependency resolution rules, and then evaluate the DAG. And then finally, mark all the roots as clean.

It is not trivial, but I think it meets your requirements.

 

the circular dependency problem is something I don't really know what to do about. Maybe we need to perform a runtime check of all possible branches whenever a stat dependency is added, in order to detect infinite loops? But then we still have the problem of deciding what happens if a circular dependency does get detected at runtime. Just cancel the effect and have players wonder why nothing is happening? Feels like there's no good solution where we have a guarantee that everything is safe & coherent, and perhaps this sort of Strength-To-Strength buff shouldn't be allowed afterall

I'm not sure how much easier the problem becomes if we can determine an order of stat calculations by type though. Since the system has to be optimized for infrequent stat changes, chances are we'd probably end up working with a buffer of dirty stat entities either way instead of scheduling a big group of one-job-per-stat-type (that all do dirty checks to see if they actually have to update the stat)

This reminds me of the existence of DataFlowGraph. A long time ago, some unity devs were describing it as a general-purpose tool that could be good at solving certain kinds of problems that don't really fit ECS well. However, I never really took the time to study it or dig into it (in fact, I barely even have a vague idea of what it does or how it's used). Now I wonder if I should attempt a DFG implementation of this stats system. Sounds like it might be able to solve graph-type problems well?

If I understand correctly, I think there are similarities between what you're suggesting and my approach. My stat entities keep a buffer of "other stats to dirty when I am dirtied". So in practice this is essentially like keeping a reference to your child nodes in the graph of interdependent stats. My stat modifiers are not represent as entities; but instead they are structs that live directly in a buffer on the stat entity. All stat modifiers of all types are in the same buffer, and they use a switch statement to determine what they do. When added, they are given a unique Id in order to be able to retrieve them later for removal when a buff ends

But your approach of building separate graphs would allow for parallel graph calculation, while my approach can only be single-thread

-------------

It's a problem that basically regroups all of my previously-mentioned DOTS difficulties. It has:

• Polymorphism
• Order
• "Reactive" things
• User-extensible things (creating new modifier types, custom OnStatChanged behaviour)
• Possible solutions where you'd want to handle order by launching one job per stat type and modifier type, but then you could end up with hundreds of jobs for your stat update

 

Ai is a good example. Our ai I think looks more like what real games look like, a lot of data from various sources and 2-3 internal flows. Like pathfinding is one, target selection another, skill selection another. Spatial grids, local and global data context's. It's all very specific to the problem.

Combat resolution is probably the next most complex system that works better outside of ECS.

Items both inventory and our building system. harvesting, crafting, guilds, groups, faction handling, chat.

Some of this is that the natural patterns that fit best aren't ECS. But what I meant by real games was mostly what do these things look like with the larger correct context applied. In some games you have persistence and network messaging, and a lot of request/response flows rather then per frame. These parts all have their own concerns that influence data structures and flow in various ways, and a lot of them are off the hotpath or maybe just partially on it.

So in fairness a lot depends on the game. Some of the above ya you could make it fit ECS without forcing it too much. Other stuff I'd say that's just wrong period. Depends on how far off it is from what fits best.

 

Entities are more flexible then gameObjects and most code can be rewritten on a per-frame basic. If you look at Unity's own code off the top of my head one example is the PhysicsWorldHistory system in networking you will find that it does not use Entities. Entities are a great replacement for gameObjects and a little bit beyond.

Unity's own code does not try to force Entities be a end all solution. If your code needs to be updated within a inner loop often Entities are not the answer and stuff like a persistent pathfinding grid should probably not be forced into a entity with limited data structure support. Unity would not code it that way and neither should you.

Most of your code can be Entities, when it does not support what you want do a burst job within a system. Use Entities as a gameObject replacement and not as parts of your algorithms and you will end up much happier. KISS, keep it simple, stupid

 

Yes. You may want to define something. It could be a simple priority system based on rarity or something (rarer items get applied later and have more pronounced impact). The same issue actually exists in OOP, it is just less obvious so people don't notice.

DFG has a few issues that make it probably a poor fit for your use case:
1) It causes a sync point.
2) Graph changes run non-Bursted on the main thread.
3) You can't turn off sub-graphs, meaning you need a bunch of graph objects (which are managed) or you pay the cost of evaluating all the stats every time.

I'd certainly use it as reference though. What I proposed will get you more parallelism, keep your component types down, and update infrequently.

I'm not convinced. None of what you said sounds like something ECS would fight you every step of the way on. Sure, there are some problems where you don't exclusively use the ECS data structure. Unity Physics is a good example of that. But it still consumes data from ECS, processes it, and spits it back out in ECS system style.

What I do think is valid is that there are some problems in game dev where there is a known solution a team is familiar and comfortable with, but that particular solution doesn't map well to DOTS. It may be cheaper for the team to build some bridge that helps this other solution domain interact with ECS from outside than to invent a new solution that maps better. But that doesn't mean that a well-mapped solution doesn't exist for these problems.

My point is that while DOTS and ECS are a relatively unexplored space, I haven't encountered a true dead-end yet that wouldn't apply to any other OOP architecture as well.

 

But there can be 907 mages in the map that just got teleported nearby by some other spell and thus all cast at once in the same frame, each casting their own. And the orb could set the slow based on how close the target is to the ball.
It's the kind of bullshit I liked to see in WC3 custom maps back in the day, of course, the numbers where way smaller.

 

Sounds like a good idea then to do what game devs have been doing for centuries and make it a per-frame loop. With DOTS you can even do that in parallel.

 

Yeah, but then imagine some units get a bonus to damage based on move speed because of a buff. The insane complexity that stat systems can get is just pure bullshit if you let your imagination go crazy enough. I don't think there is any solution that will scale.
Although there is an idea, couldn't stat changes trigger new stat changes through https://gitlab.com/tertle/com.bovinelabs.event ? There would need to be some sort of HashSet for event producers so to not trigger the same stat recalc twice on the same frame.
Then at end frame consume all events in loop until there are no more events (would cause a infinite recursion if A depends on B depends on A though).
The recursive stat recalc could happen only once per frame at the end or start of it.

 

Was this a joke lol
Someone must be pulling our legs

Also yeah, I agree polymorphism is an issue. Basically instead of having an 'Item' with extra properties, I need to add another list of 'unique type items' in my inventory. This work around is not the best. But I'll probably just separate items into entities and use components for all the extra properties. Basically it forces you to use extra components rather than morphed ones. But that's just a matter of structure.

I have hundreds of systems, and ordering does get tricky. I wish we can declare the order of the systems in a separate file.

As for reaction, I use components for events. And then remove them after using EntityQuery.RemoveComponent to do it to all of them at once. I use components for timers and remove them once they are up. Thus effecting the other queries.

User-extensible things, so basically from the editor, adding events? Or just handlers in general? To be fair, these were the worst for creating spaghetti code and tracking down bugs. They were easy to implement, but it was gonna cost you later down the line.

Lastly, for stat events.
'And finally, I want the possibility of triggering a stats update potentially several times per frame, due to some scenarios where an entity that uses stats gets instantiated & must get its stats calculated immediately before running the next system that uses those stat value.'
I don't do it in a single frame. I create event entities. For many to many events, we need separate entities per event. Then injecting them into the character or stat entity. In general this takes one frame to update modifications, but is very performant.
(if it's one to many relationship, like a button click, I add the event component to the ui entitiy!)

 

Why would it be one? I never joke about performance.

 

Respectable. But it's like using a hammer to cook scrambled eggs with. It's not meant to be used that way!
Edit: As far as I can tell EntityManager.GetComponentData<Translation> will pull every time the entire segment of data, rather than just the float3. Which is why this operation 5000x would be slow. But if you just pull the data properly with an entity query it should pull it just one time.

 

ComponentDataFromEntity is slower than operating on a query, but it's often necessary. You'll always have a bunch of systems in your games that need to read/write data on an entity other than self. And trying to avoid it at all costs can result in much worse performance in some cases

We already know that operating with a query is much faster than OOP, but we need to know how DOTS random memory access compares to OOP, because there are times where we'll need it

This is a simple use case, but there are use cases that are more complicated. Try to imagine how you'd implement an ordered events system where all events have to be processed in 1 frame. If you insist on doing this with queries, the solution to this problem will likely be scheduling 100 jobs with a sync point for structural changes inbetween each of them in order to process a list of 100 events (which would be an awful solution)

Examples of problems that would need this: reliable-ordered RPC system, some flavors of replay/time-travel systems in tactical RPGs, my stats example with ordered modifiers of different types, etc....

Imagine a third party tool that runs a complex job, but allows a function in the job that can be user-implemented through the use of generics (a bit of a dots equivalent of overriding a virtual function). But then imagine that your custom implementation needs to access a type of component that wasn't planned for in the original job. Your only option then would be modifying the source code of the tool in order to add support for accessing your game-specific component type, which means you'd be cutting yourself off from future updates of that tool

It's tricky to explain the exact specific circumstances where this becomes a problem, but it's a problem I've encountered while making my character controller asset. It's one of those cases where separation into multiple jobs wouldn't be an option, because I did the math and it would result in us having to schedule up to 80 jobs just for our character update, and the performance cost would be absurd compared to the single-job approach

I don't want DOTS to unleash a new era of sluggish unresponsive games just because frame-perfectness isn't ideal for the programming paradigm. That would be prioritizing the elegance of our code over the design of our game. All problems need a frame-perfect solution to be possible, simple and efficient, because many games will need them.

And I know that frame-perfectness is possible in DOTS and that event entities don't necessarily means waiting for the next frame; I'm just saying it would be sad if frame-perfectness becomes so tricky & tedious to pull off that most programmers will end up giving up on it. I often hear solutions to problems that involve deferring operations to the next frame, and this worries me

Just the other day I was working on an FPS controller, and the fact that Transforms were updated only once per frame meant there was a visible lag between my weapon barrel and the pos/direction of spawned projectiles while I was rotating. This is the kind of problem that could become widespread if unity doesn't provide utilities for getting/setting worldspace transforms immediately. My temporary alternative was to schedule the update of TransformsUpdateGroup at multiple points in the frame

Again; I'm not saying any of these things are impossible. I'm saying they need to not be difficult

 

I think it mostly depends on whether you do need frame perfect solution.

For the stuff that is related to rendering - the answer is... maybe?
If its not visible, or felt / heard anyway - there's no need to force sync data in the same frame.
If you think you need everything to sync perfectly - usually that is not the case.

Testing makes perfect, and switching system order is trivial. Especially if you're using system groups.
Same goes for the ECB systems.

In other cases, its completely possible to schedule jobs for data processing at the beginning of the frame, then schedule / process something different during the frame, and after simulation - sync back required data.

Tl;DR:
- Read minimum required data at the beginning of the frame (e.g. from MonoBehaviour world -> before simulation happens);
- Process read data as part of job systems, so that data change can be "chained" from one system to another by just having dependency setup.
- Write back results at the end of frame (after simulation).

Anything in between processing steps can be sorted by system execution order.

No frames skipped, minimum jobs forced to be completed. Surely it won't be as fast as "leave it for the whole frame duration", but in most cases it is enough.


Main issue with your specific case is Transform system group. Its all in a single place, where in reality it should be split into two separate read / write groups at the beginning of the frame, and end of frame ("AfterSimulationGroup") groups. Also performance wise, if you do not use conversion, its faster and wiser to sync data manually in a custom job and reduce TransformAccessArray allocations.

So, its just flawed by design, and requires rework. This is also one of the reasons why conversion sucks IMO.



In the end, goal of job balancing is to reduce amount of sync points overall, not to remove them completely.
As later tends to be impossible.

 

Hey so, I really don't see any EntityManager.GetComponentData as necessary. If it is for 5k units, you should definitely be just writing a system for it.

Well for a lot of my camera and physics stuff, I just order the systems. And it performs all the results in one frame, on the next frame. If it needs to be done on that frame, like collision responses, I use a byte to trigger it. The extra frame is only because of EntityCommandBuffers delay for adding components. But yeah, i've also implemented a lot of per frame events, but only when necessary. The component or entity creation just uses less cache. I still think many things can be delayed however, since there is 100-200 frames per second, it's not really an issue for alot of situations.

Since my game uses all my own systems, if there is a lag of one frame, I simply need to order the systems better. Alot of my camera and movement code uses data writing, and not events, this is the dynamic solution. Events are for things that update less often.

A simple example is 3 systems like this
System A -> set stats
System B -> apply damage
System C -> kill character
Where they all write to a stats component with a byte for health and one for the dead state. It can all be done in a single frame, and it's not hard either.

'In the end, goal of job balancing is to reduce amount of sync points overall, not to remove them completely.
As later tends to be impossible.'
I've removed mine completely actually lol. Ain't nothing wrong with aiming for perfection ;D I'm only really concerned with developing better algorithms and design workflows, it's good ECS handles the management. But yeah, if reworking old code, it doesn't all need to be reworked. Just using ECS where your pain points are is fine.

 

The alternatives to ComponentDataFromEntity often involve structural changes, sync points, and more jobs to schedule. There are many situations where such approaches would perform much worse than ComponentDataFromEntity (which is the same thing as EntityManager.GetComponentData)

Imagine you have "wind zones" in your game, and these wind zone add forces to rigidbodies that enter them. How would you implement this without getting PhysicsVelocity on overlapped entities and modifying their velocities? You could come up with a solution where you create ECB commands that add an "AddForce" component on overlapped entities & have another job process them, but that would just be a much more expensive solution. Or you could use a buffer of "addForce" elements on the entities and have an ECB append elements to it, but this would only start being more efficient if you think you'll have thousands of rigidbodies in wind zones & you need all this to be parallel

This is perhaps an overly-simplistic use case that I had off the top of my head, but in real life I think it's nearly impossible to imagine coding an entire game without ComponentDataFromEntity

 

For me, the best solution to this problem is well set-up ComponentGroups with CommandBuffers at the end. In my multiplayer project I have 7 stages. A command buffer runs anyway, so let it run when it needs to. Most are not caching ComponentDataFromEntity so data will never be outdated in subsequent systems.

And I agree, delaying to the next frame is lazy programming that will probably hurt the game. This act of laziness can lead to data being deferred 2-3 frames, which was a serious problem when I first started out.
For all new devs dabbling in ECS. Deferred data is a pain you need to get used to and find solutions/designs. Ignoring the fact will lead to VERY weird bugs.

 

I would add a component when they enter and leave zones, since this minimizes changes to ecs structure as much as we can. Then another system would add wind force to the entities with those components. It's not expensive at all. You only get .01 ms extra for another parallel system. Even with not many entities, this solution is very minimal in cost.

You can code it entirely with parallel systems and removed synch points actually!

'And I agree, delaying to the next frame is lazy programming that will probably hurt the game. This act of laziness can lead to data being deferred 2-3 frames, which was a serious problem when I first started out.
For all new devs dabbling in ECS. Deferred data is a pain you need to get used to and find solutions/designs. Ignoring the fact will lead to VERY weird bugs.'

It's not lazy, some solutions don't require it! It's a trade off between cache and component tags! If components add a byte in data (not sure), but there is also the fact that systems using these tags will process less data! So it's more efficient actually to add component tags and do it the next frame. But data that requires it, is related to rendering, so it important to do any render data in the same frame!

 

ok fair, I thought about this use case for a total of 0.3 seconds and it wasn't a good example

 

I believe the first year and a bit of ecs programming was painful. Especially when it was not as refined. Tutorials weren't there. Examples scattered across the forums. I do believe it's possible to solve all problems with this framework. A lot of my early pains came from trying to solve the things the older way. If you think of any more problems, I would be interested in knowing.

Polymorphism can be done with components. Although it's not ideal for human minds, but that's what the computer requires lol. The better part about that is, we can write systems to target certain item entities, and this will be more efficient then a poly-morphed object. And yeah ECS shines for large amounts of objects, but I find it better for my own management of code and bug fixing as DreamingLatios has mentioned.

 

It's a specific brand of ordered polymorphism that's complicated in ECS. I maintain that you should try to implement an ordered events system (events of all kinds of different types that can read/write all kinds of different component types and must absolutely execute in the precise order in which they were created, independently of their type) & see what you come up with. The only non-tedious solutions I can imagine have to rely heavily on codegen

And at the same time, it's also a use case where there's no escaping ComponentDataFromEntity.

Other use cases for ComponentDataFromEntity would be:

• writing a character controller, and making it so you can bounce on certain surfaces. You have to GetComponent<BounceProperties> on hit entities
• Writing a vehicle system and having to get the properties of your wheels and/or of the ground
• Removing an item from your inventory and having to recompute your total carried weight; you need to iterate on the entities of just your inventory specifically, and GetComponent<Weight> on them
• Implementing target selection for an AI, where it has a list of all targets in range and it must GetComponent<?> on targets in order to evaluate several factors
• etc

In all of these cases there would be possible solutions that don't use GetComponent, where you'd do a back & forth of ECB commands & sync points between the entities that want to interact with eachother, but it would perform significantly worse. Even worse if that logic must happen in the context of a loop with several iterations

 

What's the current verdict of generic components for many types? Simple "fat" component with condition branching or ECS filtering through components or buffers.

@PhilSA I think you have experimented a lot with this but I've lost track where you ended up. Any conclusion?

 

The best I could come up with is still to create a big "union struct" that is able to hold all possible data types across all event types, and then do a switch case. But Entity remapping makes it complicated to codegen this. Probably not impossible to solve, but I just haven't bothered to solve it yet and so I just write everything manually

I've attached as an example my "StatModifier" struct, which demonstrates the several types of stat modifiers along with special stat evaluators based on other stats (look for the "ModType.EvaluateMagicResistance" which makes a stat's value be a combination of Intelligence & Strength stats, before applying other regular modifiers)

 

Have you tried an approach that you use in: https://github.com/PhilSA/DOTSPolym...ster/Assets/_Samples/PolymorphicTest/CompB.cs

Could be a 3rd solution to this. I have not thought this through but calling an interface method without any evaluation and data conversion strikes me still as quite elegant solution. It also solves the problem of having to write ValA, ValB, etc...

I just have not used it in practice. The problem could be that it's very restrictive. When you operate on the same data, great, but when the need of more components to read/write arises it could get messy in the parameter call list with lots of overhead. But then a new PolymorphicComponent could solve this.

 

What I do with my stat modifiers is essentially this, but I write the struct manually instead of doing codegen. My PolymorphicComponent tool would work well were it not for Entity remapping issues, but unfortunately I often need to keep Entity references in these things

 

It seems these functions I made might be relevant to the thread with the while third party accessing stuff dynamically:
I don't know if it is actually burst executable, but I copied some EntityManager's functions to get this to work, and it seems to work just fine on both editor and build.
I recently started using it to let a XML configure which component a function uses as a float, for instance. No more need to use Reflection to read/write arbitrary data, you can even cache the type index.
The zip contains the code and the needed asmdef file to let the class access internals, installation **should** be just tossing the contents in your project anywhere.

Spoiler: Code

Code (CSharp):

1. using System;
2. using Unity.Collections;
3. using Unity.Collections.LowLevel.Unsafe;
4.  
5. namespace Unity.Entities.Extension {
6.     public static class UnityEntitiesExtension {
7.         [BurstCompatible(GenericTypeArguments = new[] { typeof(BurstCompatibleComponentData) })]
8.         public static void SetComponentDataReinterpret<T, E>(this EntityManager aThis, Entity entity, E Value) where T : struct where E : struct {
9.             SetComponentDataReinterpret(aThis, TypeManager.GetTypeIndex<T>(), entity, Value);
10.         }
11.         [BurstCompatible(GenericTypeArguments = new[] { typeof(BurstCompatibleComponentData) })]
12.         public static void SetComponentDataReinterpret<T>(this EntityManager aThis, int typeIndex, Entity entity, T Value) where T : struct {
13.             unsafe {
14.                 EntityDataAccess* ecs = aThis.GetCheckedEntityDataAccess();
15.                 ecs->EntityComponentStore->AssertEntityHasComponent(entity, typeIndex);
16.                 ecs->EntityComponentStore->AssertZeroSizedComponent(typeIndex);
17.  
18.                 if (!ecs->IsInExclusiveTransaction)
19.                     ecs->DependencyManager->CompleteReadAndWriteDependency(typeIndex);
20.  
21.                 var ptr = ecs->EntityComponentStore->GetComponentDataWithTypeRW(entity, typeIndex,
22.                    ecs->EntityComponentStore->GlobalSystemVersion);
23.                 UnsafeUtility.CopyStructureToPtr(ref Value, ptr);
24.             }
25.         }
26.         [BurstCompatible(GenericTypeArguments = new[] { typeof(BurstCompatibleComponentData) })]
27.         public static E TryGetComponentDataReinterpret<T, E>(this EntityManager aThis, Entity entity) where T : struct where E : struct {
28.             return GetComponentDataReinterpret<E>(aThis, TypeManager.GetTypeIndex<T>(), entity);
29.         }
30.  
31.         [BurstCompatible(GenericTypeArguments = new[] { typeof(BurstCompatibleComponentData) })]
32.         public static T GetComponentDataReinterpret<T>(this EntityManager aThis, int typeIndex, Entity entity) where T : struct {
33.             unsafe {
34.                 EntityDataAccess* ecs = aThis.GetCheckedEntityDataAccess();
35.                 ecs->EntityComponentStore->AssertEntityHasComponent(entity, typeIndex);
36.                 ecs->EntityComponentStore->AssertZeroSizedComponent(typeIndex);
37.  
38.                 if (!ecs->IsInExclusiveTransaction)
39.                     ecs->DependencyManager->CompleteWriteDependency(typeIndex);
40.  
41.                 var ptr = ecs->EntityComponentStore->GetComponentDataWithTypeRO(entity, typeIndex);
42.  
43.                 T value;
44.                 UnsafeUtility.CopyPtrToStructure(ptr, out value);
45.                 return value;
46.             }
47.         }
48.         [BurstCompatible(GenericTypeArguments = new[] { typeof(BurstCompatibleComponentData) })]
49.         public static bool TryGetComponentDataReinterpret<T, E>(this EntityManager aThis, Entity entity, out E value) where T : struct where E : struct {
50.             return TryGetComponentDataReinterpret<E>(aThis, TypeManager.GetTypeIndex<T>(), entity, out value);
51.         }
52.  
53.         [BurstCompatible(GenericTypeArguments = new[] { typeof(BurstCompatibleComponentData) })]
54.         public static bool TryGetComponentDataReinterpret<T>(this EntityManager aThis, int typeIndex, Entity entity, out T value) where T : struct {
55.             unsafe {
56.                 EntityDataAccess* ecs = aThis.GetCheckedEntityDataAccess();
57.                 if (!ecs->EntityComponentStore->HasComponent(entity, ComponentType.FromTypeIndex(typeIndex))) {
58.                     value = default;
59.                     return false;
60.                 }
61.                 ecs->EntityComponentStore->AssertZeroSizedComponent(typeIndex);
62.  
63.                 if (!ecs->IsInExclusiveTransaction)
64.                     ecs->DependencyManager->CompleteWriteDependency(typeIndex);
65.  
66.                 var ptr = ecs->EntityComponentStore->GetComponentDataWithTypeRO(entity, typeIndex);
67.  
68.                 UnsafeUtility.CopyPtrToStructure(ptr, out value);
69.                 return true;
70.             }
71.         }
72.     }
73. }

UnityEntitiesExtension.zip

 

It would be nice for Jochim Ante to give another update as to the State of ECS and DOTS since that is the title of this thread and we haven't had any official updates in a while.

 

Speaking of which. Maybe not as detailed as we would have hoped, but eh, that's something.

 

Time to get myself a DOTS tattoo DOTS for life

 

Obviously polka DOTS!

 

I've tried to install ECS in Unity 2022. Got 165 errors. How can I fix it? =/

 

Change version of unity

 

Make sure collections and other packages are matching ECS package use. They may need to be downgraded.

 

How? I've tried to use old versions in manifest:

Code (CSharp):

1.  
2.    "com.unity.burst": "1.4.1",
3.    "com.unity.collections": "0.15.0-preview.21",
4.  

But Unity installs newer versions anyway.

 

We're on 2021.2.7f1 currently and we use these packages:

Code (CSharp):

1. "com.unity.burst": "1.6.4",
2. "com.unity.collections": "0.15.0-preview.21",
3. "com.unity.entities": "0.17.0-preview.42",
4. "com.unity.mathematics": "1.2.5",
5. "com.unity.rendering.hybrid": "0.11.0-preview.44"

But be careful, collections require small changes for this version (You need to change the allocator type for NativeList, you can read about that here https://forum.unity.com/threads/2021-2-8f1-broke-asdeferredjobarray-and-deferred-jobs.1225635/ otherwise you'll have invalid deferred arrays)
Also if you have generic Blobs then you also need to modify the Entities package a bit (as it has type restrictions since 0.17 (or even 0.16 can't remember) which will prevent you from having generic utils for Blobs)
И да Андрей, когда уже КМ заведешь

 

@hippocoder Hmmmm

 

@eizenhorn I have freed you I believe I'm seeing more Cyrillic stuff flagged for mod queue.

 

Ah... I see... the first time tried my native language and got moded... you don't like Cyrillic okaaay okaaaay

(it's a joke)

 

well for me the current Entity workflow is fantastic

there are some pitfalls wen comming to conversion wich i solve with my package
https://github.com/Extrys/Hybrid-EZS
this is what im using and updating
(so it stills in wip also the documentation but is usable currently)

Such thing as cherrypicking Exclusively which components will be converted from a game object and how to convert them, has greatly benefit my workflow,
since the biggest problem with "Convert to Entity" or "SubScene Workflow" for me is that,
It convert the entire gameobject or gameobject set,
when i just may want to convert a single thing from that gameobject
or just a set of selected components

in this way I just drag and drop what I want to convert

(Custom Data Autors (also will be explained in docs) adds itselves to the data autors list when added to a gameobject and if there is not entity injector, then is created, there is an example of Projectile Authoring in the package)

Referenced entity injectors field is just for adding an entity buffer to that entity containing all primary entities from that deferenced entity injectors

most of the time i dont want to convert the rigidbody or colliders, or even the translation, i just want to sync an entity data that is linked to a gameobject in this way i have alway access to the same entity and can also integrate new components or check components from it from a non system class

so its in fact like "gameobjects with entity components wich i can iterate and burst in systems"
since the entity injector form that gameobject hold the entity reference i can also just access to the primary entity, this workflow, made lot easier for us to work with entities, and also we have faced several performance issues thanks to this

What i feel i need to do now is somewhat, preview what entities will be converted like normal entity conversion does
i think that would be the perfect improvement for my workflow, is there some way so i can show that conversion window in the editor? maybe a custom editor or something like that to access that preview window?

if someone knows it would be awesome to knoe for me, thanks

 

"When you say "enjoy programming" do you mean "comfortable with" or "prefer over C++, C#, Java, Python, ect"?"

He or she said enjoy programming assembly.

"What evidence have you seen that suggest that someone who loves DOTS is perhaps incompetent in traditional OOP projects?"

Well if people who defend dots usually include in the list of "why dots", issues that an experienced programmer does not have. Then he/she is just making things up in defense of the his/her way of doing things.

My own issue with Unity goes deeper.

A lot of the times it feel they don't really care about the people who use Unity.
First market ECS as something great all over the world. Then when you start using, then it mark it as preview, make development on it almost impossible and say its you fault if you failed.

For example I went to 2021 because my project didn't work on 2020. Now they blocked working 2021 combination of packages from package manager. You will have to edit manifest with text editor to get it working. All this countless of hours just figuring and trying to get it installing the packages most of the time failing.
I complain to their bug team. They just say 2021 is not supported. Use 2020. Well I can't use 2020 because rendering issues and errors in console.

For this I have come to conclusion that you shouldn't listen or watch their marketing videos because those guys will screw you over and just stick with game objects, because there is no guarantee that even those work.

Unity is the only software company that I know who does this.

 

ECS is great (my opinion), but it was marked as experimental from the start, not just after you use it (I don't really understand your argument)? And it doesn't make development impossible, there are already games that use ECS. Unity doesn't stop you, they just don't recommend it.

If it says it is not supported, that also means supported, but if you then install it manually, you shouldn't complain if something doesn't work. If you want to use 2021 you have to wait for 0.51.

 

"there are already games that use ECS."
Yea, I got that. That message as far as I know is for 0.50.
Why block 0.17. intentionally or unintentionally when it was working.

Edit. my point is that if you look at what it was then and you have a good idea.
You want to implement it, but end up as a beta tester for a package that gets moved to preview and gets locked out after negative feedback. Dirty play.

This is also a common complaint what I hear from people switching to Unreal.

 

There can be many reasons why you don't release a packet for a unity version, e.g. it makes little sense to release entities 0.50 for 2021 if other packages like hybrid 0.50 don't work with 2021. And if 0.50 doesn't work with 2021, you can't expect 0.17 to work with 2021 either, since 0.17.0 was released when 2020.2 was still in beta.

Actually, Unity shouldn't even allow you to manually install packages whose version constraints doesn't fit. I work a lot with PHP and composer (packet manager for PHP). And it is not possible to install packets that are not compatible.
And here I also see very often that certain packet versions are only available with certain framework/app versions, this is normal and Unity is no exception.

 

"Actually, Unity shouldn't even allow you to manually install packages whose version constraints doesn't fit. I work a lot with PHP and composer (packet manager for PHP)."

This I agree completely, but in my case my issue with rendering was fixed in 2021.
0.17 was completely working back then with 2021, but either intentionally or unintentionally they blocked it after collections package got released and 0.50 came around.

This preview package thing and the way it handled. For example the demos. They don't update them to work with new versions as far as I know and have tested. They are like so focused on developing the engine that it feels like they forgot that there are actually people whose life depend on making money with it

 

There's a difference between enjoying solving problems that require assembly and enjoying using assembly in spite of the problem. I was curious to know which group of people they were implying.

You have to be very careful with this argument. First, unless I am misunderstanding you, you are implying that someone who struggles or "has issues" that is born out of discomfort may be a sign of incompetency. Yet in my professional career, I have found personality has just as much effect on one's problem solving skills as does their competency. Second, I and many other experienced developers don't face the issues you are encountering with packages because we are using 2020.3. I could suggest by your own argument that you are incompetent, but I won't. I will instead suggest that any evidence to support your argument must be an apples-to-apples comparison. So I will ask you this: What evidence have you seen from someone who loves DOTS implied DOTS solves a problem you have seen an experienced developer solve without DOTS? I love DOTS, so call me out.

 

"I and many other experienced developers don't face the issues you are encountering with packages because we are using 2020.3."

My dots is working. I personally don't have any issues with dots and 2021. The issue what I'm talking is how they handle things and especially the package manager. Why include 0.17 and 0.51 in the package manager if it doesn't work? Why block 0.17 that was working from the package manager, but still leave the rest of 0.17 there, is a sign that unity doesn't care about their customers time. Its completely ok for them that people waste countless of cumulative hours on something that is not even suppose to work. "And just look the forums" is exactly the kind of answer that pisses me off.

Just blocking it completely would be enough of a sign that its not supported. Instead I ended up in a position where I'm in 2021 and can't go back and also, if anyone actually want s to buy and use my plugin they have to use the manifest to install it.

Big brain time xD

edit. Ok I have't tested in like a month, if I can actually go back. Like really tried, but then again I'm not in a mood for having another wrestle with unity just see another obstacle. And I did have one buy for the plugin and luckily the programmer was good enough to solve most of the issues by himself.

And also regarding the oop comment.
There is a difference in things you can control over the things you cannot control.
You can always solve the issues that you have control over, but in the case of package manager and how this is handled I have zero control.

That's why recommending dots in its current state as a mean to solve issue, which can also be solved by traditional means in oop environment is not a valid reason to go for dots.

Its like look how it easy it is in the dots. It only took me years to learn this stuff and lost half of my hair when fighting with the package manager.

Yes, if you use this kind of argument it is a sign of biased recommendation or incompetency.

If I could make a time machine, I would go and warn me not to go with the ecs. Just use Burst and jobs system and you are ok.

It really makes me sad to say this, but after more than year of learning ecs I regret opening this door.

I have made a lot of tests and just burst and job system alone is enough to handle data oriented tasks.

I never would have needed to learn ecs and I'm starting to lean over to the scientists and mathematicians in the seventies (or close to that period) that developed OOP as a humane and natural way of programming.

The thing that I'm happy about is the safety system. Without it I wouldn't even made it this far

 

I don't know if you guys and girls remember the comment Unity made in the ecs presentations in unite.

No game programmer wants to wake up in the morning to code a pooling system. or something similar

Well here some of us are. Using the amount of time that it makes to make a full game just to learn how to program in a data oriented way in ecs environment.

I mean come on and please don't get offended on anything what I say. Its feedback with tears and salt.

 

OK so having used DOTS (ECS+Jobs) a few years ago and now a couple of weeks back into it from 0.50, I think I can contribute at least some impressions on the state of it, but please know I am very inexperienced and humble enough to accept corrections (invited actually):

• Conversion workflow is still awkward, because code is all over the place. I see many different ways to convert but worry if the one I will choose is optimal.
• Sync points and ECB make sense, but aren't really things I feel confident working with yet - maybe I need time?
• Absolutely love ECS as it's own 'pattern'. Doing things the ECS way, I fully believe is the future of game development. There's also a terrific advantage in hybrid workflows (say input is passed into the ECS framework and from there you continue as usual).
• Systems make me feel like some kind of dark god. I'm able to toggle them and mutate others. I have a lot of power but not much confidence in knowing if I am doing it right or not.
• Components being mixed up with Authoring throws me a lot. I wish it was clearer or a forced separation, but I think I will change my tune as I gain experience and familiarity.

In general, I am in love with DOTS and ECS. I think the entire DOTS team deserves a medal. I needed something to wake me up from my slumber, and this has become important to me.

I hope though that it becomes clearer to know if I'm doing a thing right or wrong. My self-confidence in coding with DOTS is all over the place and it's hard to just bang out code without fear. I don't have any trouble working out a way to do it (I have coded since '82 after all) but I do have a lot of insecurity if it's right or not.

I guess more documentation would help. Unity is purposefully quiet about the one true way to do things, I suspect, because they want to learn from our efforts. A good call, but still a barrier to entry for newcomers.

 

"In general, I am in love with DOTS and ECS. I think the entire DOTS team deserves a medal. I needed something to wake me up from my slumber, and this has become important to me."

No doubt about that. Personally i have to drop from the ECS world until its finished. Maybe just do mandatory conversion to 0.51 at some point, but the situation is way too unstable to build anything solid for me and I lost a lot of the trust I had with Unity. My original plan was to make the targeting part of my game as a plugin and share it, so everyone could use it. When I started creating it the other part of it, the ecs had been over a 1-2 years in the development. I was expecting the they could have a version in 3 years that I could use to fund my game development partly, if not completely with the plugin.

Now it looks like I'm starting a position working in a game engine tools for robotics.They don't use Unity and I'm not sure if I want to use my spare time on this. Actually quite sad that it didn't work out. So I guess this might be the end of my presence here.

Don't think people will remember me as a nice person though, since I just had so many issues that where out of my control and filing bug reports on the issues where useless for the issues I had.First rendering issue with 2020 that was fixed in 2021 and then the sudden message they drop the support for 2021 (or maybe there never was one).

 

@mikaelK It took me a while to understand your viewpoint, but I think I understand it now.

It is important to recognize what problem Entities actually solves that set it apart from the rest of DOTS. It is essentially a flexible Burst-compatible parallel superstructure and a set of APIs for interacting with it. If you don't need your game state primarily stored in a Burst-compatible superstructure, you don't need Entities. All other aspects of Entities that people praise it for are just nice-to-haves that can be easily implemented in MonoBehaviour if you know what you are doing. Some people pursue those nice-to-haves simply because they find OOP counter-intuitive. I don't judge them, nor do I judge you for preferring OOP. People are different. And I always encourage people to consider the team they are working with before embarking on a DOTS journey. As for me, I actually need Entities. My interest is in tight integration and orchestration of animation, gameplay, and physics. To do that at playable framerates, I need to be able to execute expensive algorithms (Burst) in an immediate context with directly accessed state (Burst-compatible superstructure AKA Entities). It is also difficult to get these complex things to share state without a DoD approach. Maybe you can think of a slick way to do that in OOP. I couldn't. Of course with Entities, one gives up the local abstract reasoning abilities that OOP provides. I know people who need that to be effective, and I don't judge them. I'm not one of those people.

I'm sorry I didn't challenge you with the right questions when you embarked on your "adventure" with Entities. But hopefully you can at least walk away from this respecting it for what it is and what purpose it really serves (and not what the marketing and hype people advertise it for) and also walk away knowing it isn't right for you and that's okay.

Give yourself some credit. Your gut feelings are a testament to your wisdom despite this still being somewhat new to you.

It is okay to feel lost here. The APIs are pretty chaotic and messy. Unity is renaming and rewriting this mechanism for 1.0. Hopefully they fix the issues I typically run into while also allowing me to not execute code I don't want to be executed.

It takes time, but also a project where it matters. When you look at the timeline view of the profiler and see "IDLE" in the worker threads, that's wasted performance. I call them "bubbles". Sync points are one source of these bubbles. You can win back performance by either eliminating bubbles, or by filling them. Naturally, big bubbles are easier to fill than small ones. But filling bubbles in general typically doesn't happen until a ways into the project when the extra features that can fill them are being built. There's also another way to create bubbles that aren't sync points, and that's if you alternate between single-threaded and parallel jobs, each dependent on the previous. But until you are at the point where you need the performance lost to bubbles, it can be difficult to consider them, and my guess is a lot of games are perfectly fine shipping without ever addressing them.

That "god-like" perspective when writing systems is a signature of DoD programming. Try to avoid coupling systems and instead pass data between them through entities. Otherwise when you need to replace systems or share systems across multiple projects, you will encounter a lot of pain. I tend to have a more extreme view of this than most, to the point where I tie collections to entities and use explicit top-down system ordering. Unity themselves are much more lax about this, which has been a constant pain point in developing my framework when I need to fix transforms or rendering but don't want to ship modified packages.

This one will probably come with time. Consistent naming conventions and code organization will help.

The "working out a way to do it" is the part that trips up most people, so you are doing pretty well! Let performance and simplicity guide you until you encounter pain. Then take a step back an address it (and ask on the forums if necessary). Then repeat. For the amount of time I have been working with Entities, this has not mislead me once.