Question System lookups, options & burst compilation

Preivously in 0.51 I was using SystemBase with custom native containers inside. E.g. NativeOctree.

These containers were accessed inside other ISystems via SystemBase managed reference, dependency was combined to ensure safety, and then containers were used inside related job.

The question, how to properly store and access these types of containers?
ISystem no longer can contain managed reference types. Can store SystemHandle & resolve it via

state.WorldUnmanaged.GetUnsafeSystemRef<T>()

, but there's a big "no you're doing something wrong" tooltip.

So, how to do it properly, and what are the options?

What I was thinking is something similar to ECB.Singleton binding of system fields to the IComponentData.
But, it requires an unsafe collection, and I'm not sure it will work. Plus, all safety would be lost.

In theory I could:
- Strip safety of the collection & make an unsafe version of it;
- Add it to the ISystem;
- "Register" a singleton entity with IComponentData that will store reference to the unsafe version of the collection;
- Access it similarly to the ECB.Singleton via SystemAPI;
- This should be BurstCompile-able for both lookup system & reading / writing system fetch
(since systems no longer interact with each other directly);

But I'm not sure whether it would be safe to read / write it across multiple systems.
If that collection is only accessed via SystemAPI.GetSingletonRW() it should ensure that dependencies are combined / correct automatically, right?

Ideally, I'd like to do the following:
1. Bind persistent collection to system somehow;
2. Access it for Read / Writes in other systems in async way from jobs;
3. With at least some kind of safety;
4. (Optionally) No manual dependency management;

Any suggestions?

 

I believe what you want is possible with singletons, but someone who actually understands that mess will have to explain it to you.

This is what I use: https://github.com/Dreaming381/Lati...e/Collection and Managed Struct Components.md
You can attach collection components to system entities, blackboard entities, or whatever other entities you want. And JobHandles are managed fully automatically be either merging into or reading from SystemState.Dependency behind the scenes. Works perfectly in Burst-compiled ISystem and you can pass the containers inside the collection components into jobs. And all the safety is there with the safe native collections.

 

Yeah, that's what I've been trying to figure out.

ECBSystem pending buffers are fetched and changed on the main thread.
I'd want to be able to access data from the job, like LookupEntity (from query) -> ComponentLookup -> Pointer -> actual data.

CollectionComponent is cool concept, but it seems like have the same issue as just managing singletons.
That is - it needs to be fetched from main thread in order for safety to work properly. And they're "managed", so e.g. OnUpdate won't be burst compiled either. Ideally, I'd want to burst compile the scheduling part as well.

Regarding Singletons / unsafe collections inside component.
If component type is added to the system, it should combine dependency based on read / write, right?

So why not just put unsafe collection pointer inside the IComponentData and let Entities manage dependencies instead on component access?

If pointer / collection is not shared elsewhere, shouldn't it be pretty safe to do?
What other edge cases there might be other than pointers being shared to other components or changed by other logic?

 

This is always true, because you can't use nested NativeContainers in jobs.

This is not true at all. This attribute ain't there for show: https://github.com/Dreaming381/lsss...SubSystems/Gameplay/BulletVsWallSystem.cs#L29

Because the job system doesn't know what the pointer points to, and doesn't know how to patch safety on the AtomicSafetyHandle.

Yes, but the safety system doesn't know that.

Parallel writers don't get the thread index injected.

 

Docs needs to be updated, but, that's good to know.

Alright, makes sense. Basically, one "block" of memory would work if used sequentially, but not in parallel.
Currently I've managed to:
- Re-create most of the lookup data as native containers + unsafe counterparts;
- Attached to the system in following way:
System -> NativeContainer -> Singleton -> IComponentData + UnsafePtr (to the data storage);
- Grab SingletonEntity + ComponentLookup on the main thread;
- In job, access ComponentLookup via SingletonEntity;
- Grab actual [unsafe] data storage via ComponentData Ptr;
- Perform operations;
TL;DR: Blobs, but mutable.

This is all theoretical compiled code, but I haven't run anything properly and now I know why nobody does this.

Pros - it should BurstCompile and automatically manage dependencies.
Con - its absurdly hard to write & read cognitively and, as a result - error prone. Writing native containers is pure pain, especially at high quantity. Plus the need for two variants - one NativeContainer (for the system to keep pointer to the unsafe data storage to cleanup) + actual UnsafeData container. More data types == more code == more issues.

So in the end, its probably going to implode and burn down. I guess I'll try on monday.
If nothing else works, got one extra hack instead of NativeContainers - DynamicBuffers as intermediate layer / storage for the pointer / data. Still super hacky, but DynamicHashmap works wonders. Main con - its always RW, due to DynamicBuffer being always RW.

In any case, Entities needs a better collection <-> Entity relation.

 

Well, that's what I try to do. Except previous version data is not a NativeContainer.
Usually its a struct with a bunch of NativeContainers that aren't physically allocated as one data reference pointer.

So now it has to be converted to NativeContainer first. That's 8 x 2 containers.
x2 because can't grab NativeContainers from entities in a job.

Spoiler: On topic of NativeContainers

Then there's an inconsistent and internal gated allocation related API.
UnsafeList<T>.Create exists.
UnsafeParallelSet<T>.Create - does not.
Neither do UnsafeParallelMultiHashMap<TKey, TValue>.Create or UnsafeParallelHashMap<TKey, TValue>.Create.

Internally unsafe collections can be allocated with AllocatorHandle.
Which is not accessable in user code.

So, extension methods has to be written first.
Thankfully MallocTracked / FreeTracked now exists, so hopefully less random crashes.

Reading / Writing component directly on main thread == stalling previous jobs.
Which is why entity + IComponentData bypass.
Then systems with logic has to be rewritten to grab singletons.
And its ~50 systems ~ a lot of code.

Whole thing sucks majorly. Attaching collections to entities shouldn't be this hard.
Sequential (non-parallel) reads / writes from different systems to the same data block shouldn't be this hard.

Worst part - I don't even know whether it will work, or just hard crash in the end.
Or get a random deadlock because of forgotten RW somewhere.

Judging by ECBSystem @ 1.0.11 state - nobody actually knows how to do this properly either.
A random pointer is thrown for pending buffers and that's about as good solution as none.

It really feels like Collections are written to be a separate package from Entities.
And well, Entities are separate from Collections. There's no inbetween.

 

That's the point I've missed completely. Thank you.

I'll try putting NativeContainers directly.
Previously containers inside IComponentData wasn't supported, so I thought this is still the case.
SingletonRW hack is definitely useful to grab those references.

 

GetSingletonRW doesn't combine dependencies if different systems write to the native container.
How to address this case?
Store JobHandle inside the struct itself or there's a better way to chain them?

E.g.

Code (CSharp):

1.  
2.       public void OnUpdate(ref SystemState state) {
3.           ...
4.  
5.          var spawnProjectileRequests = _spawnProjectileRequestsQuery.GetSingletonRW<SpawnProjectileRequests>().ValueRW;
6.    
7.          new SpawnWeaponProjectileJob
8.          {
9.             AimLookup = lookup,
10.             SpawnRequests = spawnProjectileRequests
11.          }.Schedule();
12.  
13.          ...
14.       }

Will cause:

Code (CSharp):

1. InvalidOperationException: The previously scheduled job SpawnWeaponProjectileJob writes to the Unity.Collections.NativeList`1[Projectiles.ProjectileSpawnRequest] SpawnWeaponProjectileJob.JobData.SpawnRequests.Requests. You are trying to schedule a new job SetupRequestsJob, which writes to the same Unity.Collections.NativeList`1[Projectiles.ProjectileSpawnRequest] (via SetupRequestsJob.SpawnRequests.Requests). To guarantee safety, you must include SpawnWeaponProjectileJob as a dependency of the newly scheduled job.

When attempted to be read / written from a different system.

Manual points to EntityManager.CompleteDependencyBeforeRW.
Thing is, I don't want to force complete jobs. I want to chain them one after another without main thread stalls.

 

Its a simple IJob with ComponentLookups to fetch & update random access data.

Since there's no way to tell what job needs, I've ended up adding Dependency field to the "LookupReference" struct, and manual dependency management works.

Still awkward manual dependency management but, at least it burst compiles and [partially] decoupled from the system.

Overall, safe RW access look like this:

Code (CSharp):

1. ref var lookupRef = ref _lookupQuery.GetSingletonRW<TLookup>().ValueRW;
2. // Minimum RO
3. state.Dependency = JobHandle.CombineDependencies(state.Dependency, lookupRef.Dependency);
4.      
5. state.Dependency = new SomeJob
6. {
7.    Lookup = lookupRef.Lookup
8. }.Schedule(state.Dependency);
9.      
10. // For RW
11. lookupRef.AddWriteDependency(state.Dependency);

Also, yeah, empty IJob.Schedule() vs IJobEntity.Schedule() is a separate codegen gotcha.

 

Yep, it does throw (e.g. if dependency read taken out, or RW taken out when RW is required).
I'm pretty sure _query.GetSingletonRW doesn't codegen anything dependency related to the singleton at all.
Maybe its due to its hack nature.

Second system is IJobEntity system, generated code shows it adds state.Dependency automatically.

Code (CSharp):

1. state.Dependency = __ScheduleViaJobChunkExtension_0(new SpawnWeaponProjectileJob { AimLookup = lookup, SpawnRequests = spawnProjectileRequests }, __TypeHandle.__Weapons_SpawnWeaponProjectileJob_WithDefaultQuery_JobEntityTypeHandle.DefaultQuery, state.Dependency, ref state, false);

Plus, there are more jobs that could potentially write to the same container (through GetSingletonRW) which fails without dependency management.

Previously I was using SystemBase.Dependency directly to combine them and updated Dependency via custom method. Same approach, different JobHandle location. No big deal.

 

Got one more question.
Shouldn't this be safe to read before write happens, but I guess not anymore or this case wasn't covered previously?

The following lookup case looks like this:


If RO job isn't added as dependency, safety freaks out:

Code (CSharp):

1. InvalidOperationException: The previously scheduled job ProcessExplosion_SetupCollisionTestsJob reads from the Octree.NativeOctree`1[HitsGathering.Targetable] ProcessExplosion_SetupCollisionTestsJob.JobData.Lookup. You are trying to schedule a new job GatherDynamicTargetable_BulkInsertJob, which writes to the same Octree.NativeOctree`1[HitsGathering.Targetable] (via GatherDynamicTargetable_BulkInsertJob.Lookup). To guarantee safety, you must include ProcessExplosion_SetupCollisionTestsJob as a dependency of the newly scheduled job.

Correct solution is to move read system (group in this case) after PostTransformationGroup, but I'm still curious.

Theoretically read may somehow happen after write even if first system RO scheduling first before second RW system?

Edit: Its probably not safe because job system cannot assume time period before read and write happens.
Even if group is moved to be later on (RW first, RO later) RO still requires to be added as dependency. Weird.

 

First one is a SystemBase (ProcessExplosionSystem [RO]):

Code (CSharp):

1. ref var lookupRef = ref _targetableLookupQuery.GetSingletonRW<TargetableLookup>().ValueRW;
2. ref var markupLookupRef = ref _markupLookupsQuery.GetSingletonRW<ColliderMarkupLookups>().ValueRW;
3.    
4.  Dependency = Dependency.CombineWith(lookupRef.Dependency);
5.          
6. new ProcessExplosion_SetupCollisionTestsJob
7.          {
8.             Lookup = lookupRef.Lookup,
9.    
10.             RayCmds = rayCmds,
11.             QueryParameters = queryParameters,
12.             MaxExplosionHits = MaxExplosionHits
13. }.Schedule();
14.  
15.  
16.     ...
17.  
18.     var deps = Dependency;
19.  
20.    // Will trigger safety if removed, basically combines lookupRef.Dependency + deps via JobHandle.CombineDependencies
21.    lookupRef.AddDependency(deps);
22.  

Generated code for it looks like this:

Code (CSharp):

1. ref var markupLookupRef = ref _markupLookupsQuery.GetSingletonRW<ColliderMarkupLookups>().ValueRW;
2. Dependency = Dependency.CombineWith(lookupRef.Dependency);
3. Dependency = Dependency.CombineWith(markupLookupRef.Dependency);  
4. this.CheckedStateRef.Dependency = __ScheduleViaJobChunkExtension_0(new ProcessExplosion_SetupCollisionTestsJob { Lookup = lookupRef.Lookup, RayCmds = rayCmds, QueryParameters = queryParameters, MaxExplosionHits = MaxExplosionHits }, __TypeHandle.__Damage_ProcessExplosion_SetupCollisionTestsJob_WithDefaultQuery_JobEntityTypeHandle.DefaultQuery, this.CheckedStateRef.Dependency, ref this.CheckedStateRef, false);

So dependencies are included.

GatherDynamicTargetableSystem [RW] looks like this:

Code (CSharp):

1. ref var lookupRef = ref _lookupQuery.GetSingletonRW<TargetableLookup>().ValueRW;
2. state.Dependency = state.Dependency.CombineWith(lookupRef.Dependency);
3.  
4. state.Dependency = new GatherDynamicTargetable_BulkInsertJob
5.                             {
6.                                Buffer = buffer.AsDeferredJobArray(),
7.                                Lookup = lookupRef.Lookup
8.                             }.Schedule(state.Dependency);
9.  
10. lookupRef.AddDependency(state.Dependency);

Which generates pretty much identical code:

Code (CSharp):

1. ref var lookupRef = ref _lookupQuery.GetSingletonRW<TargetableLookup>().ValueRW;
2. state.Dependency = state.Dependency.CombineWith(lookupRef.Dependency);
3. state.Dependency = new GatherDynamicTargetable_BulkInsertJob
4.             {
5.                 Buffer = buffer.AsDeferredJobArray(),
6.                 Lookup = lookupRef.Lookup
7.             }.Schedule(state.Dependency);
8.  
9. lookupRef.AddDependency(state.Dependency);

Overall it works, but I have no clue why it requires RO job to be part of the chain if it goes RW -> RO.
RO -> RW makes at least some sense why would it trigger safety (in theory).

My best guess is to safeguard against jobs which run for too long & persist to the beginning of the next frame.
I might be wrong though, and it could just be false positive.