Showcase Core

Promised for ages I'd publicly post my core library and I finally got around to it
https://gitlab.com/tertle/com.bovinelabs.core
https://openupm.com/packages/com.bovinelabs.core.html

It's a couple of years of extensions, tools and utility mostly focused on DOTS but also just general Unity stuff. This will be well maintained as ~15 of my packages and 2 games rely on it, but I do not guarantee there won't be breaking changes especially with 1.0 coming.

It should be non-destructive in the sense that adding it to your project should change nothing by default (if you open settings window though it will generate a scriptable object)

Some highlights

# Assembly builder


# A hell of a lot of extensions, many for extreme performance optimizations. These include a whole suite of extensions for populating collections including hash maps, lists etc magnitudes faster.

# An extremely fast spatial map - populates 200k positions in 0.2-0.3ms

# Settings workflow which ties directly into conversion and entities (see below screenshot)

# K - a replacement for Enums or LayerMask.NameToLayer. It let's you map strings to bytes/ints in settings and access them from code, even inside burst jobs.
var state = K<UIStates>.NameToKey("build")


# Entity State System, it's great. Check it out.

# DynamicHashMap

# And a lot more stuff I can't be bothered detailing, for example a collection of allocators (NativeSlabAllocator, UnsafePoolAllocator, UnsafeParallelPoolAllocator etc)

 

Wow, amazing. Another DOTS framework released y the community. Great job tertle.

 

Awesome! I've been using some of your libraries already.

 

Nice, thank you!

 

I use it in my project . Can't live with DynamicHashMap now. Other stuff is very good too .

 

Looks interesting. Got a question, what's

Code (CSharp):

1.         byte Value { get; }

in IDynamicHashMapBase<TKey, TValue> for?

It doesn't seems to be used anywhere (at least IDE doesn't pick it up).
Is it a leftover, and could be removed or I'm missing something?

Edit: Also, DynamicHashMap does not render via DOTS editor correctly as well, but I guess its expected & due to custom struct layout instead of being buffer. Still, would be cool to have it rendered.

 

Ensures the size of the struct is 1.
And yeah, unfortunately I've tried but how Inspector works there's no way for me to implement a custom drawer for it.

 

That's interesting. I've tried without it, and it still compiles & runs fine. But, haven't checked if actual data is correct.

Some kind of edge case safety if some other data with zero size is added to the hash map?

 

Not sure how you'd compile without this property? It's part of the interface.
You can add any size data the hashmap, this property is just saying that buffer component needs to be 1 byte. The buffer data is reinterpreted into whatever size it requires.

 

I've removed it from the interface

Alright, got it.

 

https://gitlab.com/tertle/com.bovinelabs.core/-/tree/0.9
0.9 branch is updated to entities 1.0 but extensions are not completely tested. This will be done as I update libraries that use the extensions.
Currently requires old transform which is required by physics/entities anyway but i'll conditionally support both soon

 

Hi. Are there any chances for any documentation in the future?

At the moment I'm trying to understand how SpatialMap works, would really appreciate a script as example on how to build a SpatialMap and how to make basic query with it.

 

Yeah unfortunately I don't really have time to document this, I have other libraries I that are much more important for me to document first that I've put off for months *cough* saving/serialization *cough*.

However here is a quick example for you, the easiest way is to combo it with PositionBuilder if all you want to do is use Local/WorldTransform

 

That's very kind of you, thank you for the example and for the answer.

 

Note I realized there was a typo in the sample

Code (CSharp):

1. var query = SystemAPI.QueryBuilder().WithAll<GhostComponent, LocalTransform>().Build();
2. this.positionBuilder = new PositionBuilder(ref state, query);

should be

Code (CSharp):

1. this.spatialQuery = SystemAPI.QueryBuilder().WithAll<GhostComponent, LocalTransform>().Build();
2. this.positionBuilder = new PositionBuilder(ref state, this.spatialQuery);

 

Finally started upgrading from 0.51 to 1.+. Got a question.
What's the most compatible branch with 1.0.11? 1.0.0-pre.1?

Also, any chance DynamicHashMap gets moved to the separate library so it could be included as dependency for other libs? Something like Bovinelabs.Collections.

Whole toolset is nice, but kinda a waste to include if you only need one collection.

 

Hey, I have a nooby question. In SystemStateExtensions you have functions like this one:

Code (Boo):

1. public static bool HasSingleton<T>(ref this SystemState state)
2. {
3.     var query = new EntityQueryBuilder(Allocator.Temp).WithAll<T>().WithOptions(QueryOptions).Build(ref state);
4.     return query.CalculateChunkCount() == 1;
5. }

I assume this is used in ISystem.OnUpdate e.g., if (state.HasSingleton<MyComponentType>()) ...;
(or the others - but my question is for OnUpdate)

Allocator.Temp? Isn't building an EntityQuery expensive? And aren't they somewhat heavyweight objects with lots of state and buffers? It was in various other ECSes I used, so I always store my queries as early as possible. It's a major pain in the back, much more than even the comically verbose code this extension function wraps.

Or is it okay like this because it's only used at what I call "Schedule-Time", not "Run-Time" for the System's jobs?

(then, why isn't this a convenience method out of the box, would be the first thing I'd write if I were Unity)

Super insightful code base, I greatly appreciate you sharing it! So good to read some actual, real life use case code.

 

Master is fine but the 1.0.0 pre branch is the latest and will be merged in about a week, but I just need to fix up some defines.

I'm going to 1.0 as I'm starting to document what I can.

As for splitting off, no. I have too many libraries a is and it was becoming a pain to maintain so I'm actually doing the opposite with the concept of extensions. Merging other libraries in but making them conditionally optional.

Core is designed to have no impact on your project if you just so it's really not a big deal if to add it as a dependency.

Also it's MIT licences so if you really want you can strip anything out yourself and do pretty much anything with it as long as you leave the original licence info somewhere.

 

Expensive is relative and you shouldn't be doing this every onupdate but maybe you can have a case where SystemAPI isn't available or you simply don't want to add a dependency to a system.

This said this method looks very old and untouched as there is a better way to check this.

 

v1.0.0-pre.1 released
the focus of the final 1.0 release is a bit more documentation

### Added
* Added GetEnableablMask to archetype extensions for IEnableabale extensions
* Support for ObjectDefinition to support child ScriptableObjects
* Added support to edit other package components to make them IEnableabale
* Added WorldSafeShutdown to extensions that ensures a deterministic shutdown and systems are stopped before subscenes are unloaded
* Added StatefulTriggerEvent and StatefulCollisionEvent to extensions. They provide the same functionality as Unity's implementation except rewritten to be parallel providing 5x+ performance gains
* Added NativeMultiHashMap
* Added UIDCreateAttribute to adding an easy right click create menu for ObjectManagement

### Changed
* Renamed CopyEnablable to CopyEnableable
* Renamed BufferCapacitySettings to TypeManagerOverrideSettings. You'll need to update the file if you use this.
* Rewrote DynamicHashMap back end for performance improvements
* Reworked asset creator to now not require inheritances, instead looks for the AssetCreatorAttribute

### Fixed
* Fixed NativeThreadRandom seeds all being the same
* Added a workaround for unity disposing log handle before world shutdown
* Fixed missing define in Core.Tests causing unit tests to incorrectly appear in runner
* Fixed ObjectGroupMatcher not being initialized
* NetCode breaking VirtualChunks if systems landed out of order
* ObjectCategories incorrectly bringing in baking GameObjects into the build
* TypeMangerEx loading wrong file in builds

### Documentation
* Added documentation for K
* Added documentation for States
* Added documentation for Jobs
* Updated outdated setup documentation for Object Definitions

 

I see what's going on now. UT added a sanity check in 1.0 that messes around with DynamicHashMap.

TypeManager.BuildComponentType throws by reflection check:

Code (CSharp):

1.  // Empty types will always be 1 bytes in size as per language requirements
2. // Check for size 1 first so we don't lookup type fields for all buffer types as it's uncommon
3. if (elementSize == 1 && type.GetFields(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance).Length == 0)
4.   throw new ArgumentException($"Type {type} is an IBufferElementData type, however it has no fields and is thus invalid; this will waste chunk space for no benefit. If you want an empty component, consider using IComponentData instead.");

Code (CSharp):

1. ArgumentException: Type T is an IBufferElementData type, however it has no fields and is thus invalid; this will waste chunk space for no benefit. If you want an empty component, consider using IComponentData instead.

Check wasn't included in 0.51. Hence why it wasn't an issue. A shame, extra boilerplate code.

Wish there was a default interface field implementation, or at least UT checked against properties as well.
That way this issue could be bypassed without extra LOC in inherited lookups.

 

Got another question. Entities 1.0.11 + Core / DynamicHashMap 1.0.0-pre.1;
This exception throws:

Code (CSharp):

1. InvalidOperationException: Buffer not initialized
2. BovineLabs.Core.Iterators.DynamicHashMap`2[TKey,TValue].CheckSize (Unity.Entities.DynamicBuffer`1[T] buffer) (at ./Packages/com.bovinelabs.core/BovineLabs.Core/Iterators/DynamicHashMap/DynamicHashMap.cs:274)
3. BovineLabs.Core.Iterators.DynamicHashMap`2[TKey,TValue]..ctor (Unity.Entities.DynamicBuffer`1[T] buffer) (at ./Packages/com.bovinelabs.core/BovineLabs.Core/Iterators/DynamicHashMap/DynamicHashMap.cs:32)
4. BovineLabs.Core.Iterators.DynamicExtensions.AsHashMap[TBuffer,TKey,TValue] (Unity.Entities.DynamicBuffer`1[T] buffer) (at ./Packages/com.bovinelabs.core/BovineLabs.Core/Iterators/DynamicHashMap/DynamicExtensions.cs:66)

This happens when DynamicBuffer is attempted to be used as DynamicHashMap generated via ECB.
(Basically

ecb.SetBuffer<T>(entity).AsHashMap<T, TKey, TValue>();

)

Is it intentional that ECB generated buffers cannot be used now directly to write hash map data?
Previously it was working in 0.51.

Edit: In previous version this check looked like this:

Code (CSharp):

1. [Conditional("ENABLE_UNITY_COLLECTIONS_CHECKS")]
2.         private static void CheckSize(DynamicBuffer<byte> buffer)
3.         {
4.             if (buffer.Length != 0 && buffer.Length < UnsafeUtility.SizeOf<DynamicHashMapData>())
5.             {
6.                 throw new InvalidOperationException($"Buffer has data but is too small to be a header.");
7.             }
8.         }

So its probably a mistake.

Edit 2: Nvm, it seems like InitializeHashMap has to be called now before access attempt.

 

Yep as you figured out I made the Initialization call seperate and explicit.

Previously it just checked every time you called ashashmap but this caused errors if your first call was in a job with a readonly buffer. I was also able to do a bunch of optimizations by separating this.

 

While this may yield some performance gains, its also a major pain to debug where and why HashMap became zero length. Especially when upgrading from previous versions.

There's a hybrid case where dynamic buffer initialization is delayed.
E.g. entity is created with EntityArchetype of DynamicBuffer<TLookup>, but its zero until ECB is received with SetBuffer<TLookup>().InitializeHashMap<TLookup, TKey, TValue>.

So, the solution is to filter out in job alike

if (buffer.Length == 0) return;

.
But its still user error prone, since user may forget to initialize hashmap initialially and job will never work for this reason (fail silently). Which is even worse.

Consider adding something like AsHashMapSafe that initializes HashMap in case if its not initialized yet.
Extra branch sucks, but not as much as critical hard to track bugs.

 

I consider it a critical error not to instantiate it in baking (or at the point you do runtime entity creation.) A user shouldn't need to do it in instantiation or any other time except this.

We found this to pretty much eliminate bugs vs having issues randomly pop up due to system order timing.

Don't get what you mean by this? It works yell at you very loudly if you ever tried to use the hashmap.

 

Folks that do not use baking at all would disagree. Myself included.
User could add buffer directly via EntityManager, but then benefits of writing via ECB would be lost.
Not worth to trade main thread vs random job, unless specific optimization is required.

Pretty much any hashmap buffer that gets added in runtime. Exception points to the job that runs into AsHashMap, but not into the code that adds the buffer itself. Or, clears. Which makes quite a challenge to find all possible cases.

In any case, I've added separate extension for this, it's pretty straight forward.
Way simpler than looking for every possible edgecase.

 

I recently discovered the BovineLabs repository, and it's a real gem. THANKS! This repository will undoubtedly prove very valuable to many DOTS developers, saving them a lot of time.
I have a particular affinity for the IEntityCommands feature, which has proven to be exceptionally beneficial within my GameplayAbilitySystem package. Additionally, I quickly created my own DynamicHashMap implementation due to time constraints, but your implementation far more sophisticated. Unfortunately, I can't integrate it into my project due to its lack of compatibility with netcode.
Thanks Again!

 

Surprised you found IEntityCommands since I've never mentioned it anywhere but yeah it's very useful, I use it for a similar case.

 

I was reviewing your repository's code to check for any recent additions, and I casually saw it. By the way, I've been attempting to use the IEntityCommands inside a job, and I encountered the following error: "Burst error BC1020: Boxing a valuetype CommandBufferParallelCommands to a managed object is not supported." In the job, I invoke a utility function `public static void ExecuteEventTasks(IEntityCommands entityCommands, ... )`.

 

You can't use interfaces in jobs, instead you have to use it like

Code (CSharp):

1. public static void ExecuteEventTasks<T>(T entityCommands, ... ) where T : unmanaged, IEntityCommands

 

I was so exhausted yesterday night that I had a brain glitch! I thought I couldn't use the generics for my whole project, but it is only limited to Bursted Function Pointers, and sadly my package is heavy based on it.
However I already found a workaround
Thanks again for your help !