Official New SubScene & ConvertToEntity workflows

I'm trying to adopt NetCode usage.
i don't know how to convert any gameobject to entity without "ConvertToClientServerEntity" (it's deprecated) it's says use sub scene conversion workflow, I Don't Know what it is!
Can someone explain it please?

 

Hi, in your hierarchy you can right click and create a new sub empty subscene or do the same from any game object in the hierarchy to "turn" them into a subscene.
Everything that is under that subscene will automatically be converted to an entity either when you close it from it's inspector or when you hit play if the subscene is open at the time.

Main benefit is to be able to split you scene into several subscene and have every closed subscene work in entity performent World while you work/edit open subscene that will be in to game object world as long as they are opened.

https://unitycodemonkey.com/video.php?v=91kJtsDLTyE

 

thanks. but how do i manage which world it uses? server? client? default???

 

https://docs.unity3d.com/Packages/c...manual/ghost-snapshots.html#prespawned-ghosts

 

The quote from the docs doesn’t make sense to me at all could you help a noob out and translate it to English?? Haha

what do sub scenes have to do with ghosts? What are prespawned ghosts and how do they differ from predicted/interpolated ghosts?

 

When I drag a subscene into an existing scene (unity 2020.1.15f1), it doesn't open it as a subscene but rather as an additive scene (no conversion is performed).

When I create a new one directly in the scene all works as expected (until I remove it and try to add it back by dragging).

Is there a way to have subscenes loaded as subscenes instead as full scenes? (is this an editor bug, something I am missing or something else)?

 

Create an empty gameobject in your scene with the subscene component, then drag your subscene into the scene field.

 

Here are some general architectural question about subscenes before I take the plunge and convert my project to use them....

Question 1: Will subscenes improve performance?
Most of my levels are usually created on 500x500 terrains and I don't think I'll be trying to dynamically load subscenes based on player position. I read that subscenes can improve load times by storing the scene's contents in an easily loadable binary form, which seems useful, especially when targeting mobile platforms. On the other-hand, there are a handful of people in this thread who have experienced FPS drops after converting to subscenes.

Question 2: How do subscenes work with Addressables?
It seems like subscenes save/cache the subscene in binary format. Does that conflict with Addressable assets or do they somehow work together? For example, what happens if the subscene contains a necromancer model and its associated textures, and then in code at runtime addressables are used to load and spawn an instance of that necromancer from a bundled prefab asset.... does that mean there are two instances of each texture in RAM? Are there two instances on the hard drive?

Question 3: How difficult will it be to convert the workflow to subscenes?
My current code uses ConvertToEntity (on prefabs), IConvertGameObjectToEntity, IDeclareReferencedPrefabs, GameObjectConversionSystem, AddHybridComponent (for particle systems), and [GenerateAuthoringComponent]. Some models prefabs are placed in the scene editor, and others are spawned at runtime. Can I re-use most of that code? Will some things need to be added to a subscene, but others need to stay separate because they're not supported?

Question 4: What does the roadmap look like?
Is future development going to emphasize subscenes and deprecate ConvertToEntity or will they both remain as complementary technologies?

So what's your vote? Are games with small levels better with subscenes, or is a regular scene with "ConvertToEntity" scripts preferable in that case?

 

1. It will surely improve load times, as runtime conversion is SLOW when compared with the subscene loading times

2. Same way as normal scenes in that part... you will end up with duplicates unless you spawn at runtime from the loaded addressable (and unlike with scenes, there is no way to have an addressable subscene currently, but I think that this is planned for the future somehow)

3. Considered all that you mentioned, just drag your stuff into a subscene and done

4. ConvertToEntity and Subscenes work together (I mean, Subscenes relies on ConvertToEntity, Subscene is just "converting stuff at edit time instead of at runtime"), what I would guess that is going away is the runtime conversion, but probably only after we have Addressables compatibility

5. This is up to you, Subscenes doesn't support Convert And Inject for example, so if you rely on the GameObject world somehow that may not even be an option for you. Also the Addressables stuff is not that easy to get it working with Subscenes, you will need to hack you way for that.

 

https://docs.unity3d.com/Packages/com.unity.entities@0.16/manual/conversion.html

According to the doc, Subscene is just a monobehaviour. It gives the editor a scene GUID. and editor will use a special world to convert the scene to an EntityScene. With all Entity and unmanaged component stored in binary form. all the referenced managed objects/assets will be collected and build into an asset boundle. So when loaded this managed data can be found.
I'm not using Addressables in subscene so I can not say anything about it.

But for the binary data part there are a couple of thing that may lead to problems.
1. Type remap, in ECS Type is managed by TypeManager and ultimately assigned a TypeIndex and be referenced mostly by TypeIndex. ECS manage this typeindex internally so most of the time you dont need to worry about it. but when you store type related data as component or BlobAsset, via TypeIndex or simply typeIndex it self, this index could change from time to time. It is not noticeable in the editor because TypeManage is static and it's known types and typeIndex does not change unless you change your code. But when you build your project. TypeIndex will change as runtime type does not match runtime type (with all editor related type wiped out). So you need to remap typeIndex. Some hint can be found here:
https://forum.unity.com/threads/subscene-workflow-typeindex-remapping-how.1017673/

2. Entity remap, subscene binary data is built in editor and sort as binary data in Streaming Asset folder. the Entities in the subscene could have Entity ID/Version conflict with the world that you are loading the subscene into. So to ensure a valid entity all entity need to be remapped as well as entity reference in components. This is handled by Entity Scene loading relatied systems. But I am digging a bug that can be caused by Entity reference stored in BlobAsset, It may lead to missing Entitiy reference when loaded from subscene.

So Basically, when using ECS strickly the Standard+Native+Safe way subscene should work perfectly well. But if you are using special data, There could be many problems that you have not think of yet, and have to handle on your own.

where using special data means:
1. Storing typeIndex as data(in Component or Blob), or store any data that depends on TypeIndex(ComponentType, ComponentTypeTypeHandle, [these are bad examples they use typeindex but you would never want to store them as data, but using TypeIndex is a somehow internal trick so you are probably making your own type as I did]) etc.

2. Storing Entity reference in a place that ECS can not find them. (I don't have a full list, I'm testing Blob asset, And I don't know how to look form them yet) https://forum.unity.com/threads/how...ose-entity-in-blobasset-get-remapped.1020391/
Entity field in Components are fine, ECS takes care of remapping them as my test tells me...

3.Store any pointer in any unmanaged Component, pointer refer to runtime memory address which can not be serialized to binary file form. And if any of you component has a pointer in it, most likely nested in some data type form another lib for example CollisionEventData from physic, your serialize process will fall without conversion log in SubScene(monobehaviour) inspector. You need to check the log file in YourProject/Logs/AssetImportWorker[N].log

Edit as for scene loading speed.
Entity Scene can be pre-loaded and kept in loading world and exported to running DefaultWold in a blink. But preloading a SubScene involves heavy file I/O (loading streamed binary data and asset bould). So I can not say loading a GameObject scene(running mono logic and not using ECS at all) is much slower than loading an EntityScene. But loading a GameObjectScene and live convert those GameObjects to Entity is definitely slower than loading an Entitiy Scene. And live convert has the side-effect of not being able to unload live converted entity with one SceneTag query.

 

Thanks so much for your answer brunocoimbra!! Also thanks Lieene-Guo for the great info!

brunocoimbra - So when you say that I'll get duplicates by using an object in the subscene (or scene) and also by spawning at runtime... I'm assuming that means that the data exists twice in RAM, and twice on hard drive? Is it correct to say that if I then spawn 500 copies, there will still only be two duplicates? What do people normally do in these situations - just accept that there will be some duplicates, or is there a common workaround?

Lieene-Guo - I read your posts on typeindex remapping. I haven't used typeIndex or BlobAssets yet (I'm not that advanced yet), but if I ever run into something similar I'll at least know where to start. The ability to unload a whole subscene full of entities+data with a single query will be very handy... cool!

I decided to make a git branch and give subscenes a shot. I ran into a few issues, but after an afternoon of troubleshooting everything was resolved.

Just a few observations about subscenes vs normal-scenes+ConvertToEntity...

• I really like how during play mode I can click any of the subscene objects in the hierarchy view/tab and it will highlight them in the scene view/tab. After working with "ghost" entities for so long, I almost forgot how useful that can be. I hope someday we'll be able to click entities in the Entity Debugger and see them highlighted in the scene view as well.
  
  
• Scene loading is faster. Much faster!
  
  
• I could have saved a lot of time if I had known that subscenes can't be built with the normal build process. A small tutorial explaining that and another quick tutorial explaining how to setup a "windows classic build configuration" would have saved me about 5 hours. The new build pipeline looks pretty cool... I glanced over some info on how it could be used with addressables and bundled assets and I'll dive into that more at some point.

----------------------------------------------------------------------------------------------------
You can stop reading here... below are just some troubleshooting issues that someone might find useful if they're searching the forums for a particular keyword.
----------------------------------------------------------------------------------------------------

1. One thing that broke was certain monobehaviours that depended on being able to query for a particular entity. The timing was thrown off slightly and suddenly some gameobjects were trying to query-for and use entities before they were created. I fixed this by creating a system that sends out events when the entities load.
   
   
2. Another thing that broke was the pathfinding. At first I tried to put the terrain mesh in the subscene and leave the A* Pathfinding Project gameobject at the root of the main scene, but the A*PFP gameobject didn't like it - it needed the terrain mesh object to scan it and create the navmesh. So I tried to put the A*PFP gameobject inside of the subscene, but the subscene tries to convert everything to entities and components and so it would just delete the gameobject when the game ran... and then that broke all the pathfinding code. I searched the forums but couldn't find a way to exclude a specific gameobject from being converted in a subscene. Eventually, I gave up and had to move the terrain and A*PFP gameobject out of the subscene and add ConvertToEntity back on to the terrain object. I would have preferred putting the terrain in the subscene, but I guess it's not really a deal-breaker.
   
   
3. In the editor, everything worked great, but when I built a standalone Windows version, nothing in the subscene showed up. I tried several approaches, but reached several dead-ends. Eventually, I found that subscenes require the "new scriptable build pipeline". It took a while to figure out that I needed to download a platform.windows package and then create a Windows Classic Build Configuration.

 

You got it! There is no common workaround yet that I am aware off and Addressables is supposed to be supported in the future, so unless you find performance issues specifically related to that I would just not care to try to fix it.

 

@Iclemens, I got some questions: How are you using A*PFP with Entities? How do you handle all the callbacks? Do you use Seeker?

 

Unfortunately, I was not able to get it working with Seeker or any of the AI scripts.

I described the technique in this post here:

https://forum.arongranberg.com/t/compatible-with-entities-ecs-and-dots-framework/7210/16

There is a guy there named HeroSyndromeTheGame who tried a different idea where he used hybrid gameobjects to get it working. It might not be quite as fast with that method since the entities are being driven from the monobehavior side instead of a bursted parallel ecs loop, but I'm betting it's still quite fast and it lets him use the seeker/ai code and even the the RVO algorithm. One of these days I might try out that approach just for comparison.

 

Does anyone know if nesting Subscenes is supported and if not if it's planned?

 

I've got a question about subscenes and creating entities in editor LiveLink (through code), such that they show up in the DOTS editor.

When creating entities during the conversion process (using IConvertGameObjectToEntity), they don't show up in the Editor World, but only in the Converted Scene world.

Is this intended behaviour? This makes it harder to test the Live Link functionality as the generated state of the world isn't visible using the DOTS editor. These entities also aren't automatically cleaned up each time the subscene is reloaded, and as such duplicates are created each time the scene is Live Linked.

Any advice on how to create entities through code in subscenes using LiveLink such that they show up in the Editor World?

 

How can I call this list from an entirely different system?

 

I have a GameObjectConversionSystem running in
GameObjectAfterConversionGroup (but that is irrelevant), and it works fine for normal GameObjects, but it is not run for subscenes the moment these get serialized.

Its Query is one of those for Authoring components:

Code (CSharp):

1. Entities
2.                 .ForEach(
3.                          (BubbleAuthoring input)

The only way to get it to run for subscenes is to keep them open for editing:


But that can't be the purpose? What do I need to do to make a conversion system run when I close and serialize a Subscene?

EDIT: Actually, reimporting the scene like so:

... does result in the expected conversion log (down to the line where the components area dded), but the functionality still isn't there in the game.

It also doesn't hit the breakpoint I set (it does so when I enter play mode, but not on converting the subscene).

 

This probably not it, but have you tried "reimporting" your subscene? I believe the Subscene does the conversion either when it is open on "play" or when it is closed (and needs to be reimported with its new converted binary representation)

I ran into the same issues with items in the subscene not converting and sometimes I resort to "Reimport All" in assets.

 

Thank you! Yes, I found that reimport button, and the system actually seems to run as intended. (I added a timestamp and playmode test to see in what mode it converts, and it's correct).

Breakpoints don't trigger in that conversion code if it is called outside of playmode, though.

That led me to believe the system didn't run.

Now what's not working are the colliders, everything else shows up correct for the most part. But the colliders are borked, they don't interact with an entity that was converted "outside" (by a classic conversion behaviour and some other conversion systems, including the one that processes the things in the subscene; it runs correctly.).

I need to test if entities actually collide with each other inside... hmm. But so far it seems all colliders, shapes as well as mesh colliders, have hashes but don't seem to work.

For a while I suspected it was uniqueness (I have the same mesh in multiple scales), but setting them to unique didn't help, either.

 

Thygrr:

- Do you have the subscene set to "autoload"?
- How many "worlds" do you have in your project?

I ran into this issue as well when I learned that if when you load a scene with a converted subscene, and also create worlds, the subscene entities might not make it into your new created worlds.

Do you might posting what worlds exist in your Entity Debugger before and after you hit play and before and after you enter the scene? It might be a case of your systems and components not making it into the worlds that are running the systems you want.

 

I am actually a lot closer (after spending hours on this so far).

• I have 1 Subscene.
• I have a small number of other gameobjects that get converted by ConvertToEntity (including the player).

Physics Debug will show the colliders as being exactly where they are supposed to be.

Interesting stuff:

• If the subscene is open for editing, everything converts as if it were a gameobject with ConvertToEntity, and everything works, inside and outside.
• If I close the subscene, the player doesn't collide with anything in the scene.
• If I place the player object IN the Subscene and close/reimport it, it actually collides.
  (if I could actually CONTROL this behaviour, it's exactly what my physics culling code does)
• If I place the player object in another subscene, it does not collide.
• I debugged and watched the colliders of the involved objects, nothing indicates that these shouldn't collide, unless there's a hidden data field somewhere even deeper in the collider resources.

I have 1 World through I have some custom culling and physics culling going on; not only have I checked all components in Entity Debugger, but I also disabled that code as a test. Behaviour is the same.


What do SceneSections do?

Code (CSharp):

1. [Serializable]
2. public struct SceneSection : ISharedComponentData, IEquatable<SceneSection>

EDIT: More progress, and found the culprit (but not a solution):

• sometimes, my subscene entities with mesh colliders get a NonUniformScale component with all zeroes. This happens when they are scaled. If I use identity scale, I can collide with them, and they have no NonUniformScale.
• the player gets this too when I place it in the subscene. I reset some transforms (but scales were 1/1/1 identity all along) and now it doesn't happen. Now the player in the Subscene also no longer collides.
• if I then scale the player a little bit, e.g. to 1.1/1.1/1.1, it will collide again.
  
• generally, none of these scales are Non-Uniform though, I work to avoid that in general. they're all x==y==z.
  
  
  
  I mean I could just strip NonUniformScale from the objects... that seems to give me the right collisions, but not the right rendering size.

 

That is the edtior drop down for worlds before "play" is hit, what is it when the scene is playing?

I am wondering if it is the world creation and the gameobject coversion is not allowing the EntityManager of the World to keep track of all the entities properly...

 

So from the docs:

This is a very lame answer but I am guessing maybe cause Unity is bailing from ConvertToEntity that maybe not enough love was continuing to be put in so it functioned right? (with 0 evidence this being the case other than the documentation above saying that it will be phased out soon).

Maybe try switching your "ConvertToEntity" components on those objects to the IConvertGameObjectToEntity interface instead? Seems like a big ask and a hail mary but it might be worth a shot.

 

Yes, this is the worlds.

I think what's going on is that the Subscene conversion fusses about with NonUniformScale for scaled (and some unscaled...) objects.

 

Thygrr the worlds change when you hit "play", Editor World no longer exists when you hit play

 

I appreciate your answer.
All my authoring components are IConvertGameObjectToEntity or use the tick box at the top of the inspector (which is necessary).

The objects are fine, too, it's the ones in the Subscenes that get mangled (and I think it's their colliders getting the scale applied twice / multiplied by 0 / something). It doesn't seem like applied twice though, need to test that hypothesis.

 

You're right, of course EditorWorld will be gone.

 

Thygrr, I am also wondering if you need to use IDeclareReferencedPrefabsInterface for the conversion to happen properly?: https://docs.unity3d.com/Packages/c....html#the-ideclarereferencedprefabs-interface

I personally don't understand exactly what that interface does or why it is necessary. I found I was able to programmatically spawn entities without it: https://forum.unity.com/threads/prefab-workflow-and-managing.968299/#post-6675886

but really smart people on this forum said it is important to use it for ECS to "know" about prefabs. And it seems like maybe your project doesn't properly "know" about prefabs?

It is a shot in the dark sorry.

 

I experimented a bit with prefabs, and currently am just using sibling objects in the SubScene (prefabs are tricky if you want to instantiate them at conversion time).

I don't want to derail this thread further, but one last update:

I split the collider from the renderer for the large scaled object (a small moon / asteroid), and that now collides reliably.

But some of my other, unscaled objects also didn't collide. I narrowed it down to literally this:


Right GameObject will collide once converted. Left will not.
The above as converted entities in playmode (same subscene):

(spoiler, only the IComponentData PointOfInterest is present in the one that doesn't collide - I am investigating but this is only read by one UI behaviour)

This isn't coincidental, all PoIs don't collide. They do all share the same red material... hmm.

 

Just as a summary, I was able to narrow it down to chunk updates.
If I perform a structural change on the entities (e.g. remove the PointOfInterest ICD from even just one of them), they all start to work (all that were of the same archetype, yes even those unchanged).

I suspect the problem is they are static physics bodies, and the physics system at that stage in the conversion must have already committed to their layout as it is written into the subscene?

Adding dynamic or kinematic PhysicsBodies "helps", adding a static one is the same as without.

Interestingly, there's a structural change just before (but only SCD structure changes), and that's after the scene is loaded. I'll take this into the physics forum though.

 

@Thygrrr you really should post this in the Unity Physics sub-forum to get a proper response

 

I have found that it is difficult to get entities/components to work together when comiining

MainScene
- GameObject
- IConvertGameObjectToEntity interface
- ConvertedSubScene
- GameObject

I have checked through https://docs.unity3d.com/Packages/com.unity.entities@0.16/manual/conversion.html but haven't been able to elucidate why those 2 types of conversions (one in a normal scene with an IConvertGameObjectInterface and one in a Sub Scene without any explicit conversion scripts) would somehow not interact.

Anyone have tips on where I could look to figure that out?

EDIT: Demo of what I mean

You can see above there are 4 GameObjects with "ConvertToEntity" on them in the scene. There is also a Sub Scene in the scene called ConvertedSubScene.

Once I hit "play" the GameObjects "disappear" (I assume they are converted to Entities) but the entities are no where to be found in the Entity Debugger.

 

@Joachim_Ante
All link in first post are broken.

 

Are there any tutorials available reflect the current stage of DOTS?

Anyway:
Dots should be split into 2 categories.
1) the animated object stage
2) the static position object stage

And Dots Bridge should transfer objects between stage 1) and stage 2). If required.

Dots Stage 1) should complex as it's.

Dots Stage 2) should be exist as a different world perhaps on a different thread. Should be efficient for add and release entries when load and unload scenes and does not conflicts with the animated stage 1).

Currently while an animation process runs and it's required to load or unload scenes, Dots must stop the animation in order to mix new materials and entities into the same World.

Why not separating Dots into two stages and each stage works with freedom for it self.

Stage 2) require a consequent system to enable/disable entities without removing them from the world.

 

The point of repo, is that you can keep updating it and be able grab older version when needed.
But in particular case, directory and files have been renamed, as older version was using IJobForEach, and newer uses Entities.ForEach
Unity team breaks repo links quite often.

 

Pardon my ignorance, I'm having a tough time understanding when I should expect my subscenes to go through the conversion process. A few questions:

1. If a subscene is closed, and unloaded, is it safe to say that it should never go through the conversion process at runtime (during playmode)? am I right to think that should always happen in the editor, outside of playmode?

I'm having a very hard time getting a closed, unloaded subscene to ever go through conversion, outside of playmode. So far, I've never been able to make it happen. Should these steps be enough to trigger it?

1. Define an IComponentData, and a corresponding Authoring Component.
2. Add that authoring component to a GameObject inside the subscene.
3. Close and unload the subscene.
4. Back in VS, add some fields to the IComponentData - for example, an extra int field.
5. Focus back on Unity, so that it can reimport my code and recompile.

Would these steps be enough to cause that subscene to go through the conversion process again? From reading the Entities documentation (here, under "Type dependencies": https://docs.unity3d.com/Packages/com.unity.entities@0.17/manual/conversion.html) It seems like that should be enough.

I'm in a blank, new project, except for having imported Entities 0.17, and having gone through those steps.

Thank you very much for any help. I feel like subscenes just aren't working at all the way they are described. So I'm probably missing something.

 

Hi folks!
I remove transform components(LTW, Parent etc.) after conversion in GameObjectAfterConversionGroup, but have trouble with removing LinkedEntityGroup. Is it possible to remove/not create it?
Maybe in GameObjectExportGroup? But export group not updated during conversion in my case..

 

Hello!

I'm having troubles with achieving very simple thing. I want to:

1. Convert GameObject hierarchy (parent and its children) into entities
2. Keep parent-child relationships, so when I move parent I expect children also move
3. Add my own component during authoring process to each child

I can achieve 1 and 2 by just adding ConvertToEntity component to the parent. I can achieve 3 by just adding special authoring component to parent or to each child. But! Adding such component (implementing IConvertGameObjectToEntity) somehow breaks parent-child relationships! There are no Parent and no LocalToParent components in children.

And I can't add Parent component in authoring script by myself. If I use AddComponentData then there just no Parent component in result.

And If I do following:

Code (CSharp):

1. dstManager.AddComponent(child, ComponentType.ChunkComponent< Parent >());
2. dstManager.SetComponentData(child, new Parent { Value = entity });
3.  

then there is Parent component, but its Value is NULL Please help!

 

Looks like I found the reason. But I still don't know how to fix it.
The reason is the Rigidbody component.
Without it everything works fine.
But if GameObject has Rigidbody then the resulting entity has no Parent and LocalToParent components.

Why is it?

Please find minimal project, reproducing the issue in the attachement.
I use Unity 2019.4.22 (LTS).

 

I'm going to guess if it has a rigidbody it is converted to unity physics, and unity physics objects get unparented

 

Should this be considered a bug? It makes sense to remove the LocalToParent component (assuming the Rigidbody will never use Kinematic movement).

But Removing the Parent component would break the ability to destroy the child entity when parent entity is destroyed, producing undesired or unexpected behavior.

 

I think entity destruction is based on linked entity not actual parent /child relationship.

 

^^^^ BAD LINK ^^^^

Found the article at
[ https://laptrinhx.com/gameobject-conversion-ecosystem-code-tour-3694412874 ]
Please update link ??

 

man, subscenes are so broken. i just randomly get this towards the end of the build:

even when there are no exceptions when playtesting.
but sometimes it builds successfully

also they randomly get stuck when importing in the editor and you have to add an empty game object & save scene for them to get unstuck

and furthermore, they just outright fail 100% of the time when you have both the uielements and new input system package

subscenes are what's causing me the most suffering atm.
everything else is fine and all the other aspects of dots are fully usable for the most part

 

The UI team fixed this in 2021 which is unfortunate...

 

That is one of the reasons I ditched UI Toolkit. I really wanted to use it but it just had too many incompatibilities with unity versions and the new input package. Also, UI Toolkit's scrollview based widgets were not compatible with touchscreen interfaces. I'm not sure if that has been fixed yet or not.

 

There are so many common things UIElements was neglect. Too many to be practically used. It not even compatible with unity own rendering and material system. And so I really wonder why not just use html instead of force us developer to learn whole new language just for this underdeveloped system

 

I believe the scroll issue is fixed.

What do you use instead of UI Toolkit? Despite all the issues it has been night and day for me for making web app like UIs

 

I'm using the original UGUI. It's not as convenient for making editor tools, but for runtime I don't have to deal with all the incompatibility issues.

In the thread here https://forum.unity.com/threads/custom-listview.850603/ mousepods said touchscreen is supported in UITookit under 2021.1, however, since I'm using DOTS which is incompatible with 2021, that option is not available for me.