Search Unity

When, if ever, are Pure ECS/Dots editor tools coming?

Discussion in 'Data Oriented Technology Stack' started by thedrok_, Jun 27, 2019.

  1. thedrok_

    thedrok_

    Joined:
    Apr 24, 2017
    Posts:
    14
    In the pinned thread titled "New SubScene & ConvertToEntity workflows" It seems to be mentioned that tools for working for pure ECS, e.g. not the convert workflow, would be coming and the last comment expressing this i could find said 'soon' but that was back in late April. Is there any news or updates on the current state of the tools? The convert workflow, at least for me, seems like a round about way of doing things so a set of dedicated tools would be nice.
     
  2. Ferazel

    Ferazel

    Joined:
    Apr 18, 2010
    Posts:
    349
  3. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    5,743
    "Soon" in terms of any development, may mean anything from months to years.
    In terms of Unity releases time frames, that probably at least few releases ahead. Which considering each release per 3 months on average, that gives quite a bit of time ...
     
  4. thedrok_

    thedrok_

    Joined:
    Apr 24, 2017
    Posts:
    14
    So if I'm reading correctly if I install project tiny I can use the dots tools for normal development or only Project Tiny based projects? If so that'd be perfect for now and if not guess I'm working on my own to use until Unity's soon rolls around.
     
    foxnne likes this.
  5. DreamingImLatios

    DreamingImLatios

    Joined:
    Jun 3, 2017
    Posts:
    498
    It is just for Project Tiny at the moment. The direct DOTS editor doesn't have the tooling yet to scale to larger games. GameObjectConversion scales surprisingly well. Actually I prefer it in some sense because it keeps my authoring representation separate from my runtime data, and I personally find it easier to customize the inspector fields using a MonoBehaviour than a custom editor script. The lack of playmode iteration can be a little annoying, but there are workarounds if you know what you are doing.
     
  6. thedrok_

    thedrok_

    Joined:
    Apr 24, 2017
    Posts:
    14
    I haven't tried to Project Tiny DOTS editor yet so excuse me if this is a dumb question but are you saying that the dot's editor doesn't allow you to edit your component data without a custom editor script? Or just not while the game is running?
     
  7. Antypodish

    Antypodish

    Joined:
    Apr 29, 2014
    Posts:
    5,743
    DOTs doesn't have same editor tooling as Project Tiny. Yes, DOTs, is much more raw in terms of editor. Project Tiny probably could say, is more exploring way for the editor, with this new paradigm. When that will be solid, will be transferred as well to DOTs. Or something like that.
     
  8. DreamingImLatios

    DreamingImLatios

    Joined:
    Jun 3, 2017
    Posts:
    498
    GameObjectConversion limitation. Tiny DOTS has different issues which are bigger show-stoppers for me. Tiny DOTS doesn't have prefabs (though you can kinda fake them), and the inspectors for components and buffers expose the runtime data directly which makes for a rather awkward authoring workflow. I know Unity is working on tools to address those issues, and likely when those tools reach a state Unity feels comfortable sharing, that's when we will likely see the Tiny DOTS editor become the not-Tiny DOTS editor too.
     
  9. rizu

    rizu

    Joined:
    Oct 8, 2013
    Posts:
    1,177
    Tiny's DOTS editor is called DOTS Mode, it's not Tiny Mode and according the staff comments, it will eventually work for all DOTS related things.

    Do note that you CAN use some of the full ECS packages in current DOTS editor but since most are currently designed to be used with the conversion workflow, that's not going to help much. Nor is the fact that you can't use hybrid renderer or 3D cameras on DOTS Mode atm.
     
  10. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    4,717
    Our focus is on making the GameObject -> Entity conversion workflow production ready.

    We will do a lot with visualizing converted state, warnings / error overlays in inspector, live conversion, asset database caching, player live connect over the next half year.
     
    Jes28, sstrong, psuong and 1 other person like this.
  11. alexnown

    alexnown

    Joined:
    Oct 25, 2018
    Posts:
    20
    Any ETA when will it be possible to serialize scenes for 32 bit os? Requirement support for Arm-v7 devices prevents us from using the SceneConversion workflow in production.
     
    ZhavShaw and psuong like this.
  12. Ramobo

    Ramobo

    Joined:
    Dec 26, 2018
    Posts:
    18
    @Joachim_Ante
    Please tell me that Entities is not going to be released until there's a stable pure entity editor. The plan for leaving preview is 2020.1, right?
     
  13. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    4,717
    Our editing tools are game objects with a conversion pipeline. We are heavily investing into integrating all that deep into the editor, with visualization, picking, live link etc. As well as a lot of work to reduce boilerplate from the conversion workflow.

    The authoring scene format for DOTS remains the same game object based scenes we have today. If you are judging it based on what is there today, what we are talking about is a very different workflow.
     
    Zoey_O likes this.
  14. Ramobo

    Ramobo

    Joined:
    Dec 26, 2018
    Posts:
    18
    Well, what I mean here is a two-way link between the authoring GameObject and the entity. e.g. The entity's transform is updated and that change is reflected in the authoring GameObject's transform. Also, for that, the entity not being destroyed if its subscene is opened for editing in play mode.
     
  15. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    4,717
    Both of those things are planned. The second one is how it works in the next release.
     
    Ramobo likes this.
  16. DGordon

    DGordon

    Joined:
    Dec 8, 2013
    Posts:
    416
    Will the normal Unity Editor api still be used for creating custom editors with full access to C#? We have an extensive suite of custom editors here spanning years of work, and I cant see us being able to make a switch to dots someday unless we can still do everything our current editors do (besides game related stuff like dialogs and quests we also use services like Amazon Polly to auto create placeholder VO, etc). Im using the editors to create c# files, so im hoping the editors can roughly stay the same but write appropriate dots files instead.

    If we can still use the old EditorGuiLayout stuff Ill be thrilled since I could probably just change our save/load code and reuse the editors as is.
     
    Last edited: Oct 10, 2019
  17. siggigg

    siggigg

    Joined:
    Apr 11, 2018
    Posts:
    131
    In the Tiny talk from Unite 2019, they mention they are changing Project Tiny to use the conversion workflow in their next release.. so I guess "DOTS mode" editor has been cancelled and they are going full steam towards the conversion pipeline.
     
  18. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    4,717
    thats accurate.

    Inspectors or any editor code is thus pretty much entirely unaffected by anything DOTS related. The same is true for the MonoBehaviours that are used as the authoring representation. Commonly you would use the MonoBehaviour representation that was used as the runtime representation in the existing Unity, as the authoring representation for DOTS. This makes it so you wouldn't even have to change any prefabs / scene data...
     
  19. SubPixelPerfect

    SubPixelPerfect

    Joined:
    Oct 14, 2015
    Posts:
    193
    @Joachim_Ante
    Are there any plans on improving editor tools for directly manipulate entities? Two things I dream about are:
    1. Better entity inspector. Right now when you select entity in editor all its values are read-only in the inspector, would be very helpful to be able to tweak data in the inspector, and also to be able to add/remove entity components
    2. Single entity serialization. Are there plans to have serialized entity-prefab-files?
     
    Cynicat, Jes28 and optimise like this.
  20. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    4,717
    There are no plans for that. Our authoring format is game objects and it already supports add / remove component.

    Being able to do tweaks that are not persisted on components could make sense.
     
  21. Creepgin

    Creepgin

    Joined:
    Dec 14, 2010
    Posts:
    297
    I think this is fairly easy to achieve. I'm doing it with Odin right now as standalone tooling in addition to the Entity Debugger. It supports querying entities, adding/removing/setting componentData, creating/destroying entities on the fly, and multiple Worlds. With Odin, it's very straightforward to implement.

    Direct Entity Debugger support will ofc still be nice.
     
  22. SubPixelPerfect

    SubPixelPerfect

    Joined:
    Oct 14, 2015
    Posts:
    193
    IMHO the problem with conversion workflow is that entities data is a mirror of authoring game objects, but that is a one-way mirror, you can't convert back from entity to a game object. For example, if I created an entity with code, or want to use the current runtime entity state later, there is no way to save it as a prefab file.

    What I imagine is a way to serialize selected in the inspector entity to the file and later on to be able to load it to the world. Together with non-read-only entity inspector, this will create pure entities authoring with the following workflow:
    - you define your data components with code
    - you construct and configure an entity with entity inspector
    - save configured entity to file (or a subset of entities)
    - load entity from file
    - ...
    - profit
     
    JoNax97, Cynicat, Jes28 and 2 others like this.
  23. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    4,717
    i don't understand why that is useful?
     
  24. SubPixelPerfect

    SubPixelPerfect

    Joined:
    Oct 14, 2015
    Posts:
    193
    1. transparency: to configure an entity you edit entity data directly - no hidden logic with proxy objects involved
    2. live modification: you change something - you see the result instantly, without the need to go through conversion after each change
    3. consistency: the same approach for both runtime and authoring time manipulations - you can tweak not only the initial state but the current state as well, and you can write current state to file (for example if you have an in-game level editor you can save levels the right same way you do during authoring time)
     
  25. SubPixelPerfect

    SubPixelPerfect

    Joined:
    Oct 14, 2015
    Posts:
    193
    conceptually this is very similar to presets, you can write state data to file and load that state whenever you need
     
  26. Jes28

    Jes28

    Joined:
    Sep 3, 2012
    Posts:
    395
    can be done now

    just add authoring attribute so Unity create monobeh for it and you can configure with inspector

    save sub scene (use like netity prefab of just scene)

    load subscene and may be instantiate entity prefab from it.

    I think I'm one step closer to understand entities authoring in future Unity :)
    One thing from everyday work is modify entity state in runtime for debugging purposes (including add/delete components because tags make a magic) and copy found runtime state back to authoring.
     
  27. SubPixelPerfect

    SubPixelPerfect

    Joined:
    Oct 14, 2015
    Posts:
    193
    @Jes28 I know how to use subscenes and conversion workflow, it is a great approach to migrate existing scenes to ECS...

    but I'm talking about a workflow that does not use MonoBehaviours at all
     
  28. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    4,717
    What we are doing is building a live conversion workflow.

    The concept of directly editing the runtime data itself, independent of what tech is used for it, is in fact exactly the problem we are trying to solve. The concept of runtime data and editing data being the same is simply a bad idea. It took me 14 years to figure that out (Sorry it took so long...)

    But tying the data so closely together prevents us from making independent choices based on whats good for performance and whats good for authoring data. These are totally different optimisation axes. Eg. At edit time you always want a hierarchy. At runtime for example if objects are marked static, you definately dont, you just want everything baked out...

    There is not going to be a world anymore where unity will have the same data at runtime as it has at edit time. Better get used to it. We will do everything to make this transformation live, visible and with zero boiler plate entry points graduating to advanced data transformations when you need to optimise your data at conversion time.
     
    defic, JoNax97, florianhanke and 10 others like this.
  29. optimise

    optimise

    Joined:
    Jan 22, 2014
    Posts:
    456
    I think currently authoring a super big subscene like megacity it's still super lag. I guess the super big subscene also needs dots performance authoring. Long time ago you mention about dedicated entity authoring. Any plan to continue the development of the dedicated entity authoring after Game Object with Monobehavior authoring near to solid state?

     
    Last edited: Oct 12, 2019 at 6:31 AM
  30. SubPixelPerfect

    SubPixelPerfect

    Joined:
    Oct 14, 2015
    Posts:
    193
    Possibility to transform and optimize data from human-friendly to computer-friendly is a very good reason to go conversion way, and i'm 100% agree that this is necessary for great performance in some cases, as well as it is necessary to migrate existing projects.

    But I don't see any reason why direct data manipulation workflow can't coexist with the conversion workflow. In many cases, there is no need for any data optimization or transformation and data can be used as-is, so why not to allow defining it directly?

    And the ability to tweak game state data at runtime is a very important thing for productivity IMHO.

    Data flow from authoring to runtime for sure vital thing, but the ability to bring data back from runtime to authoring is important as well I think.
     
  31. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    4,717
    Yes, I am not claiming that we are done. Our goal is that you can edit a megacity sized level with 200k game objects and it can live convert and live link to the player at 30 FPS. I am quite confident that we can do this on top of game object based conversion and that we can likely hit that around end of year'ish.
     
    elcionap, Kender, elbows and 3 others like this.
  32. DGordon

    DGordon

    Joined:
    Dec 8, 2013
    Posts:
    416
    When you say there wont be a world anymore ... What does this mean for existing software / frameworks built in the pre dots era? Our company makes educational games that goes into schools and has years of code already set up for this. Outside of the file size for webgl, we dont really gain anything from dots since we cant make anything pushing performance anyway. So having the option to switch is great ... but being forced to would be worrisome.

    Should we be expecting this to stop working in the foreseeable future of unity? Meaning, as I add to the existing code base to allow it to handle additional types of games, am I writing for something whose forceful deprecation is already planned? I'll note that I can easily see this code base continuing to push out games for at least another ten years if not more (it has nothing to do with visuals so it can flow with the SRP changes, and stuff like that), so even five years from now would require us to start thinking about it ... soon.

    Thanks! (Sorry if this has been answered else where, feel free to just tell me that ... only asking because of the forcefulness of that statement :))
     
    Last edited: Oct 13, 2019 at 1:54 AM
  33. TheOtherMonarch

    TheOtherMonarch

    Joined:
    Jul 28, 2012
    Posts:
    87
    It means monobehaviour is here to stay. Looking forward to .net 5.0

    Personally, my sim will all be dots. But gui, networking, animation, particles and sound will all be .net

    Honestly, I have no idea what the future of physx will be. I bet monobehaviour physics will become a wrapper for dots physics by 2021-2022.
     
    Last edited: Oct 13, 2019 at 3:14 AM
  34. optimise

    optimise

    Joined:
    Jan 22, 2014
    Posts:
    456
    Ya. I need tweaks that are not persisted on components at runtime. I really want the feature that in play mode, I can see all the runtime data at Hierarchy after conversion from authoring and I can select the runtime entity that I want to inspect its components directly and tweak the value of components at Inspector. Currently at Entity Debugger, it's quite hard to understand which runtime entity correspond to which authored game object and the name of authored game object does not carry over to runtime entity. Entity Debugger also not allow you to tweak the value of components. With runtime entity selected, there is a button at the Inspector that can directly open the original authoring game object. Another nice to have feature is able to move around entity and changing its position component value just like moving around game object and changing its position of transform.
     
    Last edited: Oct 13, 2019 at 6:41 AM
  35. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    4,717
    Sorry I didn't actually mean to there won't be a world anymore. I am sometimes stuck in the new paradigms we are working with.

    What I meant is that for DOTS and any code/data/features depending on it, everything always goes through an authoring representation, converted to entity data.

    For game object / monobehaviour workflows nothing at all will change. In line with what we have said before, MonoBehaviours & GameObjects are here to stay. It would be unimaginable that we would really change anything significant there, since it would break almost every unity project...
     
    Jes28, DGordon and optimise like this.
  36. Joachim_Ante

    Joachim_Ante

    Unity Technologies

    Joined:
    Mar 16, 2005
    Posts:
    4,717
    Yes so i agree. Being able to tweak runtime values is useful. I'll bring it back to the editor team.

    They are not going to affect the authoring state and will not be persisted. If the authoring state changes later on, they will be overwritten as well.
     
    defic, Kender, rocar and 4 others like this.