Search Unity

  1. Unity 6 Preview is now available. To find out what's new, have a look at our Unity 6 Preview blog post.
    Dismiss Notice
  2. Unity is excited to announce that we will be collaborating with TheXPlace for a summer game jam from June 13 - June 19. Learn more.
    Dismiss Notice

[Released] Terra Slicer & S.A.M.

Discussion in 'Assets and Asset Store' started by gilley033, Nov 5, 2013.

  1. NDVR

    NDVR

    Joined:
    Jun 23, 2016
    Posts:
    4

    Thanks for the quick reply.

    The APV(Adaptive Probe Volume)of HDRP supports streaming, but since SAM needs to re-center the World, I am not sure whether there will be any problem. Currently, in URP, only lightprobe can be used for indirect illumination.As far as I know,World Streamer2 has the ability to cut lightprobe, but I haven't used it. Because I saw it on the screenshot of its UI .
    For culling solutions, I'm currently using Perfect Culling on assetstore and Umbra culling officially provided by unity, both of which have pre-baked data.

    It seems that the pre-baked data cannot be modified for location.So I'm a little confused about that.

    Here are some links to information.
    https://docs.unity3d.com/Packages/c...ition@16.0/manual/probevolumes-streaming.html
     
  2. gilley033

    gilley033

    Joined:
    Jul 10, 2012
    Posts:
    1,197
    Thanks for the info!

    I'm pretty sure what you're saying with this is that for large open worlds where floating point issues can arise, SAM needs to re-center the World, but I just want to clarify for anyone coming upon this thread that the re-centering feature is optional. If a large World is not large enough to introduce floating point errors, you do not need to make use of the re-centering feature.

    Also, I understand that HDRP supports camera relative rendering, which will eliminate floating point related rendering issues. Other systems like the Physics system will not benefit from this, but as I understand it there's a way to shift the Physics origin via code (need to investigate further). Again, other systems may still have issues with floating point numbers, so even if the rendering/physics systems do not require re-centering, it still may be needed by other systems.

    As far as APV's, I will probably focus on trying to add support for the traditional lightmap baking stuff first, including light probe volumes, since APV's are new and still being refined. From a Unity Dev's answer on the topic, they do not seem to be designed for dynamically loaded content, however hopefully support for such content will be added in the future, even if it is via scripting.

    I will explore this in the coming week and report my findings on what different options I see being possible.

    I'm pretty sure the user I mentioned before was using Perfect Culling, however I'm having trouble tracking down my conversation with that user. I will try to dig into the Perfect Culling documentation (assuming it is available online) to see what I can find out, and of course will look into Umbra culling as well (after exploring lighting first).

    In general though, you are right about pre-baked data. Most systems that use it, especially Unity's systems, assume that the objects contributing to the bakes will be static, and thus changing the position of the baked data is not possible (making those systems unworkable with our world re-centering feature).

    According to a Unity Dev in this forum thread (https://forum.unity.com/threads/ada...r-hdrp-in-2021-2.1238824/page-11#post-8990842), in 2023.2 they added the ability to move pre-baked lightprobe groups, however I doubt similar functionality exists for other baked data (but will look into this further). But hopefully this means they are working on supporting such functionality.
     
  3. NDVR

    NDVR

    Joined:
    Jun 23, 2016
    Posts:
    4
    I found a useful article to help us analyze what is needed for a big world map.You might be useful, too.

    Original address
    https://mp.weixin.qq.com/s/iDMWhi-YYrYnQYZpzZvuqg

    The documentation for Perfect Culling is very simple, and most of us use it through video tutorials. I can DM you if you want.
    Thank you for your reply and look forward to your good news.
     
    Last edited: Nov 28, 2023
  4. gilley033

    gilley033

    Joined:
    Jul 10, 2012
    Posts:
    1,197
    That seems like a nice wishlist, thanks! An HLOD system is definitely useful, however at this point it's not something I plan to support (although you should be able to implement your own system if you wish).

    I haven't had time to look into the Occlusion Culling or Lighting stuff yet, but I'm hoping to have time the rest of the week.
     
  5. gilley033

    gilley033

    Joined:
    Jul 10, 2012
    Posts:
    1,197
    @NDVR I have not forgotten about this, I promise!

    I was hoping to have more time to look at all of this this week, however some personal things and bug fixing left little time for research and development.

    Today I started exploring lightmapping (without the use of light probe groups). I don't have anything to report yet, and it will probably take most of next week before I have something definitive to report. However, tomorrow I will look at the Perfect Culling stuff so that I can at least give you an answer on that.

    Thanks for your patience!
     
  6. gilley033

    gilley033

    Joined:
    Jul 10, 2012
    Posts:
    1,197
    Okay, so my initial findings are that most of the systems that require baking in Unity are not well configured to work with large game worlds, as they mostly require you to load your entire game world into the editor in order to perform the bakes.

    I am still investigating if there are alternatives to this, however if this workflow can work for you (e.g., if your game world is not so large that loading the entire thing into the editor would crash Unity), you should be able to start testing out using baking based systems with SAM right now.

    Please bear in mind that all of these steps were created with very limited testing. This is more of a theoretical workflow and step-by-step guide and it is definitely not a definitive guide. However I wanted to post something so that users could try out the workflow and see how it goes. Maybe it works, maybe it doesn't (shrug). Maybe you find out that I missed a step. Maybe you find out something that works better. As such, proceed with caution!

    To get started, should setup SAM like so:
    1. Commit to using scenes for everything that you want to bake (this might not be strictly true for Perfect Culling as it seems it can bake prefabs, however for Unity's bake based systems it seems to be required, as the baked data is stored with the scenes). In terms of SAM, this means you should be using scenes for your Asset Chunks along with a Scene Chunk Streamer or an Addressable Scene Chunk Streamer.

    2. On all Scene Chunk Streamers that will load scenes with baked data, enable the "Keep Scenes Intact" option. This ensures that the additively loaded scenes will remain loaded (with this option disabled, the scenes are unloaded once the scenes Asset Chunk has been been moved to the main scene).

    3. Disable Hierarchy Organizers for any LOD Groups using a Scene Chunk Streamer that you have enabled the "Keep Scenes Intact" option for. You can disable Hierarchy Organizers for the entire world by nulling out the Hierarchy Organizer field under the World's main "Optional Components" window, or you can disable it on a per Grouping or per LOD Group basis via the Overridable Settings system.

      To be fair I am not positive if this step is required. It will basically keep each root game object within its containing scene, which I assume is needed not to break the baked data. But I could be wrong.
    Now, to actually perform a bake, you will:
    1. Setup your main scene with whatever settings are needed for the bake. For instance, you can assign a Lighting Data asset to the main scene. It's important to perform this setup now before loading the other scenes in your game world, as adjusting the settings while the other scenes are open will cause them all to be dirtied, which will slow the process down. Also disable any auto baking/generate options for the bake at this time.

    2. Launch the World Designer Tool. Ensure the Tool's Origin Row/Column (and possibly Layer) are configured correctly (ideally so that the game world is centered around 0,0,0).

      Load every Streamable Cell that is using scene based asset chunks and that you want included in the bake. Do this for every Grouping that you want included in the bake. It might be a pain if you've got a lot of Cells/Groupings, but it's the only way to get this done at the moment.

    3. Navigate to the Global Settings tab of the World Designer Tool's Main Controls window, click the World Regions dropdown, and click on the "Create New World Region" option. Rename this World Region to indicate that it encompasses the entire game world. This step will make one of the next steps much easier, and will also make performing additional bakes down the road much easier!

    4. Set "Unsaved Asset Handling Method" to "Auto Save". This will make a future step much faster.

    5. Close the World Designer Tool (having it open will just slow down the bake).

    6. Perform the bake. This is going to take some time! Maybe an hour or more. Unity will also probably spit out a warning about settings being different between one or more scenes. I'm pretty sure you can ignore this warning, but I'm still looking into it.

    7. Once the bake has completed, you will notice that all the loaded scenes are dirty and need to be saved. You can probably right click the main scene and use the "Save All" button, however I used the approach in step 8 and can confirm that it works.

    8. Re-launching the World Designer Tool and go to the Global Settings window if needed. Confirm "Unsaved Asset Handling Method" is set to "Auto Save" and that the World Region selected via the dropdown is the one representing your entire game world.

      Click the "Unload Region" button, which will unload all the scenes (saving them before doing so).
    And that's it! You should be able to enter play mode and see the baked data "working", at least if you are using a "fixed" world (i.e., the Streamable Cells are being loaded at the same location as they were when the bake was performed).

    If the Asset Chunks belonging to the Cells are moved (as a result of using an Origin Cell value that is different than the Origin Cell set on the World Designer Tool at the time of the bake), I think the baked data might get messed up, but I still need to test this further to confirm.

    Again, please remember that this is a work in progress. I apologize in advance if this breaks something!
     
    tmcz_911 likes this.
  7. NDVR

    NDVR

    Joined:
    Jun 23, 2016
    Posts:
    4
    Thank you for the theoretical ideas and steps. It should work in theory, as long as the map isn't too large. In game development, we also take into account chunk updates and modifications, which can affect the size of game update packages. During operation, if the target chunk is to be baked, eight chunks around it must be baked together. We want to keep only the light data of the target chunk.I can't seem to do that right now.Could The World Designer Tool have this support?

    In addition, it is suddenly considered that the navigation grid cannot be cut and streaming.
     
    Last edited: Dec 6, 2023
  8. gilley033

    gilley033

    Joined:
    Jul 10, 2012
    Posts:
    1,197
    That would be the ideal way to do things, and that's what I am currently exploring to see if it's possible.

    I just discovered that there is a Additive Scene Loading Lighting tutorial available from Unity on the Asset Store, so I'm going to explore that today or tomorrow and hopefully will have a better insight into all of this.

    I assume you are talking about Unity's NavMesh? If so, you are correct, there is no cutting technology built into SAM at the moment, and I will likely try to focus on how to bake NavMesh data when using a multi-scene/asset chunks setup (like the lighting stuff) rather than how to cut existing navmesh baked data. But assuming it's possible to get the baking working and to end up with each base chunk having separate navmesh data, I do believe it should be possible to stream this data in and connect the separate navmesh data using NavMeshLinks.

    Thanks for your patience. If you'd like, you can use the World Designer Tool to load groups of Asset Chunks (for groups of Streamable Cells) and try to test out baking lighting/navmesh/occlusion data for each set of chunks, with the center Base Grouping Chunk being set as the active scene. I don't know how well this will work, as my assumption is that when you perform subsequent bakes, some lighting data will be overwritten. However, you can do some experimentation and report your findings here, if you'd like.
     
  9. Sholms

    Sholms

    Joined:
    Nov 15, 2014
    Posts:
    85
    does this system also convert the terrain to mesh? it could be a nice feature, also export normal maps.
     
  10. gilley033

    gilley033

    Joined:
    Jul 10, 2012
    Posts:
    1,197
    Not currently, although this is something I may add in the future (the terrain to mesh feature at least).

    There are free solutions to this I believe (https://github.com/jinsek/MightyTerrainMesh for one). I don't recommend any of them as I haven't used them, but they may be worth a try.
     
    Last edited: Jan 1, 2024
  11. gilley033

    gilley033

    Joined:
    Jul 10, 2012
    Posts:
    1,197
    Hey everyone!

    Happy New Year!

    I just wanted to make a note that with the next update, the In-Editor Guide is going to be removed, as I have recently moved this documentation over to my website.

    SAM: https://deepspacelabs.net/html/sam/docs/ProductOverview_Introduction.html
    Terra Slicer: https://deepspacelabs.net/html/terrain_slicing/docs/ProductOverview_Introduction.html

    It's a work in progress. I am slowly adding more and more visual aids (like pictures and GIFs), you may find some GIFs are missing at the moment, and some information may need to be updated, however the information is now out there in the wild for existing and prospective users to peruse at their leisure. Enjoy!

    At the moment, the only Chapter from the SAM documentation that is complete (with visual aids) is the Scene Conversion tutorial. This week I will finish the Product Overview Chapter and Getting Started Tutorial Chapter (by adding visual aids).
     
  12. gilley033

    gilley033

    Joined:
    Jul 10, 2012
    Posts:
    1,197
    New updates for S.A.M. (update 1.4.0) and Terra Slicer (update 1.2.0) are now live on the Asset Store!

    With this update, the In-Editor Guide has been all but removed in favor of the online documentation, which if you don't know can be found on the website:

    S.A.M. Docs
    Terra Slicer Docs

    Here are some of the main changes for each product:

    S.A.M.
    1. Re-worked World Designer Tool UI to make it a bit less cluttered.

    2. New options for choosing how objects are assigned with the Assignment Operation (this is mainly used for breaking down scenes), including a custom option which you can use to implement your own custom logic (scripting knowledge required).

    3. Deprecated the LODCrossFadeTransitioner and moved its functionality into the MatPropBlockTransitioner and PerMaterialTransitioner components.

    4. A ton of bug fixes.
    Terra Slicer
    1. Official support added for all 2023 Unity versions (note, S.A.M. should also work on these versions, however that is untested).

    2. Newly designed inspector for the Slicer that makes use of tabbed windows.

    3. Added a new method for slicing Alphamaps that should result in more accurate positioning of alphamap features. While this new method can result in a slight loss of fine details, it also does a better job at matching up the alphamap along slice borders, meaning alphamap blending should not be required when using it (you also still have the option of using the previous alphamap slicing method).

    4. You can now generate slices for "3D" terrain groups (groups whose terrain names have a layer value). You can also now slice terrain where the terrain component is not on the root prefab(s) you are slicing. These changes allow you to slice terrain prefabs that have created using S.A.M.'s World Designer Tool.

    5. Terrain Data names no longer need to follow a specific naming convention when slicing terrain groups. In addition, the slicer will now skip over missing prefabs in the group instead of throwing an exception.

    Full S.A.M. Change Log
    Full Terra Slicer Change Log
     
  13. gilley033

    gilley033

    Joined:
    Jul 10, 2012
    Posts:
    1,197
    Hey, I just wanted to say that I have not forgotten about this! I have some other users who have requested support for a third party asset (MicroVerse), so I am currently working on that. Afterwards, I will be returning to work on the stuff we have discussed. Sorry for the delays!
     
  14. gilley033

    gilley033

    Joined:
    Jul 10, 2012
    Posts:
    1,197
    Hey there, it's been a while so I wanted to provide you (and others) an update on what I am currently working on, as it pertains to what we have discussed.

    I am finally getting around to researching the various systems we have discussed in earnest (Lightmapping, Static Batching, Occlusion Culling, etc.), and I have some good news.

    First, lightmaps on their own do not present any issues with floating origin based Worlds. The disclaimer about lightmaps only being for static objects specifically is in reference to objects that move in relation to other objects. Things like the terrain, houses, scenery, etc. can be lightmapped and used without issue, as their positions remained fixed relative to other "static" objects. This means that even if you move these objects around the scene, so long as other lightmapped stuff moves the same amount, everything will look correct.

    I am still looking into Light Probes to see if this holds true for them, but in the mean time I do have some good news with regards to them. I have just implemented a solution for "slicing/cutting" existing light probe groups so that they can be split into discreet groups on a Cell by Cell basis. This basically involves taking the original group and calculating the Cell that each probe in the group falls within, then assigning all probes that fall within the same Cells to new Probe Groups.

    I'm still testing but it seems to be working fine.

    As a result of all of this research, I have also decided that a means of automating various operations so that they can be performed on multiple Cells/Asset Chunks in one go would be very useful. For example, performing a lightmapping automation to bake light data on one or more regions of the World. I am in the beginning phases of implementing this, but I do not foresee any issues. The system will be extensible as well, so that you can write custom code to automate whatever task you want, such as task for third party assets.

    At the moment, I am still trying to figure out the best way to handle lightmapping for dynamic Worlds, because while theoretically it is possible to use them for floating origin based Worlds, Unity still does not make it easy to bake individual parts of the World. Hopefully I will have some good news on this front soon (I have some ideas on how to handle this that may or may not work).

    Thanks for listening!
     
    one_one likes this.
  15. gilley033

    gilley033

    Joined:
    Jul 10, 2012
    Posts:
    1,197
    I just uploaded a new video previewing the upcoming Automation System. If anyone has any questions, let me know!