Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

World Building [RELEASED] CityGen3D | Procedural city generation from map data

Discussion in 'Tools In Progress' started by CityGen3D, Jan 28, 2018.

  1. chmodseven

    chmodseven

    Joined:
    Jul 20, 2012
    Posts:
    115
    One more thing. I've been spotting this warning showing up now and then when I click a top level generated building's gameobject in the hierarchy:

    2020-07-02 18_07_08.png

    Clicking one of them points to this texture with its settings like this:

    2020-07-02 18_09_43.png

    Anyway it's just a warning and nothing apparently serious - if you drop down from Normal Quality compression to None just like the warning suggests doing it goes away - but I thought I'd report it so that you can look into whatever that EditorLevel script is doing in case it's masking a more serious issue.

    Oh, and I also get this cropping up sporadically when I'm in the editor:

    2020-07-02 18_14_33.png

    But unfortunately I have no further clues as to when or why, and no line number to work with. One of the custom inspectors if I had to guess.

    Cheers!
     
    CityGen3D likes this.
  2. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    681
    Cool, thanks.
    Should be fine to take the Static flag away.
    I’ll do the same for next update.
     
  3. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    681
    Thanks, I’ll check the import settings on those images.

    Strange call stack on that other one, I’ve not seen it before.
    As you say, probably linked to custom editor windows so will do some investigation there.
    In the meantime please let me know if you spot a specific trigger to reproduce it.
     
    chmodseven likes this.
  4. chmodseven

    chmodseven

    Joined:
    Jul 20, 2012
    Posts:
    115
    Here's another possible bug report. I decided to try out creating my own facades and put together a crude PBR window frame material for a test. These are just 1 by 1s using your facade and facade simple shaders:

    2020-07-04 09_04_34.png
    I set this in the Plan as Facade Atlas size of Single Texture, on the assumption that that's the correct setting for a 1 by 1. However when generating buildings, this results in an editor hang:

    2020-07-04 08_56_20.png

    I left that running for a good half an hour and it never budged from that point, and the editor became unresponsive and had to have its process killed.

    If I set the facade atlas size for those plans to Two by Two, however, it does complete. So it would seem there's an issue with Single Texture, or perhaps there's some related setting to be used in conjunction that I didn't know to adjust. Anyway it chops it up according to the "atlas" as can be seen here:

    2020-07-04 09_03_56.png

    I imagine that if I tiled it as an actual 2x2 in the material it would tile properly on the meshes, but at any rate I thought I'd let you know what came out of the test in case it's an actual bug.
     
    CityGen3D likes this.
  5. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    681
    I'm in the process of writing the Building documentation now, so this should help going forward. :)

    In the meantime please take a look at the Glass Office Block prefab. This uses the Glass 01 Plan for it's upper storeys, which is a single texture.
    So it should be possible for single texture materials to work, and that may help you setup yours correctly.

    Having said that, I obviously don't want the Editor to hang even if it's given erroneous input, so I'll try and identify the cause of that and protect against it.

    Thanks again for the all feedback on this!
     
  6. chmodseven

    chmodseven

    Joined:
    Jul 20, 2012
    Posts:
    115
    Okay, using a duplicate of Glass Office Block (and its respective Plans) seemed to fix it. Previously I had duplicated Church and its plans, and then changed the material and used Single Texture in the dropdown.

    I'll hold off doing any more until you've finished the updated doco, as that'll no doubt answer most of the remaining questions I still have about the workflow of configuring the various elements. By the way, did you create the PBR textures yourself, and if so, I'd be curious to know what software and workflow did you use to combine surface materials, draw windows and doors and ageing decals on top and so on. Also if there's any special tricks around the UVs used by the generator to try to get the seams right, I'd appreciate any guidance you might have there.
     
    CityGen3D likes this.
  7. IJas13

    IJas13

    Joined:
    Jul 27, 2017
    Posts:
    19
    Question if you could implement this small feauture: Is it possible to make a option for seperating all generated objects by materials?
     
  8. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    681
    So with CityGen3D procedural buildings I wanted to make it possible for coders with little artistic ability (like me!) to create their own building types without having to create meshes in a 3D modelling app.

    The materials for the procedural buildings were created in Substance Designer like this:

    facade wip.png
    Note that this material is being rendered on a flat cube, all the detail comes from the heightmap.
    I really enjoy the concept of creating geometry through materials with tessellation.
    I would really struggle to create something like that in Blender, but find the node based system of Substance easier to work with.

    Variations on the same theme are then just atlased together on the same material (also using Substance Designer), so you end up with a single material that results in few draw calls when rendered in Unity.

    If there's enough demand, I'll make more materials available in the future.
    Although I'm hoping when the community starts playing around with it a bit more there'll perhaps be third party releases of building materials as well - that would be really cool!

    To help with this, I will try and release a substance material and tutorial video at some point showing my workflow.
    But I'm a novice, and for people interested in learning more about creating architectural materials for CityGen3D (or any other project), I highly recommend checking out the work of Jonathan Benainous. Some of his materials are available for free so you can see how a AAA artist uses Substance to create architectural materials.
    https://magazine.substance3d.com/substance-source-3-architectural-worlds-by-jonathan-benainous/
     
  9. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    681
    Please could you clarify exactly what you mean with a specific example?
    Do you mean you want the option of being able to batch all street lights (same material) together in one mesh?
     
  10. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    681
    CityGen3D is currently in submission with Unity for release on the Asset Store. :D

    Given the large scope of the project I'm not expecting imminent approval, but with a bit of luck I hope to make it through this month. I'll let you know how it goes!

    screenshot0.png
    screenshot1.png
     
    Bartolomeus755 and chmodseven like this.
  11. chmodseven

    chmodseven

    Joined:
    Jul 20, 2012
    Posts:
    115
    Thanks for that link, I'm checking it out now.

    I've been reluctant to get into Substance due to the whole subscription model controversy, but I've found the alternatives like Quixel Mixer to be less useful than I'd hoped. I might have to dive in and embrace the pain, however, though I'll probably hold off until I'm closer to the point where I need to think about production assets. If the Steam discounts had been a little better this sale, I might have got the persistent licences instead, but alas no. Anyway thanks for the details; it does look like you can get some good, easy results with that workflow.
     
  12. IJas13

    IJas13

    Joined:
    Jul 27, 2017
    Posts:
    19

    Yes kind of. The most inefficient things in my project are buildings. Each building has different materials (->shader). I have made a tool which combines materials only with same shader. Some of the objects are using the Standard shader, which could be combined into one materiel.

    For now, I dont know if it also combines your own CityGen-shader, but if so it would drasticaly boost the performance.
     
  13. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    681
    Have you tried some of the suggestions mentioned in this post?:
    https://forum.unity.com/threads/bet...ion-from-map-data.514677/page-11#post-5995253
     
  14. IJas13

    IJas13

    Joined:
    Jul 27, 2017
    Posts:
    19
    Yeah already tried this. My whole scene needs 230 batches where only the building takes up to 200. If I try to batch every material its looking weird. I am goinig to look this week more intensively on optimising my game. Will let you know when I get any trick on how to optmise it more.
     
    CityGen3D likes this.
  15. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    681
    Optimization is a tricky subject because the correct approach will sometimes vary depending on your project.
    Just for reference, here's the stats from the default location with default settings with everything in view:
    stats.png

    As I say, CityGen3D attempts to guess what you may want to do by batching unique geometry together into single meshes for you in some places (eg road markings, sidewalks), but not batching lots of identical objects so you can use Instancing (eg street lights).

    If you have a lot of (especially tall) buildings and your camera is mainly at ground level then you may want to look at occlusion culling. This can prevent overdraw so that the GPU doesn't waste time drawing stuff that's hidden behind a large building in the foreground.
    Game levels are often specifically designed in such a way that there is limited far rendering to take advantage of this optimization.

    However if your camera is quite high up (as in the screenshot), additional batching beyond what CityGen3D already does is definitely worth looking at.
    For instance from this angle, occlusion culling isn't going to help, but batching all the buildings together into fewer meshes may yield positive results.

    Please let me know how you get on. I will experiment with further batching and perhaps add some more options into CityGen3D to expose the built-in mesh batching that it's already capable of doing, just to give you a bit more control and flexibility in the future.
     
    IJas13, chmodseven and Bartolomeus755 like this.
  16. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    681
    CityGen3D v1.00 is now available to purchase from the Asset Store here: http://u3d.as/1KRz

    A big thank you to everyone that has supported the project during the Beta to get us this far! :)

    All existing owners of CityGen3D will continue to get updates from the Download section of citygen3d.com.
    All new users will be able to purchase from the Asset Store and download updates from there.

    A new thread in the Assets section of the forum has been set up, so while I'll try and respond to any questions here, I advise using the new thread in future:
    https://forum.unity.com/threads/rel...from-real-world-map-data.927984/#post-6069897
     
  17. chmodseven

    chmodseven

    Joined:
    Jul 20, 2012
    Posts:
    115
    Great stuff! Congratulations on hitting 1.0 and I wish you all the best in making lots of sales on the store.
     
    CityGen3D likes this.
  18. IJas13

    IJas13

    Joined:
    Jul 27, 2017
    Posts:
    19
    First of all congratulations to make it to the full release and I wish you all the best with this great asset.

    I recently found out that there are some shader problems when using it on iOS devices. Therefore when using it for iOS with the Auto Graphics API selected(see: Player Settings) some shaders appear white (Terrain, Road). Changing it to OpenGLES 2/3 resolves it, but its also marked as Deprecated so for now its an easy workaround but could give some users some problem later on.

    Again congrats and all the best!
     
    CityGen3D likes this.
  19. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    681
    Thanks for the best wishes and information.
    CityGen3D doesnt use a custom terrain shader, so not sure what's going on there. But thanks for the heads up, I will look into it. :)
     
  20. overthrowrobotics

    overthrowrobotics

    Joined:
    Jun 19, 2015
    Posts:
    47
    Running into some issues in 1.0 that I didn't see in 0.94. Load Generator doesn't show all the individual items like HeightMap, SplatMap, Water, Highways, etc.

    I go into Generator and Heightmap and click Generator Terrains. I get Landscape 0, Apply Heightmap and step through the other buttons but nothing is showing up under Landscape 0. Console Logs show a lot of activity and no errors. All I end up with is a tan and gray checkboard terrain with no features. 2D Map (Ctrl-M) shows up ok. I have SRTM data downloaded into the correct folder.
     
  21. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    681
    I think this just may be the result if you forget to Process the map? Please make sure you have 2D map data loaded and processed for the said terrains.
     
  22. Panzerking

    Panzerking

    Joined:
    Jul 24, 2014
    Posts:
    5
    Free ideas time:):
    I was just watching the custom 2D map editing video on youtube, and an idea for a feature popped into my head.
    What if you could load a graphic of a historical/fantasy map, then overlay the CityGen 2D map with an adjustable transparency on top of it, so you could use the historical/fantasy map as a guide while you're drawing the CityGen 2D map? I think that would be cool. Just an idea.
     
    CityGen3D likes this.
  23. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    681
    I actually did that in testing!
    I overlaid PUBG's Karakin map and tried tracing it with CityGen3D map tools. It worked rather well! :)

    I suppose it would be nice to have a built in system for doing it, but it's actually very easy to do anyway.
    Just drop the image you want to trace into your project and create a new material for it, setting transparency to an appropriate value.
    Then add a Plane gameObject of desired scale with this image attached and position it appropriately.
    Then it's just a case of drawing out the map on top of the image.

    As I say, the Map Editing in general needs a few usability improvements to make it really good, like being able to easily join up nodes (eg so two roads at a junction can share exactly the same position). Although existing snap to grid implementations may actually be all that's required, I'll look into it.
    But anyway, it is still fun to play around with even now :)

    And its obviously very useful for editing real world data from OSM to suit your needs too.
     
    Panzerking likes this.
  24. jeffmorris1956

    jeffmorris1956

    Joined:
    Jul 3, 2012
    Posts:
    276
    How can I overlay an image file on a plane in the 2D Map view? I think that the plane doesn't show up in the 2D Map view.
     
  25. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    681
    You just have to assign it to a 2D Layer (eg Map Surface).
     
  26. jeffmorris1956

    jeffmorris1956

    Joined:
    Jul 3, 2012
    Posts:
    276


    The above image is 256 pixels by 256 pixels. What's the size of the plane should I create? I tried 512x512 and 1024x1024 planes.
     
  27. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    681
    Isn't that an OpenStreetMap image?
    With OSM maps you'd just use the normal workflow for importing them and CityGen3D will do all the work for you.

    But Unity has a 1 unit to 1 metre scale.
    The size of the image isn't important, although its just good to have a really hi-res one. It's the scale that you worried about as you want that to match up in Unity.
    I mean a 256x256 pixel image could be showing you a map size of 256 metres or 256 miles, so you need to know the scale as that will determine the size of your plane, not the resolution of your source image.

    When I tried the Karakin map (the smallest PUBG map) I worked on the basis that the map was approximately 2km x 2km in game.
    So I created a plane of 2048x2048 with the intention of using area size of 2 and 1024 terrain size (so 3D environment of 2048x2048 in total).
    Then I just position the bottom-left corner of the plane at [0,0,0] to ensure everything lined up and I would be drawing out my map to correct scale within a 2km region.
     
  28. overthrowrobotics

    overthrowrobotics

    Joined:
    Jun 19, 2015
    Posts:
    47
    I bought EasyRoads3D. I tried their OSM import and the scale of the roads is 2X off (I have to 2X X,Y,Z to fit CityGen map). With all of the intersections not connected it's a bit of a mess and when I try to manually connect them the intersections kind of go crazy. I'd like to try to build the roads by calling their API from the road data that CityGen3D imports.

    Do you have any example code on how I can read the road position information?
     
  29. overthrowrobotics

    overthrowrobotics

    Joined:
    Jun 19, 2015
    Posts:
    47
    After playing with EasyRoads3D API it doesn't allow you to make intersections but RoadArchitect does. Playing with their API right now but it looks pretty comprehensive and it's open source.
     
  30. overthrowrobotics

    overthrowrobotics

    Joined:
    Jun 19, 2015
    Posts:
    47
    Did a test with the RoadAchitect API along with CityGen3D for everything else. Traffic lights, signs, road markings, complex bridges. It's pretty solid.
     
  31. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    681
    You can try this:

    Code (CSharp):
    1. foreach ( MapRoad mapRoad in Map.Instance.mapRoads.GetMapRoads() )
    2. {
    3.     List<Vector3> roadWaypoints= mapRoad.GetPositions();
    4.    
    5.     List<List<Vector3>> laneWaypoints= mapRoad.GetLanes();
    6. }
    Note although Vector3 is returned I think y-axis is just set to zero so you'll need to align to terrain manually where required.
     
  32. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    681
    This is great work, thanks for sharing!

    It's definitely worth checking out RoadArchitect, even just for the street sign prefabs and props, which will hook up to CityGen3D really nicely as well.

    I also intend to use RoadArchitect as a reference for the automated bridge building functionality that I want to add into CityGen3D in a future update.
     
    hopeful likes this.
  33. overthrowrobotics

    overthrowrobotics

    Joined:
    Jun 19, 2015
    Posts:
    47
    I'll post another video tomorrow. I got the road to almost match the height of the terrain. It cuts into it slightly but that might be a setting I can disable. The problem is that CityGen seems to be storing a lot more road information in Map.Instance.mapRoads.GetMapRoads() than the visible map in the Data tab.
     
    Last edited: Jul 22, 2020
  34. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    681
    Map.Instance relates to the processed 2D map data held within the CityGen3D gameObject.
    So the above code should return data for the same location that you see an interactable 2D map for while in Map view.

    If this is not for the location you want, and you have already downloaded the data, you can select Database as Data Source and assign the appropriate database in the input field. Then Process map as normal to give you data for that location.
     
  35. overthrowrobotics

    overthrowrobotics

    Joined:
    Jun 19, 2015
    Posts:
    47
    It's downloaded the data I want but also a significant amount of extra data. In this video you'll see that some of the roads extend out for 10-20 medium sized blocks. I'm seeing exactly 4600 road segments in this medium block.
     
    CityGen3D likes this.
  36. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    681
    Thanks for your feedback here. The GetPositions method is referencing original OSM way data and not the processed data. This will be fixed in an improved GetPositions function for v1.01.
    Worth mentioning that the function wasn't being used anywhere in CityGen3D until we tried to use it here, so it doesn't cause any other problems.

    In the meantime you can use something like this to get the road positions from a mapRoad instance.
    Note that it's technically possible to have more than one path returned from a single mapRoad, due to previous clipping operations on the road, hence the multidimensional array. But in most cases you'll probably just get one List of points back for each mapRoad.

    Code (CSharp):
    1. List<List<Vector3>> positions = new List<List<Vector3>>();
    2. foreach ( Shape shape in mapRoad.GetComponents<Shape>() )
    3. {
    4.     for ( int line = 0; line < shape.GetPathCount(); line++ )
    5.     {
    6.         List<Vector3> outline = shape.GetPositions( line );
    7.         positions.Add( outline );
    8.     }
    9. }
    Let me know if this helps!
     
  37. jeffmorris1956

    jeffmorris1956

    Joined:
    Jul 3, 2012
    Posts:
    276
    Where can I put the the code shown in the above message? That video shows me why processing of data takes a long time.
     
  38. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    681
    Hi Jeff,

    That code is for getting road position data from already processed OSM data.

    As I mentioned in the post, the method being called in that test isn’t used internally by CityGen3D anywhere, so it’s not linked to any normal processing overhead.

    It’s just a public API function, which was erroneously returning unprocessed data from OSM download instead of the already clipped waypoints stored by CityGen3D.

    However we are always looking at ways to speed up the actual processing of OSM data (like the changes made near the end of the beta with the buildings optimisations).
     
  39. overthrowrobotics

    overthrowrobotics

    Joined:
    Jun 19, 2015
    Posts:
    47
    Is this why it's a List inside a List? To get the actual X, Z coordinates I'll need to foreach through 'positions' twice?

     
  40. overthrowrobotics

    overthrowrobotics

    Joined:
    Jun 19, 2015
    Posts:
    47
    Can I use shape.GetPosition() instead? It's asking for pathIndex and Index. What's the index? I'd prefer to have all the Vectors in a single List so I can use the for loop to update the Terrain Y in a single for loop.
     
  41. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    681
    You can iterate like this:

    Code (CSharp):
    1. foreach( List<Vector3> path in positions )
    2. (
    3.     foreach( Vector3 pos in path )
    4.     {
    5.         // do stuff with pos waypoint
    6.     }
    7. )
     
  42. jeffmorris1956

    jeffmorris1956

    Joined:
    Jul 3, 2012
    Posts:
    276
    Richard, Please test CityGen3D on Unity3D 2020.1.0f1 that is released today. I think that Unity3D 2020 is a little bit faster than Unity3D 2019. Note that the Package Manager is little bit different. I created 4 groups of 4X4 terrains and 1 group of 2X2 terrains in a few hours.
     
    CityGen3D and chmodseven like this.
  43. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    681
    Yeah, I posted this on twitter yesterday :)
    CityGen3D v1.0 seems to work fine with Unity 2020. I haven't profiled speed differences yet, but that would be great if there was a boost.

    For stability it's advised to use 2019 LTS, but for those that do want to play with the new Unity features in 2020 alongside CityGen3D, then please let me know if you do run into any problems.
     
  44. jeffmorris1956

    jeffmorris1956

    Joined:
    Jul 3, 2012
    Posts:
    276
    I found two video tutorials on importing OSM data into Unity3D.





    While the OSM import tools can load small sample files, they won't accept my files.
     
  45. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    681
    What are you trying to do?
    CityGen3D already uses OpenStreetMap (OSM) and downloads the required data for your chosen terrains when you click Download button.
     
  46. chmodseven

    chmodseven

    Joined:
    Jul 20, 2012
    Posts:
    115
    I can confirm that CityGen3D seems to work quite well on a fresh 2020.1.0f1 project, and although I haven't measured it, it did seem to run faster. My test New York scene came out looking just fine and all lined up properly with StreetView.

    EDIT: Ignore that maths package stuff I wrote if you saw it before: I am a wally. Just realised that installing Burst will also install the hidden com.unity.mathematics package, so it's fine.
     
    Last edited: Jul 25, 2020
    CityGen3D likes this.
  47. jeffmorris1956

    jeffmorris1956

    Joined:
    Jul 3, 2012
    Posts:
    276
    overthrowrobotics found the problem that took Unity3D so long to process the OSM data. If I process first new group of 4X4 terrains, the data is shown in 2D map within the terrains. If I process another group of 4X4 terrains, the data from both groups are shown in 2D map. I think that Unity3D tries to process data from all groups of 4X4 terrains instead of the new group. I could post a screenshot but I deleted the project.
     
  48. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    681
    You just need to press Clear before Process if you already have 2D map data in memory.

    As I've said previously, what was demonstrated in an earlier video by Chris was an issue with an API function that isn't used internally by CityGen3D at all.
    (And it’s already fixed ahead of v1.01).

    So if you are experiencing particularly slow processing, please give me specific details of the Location settings you used so I can investigate further.
    Then I can look at the profiling in testing and see where to focus optimisations for you.
     
    Last edited: Jul 25, 2020
  49. jeffmorris1956

    jeffmorris1956

    Joined:
    Jul 3, 2012
    Posts:
    276


    I don't know that I have to clear the old data before processing new data. I'll try this and let you know
     
  50. jeffmorris1956

    jeffmorris1956

    Joined:
    Jul 3, 2012
    Posts:
    276
    Clearing old data before processing new data worked! Thank you very much.
     
    CityGen3D likes this.