Search Unity

Assets [BETA RELEASED] CityGen3D | Procedural city generation from map data

Discussion in 'Works In Progress' started by CityGen3D, Jan 28, 2018.

  1. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    186
    Yes, how I see it working is a bit like around 1 minute into that video.
    You'll be able to draw all the different types of CityGen3D objects out in 2D (such MapSurfaces and MapRoads).
    Then any generator settings and functionality would apply to these just as they do for those created by OpenStreetMap at the moment.

    Hypothetically if you draw a mesh even now, say using Pro Builder, and assign a MapSurface component to it, the generator would see that just as if it had been data from OSM. But obviously its really unintuitive at the moment. So this feature will be about providing a user friendly way of being able to draw the 2D maps using a toolbar and clicking out shapes and lines in Scene view (top-down) to make your residental zones, roads and whatever else.
     
  2. jeffmorris1956

    jeffmorris1956

    Joined:
    Jul 3, 2012
    Posts:
    189
    When using CityGen3D to create 3D models of cities, don't try to create groups of terrains larger than 5X5 or you'll get messages about files larger than 4GB.
     
  3. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    186
    Unity has a 4GB scene limit.
    If you are processing very large areas in one go, after creating the terrains click the Convert To Scenes button (under Generate Terrains), which will save down each terrain as a separate scene.

    However, since the terrain tiling workflow was introduced you also have the option of working on smaller areas at a time.
     
  4. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    186
    Here's a work-in-progress look at the new "Features" module for mesh cloning and extrusion along OpenStreetMap ways.
    Perfect for procedural generation of railways, fences, walls and hedges.

    editor_features.png
     
  5. jeffmorris1956

    jeffmorris1956

    Joined:
    Jul 3, 2012
    Posts:
    189
  6. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    186
    Hi Jeff,

    I'd really like to fix the issue you are having so please let me know the Lat/Lon of the 3x3 area that failed so I can reproduce.
    I have no outstanding bug reports from any other user, so I can look into your issues as high priority with a bit more info.
    Thanks!
     
  7. jeffmorris1956

    jeffmorris1956

    Joined:
    Jul 3, 2012
    Posts:
    189
    Is it possible that I may have bad OpenStreetMaps data? What's the best OpenStreetMaps data?
     
  8. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    186
    I'm not sure what you mean by "best OSM data". Some areas are more densely mapped than others. An area like Staten island is great because it has lots of detail in the mapping, but this does mean it can take longer to process.
    However, if it's taking an excessively long time then that may indicate that there's something unique in the data set of a particular location that is taking really long to process.

    Some OSM ways and relations (group of ways) can be huge, and it may be that CityGen3D is trying to process something it doesn't need to, as was previously the case with the Place-Island relation.
    So knowing the Lat/Lon coords is really helpful as it allows me to investigate the exact cause and come up with a fix.

    Also note that you can speed up data processing by ignoring certain types of data on the Processing Filter on Location panel (v0.64+).
    For instance there's not much point processing Buildings and Features yet, because the modules that would utilise the data isn't released yet.
    You can also speed up processing fairly significantly by ignoring coastline processing if you know your current area selection doesn't cross a land/sea border.

    But as I say, if you having to wait more than a few minutes after clicking Process then please give me Lat/Lon cords as an example and I will look into this further as priority.
     
  9. jeffmorris1956

    jeffmorris1956

    Joined:
    Jul 3, 2012
    Posts:
    189
    Status report on CityGen3d?
     
  10. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    186
    Hi Jeff,

    Next release probably a week or two away, but will include some good stuff including more 3D assets to demo the new mesh extrusion feature.
    I'll try and release more screenshots over weekend.

    Planned roadmap at the moment for major iterations to give you a better idea:

    0.7 - Mesh extrusion along ways such as railway, hedges, walls, fences. (Hopefully early August)
    0.8 - Buildings generated from OSM data (either BuildR2 integration, custom solution, or both).
    0.9 - Fixes for Asset Store approval (Undo functionality on Editor windows, etc).
    1.0 - First Asset Store release, if approved.

    After Asset Store Release:
    * Support for 2D map editing and creating custom maps without OSM data
    * Scene streaming solution
    * Support for procedural bridges
    * Start work on separate AI add-on for CityGen3D to add traffic and pedestrians
     
    trig-ger, hopeful and Bartolomeus755 like this.
  11. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    186
    Here are some more test of the upcoming module in CityGen3D allowing you to extrude and clone prefabs along OSM ways, used in these screenshots for the hedges, wooden fences, and chain-link fencing.

    screenshot1.png
    screenshot2.png
     
  12. Bartolomeus755

    Bartolomeus755

    Joined:
    Jun 20, 2013
    Posts:
    240
    Looks great. :)
     
    CityGen3D likes this.
  13. DiscoJustice

    DiscoJustice

    Joined:
    Aug 21, 2013
    Posts:
    8
    Looks really good!
     
    CityGen3D likes this.
  14. PhilipZ0

    PhilipZ0

    Joined:
    Mar 5, 2019
    Posts:
    16
    Any updates on when 0.7 version might come up? The plugin seems amazing.
     
    CityGen3D likes this.
  15. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    186
    Thanks. It’s in test now so hopefully at some point over next few days if nothing unexpected comes up.
    I’ll let you know here as soon as there’s any definite news.
     
  16. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    186
    Beta version 0.7 is now available from www.citygen3d.com

    I'll be updating the manual and website as soon as possible, but the new Features module is setup with lots of defaults for you to play with to give you a good idea of what you can do with it.
    Here are some more screenshots:

    screenshot700_s.png
    screenshot701_s.png
    CityGen3D includes some great prefabs and materials that various artists have kindly donated.
    Please see https://www.citygen3d.com/links for more info.


    The mesh extrusion isn't perfect yet, but it's a really good starting point and a very versatile system. I'll be working on improvements to it while I use it to implement the OSM to Building generator module, so please report any specific issues as normal so I can fix them during beta. Thanks!
     
    Last edited: Aug 25, 2019
  17. jeffmorris1956

    jeffmorris1956

    Joined:
    Jul 3, 2012
    Posts:
    189
    I think that CityGen3D V0.7 doesn't generate train tracks. Did I do something wrong?
     
  18. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    186
    The new Features panel on the Generator is where you'll be able to generate railway on selected landscapes.

    But first you'll want to ensure that the railway is visible on 2D Map and listed in the hierarchy (see left on screenshot).
    If you cant see anything under MapFeatures object you probably just need to reprocess the data to include them.

    You can do this by selecting Features in the drop-down Filter box in the Processing section of the Data->Location panel and clicking Process.

    test.png
     
  19. jeffmorris1956

    jeffmorris1956

    Joined:
    Jul 3, 2012
    Posts:
    189
    Is there difference between railway and track?
     
  20. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    186
    A railway, track, wall or fence is just a series of points called a Way in OpenStreetMap.

    They are differentiated from each other by a key/value combination tag that says what that way represents.

    For example:
    fence/chain-link
    wall/brick
    railway/tram

    You can see which key/value combinations your scene is using by looking at Data-Features panel.

    So if there’s a specific rail track type that isn’t being identified by CityGen3D that you want to use then you are able to add this in so it gets found.

    You’ll just need to re process the data after you have added the new tags.

    Most other modules work in a similar way and you have full control over what data is used in each instance.

    Once the data is present, you can then use the Generator to associate different prefabs to the new feature type so they appear in 3D in your scene.
     
  21. jeffmorris1956

    jeffmorris1956

    Joined:
    Jul 3, 2012
    Posts:
    189
    I think that CityGen3D expects the tracks to be called Railway (UK) and not Railroad (USA).
     
  22. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    186
    Yeah, it's all about knowing how it's tagged in OSM.

    You can check this yourself by loading up www.openstreetmap.org in your browser and finding your location.
    Zoom in close enough that the query tool is available to use and click it (Thats the button with the arrow and question mark icon).
    Then click over the railway in question.
    You should then see a list of all the tags associated with it.
    The recommended tag (key/value) for railways is "railway/rail".
    But it's plausible "railroad" or something else is being used in some places.
    If that's the case, you simply need to add in the tag to CityGen3D as described above.

    If preferred, if you send me the lat/lon coords for your location I'd be happy to investigate this for you.
     
  23. jeffmorris1956

    jeffmorris1956

    Joined:
    Jul 3, 2012
    Posts:
    189
    On Open Street Map website, enter the lat/lon coords:
    Top: 40.51416
    Left: -74.25230
    Right: -74.24856
    Bottom: 40.51256

    You should see Staten Island Railway. Adding "Subway" tag to Data-Features panel didn't work.
     
  24. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    186
    Most of that railway seems to be tagged as "railway"-"subway".
    So if you add it to the Data-Features panel as shown in screenshot it should work.

    features panel.png

    Just make sure that Features are active in the processing filter as shown before clicking Process.

    processing filter.png

    Once you've done that, railway objects should appear parented to MapFeatures (for the 2D map).
    You can then use the generator to add them to your 3D scene.

    Of course, subways are underground in real life and CityGen3D doesn't support procedural tunnelling yet, in the same way it doesn't support bridges. ( I hope to in the future, but naturally we have to start simple :) )
    So if you add subways in as described, the railway will appear at ground level, although you could move it below the terrain and use it as the basis for an underground network, adding in appropriate tunnel geometry manually.
     
  25. jeffmorris1956

    jeffmorris1956

    Joined:
    Jul 3, 2012
    Posts:
    189
    I followed your instructions and it worked. As for real-life subways, they are usually not 100% subway. A subway system can have elevated track structures, opencuts, embankments, and on ground.
     
    CityGen3D likes this.
  26. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    186
    Great, happy to hear you got it working.

    Yes, I work in London so all too familiar with subways! :)
    It's quite possible to map that level of detail in OSM. Different segments of a subway could be tagged with "layer/-1" or "layer/0" depending if they are above or below ground.

    Roads (and everything else) are layered in the same way, which is how CityGen3D knows not to generate underground road segments or bridges as if they were ground level.
    That's eventually how it will know when to generate tunnels or bridges, once we support that level of detail.
     
  27. jeffmorris1956

    jeffmorris1956

    Joined:
    Jul 3, 2012
    Posts:
    189
    After generating water, is water supposed to be tan instead of blue-green?
     
  28. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    186
    Here's a screenshot showing water from a few months ago:
    https://forum.unity.com/threads/bet...tion-from-map-data.514677/page-4#post-4287388

    Remember to "Apply Modifiers" to the heightmap (via Generator->Heightmap panel) to reduce terrain height where water is. If it looks brown-ish that may just be the muddy terrain rather than the water mesh.
     
  29. jeffmorris1956

    jeffmorris1956

    Joined:
    Jul 3, 2012
    Posts:
    189
    I applied modifiers to heightmap but the tan areas (I think beaches) are below the ground and the ocean.
     
  30. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    186
    The Modifiers section of the Heightmap panel allows you to offset the height of different surface types.
    By default, this includes negatively offsetting sand terrain, but you can customise this how you want.
    If you don't want that to happen you can remove it from the modifiers list by clicking the appropriate [X] (next to the Sand modifier).

    It's also a good idea to use the heightmap Smoothing after using Modifiers and to generate the water mesh after you have finished your heightmap modifications and smoothing.

    Here you can see how I tweaked the settings to get the same location looking like this.
    You may need to adjust the y-axis transform of your water planes manually to get it precisely where you want.

    buffalo street.png
     
  31. A-S-U

    A-S-U

    Joined:
    Sep 2, 2019
    Posts:
    3
    Great work, my first tests with CityGen3D v0.7 are working fine.
    But depending on the latitude and longitude I'm testing, some areas display elevation artifacts when there is no data in SRTM files.

    For example in this area 49.437267303, 1.0580348968: upload_2019-9-2_16-3-22.png

    The corresponding SRTM data (from https://dds.cr.usgs.gov/srtm/version1/Eurasia/N49E001.hgt.zip ) is missing in some area:
    upload_2019-9-2_16-3-22.png

    The interpolation is using the missing data and it seems to use boundaries values to create the elevation map, producing fun maps :) :
    upload_2019-9-2_16-3-22.png

    I'm trying to find elevation data from other sources, howerver maybe the interpolation should not use boundary values from missing data.
     
  32. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    186
    Thanks for your message.

    Coincidentally I came across the same issue the other day when testing a location for Jeff, so I'll try and release a fix for it in 0.71.
    In the meantime you can change your elevation source to Bing or MapQuest which may suit your needs better until this is fixed.

    These are both integrated into CityGen3D and you just need to select it from the Heightmap listbox, which will say SRTM by default.
    You will need obtain a free key from the relevant provider and enter this in the key field which will appear under the listbox when either of these are active.
    Then just Generate Heightmap as normal.

    Once I know SRTM works fully I may drop support for the other two sources, but it's quite a handy alternative in the meantime.
     
  33. A-S-U

    A-S-U

    Joined:
    Sep 2, 2019
    Posts:
    3
    Thanks for the quick answer.

    However if I try to use either Bing Map or MapQuest, the first one fail with a http 500 error and the other with a 504 gateway timeout.
    Looking at the http requests it seems the problem comes from the serialization of the parameters/post data in the http request send to servers. I use a french locale on my computer and floating numbers are formated with coma instead of dot.

    When serializing the request, I can see in logs that floating numbers stiil use coma instead of dot. Same problem when the response comes from the MapQuest server, it parses strings using the computer local instead of the API floating number locale.
    I wrote a quick proxy to manage those requests/responses rewrite but a fix would be better :)
     
  34. jeffmorris1956

    jeffmorris1956

    Joined:
    Jul 3, 2012
    Posts:
    189
    I think that there are problems with the Bing Maps server today.
     
  35. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    186
    Thanks for the info. I fixed a similar locale parsing issue on the OSM requests a while back and I wasn't aware of a similar issue in the heightmap requests. I'll look into it.
     
  36. A-S-U

    A-S-U

    Joined:
    Sep 2, 2019
    Posts:
    3
    After some tries, I can't manage Bing servers to answer free requests on too small area.
    I tried the same request on a larger area and Bing server answered correctly.
    Maybe a change in the way elevation data is provided for free accounts ...

    Anyways, I succeed to use elevation data on my area. But interpolation results are strange; here are result with SRTM 30m (with smoothing and flattening):

    I check SRTM elevation data using QGIS 3D view, there seems to be less waves/staircase effects:

    I also got elevation data from another source on the same area with a 25m step, artifacts completely disappear at this level:

    I also got even more precise (5m) elevation data but is there a way to load it with CityGen3D ?
     
    Last edited: Sep 3, 2019
  37. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    186
    From your message yesterday I identified a potential locale issue in the MapQuest server requests, so I should be able to fix that in next update.

    Not sure what the issue is with Bing. I couldnt see similar potential for a locale issue, so may just be server issue. Bing was the first one I supported but it was a bit unreliable which is why I ended up looking at alternatives.
    If anything, it's normally happier with smaller requests because the server can get upset with lots of requests at once and there is one request sent per terrain tile.
    I will try it out on your location here and see if I can get to the bottom of it.

    I'll spend most of my efforts fixing up any issues you experience in SRTM data though, because I probably see that as the long term provider for CityGen3D. So likewise, I'll run some tests here and see what's going on with the interpolation.

    There's currently no support for importing other data sources, but if you can point me in the right direction regarding your source for the 5m resolution data, I'd be happy to look into it.
     
  38. scarfell

    scarfell

    Joined:
    May 16, 2017
    Posts:
    5
    Hi there

    Just got CityGen3D and its amazing. had a lot of fun playing with all the different options and results

    I also had 2 questions.

    I noticed on your website you could gen houses with gardens and fences. I happen to use rik4000's UK Asset Collection and have a lot of the same models. How could a get results like this?
    upload_2019-9-8_11-52-9.png
    Second was about features generating time. it takes quite long for each tile, about 30min. Is that normal?

    Thanks and keep up the great work :)
     
    CityGen3D likes this.
  39. jeffmorris1956

    jeffmorris1956

    Joined:
    Jul 3, 2012
    Posts:
    189
    CityGen3D, I was able to create 15 separate scenes. Each scene has 5X5 terrains. I created a new blank scene and tried to import 15 scenes into the new scene but Unity3D crashes to desktop when trying to import 4th scene from Project window to Hierarchy window. I tried Scene Streamer from Asset Store but it doesn't work with Unity 3D 2019.2. They created MegaCity demo but I don't know how to set up scene streaming. How can I create large game worlds?
     
  40. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    186
    Hi,

    Happy to hear you are enjoying it!

    30 mins does seem excessive for Feature generation.
    I think there's an issue where each terrain currently wastes some time on processing stuff outside it's bounds.
    In other words, the exact same location with an active area size of only one will at the moment be much faster, most likely.
    I hope to have a fix for that in next update.

    Regarding generating houses with gardens and fences, at it's most basic level it's really just a case of adding additional meshes to a prefab, then assigning that prefab to a Roadside-Prefabs slot.
    As long as each mesh in your prefab has a collider, CityGen3D will ensure it doesn't overlap another object in any layer specified in the Avoid drop-down box in Roadside-Distribution rules.

    In this screenshot you can see I'm working on a House With Garden prefab. I will include a few new prefabs in 0.71 to demonstrate this workflow more clearly.

    house with garden.png

    So I should have some exciting info to share on this subject later this month, including how to ensure your props (such as fences and walls) in roadside prefabs align to terrain and how you can automatically add texture and detail to your gardens. Just need to put in a few fixes first to support all this functionality properly.
     
    scarfell likes this.
  41. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    186
    Hi Jeff,

    Yes, loading lots of scenes manually into the Editor won't work as you'll run out of memory.
    The main idea behind scene streaming is to asynchronously load and unload scenes as you need them, while the player moves around the environment.

    So you add a reference to each scene in your build settings and use scripts to manage this process at run-time.
    As you have mentioned, the free Scene Streamer asset demonstrates how you can achieve this effect in Unity.
    This asset demo works for me in 2019.1, but I've not tried specifically in 2019.2.

    As noted previously, I am working towards providing in-built support for this in CityGen3D so you won't have to code anything yourself to get it to work.
    Recent changes to CityGen3D, such as being able to save terrains as individual scenes, are steps towards this goal.
    However, we aren't quite there yet, so you do have to rely on third-party assets in the short term.

    Next stages towards this aim in CityGen3D will be to add support for smaller terrain tiles (eg 512x512 and maybe 256x256 instead of fixed at 1024x1024).
    This should make it easier to work with and offer more flexibility when setting up larger worlds.

    I will let you know as soon as I have more news in this area.
     
    StevenPicard likes this.
  42. jeffmorris1956

    jeffmorris1956

    Joined:
    Jul 3, 2012
    Posts:
    189
    I get type or namespace errors in "First Person Controller" script when I try to load the Scene Streamer demo into Unity 3D 2018 and 2019.
     
  43. CityGen3D

    CityGen3D

    Joined:
    Nov 23, 2012
    Posts:
    186
  44. matidzeramz

    matidzeramz

    Joined:
    Oct 5, 2016
    Posts:
    4
    amazing asset
     
    CityGen3D likes this.