World Building [RELEASED] CityGen3D | Procedural city generation from map data

Yes, how I see it working is a bit like around 1 minute into that video.
You'll be able to draw all the different types of CityGen3D objects out in 2D (such MapSurfaces and MapRoads).
Then any generator settings and functionality would apply to these just as they do for those created by OpenStreetMap at the moment.

Hypothetically if you draw a mesh even now, say using Pro Builder, and assign a MapSurface component to it, the generator would see that just as if it had been data from OSM. But obviously its really unintuitive at the moment. So this feature will be about providing a user friendly way of being able to draw the 2D maps using a toolbar and clicking out shapes and lines in Scene view (top-down) to make your residental zones, roads and whatever else.

 

When using CityGen3D to create 3D models of cities, don't try to create groups of terrains larger than 5X5 or you'll get messages about files larger than 4GB.

 

Unity has a 4GB scene limit.
If you are processing very large areas in one go, after creating the terrains click the Convert To Scenes button (under Generate Terrains), which will save down each terrain as a separate scene.

However, since the terrain tiling workflow was introduced you also have the option of working on smaller areas at a time.

 

Here's a work-in-progress look at the new "Features" module for mesh cloning and extrusion along OpenStreetMap ways.
Perfect for procedural generation of railways, fences, walls and hedges.

 

Unity 3D stops responding when trying to process 10th group of 5X5 terrains. I tried a group of 3X3 terrains and it didn't work. I will no longer use your CityGen3D addon. I tried OpenStreetMaps addon for Blender3D shown at https://lesterbanks.com/2016/06/realistic-city-openstreetmap-blender/ and it worked.

 

Hi Jeff,

I'd really like to fix the issue you are having so please let me know the Lat/Lon of the 3x3 area that failed so I can reproduce.
I have no outstanding bug reports from any other user, so I can look into your issues as high priority with a bit more info.
Thanks!

 

Is it possible that I may have bad OpenStreetMaps data? What's the best OpenStreetMaps data?

 

I'm not sure what you mean by "best OSM data". Some areas are more densely mapped than others. An area like Staten island is great because it has lots of detail in the mapping, but this does mean it can take longer to process.
However, if it's taking an excessively long time then that may indicate that there's something unique in the data set of a particular location that is taking really long to process.

Some OSM ways and relations (group of ways) can be huge, and it may be that CityGen3D is trying to process something it doesn't need to, as was previously the case with the Place-Island relation.
So knowing the Lat/Lon coords is really helpful as it allows me to investigate the exact cause and come up with a fix.

Also note that you can speed up data processing by ignoring certain types of data on the Processing Filter on Location panel (v0.64+).
For instance there's not much point processing Buildings and Features yet, because the modules that would utilise the data isn't released yet.
You can also speed up processing fairly significantly by ignoring coastline processing if you know your current area selection doesn't cross a land/sea border.

But as I say, if you having to wait more than a few minutes after clicking Process then please give me Lat/Lon cords as an example and I will look into this further as priority.

 

Status report on CityGen3d?

 

Hi Jeff,

Next release probably a week or two away, but will include some good stuff including more 3D assets to demo the new mesh extrusion feature.
I'll try and release more screenshots over weekend.

Planned roadmap at the moment for major iterations to give you a better idea:

0.7 - Mesh extrusion along ways such as railway, hedges, walls, fences. (Hopefully early August)
0.8 - Buildings generated from OSM data (either BuildR2 integration, custom solution, or both).
0.9 - Fixes for Asset Store approval (Undo functionality on Editor windows, etc).
1.0 - First Asset Store release, if approved.

After Asset Store Release:
* Support for 2D map editing and creating custom maps without OSM data
* Scene streaming solution
* Support for procedural bridges
* Start work on separate AI add-on for CityGen3D to add traffic and pedestrians

 

Here are some more test of the upcoming module in CityGen3D allowing you to extrude and clone prefabs along OSM ways, used in these screenshots for the hedges, wooden fences, and chain-link fencing.

 

Looks great.

 

Looks really good!

 

Any updates on when 0.7 version might come up? The plugin seems amazing.

 

Thanks. It’s in test now so hopefully at some point over next few days if nothing unexpected comes up.
I’ll let you know here as soon as there’s any definite news.

 

Beta version 0.7 is now available from www.citygen3d.com

I'll be updating the manual and website as soon as possible, but the new Features module is setup with lots of defaults for you to play with to give you a good idea of what you can do with it.
Here are some more screenshots:

CityGen3D includes some great prefabs and materials that various artists have kindly donated.
Please see https://www.citygen3d.com/links for more info.

The mesh extrusion isn't perfect yet, but it's a really good starting point and a very versatile system. I'll be working on improvements to it while I use it to implement the OSM to Building generator module, so please report any specific issues as normal so I can fix them during beta. Thanks!​

 

I think that CityGen3D V0.7 doesn't generate train tracks. Did I do something wrong?

 

The new Features panel on the Generator is where you'll be able to generate railway on selected landscapes.

But first you'll want to ensure that the railway is visible on 2D Map and listed in the hierarchy (see left on screenshot).
If you cant see anything under MapFeatures object you probably just need to reprocess the data to include them.

You can do this by selecting Features in the drop-down Filter box in the Processing section of the Data->Location panel and clicking Process.

 

Is there difference between railway and track?

 

A railway, track, wall or fence is just a series of points called a Way in OpenStreetMap.

They are differentiated from each other by a key/value combination tag that says what that way represents.

For example:
fence/chain-link
wall/brick
railway/tram

You can see which key/value combinations your scene is using by looking at Data-Features panel.

So if there’s a specific rail track type that isn’t being identified by CityGen3D that you want to use then you are able to add this in so it gets found.

You’ll just need to re process the data after you have added the new tags.

Most other modules work in a similar way and you have full control over what data is used in each instance.

Once the data is present, you can then use the Generator to associate different prefabs to the new feature type so they appear in 3D in your scene.

 

I think that CityGen3D expects the tracks to be called Railway (UK) and not Railroad (USA).

 

Yeah, it's all about knowing how it's tagged in OSM.

You can check this yourself by loading up www.openstreetmap.org in your browser and finding your location.
Zoom in close enough that the query tool is available to use and click it (Thats the button with the arrow and question mark icon).
Then click over the railway in question.
You should then see a list of all the tags associated with it.
The recommended tag (key/value) for railways is "railway/rail".
But it's plausible "railroad" or something else is being used in some places.
If that's the case, you simply need to add in the tag to CityGen3D as described above.

If preferred, if you send me the lat/lon coords for your location I'd be happy to investigate this for you.

 

On Open Street Map website, enter the lat/lon coords:
Top: 40.51416
Left: -74.25230
Right: -74.24856
Bottom: 40.51256

You should see Staten Island Railway. Adding "Subway" tag to Data-Features panel didn't work.

 

Most of that railway seems to be tagged as "railway"-"subway".
So if you add it to the Data-Features panel as shown in screenshot it should work.



Just make sure that Features are active in the processing filter as shown before clicking Process.



Once you've done that, railway objects should appear parented to MapFeatures (for the 2D map).
You can then use the generator to add them to your 3D scene.

Of course, subways are underground in real life and CityGen3D doesn't support procedural tunnelling yet, in the same way it doesn't support bridges. ( I hope to in the future, but naturally we have to start simple )
So if you add subways in as described, the railway will appear at ground level, although you could move it below the terrain and use it as the basis for an underground network, adding in appropriate tunnel geometry manually.

 

I followed your instructions and it worked. As for real-life subways, they are usually not 100% subway. A subway system can have elevated track structures, opencuts, embankments, and on ground.

 

Great, happy to hear you got it working.

Yes, I work in London so all too familiar with subways!
It's quite possible to map that level of detail in OSM. Different segments of a subway could be tagged with "layer/-1" or "layer/0" depending if they are above or below ground.

Roads (and everything else) are layered in the same way, which is how CityGen3D knows not to generate underground road segments or bridges as if they were ground level.
That's eventually how it will know when to generate tunnels or bridges, once we support that level of detail.

 

After generating water, is water supposed to be tan instead of blue-green?

 

Here's a screenshot showing water from a few months ago:
https://forum.unity.com/threads/bet...tion-from-map-data.514677/page-4#post-4287388

Remember to "Apply Modifiers" to the heightmap (via Generator->Heightmap panel) to reduce terrain height where water is. If it looks brown-ish that may just be the muddy terrain rather than the water mesh.

 

I applied modifiers to heightmap but the tan areas (I think beaches) are below the ground and the ocean.

 

The Modifiers section of the Heightmap panel allows you to offset the height of different surface types.
By default, this includes negatively offsetting sand terrain, but you can customise this how you want.
If you don't want that to happen you can remove it from the modifiers list by clicking the appropriate [X] (next to the Sand modifier).

It's also a good idea to use the heightmap Smoothing after using Modifiers and to generate the water mesh after you have finished your heightmap modifications and smoothing.

Here you can see how I tweaked the settings to get the same location looking like this.
You may need to adjust the y-axis transform of your water planes manually to get it precisely where you want.

 

Great work, my first tests with CityGen3D v0.7 are working fine.
But depending on the latitude and longitude I'm testing, some areas display elevation artifacts when there is no data in SRTM files.

For example in this area 49.437267303, 1.0580348968:

The corresponding SRTM data (from https://dds.cr.usgs.gov/srtm/version1/Eurasia/N49E001.hgt.zip ) is missing in some area:


The interpolation is using the missing data and it seems to use boundaries values to create the elevation map, producing fun maps :


I'm trying to find elevation data from other sources, howerver maybe the interpolation should not use boundary values from missing data.

 

Thanks for your message.

Coincidentally I came across the same issue the other day when testing a location for Jeff, so I'll try and release a fix for it in 0.71.
In the meantime you can change your elevation source to Bing or MapQuest which may suit your needs better until this is fixed.

These are both integrated into CityGen3D and you just need to select it from the Heightmap listbox, which will say SRTM by default.
You will need obtain a free key from the relevant provider and enter this in the key field which will appear under the listbox when either of these are active.
Then just Generate Heightmap as normal.

Once I know SRTM works fully I may drop support for the other two sources, but it's quite a handy alternative in the meantime.

 

Thanks for the quick answer.

However if I try to use either Bing Map or MapQuest, the first one fail with a http 500 error and the other with a 504 gateway timeout.
Looking at the http requests it seems the problem comes from the serialization of the parameters/post data in the http request send to servers. I use a french locale on my computer and floating numbers are formated with coma instead of dot.

When serializing the request, I can see in logs that floating numbers stiil use coma instead of dot. Same problem when the response comes from the MapQuest server, it parses strings using the computer local instead of the API floating number locale.
I wrote a quick proxy to manage those requests/responses rewrite but a fix would be better

 

I think that there are problems with the Bing Maps server today.

 

Thanks for the info. I fixed a similar locale parsing issue on the OSM requests a while back and I wasn't aware of a similar issue in the heightmap requests. I'll look into it.

 

After some tries, I can't manage Bing servers to answer free requests on too small area.
I tried the same request on a larger area and Bing server answered correctly.
Maybe a change in the way elevation data is provided for free accounts ...

Anyways, I succeed to use elevation data on my area. But interpolation results are strange; here are result with SRTM 30m (with smoothing and flattening):

I check SRTM elevation data using QGIS 3D view, there seems to be less waves/staircase effects:

I also got elevation data from another source on the same area with a 25m step, artifacts completely disappear at this level:

I also got even more precise (5m) elevation data but is there a way to load it with CityGen3D ?

 

From your message yesterday I identified a potential locale issue in the MapQuest server requests, so I should be able to fix that in next update.

Not sure what the issue is with Bing. I couldnt see similar potential for a locale issue, so may just be server issue. Bing was the first one I supported but it was a bit unreliable which is why I ended up looking at alternatives.
If anything, it's normally happier with smaller requests because the server can get upset with lots of requests at once and there is one request sent per terrain tile.
I will try it out on your location here and see if I can get to the bottom of it.

I'll spend most of my efforts fixing up any issues you experience in SRTM data though, because I probably see that as the long term provider for CityGen3D. So likewise, I'll run some tests here and see what's going on with the interpolation.

There's currently no support for importing other data sources, but if you can point me in the right direction regarding your source for the 5m resolution data, I'd be happy to look into it.

 

Hi there

Just got CityGen3D and its amazing. had a lot of fun playing with all the different options and results

I also had 2 questions.

I noticed on your website you could gen houses with gardens and fences. I happen to use rik4000's UK Asset Collection and have a lot of the same models. How could a get results like this?

Second was about features generating time. it takes quite long for each tile, about 30min. Is that normal?

Thanks and keep up the great work

 

CityGen3D, I was able to create 15 separate scenes. Each scene has 5X5 terrains. I created a new blank scene and tried to import 15 scenes into the new scene but Unity3D crashes to desktop when trying to import 4th scene from Project window to Hierarchy window. I tried Scene Streamer from Asset Store but it doesn't work with Unity 3D 2019.2. They created MegaCity demo but I don't know how to set up scene streaming. How can I create large game worlds?

 

Hi,

Happy to hear you are enjoying it!

30 mins does seem excessive for Feature generation.
I think there's an issue where each terrain currently wastes some time on processing stuff outside it's bounds.
In other words, the exact same location with an active area size of only one will at the moment be much faster, most likely.
I hope to have a fix for that in next update.

Regarding generating houses with gardens and fences, at it's most basic level it's really just a case of adding additional meshes to a prefab, then assigning that prefab to a Roadside-Prefabs slot.
As long as each mesh in your prefab has a collider, CityGen3D will ensure it doesn't overlap another object in any layer specified in the Avoid drop-down box in Roadside-Distribution rules.

In this screenshot you can see I'm working on a House With Garden prefab. I will include a few new prefabs in 0.71 to demonstrate this workflow more clearly.



So I should have some exciting info to share on this subject later this month, including how to ensure your props (such as fences and walls) in roadside prefabs align to terrain and how you can automatically add texture and detail to your gardens. Just need to put in a few fixes first to support all this functionality properly.

 

Hi Jeff,

Yes, loading lots of scenes manually into the Editor won't work as you'll run out of memory.
The main idea behind scene streaming is to asynchronously load and unload scenes as you need them, while the player moves around the environment.

So you add a reference to each scene in your build settings and use scripts to manage this process at run-time.
As you have mentioned, the free Scene Streamer asset demonstrates how you can achieve this effect in Unity.
This asset demo works for me in 2019.1, but I've not tried specifically in 2019.2.

As noted previously, I am working towards providing in-built support for this in CityGen3D so you won't have to code anything yourself to get it to work.
Recent changes to CityGen3D, such as being able to save terrains as individual scenes, are steps towards this goal.
However, we aren't quite there yet, so you do have to rely on third-party assets in the short term.

Next stages towards this aim in CityGen3D will be to add support for smaller terrain tiles (eg 512x512 and maybe 256x256 instead of fixed at 1024x1024).
This should make it easier to work with and offer more flexibility when setting up larger worlds.

I will let you know as soon as I have more news in this area.

 

I get type or namespace errors in "First Person Controller" script when I try to load the Scene Streamer demo into Unity 3D 2018 and 2019.

 

This can be installed separately from Unity Standard Assets
https://assetstore.unity.com/packages/essentials/asset-packs/standard-assets-32351

 

amazing asset

 

Hi, I’m wondering will this application have the ability to auto gen terrain textures that match actual satellite imagery, for example, field borders, separated with ditch lines or even mounds like in this image.
I think this is a great tool and it’s got huge potential, keep up the great work

 

Hi,

Thanks for your interest in the project.

CityGen3D textures terrains by texturing different OSM map regions (we call Surfaces) and allowing you to assign one or more textures to these regions.

Some terrain textures are supplied with the asset for demonstration, but you can use any seamless textures you have.

Using the built in filters it’s also possible to texture the terrains using Perlin Noise, assigning different textures to the output within a surface.

So there’s already quite a lot of control available to you to get certain effects, but as with a lot of the more advanced features there’s a lack of demos and tutorials to demonstrate the workflow, which I hope to address as we approach first full release.

But this general approach works very well where you need hi-res texturing, such as in a first person game, and you don’t have some of the issues associated with using satellite imagery, such as unwanted 3D details on the terrain texture (eg. buildings and shadows).

However I do like the idea of integrating support for satellite imagery and using that for the texture map. In some instances that may be a preferable approach (eg. flight sim where you aren’t close to the terrain).

There are also long term plans to allow you to draw your own surfaces in 2D so you could map different parts of your terrain before texturing them appropriately.

 

Some news on development of a run-time scene streaming solution in CityGen3D.

Here you can see the environment being loaded and unloaded in the background while moving around the world.

Almost no setup required because it doesn’t rely on colliders to trigger scene transitions. It’s simply a case of setting your terrain size before downloading OSM data, and setting loading range for run time use.

Original roadmap was to release this much later but I may bring it forward to get some feedback during the beta.

Next stage is to implement origin resetting at runtime to support very large worlds properly.

 

While searching the Internet for ways of generating cities from map data, I found https://github.com/Microsoft/MapsSDK-Unity and it looks interesting.

 

I really need Buildings generated from OSM data with my own prefabs

 

Buildings from OSM data is last major feature planned before full release of CityGen3D, now planned for v0.9.

I swapped it around with scene streaming, which is coming very soon in v0.8, just because that ended up being much quicker to implement than I had imagined.

But the buildings feature is already quite far along too. I will start posting updates on this once v0.8 is out