Question Crash (out of memory) during large raycast

Hello

Long story short, I end up using jobs to raycast about 45 million times and then generate 45 million structs called SourceVertex for grass positions. I have strenuously optimized this so that it runs performantly on my MacBook Air (16gb ram) and only takes about 10 seconds to load. This only happens once at the start of the game. And it is completely optimized during game so that only about 40k points are even rendered/handled at once. The world is divided into cells called Areas and so technically I wouldn’t think the other 44.94 million would be kept in ram???

Anyways, I build it on my iPhone 13 mini and as soon as the raycast task starts, the phone crashes with the warning:

Thread 1: EXC_RESOURCE (RESOURCE_TYPE_MEMORY: high watermark memory limit exceeded) (limit=2098 MB)

So I split this large task up into an IEnumerator and it runs for about 1.5 min without crashing then suddenly crashes. This proves that it’s not the large amount of raycasts or the large jobs but that after enough SourceVertex structs have been created, somehow memory runs out. This confused me cause I would think once the array of structs is created it should be removed from memory and so there should only be one in memory at a time. And even if all 45 million structs are in memory that still is only 1.3 gb which even with struggle the phone should be able to handle right???

Please help! Any suggestions or ideas would be super helpful! Thanks!

 

Memory isn't "removed" like that: it simply becomes no longer used (and this must mean that NOTHING and NOBODY retains that reference!) and thus becomes eligible for garbage collection and return to the pool at some indeterminate point in the future.

If it's one giant array, it's either all there or all gone.

Unity should be responding to low-memory alerts from the OS and it should use that to unload unused stuff and GC.

So find out if it is getting returned... you should be able to easily see that large a chunk profiling on the MacBook itself, and verify that you're not growing to something massive-sized.

 

Thank you for your reply! I am definitely not claiming to know everything about memory lol. I am acty not very versed with the understanding of how memory works.

The 45 million is split into cells of 45k by objects with an array of 45k SourceVertex structs. So there should never be one array larger than 45k.

I have tried using the Memory profile and I can't really figure out what it is telling me. Here are some screenshots of memory: The first image shows the ComputeDetails.SourceVertex[] as 1.19gb so I don't know if that is a problem or what lol? And what is the 14.93 gb of Reserved Native Memory? I am pretty new to the Jobs/Burst/Memory stuff so forgive me if I am being naive.

 

It looks like I hit this error in Xcode

UnityGfxDeviceWorker (39): EXC_RESOURCE (RESOURCE_TYPE_MEMORY: high watermark memory limit exceeded) (limit=2098 MB)

as soon as the ComputeDetails.SourceVertex arrays (all added together) take up about 0.65gb of memory.

 

As long as each of those cells is kept track of by your array of them, none of them are eligible for deallocation / cleanup.

 

Ok, that does make sense. However, even if that's slow and takes up about 1.3 GB of memory, shouldn't my iPhone slow down, not crash? And my iPhone has 4 GB of ram so shouldn't that be more than enough?

So, generally, I have to find some way to store the SourceVertex arrays in hard memory, then load them when I need them so that only a few are in present memory at a time? Thanks for the help

 

Once stuff is in memory, it takes no more CPU to keep it there.

Your program is technically not crashing. Your program is receiving low memory warnings from the OS and then not doing "enough" about the warnings to keep the OS happy, and finally the OS is either SIG-KILL or SIG-TERM-ing your process in order to save the rest of the computer from your rampant memory usage.

It's not all for you to use... the OS hogs a lot of it for its own business of operating the system... hence "OS."

There are also other services and programs that might need RAM and the OS has to keep those in mind as well: inbound network traffic, phone calls, alerts, anything else the OS could bring up while your app is running.

Or recomputing them. Or computing them only as you need them. Or computing them as part of your build process and storing the arrays as game assets and loading only the ones you need while releasing references to the ones you don't need anymore, or not even loading them in the first place.

Also... shouldn't grass just be built into your app's level data in some way??

As in... "why exactly are you doing all this raycast-y work at runtime??"

 

Haha 'tis the question I have been asking myself. I am using a Compute Shader to generate blades of grass at a RWStructuredBuffer of the ComputeDetails.SourceVertex struct. So somehow I need to generate the positions where I want them. At first, I tried using a texture to hold the data and it worked well for a smaller world. But after making a much larger world and dividing the world into cell areas, the texture idea quickly became obsolete for our needs. Our designers want to be able to use vertex colors (specifically the red channel) to specify where grass should be. I could generate a blade at each vertex and just read the vertex color at that point, but the world mesh cannot contain the large number of vertices needed for enough grass. So somehow I have to get all points on the mesh that have red channel vertex color on them (and the points have to have a specified density). After a ton of thinking lol, I couldn't think of a way to do that without ray casts.

If you can think of ANY better way I am SOOOO all ears lol. Like I just can't figure out how Zelda BOTW or other games with lots of grass, handle so many points.

 

MY SOLUTION

In case anyone is having a similar problem and can't figure it out, here is how I fixed it:

I created a ScriptableObject called WorldDetailsCollectionPreset that contains a (Serializable) class called WorldDetailsCommissioner which contains a List of (Serializable) data. I also have a MonoBehavior called ComputeDetails which holds a reference to a WorldDetailsCommissioner and, if its reference isn't null, it renders all the data of the given WorldDetailsCommissioner.

Each ComputeDetails in the world calculates LOD and if the data should be rendered then it executes an Async Load to grab the WorldDetailsCollectionPreset from hard memory. If no ScriptableObject is found, then it creates its own WorldDetailsCommissioner and generates it with new according data (my data were points generated by casting rays–but you could do this however you want). If the ComputeDetails gets out of LOD range then it drops its reference (sets to null) to the WorldDetailsCommissioner; therefore, removing it from memory and not rendering the data any longer. Since I will make sure to have all ScriptableObjects created previous to building the game, the data generation (i.e. ray casting) will NEVER be done during runtime and will have already been baked into WorldDetailsCollectionPreset objects beforehand.

I have an editor script which, on MenuItem executed, bakes all live in-game instances of WorldDetailsCommissioner into WorldDetailsCollectionPreset ScriptableObjects. So I first run the game (which generates the data), then bake them, and then if I run it again, it will just load the baked objects instead of generating the points again.

Please, let me know if there are any flaws in my solution because I am by no means perfect at this kind of stuff and I don't want to lead people down a wrong path.