iPhone 8 / A11 / GPU Hang Error / Command Buffer

Hello,

currently our game isn't working on the iPhone 8. After a short time playing the game (few seconds to a minute), the game freezes and the device log shows this error messages:

Execution of the command buffer was aborted due to an error during execution. Caused GPU Hang Error (IOAF code 3)​

Execution of the command buffer was aborted due to an error during execution. Ignored (for causing prior/excessive GPU errors) (IOAF code 4)​

The last message (with IOAF code 4) is then produces infinitely.

As we use command buffer to draw some meshes, the first step for me was to totally disable all command buffers. But even with all our command buffers disabled, the game freezes with the same error messages. I double checked it by displaying the commandBufferCount of the camera which was zero all the time while testing.

Our game is currently in the app store using Unity 5.5.4p3 and has this error. But we are internaly updated to Unity 2017.2.0f3 and also 2017.3.0b4, but both produce the same error.

We couldn't reproduce this problem on iOS Devices with older GPUs. So e.g. the iPhone 7 is working without any problems.

Do you have any suggestions what I could test for further investigation?

 

Thank you for your answer!

If I enable the Metal API Validation, I get a signal SIGABRT with the error message while starting the game (occures during SplashScreen):

Thread: UnityGfxDeviceWorker
Method: DrawBufferRanges()
Message: validateFunctionArguments:3379: failed assertion `Fragment Function(xlatMtlMain): The pixel format (MTLPixelFormatR16Unorm) of the texture (name:UnityAttenuation) bound at index 1 is incompatible with the data type (MTLDataTypeHalf) of the texture parameter (_LightTextureB0 [[texture(0)]]). MTLPixelFormatR16Unorm is compatible with the data type(s) (
float
).'

I added a bug report (https://fogbugz.unity3d.com/default.asp?959930_q59tm1l8k3s3tnpo).

In the meantime: Any further things I could test? I also will try to reproduce this in a smaller project, as our game is to big to upload it.

 

I tried sampler2D_float and sampler2D_half for both samplers with the name "_LightTextureB0" in AutoLight.cginc but I'm still getting the "incompatible with the data type" error.

 

Ok, the game it's working now with extended Metal API Validation.
I had to change sampler2D to sampler2D_float for both _LightTextureB0 in the AutoLight.cginc file and also for unity_NHxRoughness in the UnityStandardBRDF.cginc file.

The game is now starting with "GPU Frame Capture" set to "Metal" and "Metal API Validation" set to "Extended". I'm sure it is enabled because it logs:

Metal GPU Frame Capture Enabled
Metal API Extended Validation Enabled

I played the game until it freezes again. But there are not additional logs to the "GPU hang errors" mentioned above.
Do I have to look anywhere else? Do you have any other suggestions?

I can't find out what exactly is causing the problem. So currently I can't reproduce the problem in a smaller repro.
I made a build with a test scene where I added cubes with all of our materials to find out, if this is a problem of a specific shader. But the game didn't freeze.

Any other ideas?

Thank you for your help!

 

While driving around in the game I got another API Validation Error:

-[MTLRenderPipelineDescriptorInternal validateWithDevice:]:2150: failed assertion `vertexFunction is associated with a different device'

Spoiler: Xcode Debug View

 

Any further ideas what I could test?

 

Not to derail this thread. But I had/have a similar issue with my app as well. Only occurs on iPhone 5s.
Impossible to find any information about this error.
I grabbed the logs from the settings: Privacy -> Analytics -> Analytics Data

https://forum.unity.com/threads/hang-during-play-gpuevent-firmware-detected-lockup.489725/

 

Just as info for other developers:

Together with the Unity Support we found out, this is really a driver problem of the devices.
The way matrices in shaders are handled have changed on newer apple devices.

The fix for us was to make sure, indices to handle matrix "cells" are valid, so a simple clamp between max and min possible values fix this problem.

Our small test project will be handed over to Apple for deeper investigation of the problem.

 

Hi Thomas,
I have a similar issue & would like to clamp my matrix indices.
Where did you do this? Was it in a shader? If so which shader?

I'd like to avoid inducing this issue if possible.

Thanks

 

We have very special self written shaders which are using a texture array for lightmapping. The matrix defines the index of the lightmap in the texture array.
The index is calculated in the shader based on the world position. We just make sure the calculated indices are not out of bounds of the matrix:

Code (CSharp):

1. // iPhone 8 fix
2. gridOffsetX = max(0 , min(3, gridOffsetX));
3. gridOffsetZ = max(0 , min(3, gridOffsetZ));

where gridOffsetX and gridOffsetZ are the calculated indices.

 

Hi, I am using the metal framwork on iOS as well. I am capturing images and zipping them and saving them to the phone's document directory and and after about 300 MB are saved (using iOS 12), I get the error "Execution of the command buffer was aborted/GPU Error". This is probably stemming from the same issue you had. What do you think? Is there a work around?

 

Same issue on latest tvOS when anti aliasing is enabled.
Work around: Disable AA in quality settings

 

Hello,
we are facing the same issue with Unity 2018.2 and ios 12.1.1. Disabling Metal in Graphics API seems to fix the issue.
Is there any other way to fix this? We are using cloud build, is it possible to enable extended Metal API Validation in cloud build?

 

@martonekler I'm getting a similar crash, but we're not using Metal in Unity 2019.2.5. Is there a reason this would happen?

 

IOS：
I'm getting a similar crash, using Metal api in Unity 2018.2.21f1. Is there a reason this would happen?
shader no shadow ，no commanbuffer script no fog ，prefab no cast shaddows no receive shadows
Execution of the command buffer was aborted due to an error during execution. Caused GPU Hang Error (IOAF code 3)
Execution of the command buffer was aborted due to an error during execution. Ignored (for causing prior/excessive GPU errors) (IOAF code 4)

 

Execution of the command buffer was aborted due to an error during execution. Ignored (for causing prior/excessive GPU errors) (IOAF code 4)
Every frame appears ，Is there a reason this would happen?