Search Unity

Bug Burst AVX2 Crash using Burst 1.8.2, 1.8.3

Discussion in 'Burst' started by ed-fj, Mar 20, 2023.

  1. ed-fj

    ed-fj

    Joined:
    Mar 22, 2013
    Posts:
    13
    We have been getting a crash on builds made using 2022.2.2 and Burst 1.8.2 where in the crash dump it has indicated an AVX2 related error. This was on an i5-2400.

    The build did not crash with AVX2 disabled. Then we got 2022.2.11 and Burst 1.8.3 to see whether there is any difference, and there is, however we get a similar error at a different place.

    The error looks like this:

    ExceptionAddress: 00007ffd5faa9959 (lib_burst_generated!burst.initialize.statics.a6f5259e22ed809ef3937424c4bd686d_avx2+0x0000000000004aa9)
    ExceptionCode: c000001d (Illegal instruction)
    ExceptionFlags: 00000000
    NumberParameters: 0
     
    ed-fj, Mar 20, 2023
    #1

  2. tim_jones

    tim_jones

    Unity Technologies

    Joined:
    May 2, 2019
    Posts:
    287
    Hi @ed-fj - it's expected that a build targeting AVX2 would/could crash on an i5-2400, since that chip doesn't support AVX2. Unless I'm missing something?
     
    tim_jones, Mar 23, 2023
    #2

  3. Grigler_f

    Grigler_f

    Joined:
    May 14, 2020
    Posts:
    24
    I'm a colleague of @ed-fj .

    We've been previously building with the following target architectures on Unity version 2021.3.5f1, Burst version 1.7.4 without issue. This issue has only started happening while attempting to upgrade to the versions mentioned in the OP. While investigating the issue, we found this older bug post which looked similar which is why we believed there was a bug in Burst.

    https://forum.unity.com/threads/burst-crashes-development-build-on-specific-gpu.1364952/

    This part of the documentation does also contradict what you've said, so it would be good to get clarification on this.

    https://docs.unity3d.com/Packages/com.unity.burst@1.8/manual/building-aot-settings.html


    upload_2023-3-23_16-30-48.png
     
    Grigler_f, Mar 23, 2023
    #3

  4. tim_jones

    tim_jones

    Unity Technologies

    Joined:
    May 2, 2019
    Posts:
    287
    @Grigler_f - thanks for the extra details. I didn't know you were doing a build for multiple CPU architectures - I thought maybe you were only targeting AVX2. You're right, when you target multiple CPU architectures like that, it should be dispatching only to a version supported on the current CPU. Perhaps something is going wrong with Burst's CPU detection.

    Are you in a position to submit a bug report via Help > Report a Bug? If so, please let me know the bug number and we'll have a look.
     
    tim_jones, Mar 24, 2023
    #4

  5. tim_jones

    tim_jones

    Unity Technologies

    Joined:
    May 2, 2019
    Posts:
    287
    Another thought - we did fix a possibly-related bug in Burst 1.8.3, so please could you try upgrading to that and seeing if it still happens?
     
    tim_jones, Mar 24, 2023
    #5

  6. ed-fj

    ed-fj

    Joined:
    Mar 22, 2013
    Posts:
    13
    Thanks for your responses, Tim. It's true that we began with "Everything" enabled, and later experimented by turning individual options off.

    We've tried both Burst 1.8.2 and 1.8.3 (and both experienced the issue). So now I'm in the process of submitting a bug report for you. I'll let you know when that's done.
     
    ed-fj, Mar 27, 2023
    #6

  7. ed-fj

    ed-fj

    Joined:
    Mar 22, 2013
    Posts:
    13
    I have created a bug report:

    IN-36563

    I did not include our entire project because it was failling for unclear reasons, but it was also pretty large. Anyway if that's needed maybe I can upload that separately later.

    I included the exe and instructions so maybe we can start there anyway.
     
    ed-fj, Mar 27, 2023
    #7

  8. OndrejP

    OndrejP

    Joined:
    Jul 19, 2017
    Posts:
    304
    OndrejP, Mar 31, 2023
    #8

  9. tim_jones

    tim_jones

    Unity Technologies

    Joined:
    May 2, 2019
    Posts:
    287
    Thanks @ed-fj - the bug report has made its way to us, and we'll take a look.
     
    tim_jones, Apr 4, 2023
    #9

  10. OndrejP

    OndrejP

    Joined:
    Jul 19, 2017
    Posts:
    304
    OndrejP, Apr 5, 2023
    #10

  11. Lee_Hammerton

    Lee_Hammerton

    Unity Technologies

    Joined:
    Jul 26, 2018
    Posts:
    118
    @OndrejP Thankyou, from the crash dump and the lib_burst_generated and the debug build, I've been able to determine the cause of the issue (it is indeed a bug in burst). It will require a new burst package (I`ll work on the fix shortly, but I cannot give an ETA as yet).

    @ed-fj I'm trying to confirm if your bug is related (I suspect it is), do you happen to have a crash dump from one of the times it failed? (And a matching lib_burst_generated in case its different to the one provided in the bug). At present I don't have access to a machine that doesn't support AVX, so i can't simply reproduce and get my own crash dump. The stack you pasted in the first message, doesn't seem to resolve to an avx instruction in the lib burst generated provided in the bug report - i had to assume the base address, since i don't have a dump.
     
    Lee_Hammerton, Apr 5, 2023
    #11
    tim_jones and OndrejP like this.

  12. Lee_Hammerton

    Lee_Hammerton

    Unity Technologies

    Joined:
    Jul 26, 2018
    Posts:
    118
    Just to update, thanks to @ed-fj sending me the crash dump, I've been able to determine that indeed the OPs issue is caused by the same bug.

    I`m working on this now, with the intention to get the fix into 1.8.5, the only workaround for now, would be to disable all but the SSE2 code gen in the Burst.Aot.Settings.
     
    Lee_Hammerton, Apr 12, 2023
    #12
    forestrf, tim_jones, OndrejP and 1 other person like this.

  13. forestrf

    forestrf

    Joined:
    Aug 28, 2010
    Posts:
    231
    I also encountered this problem. Using only SSE2 does indeed fix the problem, so I will use it for now until the next patch. Thank you for the workaround!

    Edit: I'm on Unity 2021, currently on July 2023 there are no updates yet, but Burst appears updated in Unity 2022 to version 1.8.7. Manually updating the manifest.json file to point to that version, the problem seems to have been solved.
     
    Last edited: Jul 10, 2023
    forestrf, Apr 13, 2023
    #13
    Lee_Hammerton likes this.