Search Unity

AssetBundles and dependencies

Discussion in 'Editor & General Support' started by jonlab, Jan 5, 2016.

  1. jonlab

    jonlab

    Joined:
    Apr 6, 2010
    Posts:
    182
    Hi there,

    We work on a shared world project named "New Atlantis". This runs on PC/Mac standalone.

    We want the users to be able to create content in Unity 5 and upload this content to shared spaces. So everybody is working in his own Unity project with only a few shared assets (mainly the system management scripts).

    We have been using the AssetBundles as the way for users to export their objects and put them in spaces so that anyone running a standard viewer application could see them. The Viewer just load the AssetBundles that should include all the needed dependencies.

    Bundles are exported using the (now deprecated) BuildPipeline.BuildAssetBundle() method with a full dependencies export :
    Code (CSharp):
    1. BuildPipeline.BuildAssetBundle(Selection.activeObject, selection, path, BuildAssetBundleOptions.CollectDependencies | BuildAssetBundleOptions.CompleteAssets);
    This was working pretty fine until Unity 5.1, but since Unity 5.2 and especially Unity 5.3, we have issues with the custom shaders not properly exported (they are not included anymore in the
    Asset Bundles and so appear pink when loaded in the viewer).

    EDIT : actually it seems to be more a loading issue than an exporting issue : the shader seems to be properly exported with the deprecated 4.X method even if the shader is not explicitely showing as an Object when we load it. But not working correctly when loaded....

    Even the standard shaders do not work if not included in the "always included shaders" in the Graphics settings.

    I tried with the newly introcuded AssetBundle export method, but no matter the BuildAssetBundleOptions I give, it exports only the main prefab I declared in this AssetBundle name, and no depencies at all (materials, shaders, ...), so this is useless for us as the main idea is that we need these dependencies to be exported in the AssetBundle (these assets are NOT in the main project).

    Any idea ? Is there a way in Unity 5.3 to properly export an AssetBundle including all the dependencies, especially the shaders ?
     
    Last edited: Jan 8, 2016
  2. Jordi-Bonastre

    Jordi-Bonastre

    Unity Technologies

    Joined:
    Jul 6, 2015
    Posts:
    102
    In Unity 5, CollectDependencies and DeterministicAssetBundle are always enabled. I checked that it's working fine exporting a prefab with a custom material and a custom shader following the documentation:

    In the manifest file, you will only see the prefab, not the material or the shader.

    Did you create a bug report?
     
  3. jonlab

    jonlab

    Joined:
    Apr 6, 2010
    Posts:
    182
    So, if my prefab is referencing a material referencing a custom shader, the material and the shader should be automatically included ?
    The manifest should include all the exported Assets ? (in my manifest I have only the prefab).
    I tried following the documentation (new export method) but this does not work, I have only the prefab in the AssetBundle.
    If I build the asset bundle with the previous method, I have all the objects except the shader.
     
    Last edited: Jan 5, 2016
  4. jonlab

    jonlab

    Joined:
    Apr 6, 2010
    Posts:
    182
  5. Jordi-Bonastre

    Jordi-Bonastre

    Unity Technologies

    Joined:
    Jul 6, 2015
    Posts:
    102
    Until the bug is fixed, you could have a look to this forum thread with a workaround. Are your textures sprites?
     
  6. jonlab

    jonlab

    Joined:
    Apr 6, 2010
    Posts:
    182
    no, this is regular textures or CubeMaps.
    I tried this hack and it works in some cases, but not all. I can't really find the logic that is behind.
    Sometimes the shader name is "Hidden/InternalErrorShader" and so remains pink.
     
  7. jonlab

    jonlab

    Joined:
    Apr 6, 2010
    Posts:
    182
    one more thing, when I load an AssetBundle with a "Hidden/InternalErrorShader" shader pink material, the standalone app crashes (Mac OS X build run on Mac OS X 10.10.5).
     
  8. jonlab

    jonlab

    Joined:
    Apr 6, 2010
    Posts:
    182
    I am still fighting with this. Even the standard shaders when loaded from an AssetBundle appear pink, or opaque (even if set as transparent). Pausing the game and clicking on the material and reaffecting the shader makes it work (and I suppose that this is similar to the "mat.shader = Shader.Find(mat.shader.name)" hack. But doing this for every loaded material makes the build crash...
    What the hell happened with the materials and the AssetBundles in Unity 5.3 ?????
     
  9. jonlab

    jonlab

    Joined:
    Apr 6, 2010
    Posts:
    182
    Is there a way to force a material to 'update itself' (no matter what it means) ?
    (Apart from the mat.shader = Shader.Find(mat.shader.name) trick that crashes.)
    Attached is a small example of a material with the issue : everything seems to be ok, but the material is pink.
    Reselecting the same shader makes it work. Capture d’écran 2016-01-07 à 20.22.19.png
     
  10. diekeure

    diekeure

    Joined:
    Jan 25, 2013
    Posts:
    219
    I had the same issue as you have, but it was resolved in 5.3.1f1.

    There's a catch though, they changed the way shaders are packed in assetbundles: instead of containing shaders that work on every platform, the assetbundles now have shaders that only work for the platform they were build for. This means that the editor is only able to load shaders from the assetbundles if those assetbundles were build for windows standalone or webplayer (or standalone osx if you're on osx). If you have assetbundles build for another platform you won't be able to load shaders from them in the editor.
     
  11. jonlab

    jonlab

    Joined:
    Apr 6, 2010
    Posts:
    182
    I tested the same in beta 5.4.0, and this is still there : AssetBundle is loaded, all the materials are pink, reaffecting the shader in the inspector "fixes" the material.
     
  12. jonlab

    jonlab

    Joined:
    Apr 6, 2010
    Posts:
    182
    No ???? Seriously ??? I can't believe this. Why would they do that ?
    And from what I am seeing, this doesn't seems to be that, as I tested with an AB built for Mac standalone and loaded on a Mac and the issue is there too.
     
  13. jonlab

    jonlab

    Joined:
    Apr 6, 2010
    Posts:
    182
    Maybe I misread what you were explaning alexvda. Yes, I am using only standalone AssetBundles loaded from a standalone (or editor) app. So it should work.
     
  14. diekeure

    diekeure

    Joined:
    Jan 25, 2013
    Posts:
    219
    Yes it should, we have the same situation as you describe, all custom shaders in assetbundles, but it took some tinkering before we got it right.

    How do you build your assetbundles? Do you use custom code for this or do you use the editor for that?

    If custom code, can you provide an example?
     
  15. Immanuel-Scholz

    Immanuel-Scholz

    Joined:
    Jun 8, 2013
    Posts:
    217
    This may sound stupid, but... how do you build asset bundles without custom code, purely "using the editor" ?
     
  16. jonlab

    jonlab

    Joined:
    Apr 6, 2010
    Posts:
    182
    I am using custom code (the 4.X method) as the new 5.X method does not appear to correctly handle dependencies.
    (see at the beginning of this thread)
     
  17. jonlab

    jonlab

    Joined:
    Apr 6, 2010
    Posts:
    182
    the trace of the crash I have when trying to fix the materials with the hack :
    ============================================================
    Crashed Thread: 0 Dispatch queue: com.apple.main-thread

    Exception Type: EXC_BAD_ACCESS (SIGABRT)
    Exception Codes: KERN_INVALID_ADDRESS at 0x0000000000000024

    VM Regions Near 0x24:
    -->
    __TEXT 0000000000001000-00000000012c7000 [ 18.8M] r-x/rwx SM=COW /Users/USER/Desktop/NewAtlantis_v0.88.app/Contents/MacOS/NewAtlantis_v0.88

    Application Specific Information:
    abort() called

    Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
    0 libsystem_kernel.dylib 0x94fc469a __pthread_kill + 10
    1 libsystem_pthread.dylib 0x90ae5634 pthread_kill + 101
    2 libsystem_c.dylib 0x9a95fdee abort + 156
    3 libmono.0.dylib 0x018ec4f7 mono_handle_native_sigsegv + 891
    4 libmono.0.dylib 0x019283e9 sigabrt_signal_handler + 96
    5 libsystem_platform.dylib 0x9961503b _sigtramp + 43
    6 ??? 0xffffffff 0 + 4294967295
    7 libmono.0.dylib 0x01928389 sigusr1_signal_handler + 156
    8 libsystem_c.dylib 0x9a95fdee abort + 156
    9 unity.DefaultCompany.NewAtlantisNewUnity 0x00497bb9 HandleSignal(int, __siginfo*, void*) + 41
    10 libmono.0.dylib 0x01927fda mono_chain_signal + 73
    11 libmono.0.dylib 0x0187055b mono_sigsegv_signal_handler + 231
    12 libsystem_platform.dylib 0x9961503b _sigtramp + 43
    13 ??? 0xffffffff 0 + 4294967295
    14 libmono.0.dylib 0x01870474 mono_sigill_signal_handler + 59
    15 unity.DefaultCompany.NewAtlantisNewUnity 0x001dca16 CalculateSubShaderIndexToUse(Shader*, Shader*, int, int) + 102
    16 unity.DefaultCompany.NewAtlantisNewUnity 0x001ed5f4 PrepareShadowMapsJob(ShadowMapJobHeader*) + 932
    17 unity.DefaultCompany.NewAtlantisNewUnity 0x0028221e JobQueue::Exec(JobInfo*, int, int) + 94
    18 unity.DefaultCompany.NewAtlantisNewUnity 0x002821a6 JobQueue::Steal(JobGroup*, JobInfo*, int, int) + 278
    19 unity.DefaultCompany.NewAtlantisNewUnity 0x00283267 JobQueue::WaitForJobGroup(JobGroupID, bool) + 471
    20 unity.DefaultCompany.NewAtlantisNewUnity 0x00280e23 CompleteFenceInternal(JobFence&) + 51
    21 unity.DefaultCompany.NewAtlantisNewUnity 0x001f0192 RenderShadowMaps(ShadowMapJobHeader const*, ShaderPassContext&) + 354
    22 unity.DefaultCompany.NewAtlantisNewUnity 0x0020f6cc ForwardShaderRenderLoop::RenderLightShadowMaps(ForwardShadowMap&, bool, bool, ShaderPassContext&) + 284
    23 unity.DefaultCompany.NewAtlantisNewUnity 0x0020e669 ForwardShaderRenderLoop::performRendering(JobFence&, ActiveLight const*, ShadowCullData const&, bool, bool, bool, int) + 297
    24 unity.DefaultCompany.NewAtlantisNewUnity 0x00210d97 DoForwardShaderRenderLoop(RenderLoopContext&, dynamic_array<RenderObjectData, 4ul>&, bool, bool, ActiveLights&, bool, int) + 4247
    25 unity.DefaultCompany.NewAtlantisNewUnity 0x0021b146 DoRenderLoop(RenderLoop&, RenderingPath, CullResults&, ShadowMapCache&, bool) + 2198
    26 unity.DefaultCompany.NewAtlantisNewUnity 0x00189575 Camera::DoRender(CullResults&, void (*)(Camera&, RenderLoop&, CullResults&), int) + 309
    27 unity.DefaultCompany.NewAtlantisNewUnity 0x0018bd28 Camera::Render(CullResults&, ShaderPassContext&, CameraRenderingParams const*, int) + 872
    28 unity.DefaultCompany.NewAtlantisNewUnity 0x0018a28f Camera::Render(CullResults&, ShaderPassContext&, int) + 47
    29 unity.DefaultCompany.NewAtlantisNewUnity 0x001db886 RenderManager::RenderCameras(int) + 374
    30 unity.DefaultCompany.NewAtlantisNewUnity 0x00420746 PlayerRender(bool) + 310
    31 unity.DefaultCompany.NewAtlantisNewUnity 0x004216f9 PlayerLoop(bool, bool, IHookEvent*) + 2841
    32 unity.DefaultCompany.NewAtlantisNewUnity 0x00a60861 -[PlayerAppDelegate UpdatePlayer] + 273
    33 com.apple.Foundation 0x9550676f __NSFireTimer + 119
    34 com.apple.CoreFoundation 0x917a7006 __CFRUNLOOP_IS_CALLING_OUT_TO_A_TIMER_CALLBACK_FUNCTION__ + 22
    35 com.apple.CoreFoundation 0x917a6ab4 __CFRunLoopDoTimer + 1316
    36 com.apple.CoreFoundation 0x9182254f __CFRunLoopDoTimers + 351
    37 com.apple.CoreFoundation 0x9175e531 __CFRunLoopRun + 2081
    38 com.apple.CoreFoundation 0x9175daa6 CFRunLoopRunSpecific + 390
    39 com.apple.CoreFoundation 0x9175d90b CFRunLoopRunInMode + 123
    40 com.apple.HIToolbox 0x996458f8 RunCurrentEventLoopInMode + 262
    41 com.apple.HIToolbox 0x99645631 ReceiveNextEventCommon + 494
    42 com.apple.HIToolbox 0x9964542c _BlockUntilNextEventMatchingListInModeWithFilter + 99
    43 com.apple.AppKit 0x99d6db41 _DPSNextEvent + 742
    44 com.apple.AppKit 0x99d6d1e5 -[NSApplication nextEventMatchingMask:untilDate:inMode:dequeue:] + 350
    45 com.apple.AppKit 0x99d61b9c -[NSApplication run] + 907
    46 com.apple.AppKit 0x99cd6fa0 NSApplicationMain + 2082
    47 unity.DefaultCompany.NewAtlantisNewUnity 0x00a5f429 PlayerMain(int, char const**) + 713
    48 unity.DefaultCompany.NewAtlantisNewUnity 0x00002945 start + 53
    ========================================================================
     
  18. jonlab

    jonlab

    Joined:
    Apr 6, 2010
    Posts:
    182
  19. Immanuel-Scholz

    Immanuel-Scholz

    Joined:
    Jun 8, 2013
    Posts:
    217
    ..... which means you need custom scripts, just with a different API, right? (ok, you can setup the content of the bundle in-editor..)

    (Anyway, thanks for the clarification. Its not that its super-hard to call the script, just that I was looking whether I missed something about asset bundles. Am kinda new to this part of unity..)
     
  20. diekeure

    diekeure

    Joined:
    Jan 25, 2013
    Posts:
    219
    That's what I meant yes, did you setup the contents of your assetbundles via code or via the editor, excuse the confusion.
     
  21. diekeure

    diekeure

    Joined:
    Jan 25, 2013
    Posts:
    219
    But I think the 4.x method is completely not useable in 5.3.x, It was deprecated in 5.1 and didn't they remove it in 5.3?
     
  22. jonlab

    jonlab

    Joined:
    Apr 6, 2010
    Posts:
    182
    No, it still works, this is not removed (yet)...
     
  23. jonlab

    jonlab

    Joined:
    Apr 6, 2010
    Posts:
    182
    Ok, I did more tests, and it does not crash anymore when I apply the shader reaffecting hack when I disable the shadows in Quality settings.
     
  24. diekeure

    diekeure

    Joined:
    Jan 25, 2013
    Posts:
    219
    I have a development as well, I noticed that if I build the osx assetbundles on the same machine running the editor, the shaders don't become pink. But if I use our build server to build them then they do become pink... So it seems there is still a pink shader issue for OSX indeed.
     
  25. pahe

    pahe

    Joined:
    May 10, 2011
    Posts:
    424
    @alexvda You saw this? Similar problem like yours, maybe the solution is to build for the right platform as in our case. Check if you build server is building the right platform.
     
  26. diekeure

    diekeure

    Joined:
    Jan 25, 2013
    Posts:
    219
    I didn't, thanks. And at first I thought that building the assetbundles for the correct platform (OSX in this case) would be the solution. It is on windows. Unfortunately we already build platform specific assetbundles for OSX and these do not solve the problem, at least not if they are build from a (windows) build server. If we build them on the same machine that runs it, it looks ok.
     
  27. pahe

    pahe

    Joined:
    May 10, 2011
    Posts:
    424
    Command line parameter also sets correct platform? Not sure if that could be a problem, but I did it just in case: -buildTarget win64
    -buildTarget osxuniversal
     
  28. diekeure

    diekeure

    Joined:
    Jan 25, 2013
    Posts:
    219
    I've found it, via a merge in our source control there was an issue with the data path from which we loaded assetbundles, and apparently I was downloading webplayer assetbundles instead of osx assetbundles after all. It all works now, even when build from a windows build server.
    @pahe Thanks for the help and sorry for the confusion.
     
    pahe likes this.
  29. Bradamante

    Bradamante

    Joined:
    Sep 27, 2012
    Posts:
    300
    Sounds you have encountered the bug I filed here. Support said it might get fixed with Unity 5.4.
    Notice how trivial it is to reproduce. I use the built-in Default-Particle (Particles / Alpha Blended Premultiply) material and the 5.x building process. According to support, it doesn't seem to happen on non-Macs. Personally, I am using Unity on El Capitan. Curiously, only Particle shaders seem to be affected. Legacy/Diffuse or the Standard shader load fine.

    Here's the kicker ... With OpenGL 2.x it works fine. Since you can use either 2 or Core during creation or loading we get this lovely cross table: 1) OpenGL 2 -> Core, 2) OpenGL 2 -> 2, 3) OpenGL Core -> Core and 4) OpenGL Core -> 2.

    In my testing, 3) does not work, the others do. 3) does work if I force the inclusion of the shader in the Graphics settings. Curiously, Unity support never figured this out it seems. What that means for my main game I can't say since I am currently doing major re-writes.

    I have used Unity for years without actual show stopping bugs. However this bug for me proved how insufficient Unity's release testing policies are and how clueless their support is.
     
    Last edited: Feb 3, 2016
unityunity