The assetbundle build is different each time if I remove the Library folder

I am building the assetbundles to support the "hot" update the game asset. So I want to keep my asset bundle binarary consistency so thay the user will not download the whole assets.
But if I delete my Library folder without any change to my Assets folder, the build result is not the same. I have to commit the Library folder to the version control to avoid this, it really mess up the project.
I known there have a lot of discussion about this issue, but I never find a right way.
Is there any way to reslove this, or I am missing something.

 

I am using Unity 5.6 and the svn as version control. I submit my assetbundle to the svn, but each time I delete Library folder and rebuild assetbundle, I found a lot of bundle is changed.
I write a script to deal with the import setting for each folder, make sure the import setting in the right way. Different setting for different folder.

 

We also have same issue with GIT and any version of Unity 5.2+. I'm sure generated asset bundles are same, because they have same CRC but different hash in their manifest. We reduce cases with defining all unity files as "-text" in gitattributes, so git not changes the line endings. But as far as I know it's impossible to preserve actual time stamps in GIT. It's actually quite annoying because we almost never achieve to NOT generate asset bundles. There is always one developer/artist who imported the assets which differs from rest of team

Edit: The problem may not be caused from different timestamps. What we observes in our team is that we usually ends up with same binary and meta file, however the importer generates one hash, and rest of team generated another hash for same asset bundle.

 

This can also happen if even a single meta file is missing in the directory chain. IE: If you have assets/foo/bar/myfolder/mything, and the meta file for foo is not checked in, it will get generated on each machine with a different GUID and cause mything.bundle to get a different hash, even though it isn't actually different.

 

I think the build system for assetbundle should find a way to get rid of the line-end or timestamp. If the two file is the same, the assetbundle must the same.
Use other tool or script to make sure the file line-end or timestamp is so easy to broken.

 

I do some playground, I think timestamps are irrelevant. My steps are in below. Is there any possible cause you can suggest?

Unity Version 2017.2f3 - Windows
[Master] create a repo
[Master] create unity project (switch Android)
[Master] use gitignore and gitattribute files (attached to post)
[Master] initial commit
[Master] import 2 sprites assign a bundle name (sprites.bundle)
[Master] commit
[Master] update timestamp (access, and modified. attached to post)
[Master] build bundle
[Master] commit
[Master] update timestamp
[Master] build bundle (no diff)
[Master] push
[Slave-1] clone repo (open from command line with -buildTarget android)
[Slave-1] update timestamp
[Slave-1] check timestamps (identical with master)
[Slave-1] build bundles (sprites.bundle is same, Bundles and Bundles.manifest generated with CRC_NEW old CRC_OLD)
[Slave-1] commit diff
[Slave-1] build bundles (no diff)
[Master] pull
[Master] build bundles (sprites.bundle is same, Bundles and Bundles.manifest generated again with CRC_OLD)
[Slave-2] clone repo (do not sync timestamps, open from UI, switch to android from Build Settings)
[Slave-2] build bundles (sprites.bundle is same, Bundles and Bundles.manifest generated with CRC_NEW, identical with Slave-1)
[Slave-2] commit diff
[Slave-2] build bundles (no diff)
[Slave-1] pull
[Slave-1] build bundles (no diff)
[Master] pull
[Master] build bundles (sprites.bundle is same, Bundles and Bundles.manifest generated again with CRC_OLD)

In my previous experiences the problem will grow in time. For example if project 100 bundles. At first only the "Bundles" is different and then rest of the bundles become different machine to machine in time.

 

+ Ok, based on what you are saying, I take it you are building your asset bundles to a folder called "Bundles" thus you end up with a Bundles and Bundles.manifest file correct?
- Yes, It's located Assets/Bundles. My build code, activeBuildTarget is "android"

Code (CSharp):

1. BuildPipeline.BuildAssetBundles("Assets/Bundles", BuildAssetBundleOptions.None,
2.             EditorUserBuildSettings.activeBuildTarget);

+ Also just to understand, sprite.bundle is the same in all your test cases, this includes the hash and crc contained in the sprite.bundle.manifest file, but Bundles & Bundles.manifest is not the same, correct?
- Yes. sprite.bundle and sprite.bundle.manifest is same for all git clones

+ Can you send me a direct msg with the bundles built from Master & Slave-1 so I can look at the internals?
- Sent you DM

 

I also encountered the same problem, I use [Unity Assets Bundle Extractor] tool found that when packing material in prefab to AssetsBundle, leading to the two pack AssetsBundle is different, but the two AB files, unity hash value is the same, think the file has not changed, but the actual file MD5 has been different.(unity5.6.1p4)

ab file1:

Code (CSharp):

1. 0 Material Base
2. 1 string m_Name = "mat_128"
3. 0 PPtr<Shader> m_Shader
4.   0 int m_FileID = 5
5.   0 SInt64 m_PathID = 1902212346509020161
6. 1 string m_ShaderKeywords = ""
7. 0 unsigned int m_LightmapFlags = 5
8. 1 bool m_EnableInstancingVariants = false
9. 0 int m_CustomRenderQueue = -1
10. 0 map stringTagMap
11.   0 Array Array (0 items)
12.    0 int size = 0
13. 0 vector disabledShaderPasses
14.   0 Array Array (0 items)
15.    0 int size = 0
16. 0 UnityPropertySheet m_SavedProperties
17.   0 map m_TexEnvs
18.    0 Array Array (1 items)
19.     0 int size = 1
20.     [0]
21.      0 pair data
22.       1 string first = "_MainTex"
23.       0 UnityTexEnv second
24.        0 PPtr<Texture> m_Texture
25.         0 int m_FileID = 0
26.         0 SInt64 m_PathID = -325076255573081539
27.        0 Vector2f m_Scale
28.         0 float x = 1.000000
29.         0 float y = 1.000000
30.        0 Vector2f m_Offset
31.         0 float x = 0.000000
32.         0 float y = 0.000000

ab file2:

Code (CSharp):

1. 0 Material Base
2. 1 string m_Name = "mat_128"
3. 0 PPtr<Shader> m_Shader
4.   0 int m_FileID = 5
5.   0 SInt64 m_PathID = 1902212346509020161
6. 1 string m_ShaderKeywords = ""
7. 0 unsigned int m_LightmapFlags = 5
8. 1 bool m_EnableInstancingVariants = false
9. 0 int m_CustomRenderQueue = -1
10. 0 map stringTagMap
11.   0 Array Array (0 items)
12.    0 int size = 0
13. 0 vector disabledShaderPasses
14.   0 Array Array (0 items)
15.    0 int size = 0
16. 0 UnityPropertySheet m_SavedProperties
17.   0 map m_TexEnvs
18.    0 Array Array (3 items)
19.     0 int size = 3
20.     [0]
21.      0 pair data
22.       1 string first = "_MainTex"
23.       0 UnityTexEnv second
24.        0 PPtr<Texture> m_Texture
25.         0 int m_FileID = 0
26.         0 SInt64 m_PathID = -325076255573081539
27.        0 Vector2f m_Scale
28.         0 float x = 1.000000
29.         0 float y = 1.000000
30.        0 Vector2f m_Offset
31.         0 float x = 0.000000
32.         0 float y = 0.000000
33.     [1]
34.      0 pair data
35.       1 string first = "_MaskTex"
36.       0 UnityTexEnv second
37.        0 PPtr<Texture> m_Texture
38.         0 int m_FileID = 0
39.         0 SInt64 m_PathID = 0
40.        0 Vector2f m_Scale
41.         0 float x = 1.000000
42.         0 float y = 1.000000
43.        0 Vector2f m_Offset
44.         0 float x = 0.000000
45.         0 float y = 0.000000
46.     [2]
47.      0 pair data
48.       1 string first = "_NoiseTex"
49.       0 UnityTexEnv second
50.        0 PPtr<Texture> m_Texture
51.         0 int m_FileID = 0
52.         0 SInt64 m_PathID = 0
53.        0 Vector2f m_Scale
54.         0 float x = 1.000000
55.         0 float y = 1.000000
56.        0 Vector2f m_Offset
57.         0 float x = 0.000000
58.         0 float y = 0.000000

_MaskTex, _NoiseTex is what I later modified shader to join

 

@Ryanc_unity @onur-gg did you resolve this issue? We are seeing a similar issue. We are using PlasticSCM with a local server setup and have been able to replicate the issue with both Unity 5.6 and 2017.4.

The issue is that some of the asset ID's has changed and that it orders dependencies differently:

m_Dependencies (vector)
size 3 (int)
data "shaders" (string)
data "ui" (string)
data "vfx" (string)
vs

m_Dependencies (vector)
size 3 (int)
data "ui" (string)
data "shaders" (string)
data "vfx" (string)

 

bump, Because of this problem, you can not configure a new build machine.
Please let me know how this problem is solved.

 

different hash different machine, this is not what we had expected,do you find solutions??

 

bump, Using xcopy can avoid problems, but it can not be an alternative. The advantage of using SCM is eliminated.
This will result in a massive download when the live service.

 

Sorry to necro an old thread, but it looks like other people are still wondering about this issue.

I can confirm that this still happens in 2018.2.x.

This is a serious problem, given the same asset folder, asset bundle creation should produce the exact same output, regardless of the Library folder (i.e. computer, workspace, branch, stream, etc.).

 

Could you please submit a bug-report, including a project to reproduce the issue, as described in this document:
https://unity3d.com/unity/qa/bug-reporting

Submitting a bug-report is the official way to tell Unity Technologies there is a problem. The forum is not really a legit place to report bugs.

After you submitted the bug-report, you receive a confirmation email with a bug-report Case number. You can post this Case number (the number only, not the link) in this forum thread for Unity staff to pick up, in case they see it.

 

@AmaritundeOkafor how about this issue, is it reloved?

 

It's still there and it's very annoying.

 

Hi all.
Here is QA answer for one of the nasty and annoying bug with asset bundles. https://fogbugz.unity3d.com/default.asp?1183713_om4th52sputku6mh
Is there any Unity developer here to confirm that this bug is not an expected behavior?
Here is link to test project: https://www.dropbox.com/s/pasl9ljncg68otq/BundlesTest.zip?dl=1

 

I test Unity2018.4.11f1, it seems this problem was fixed. Can anyone confirm that again?

 

My solution is caching the Library folder by using git.

 

our problrv now on 2020.3.18 - different bundle size every time on same machine, with no changes at project...

 

Hi,I get differens,some one can help me?
Int unity 2021-3-7f1,use UnityCacheServer(UnityAccelerator) manager Library for Android.
But i insert 2 pictures in Assets,clear Library, Open Unity，Import Assets do downLoad from UnityAccelerator, finish import,then i build assetBundle . I find the differents 1605 numbers:
1,265 prefab
337 mat
1 shadervariants
1 fbx
bundle totol number is 5524

Animator(be binded to prefab) has different.