Search Unity

CRC stability

Discussion in 'Asset Bundles' started by redeyerulk88, Nov 13, 2017.

  1. redeyerulk88

    redeyerulk88

    Joined:
    Dec 20, 2013
    Posts:
    4
    Hi, I have an issue with crc stability. Basically asset hashes and tree hashes are the same each build but crc-s are different each time we build. Is this by design? CRC.png
    This really complicates life a bit for us, as we need to have some kind of database for distributed bundles
     
    redeyerulk88, Nov 13, 2017
    #1

  2. Ryanc_unity

    Ryanc_unity

    Unity Technologies

    Joined:
    Jul 22, 2015
    Posts:
    332
    Hey @redeyerulk88
    CRC generally is (currently) the most reliable / stable method for determining if a bundle changed. We are working on improving this. Have you run a binary compare on the two bundles to see if they are identical or different?
     
    Ryanc_unity, Nov 13, 2017
    #2

  3. redeyerulk88

    redeyerulk88

    Joined:
    Dec 20, 2013
    Posts:
    4
    They are different.
    Can you elaborate on this, please.
    We are currently building our distribution system assuming that the asset hash (AssetBundleManifest.GetAssetBundleHash() ) can be used to determine weather or not the bundle's content has changed. And I assumed that crc is just to check weather or not there were downloading errors.
    So to prevent users from downloading asset bundles after each release/patch we are re-deploying only bundles with changed hashes while preserving bundles from previous releases if the hash is the same in new release. Will this eventually cause problems if we are considering only asset hashes?

    Thanks for quick reply :)
     
    redeyerulk88, Nov 14, 2017
    #3

  4. Ryanc_unity

    Ryanc_unity

    Unity Technologies

    Joined:
    Jul 22, 2015
    Posts:
    332
    Sure.
    So first, the CRC is generated entirely based on the exact bundle that was written to disk. So if the bundle changes slightly, so does the CRC. Granted the CRC is only 32 bits, so there could be some cases of collision compared to a 128 bit value like the hash. Now there is a catch, CRC is based on post compression bundle state, so if 2 bundle contents are the exact same, but one is uncompressed while the other is compressed, you will get 2 different CRCs.

    Now the GetAssetBundleHash function returns the hash that was used during the incremental build step of the current build pipeline. The problem is that this process tries to calculate what it thinks the resulting bundle will be. It does a fairly decent job at it, however over the years Unity's assets have gotten more and more complex, and there are definite areas that this process just can't handle accurately. (I blame shaders =P, pls don't kill me graphics team....)

    So that's why CRC tends to be more accurate of an identifier to know if a bundle changed or not.

    We've talked about our rewrite of the build pipeline quite a bit at various Unites and on the forums. But to go into a bit more detail on what we are doing to resolve this issue in the new pipeline is that we are moving to a multi-pass pipeline. Where the first step is to calculate all the dependencies accurately, then from there we can use that data, plus the data of when the asset or dependent assets last changed to know exactly what needs to be rebuilt and why. Also instead of generating the hash up front, we use the written out bundle to generate the exact version hash and crc. So both will be in sync and accurate.
     
    Ryanc_unity, Nov 16, 2017
    #4
    Energy0124 likes this.