Bug Scriptable Build Pipeline 1.21.3 Bug

We found a bug in SBP introduced probably in 1.21.3, I cannot put it through Unity QA as I have no repro, just where it is in the code and how to fix it (we've patched it locally). @davidla_unity @andymilsom

CalculateSceneDependencyData::Run() added

List<CachedInfo> info

and adds cached entries to it. If it later determines that the cache is invalid and sets useUncachedScene to false, it does not remove the cached entry from Info. Later in the code the uncached entry is added to the same list, so when later in the code all the CacheInfo entries are added to the m_Results.AssetResults dictionary, it causes a duplicate key exception at CalculateSceneDependencyData.cs:225.

Our fix was to add

info.Remove(cachedInfo[i]);

at CalculateSceneDependencyData.cs:150.

 

This bug was introduced in 1.20.1

 

Hm, the exact code path that caused it for us was introduced after 1.20.1, I've just diffed them, I suspect that they fixed a bug and introduced another one in the process.

 

Just got this on Addressables 1.21.12 and SBP 1.21.5.

Build Task CalculateSceneDependencyData failed with exception:

An item with the same key has already been added. Key: 841703f74e2e442ddbceae1dab5de7a8

Do we need to submit a bug report?
I can reproduce by running all rules in the analyzer tool 2 times.

solution is to clear the build cache in-between analyzer runs.

 

Yes, please. I have locally fixed this as described in the initial post and we haven't had any failures since but didn't have a repro.

Edit: Mistakenly replied that I have submitted this already, I have not, I reported a few bugs in Addressables that I have also fixed locally lately, and getting them mixed up...

 

Unfortunately I can't reproduce in an empty project. Can't submit a bug report on my working project because reasons...

Will keep trying though

 

I had similar issues with CalculateSceneDependencyData adding null types after upgrading our project to 2022.3. In my case I was able to find an empty game object without a transform and a null component in a scene via code. After deleting it building works.
In general CalculateSceneDependencyData seems to be much more error prone in the latest version and also doesn't log any information which scene is causing the builds to fail.

 

@davidla_unity @andymilsom

 

Yeah I feel like there's some hidden issues here. I still cannot reproduce in an empty project.
We upgraded from 1.20.3 and now analysis is always failing.

 

Is your stack trace the same as mine, i.e. ArgumentException at CalculateSceneDependencyData.cs:225, or is it something else? I can have a look to get you a workaround if it’s something different.

 

Yes that was the issue we were seeing.

However, after updating to 2022.3.1, the error no longer occurs

 

Considering the root cause, I’m quite surprised at that. We’ve had no problems with this since I fixed it in the package, I wonder if that version ships with a newer version of SBP.

 

Yeah I could be wrong. Lots of changes have happened in our project since this issue was discovered.
So it might have been solved by fixing issues in scenes and/or prefabs.

 

Nope, this is still a thing. Only seems to happen when running 2+ analyze rules. I still think there’s a hidden build issue at play here.

@davidla_unity i don’t suppose you are aware of these issues?

how can I diagnose this?

@AlkisFortuneFish method does fix, but I really want to avoid having to embed the SBP package, and I feel it’s just masking a deeper issue

 

I'd recommend you embed it, and addressables. None of these packages are robust enough not to need manual fixes constantly. The fix isn't masking a deeper issue, the bug is indeed in that code, it's not a workaround. They mark a cache entry as invalid but don't remove it from the cached entry list, resulting in it ending up in both the cached and uncached entry lists, both of which get added to a dictionary with clashing keys. The fix is just to remove the entry from the cached entry list when marking it the entry as invalid.

Lower level stuff in the engine will change how often the cache is invalidated, which would explain why you get it less with a newer version of Unity, but the actual issue here is literally an elementary code bug.

 

Thanks for the opinon. In case who also want to embed the addressable package, one can follow the instruction here
https://docs.unity3d.com/Manual/upm-embed.html

 

Thanks for posting this AlkaFortuneFish. I just wanted to support that we've also hit this bug in our project (SBP 1.21.5), have reviewed the logic of the CalculateSceneDependencyData and it's definitely just a logic issue that should be simple to fix. Hopefully the Scriptable Build Pipeline team can take care of this small task. We try to avoid embedding packages as much as possible (ease of updating and all), and it's frustrating to need to embed a package for such a simple bug fix...

 

Same issue on my project, using unity 2022.3.8 and SBP package v1.21.20

Apparently it's always the same scene that is causing the duplicate key exception.
What is weird is that the bug seems to be non-deterministic: I can build addressables multiple times in a row without changing anything in the project, and occasionally it will fail due to this.

I managed to track it down to the fact that for some reason

ContentBuildInterface.CalculatePlayerDependenciesForScene

on line 163 of CalculateSceneDependencyData occasionally returns the

sceneInfoNew

where

referencedObjects

is different from the cached

referencedObjects

, even if nothing was changed between runs. I wonder if this points to a deeper underlying issue.
This is what causes the code to decide that the cached entry is no longer valid, and add the new cache entry without clearing the old one, leading to the duplicate key exception later on.

In any case I managed to "fix" this without embedding the package by purging the SBP cache before I run an addressables build.

 

Can you share the details how you fixed the issue - i.e. how to clear the SBP cache?

 

I implemented a custom build script that extends the default one and clears the cache before building:

public class AddressablesBuildScript : BuildScriptPackedMode

{

    protected override TResult BuildDataImplementation<TResult>(AddressablesDataBuilderInput builderInput) {

        Debug.LogWarning("Purging SBP build cache");

        BuildCache.PurgeCache(false);

        return base.BuildDataImplementation<TResult>(builderInput);

    }

}

You must then create an instance of this scriptable object and assign it in the addressables settings "Build and Play mode scripts", replacing the default build script.