Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

The BlobAssetReference is not valid (Serializing/Deserializing World)

Discussion in 'Entity Component System' started by merobbins5, Jun 25, 2020.

  1. merobbins5

    merobbins5

    Joined:
    Jun 16, 2020
    Posts:
    7
    I'm trying to save a snapshot of the world's physics so I can revert back to it at a later time deterministically with the entity component system. I have it working where I can save and load one time, but on after the second save when I try to load/deserialize again I get errors saying "The BlobAssetReference is not valid. Likely it has already been unloaded or released.", and "IndexOutOfRangeException: Index 1065297994 is out of range of '2' Length." How replace the original entities without losing their blobassetreferences?

    Any help would be greatly appreciated.

    Code (CSharp):
    1.     public void SerializeWorld()
    2.         {
    3.             EntityManager entityManager = World.DefaultGameObjectInjectionWorld.EntityManager;
    4.    
    5.             using (var writer = new StreamBinaryWriter(_savedWorldFilename))
    6.             {
    7.            SerializeUtility.SerializeWorld(entityManager, writer, out unityMappingTable);
    8.             }
    9.         }
    10.    
    11.         public void DeserializeWorld()
    12.         {
    13.             World.DefaultGameObjectInjectionWorld.EntityManager.DestroyEntity(World.DefaultGameObjectInjectionWorld.EntityManager.UniversalQuery);
    14.    
    15.             if (unityMappingTable != null)
    16.             {
    17.    
    18.                 World localWorld = new World("local world");
    19.    
    20.                 var transaction = localWorld.EntityManager.BeginExclusiveEntityTransaction();
    21.                 using (var reader = new StreamBinaryReader(_savedWorldFilename))
    22.                 {
    23.                SerializeUtility.DeserializeWorld(transaction, reader, unityMappingTable);
    24.                 }
    25.    
    26.                 localWorld.EntityManager.EndExclusiveEntityTransaction();
    27.                 World.DefaultGameObjectInjectionWorld.EntityManager.MoveEntitiesFrom(localWorld.EntityManager);
    28.             }
    29.         }
    30.  
     
    ssellvf likes this.
  2. ssellvf

    ssellvf

    Joined:
    Aug 3, 2018
    Posts:
    18
    Were you able to figure this out? I am having the same issue myself.

    Everything works great serializing/deserializing once. But trying to serialize after a second deserialize causes the error.

    Code (csharp):
    1. InvalidOperationException: The BlobAssetReference is not valid. Likely it has already been unloaded or released.
    2. Unity.Entities.BlobAssetReferenceData.ValidateNonBurst () (at Library/PackageCache/com.unity.entities@0.13.0-preview.24/Unity.Entities/Blobs.cs:253)
    3. Unity.Entities.BlobAssetReferenceData.ValidateNotNull () (at Library/PackageCache/com.unity.entities@0.13.0-preview.24/Unity.Entities/Blobs.cs:270)
    4. Unity.Entities.BlobAssetReference`1[T].get_Value () (at Library/PackageCache/com.unity.entities@0.13.0-preview.24/Unity.Entities/Blobs.cs:358)
    5. Unity.Physics.Broadphase+PrepareStaticBodyDataJob.ExecuteImpl (System.Int32 index, System.Single aabbMargin, Unity.Collections.NativeArray`1[T] rigidBodies, Unity.Collections.NativeArray`1[T] aabbs, Unity.Collections.NativeArray`1[T] points, Unity.Collections.NativeArray`1[T] filtersOut) (at Library/PackageCache/com.unity.physics@0.4.0-preview.5/Unity.Physics/Collision/World/Broadphase.cs:816)
    6. Unity.Physics.Broadphase+PrepareStaticBodyDataJob.Execute (System.Int32 index) (at Library/PackageCache/com.unity.physics@0.4.0-preview.5/Unity.Physics/Collision/World/Broadphase.cs:805)
    7. Unity.Jobs.IJobParallelForDeferExtensions+JobParallelForDeferProducer`1[T].Execute (T& jobData, System.IntPtr additionalPtr, System.IntPtr bufferRangePatchData, Unity.Jobs.LowLevel.Unsafe.JobRanges& ranges, System.Int32 jobIndex) (at Library/PackageCache/com.unity.jobs@0.4.0-preview.18/Unity.Jobs/IJobParallelForDefer.c
    Code (csharp):
    1.  
    2. IndexOutOfRangeException: Index 1052714770 is out of range of '257' Length.
    3. Unity.Collections.NativeArray`1[T].FailOutOfRangeError (System.Int32 index) (at <4cc8ec075538416496e5db5d391208ac>:0)
    4. Unity.Collections.NativeArray`1[T].CheckElementReadAccess (System.Int32 index) (at <4cc8ec075538416496e5db5d391208ac>:0)
    5. Unity.Collections.NativeArray`1[T].get_Item (System.Int32 index) (at <4cc8ec075538416496e5db5d391208ac>:0)
    6. Unity.Physics.BoundingVolumeHierarchy.Refit (Unity.Collections.NativeArray`1[T] aabbs, System.Int32 nodeStartIndex, System.Int32 nodeEndIndex) (at Library/PackageCache/com.unity.physics@0.4.0-preview.5/Unity.Physics/Collision/Geometry/BoundingVolumeHierarchyBuilder.cs:572)
    7. Unity.Physics.BoundingVolumeHierarchy.BuildBranch (Unity.Collections.NativeArray`1[T] points, Unity.Collections.NativeArray`1[T] aabb, Unity.Physics.BoundingVolumeHierarchy+Builder+Range range, System.Int32 firstNodeIndex) (at Library/PackageCache/com.unity.physics@0.4.0-preview.5/Unity.Physics/Collision/Geometry/BoundingVolumeHierarchyBuilder.cs:740)
    8. Unity.Physics.BoundingVolumeHierarchy+BuildBranchesJob.Execute (System.Int32 index) (at Library/PackageCache/com.unity.physics@0.4.0-preview.5/Unity.Physics/Collision/Geometry/BoundingVolumeHierarchyBuilder.cs:813)
    9. Unity.Jobs.IJobParallelForDeferExtensions+JobParallelForDeferProducer`1[T].Execute (T& jobData, System.IntPtr additionalPtr, System.IntPtr bufferRangePatchData, Unity.Jobs.LowLevel.Unsafe.JobRanges& ranges, System.Int32 jobIndex) (at Library/PackageCache/com.unity.jobs@0.4.0-preview.18/Unity.Jobs/IJobParallelForDefer.cs:62)
    Code (csharp):
    1.  
    2. NullReferenceException: Object reference not set to an instance of an object
    3. Unity.Physics.Broadphase+BodyPairWriter.Flush () (at Library/PackageCache/com.unity.physics@0.4.0-preview.5/Unity.Physics/Collision/World/Broadphase.cs:999)
    4. Unity.Physics.Broadphase+BodyPairWriter.Close () (at Library/PackageCache/com.unity.physics@0.4.0-preview.5/Unity.Physics/Collision/World/Broadphase.cs:974)
    5. Unity.Physics.Broadphase+StaticVsDynamicFindOverlappingPairsJob.ExecuteImpl (Unity.Mathematics.int2 pair, Unity.Physics.Broadphase+Tree staticTree, Unity.Physics.Broadphase+Tree dynamicTree, Unity.Collections.NativeStream+Writer& pairWriter) (at Library/PackageCache/com.unity.physics@0.4.0-preview.5/Unity.Physics/Collision/World/Broadphase.cs:1073)
    6. Unity.Physics.Broadphase+StaticVsDynamicFindOverlappingPairsJob.Execute (System.Int32 index) (at Library/PackageCache/com.unity.physics@0.4.0-preview.5/Unity.Physics/Collision/World/Broadphase.cs:1057)
    7. Unity.Jobs.IJobParallelForDeferExtensions+JobParallelForDeferProducer`1[T].Execute (T& jobData, System.IntPtr additionalPtr, System.IntPtr bufferRangePatchData, Unity.Jobs.LowLevel.Unsafe.JobRanges& ranges, System.Int32 jobIndex) (at Library/PackageCache/com.unity.jobs@0.4.0-preview.18/Unity.Jobs/IJobParallelForDefer.cs:62)
    Etc.
     
  3. ssellvf

    ssellvf

    Joined:
    Aug 3, 2018
    Posts:
    18
    So I was able to workaround this issue by re-creating my colliders on deserialization (loading the world).

    Fortunately for me it was pretty simple since I already made use of "Proxy" components for colliders as I define my "prefab" entities in JSON files (project is two years old, so predates the recent improvements regarding object/prefab conversion to entities).

    Code (CSharp):
    1. using (var deserializeWorld = new World("deserialization_world"))
    2. {
    3.     var transaction = deserializeWorld.EntityManager.BeginExclusiveEntityTransaction();
    4.  
    5.     try
    6.     {
    7.         using (var binaryReader = new Unity.Entities.Serialization.StreamBinaryReader(file))
    8.         {
    9.             Unity.Entities.Serialization.SerializeUtility.DeserializeWorld(transaction, binaryReader, unityObjects);
    10.         }
    11.     }
    12.     catch (Exception e)
    13.     {
    14.         Logger.Catch(e);
    15.     }
    16.  
    17.     deserializeWorld.EntityManager.EndExclusiveEntityTransaction();
    18.  
    19.     var removeCollidersQuery = deserializeWorld.EntityManager.CreateEntityQuery(typeof(PhysicsCollider));
    20.     deserializeWorld.EntityManager.RemoveComponent<PhysicsCollider>(removeCollidersQuery);
    21.  
    22.     EntityManager.MoveEntitiesFrom(deserializeWorld.EntityManager);
    23. }
    This removes the troublesome colliders before moving the entities into the main world. Then my systems fire off on my proxy collider components and create new PhysicsColliders for the entities.
     
  4. Scorr

    Scorr

    Joined:
    Jul 2, 2013
    Posts:
    73
    Running into this too, as well as an assertion failure where refcount is -1 in ManagedComponentStore.RemoveReference (on deserialize)
    I'm guessing this happens because of an issue with the BlobAssetOwner component that gets added on deserialization.
     
  5. tertle

    tertle

    Joined:
    Jan 25, 2011
    Posts:
    3,753
  6. s_schoener

    s_schoener

    Unity Technologies

    Joined:
    Nov 4, 2019
    Posts:
    81
    Thank you for reporting this. We're aware of the issue and are working towards a solution. No ETA, unfortunately.
     
    MNNoxMortem likes this.
  7. kanesteven

    kanesteven

    Joined:
    Aug 30, 2018
    Posts:
    45
    @s_schoener This is stalling out efforts to integrate physics with NetCode. I am attempting to construct my own physics/collision worlds inside the prediction systems used for re-simulation during backwards reconciliation on connected clients.

    The code is quite simple and looks like this:


    Code (CSharp):
    1. using Unity.Collections;
    2. using Unity.Entities;
    3. using Unity.Jobs;
    4. using Unity.Physics;
    5. using Unity.NetCode;
    6. using Unity.Transforms;
    7. using UnityEngine;
    8.  
    9. [UpdateInGroup(typeof(GhostPredictionSystemGroup), OrderFirst = true)]
    10. public class PredictionPhysicsWorldSystem : SystemBase {
    11.   public bool IsServer;
    12.   public NativeArray<int> HaveStaticBodiesChanged;
    13.   public EntityQuery PredictionPhysicsEntityQuery;
    14.   public PhysicsWorld PredictionPhysicsWorld;
    15.   public CollisionWorld PredictionCollisionWorld;
    16.  
    17.   protected override void OnCreate() {
    18.     IsServer = World.GetExistingSystem<ServerSimulationSystemGroup>() != null;
    19.     HaveStaticBodiesChanged = new NativeArray<int>(1, Allocator.Persistent);
    20.     HaveStaticBodiesChanged[0] = 0;
    21.     PredictionPhysicsWorld = new PhysicsWorld(0, 0, 0);
    22.     PredictionCollisionWorld = new CollisionWorld(0, 0);
    23.   }
    24.  
    25.   protected unsafe override void OnUpdate() {
    26.     var dt = World.Time.DeltaTime;
    27.     var physicsStep = HasSingleton<PhysicsStep>() ? GetSingleton<PhysicsStep>() : PhysicsStep.Default;
    28.     var ghostEntityCount = PredictionPhysicsEntityQuery.CalculateEntityCount();
    29.  
    30.     PredictionPhysicsWorld.Reset(0, ghostEntityCount, 0);
    31.     PredictionCollisionWorld.Reset(0, ghostEntityCount);
    32.  
    33.     var rigidBodies = PredictionPhysicsWorld.Bodies;
    34.  
    35.     Dependency =
    36.     Entities
    37.     .WithName("Build_Prediction_PhysicsWorld_Rigid_Bodies")
    38.     .WithStoreEntityQueryInField(ref PredictionPhysicsEntityQuery)
    39.     .WithAll<GhostComponent>()
    40.     .ForEach((Entity entity, int entityInQueryIndex, in PhysicsCollider collider, in LocalToWorld localToWorld) => {
    41.       var rigidBody = RigidBody.Zero;
    42.  
    43.       // rigidBody.CustomTags = customTags.Value;
    44.       rigidBody.WorldFromBody.pos = localToWorld.Position;
    45.       rigidBody.WorldFromBody.rot = localToWorld.Rotation;
    46.       rigidBody.Entity = entity;
    47.       rigidBody.Collider = collider.Value;
    48.       rigidBodies[entityInQueryIndex] = rigidBody;
    49.     })
    50.     .WithBurst()
    51.     .ScheduleParallel(Dependency);
    52.  
    53.     Dependency =
    54.     PredictionCollisionWorld.ScheduleBuildBroadphaseJobs(ref PredictionPhysicsWorld, dt, physicsStep.Gravity, HaveStaticBodiesChanged, Dependency, physicsStep.ThreadCountHint);
    55.   }
    56. }
    Unfortunately, when I try to use this CollisionWorld (PredictionCollisionWorld) to do Collider Casts I get the error:

    upload_2020-10-26_16-48-52.png

    This is likely somehow the BlobAssetReference wrapping the Collider being somehow invalid but I have absolutely no idea how to fix this. I am comfortable hacking pretty deliberately if needed to get around this but some advice as to what is even going on and what a workaround may be would be very much appreciated. @timjohansson if you should happen to know the answer to this or have been working on a similar system I would love your help as well.

    Thanks for all the awesome work. It's a blast using this stuff and I am very encouraged to keep going. Having networked, fully-animated characters doing frame-perfect collision-detection is awesome and I am so close to doing it with the full physics engine I'd love some help to get across that finish line.
     
    Last edited: Oct 27, 2020
  8. Lieene-Guo

    Lieene-Guo

    Joined:
    Aug 20, 2013
    Posts:
    547
    Unity.Physics.ConvexHull is using a custom none-generic BlobArray

    Code (CSharp):
    1.     // Non-generic temporary stand-in for Unity BlobArray.
    2.     // This is to work around C# wanting to treat any struct containing the generic Unity.BlobArray<T> as a managed struct.
    3.     // TODO: Use Unity.Blobs instead
    4.     public struct BlobArray
    Which does not support Serializing/Deserializing to/from file.

    Basically, All ConvexCollider is saved to file with its header only. All vertices, faces, edges etc. will be lost.

    To fix this, look like the physics team is waiting for c# 8.0. where BlobArray<T> is considered a valid unmanaged type.
    @s_schoener
     
    Last edited: Oct 27, 2020
  9. MicCode

    MicCode

    Joined:
    Nov 19, 2018
    Posts:
    59
    Just run into this while working with Unity Physics, any fix soon?
     
  10. MicCode

    MicCode

    Joined:
    Nov 19, 2018
    Posts:
    59
    I workaround this by removing all physics collider after load,
    Then add the physics collider converted fresh from prefab back to the entity.

    Also remove terrain entity before save and generate fresh after every load
     
  11. Rob-A

    Rob-A

    Joined:
    Aug 7, 2012
    Posts:
    33
    Still an issue 1 year later here working with Unity DOTS Physics as well. Any news on that?

    For me i had the following code.

    Code (CSharp):
    1.  
    2. using (var store = new BlobAssetStore())
    3. {
    4.     //this is code to provide ECS conversion with entity prefabs
    5.     prefabEntity = GameObjectConversionUtility.ConvertGameObjectHierarchy(Prefab, new GameObjectConversionSettings(conversionWorld, GameObjectConversionUtility.ConversionFlags.AssignName, store));
    6. }
    7.  
    i got rid of the error by modifying to

    Code (CSharp):
    1.  
    2. var store = new BlobAssetStore();
    3. {
    4.     //this is code to provide ECS conversion with entity prefabs
    5.     prefabEntity = GameObjectConversionUtility.ConvertGameObjectHierarchy(Prefab, new GameObjectConversionSettings(conversionWorld, GameObjectConversionUtility.ConversionFlags.AssignName, store));
    6. }
    7.  
    you see the using had to go but now i have leaks of course.
    Im probably doing it wrong anyways and should store the BlobAssetStore for the runtime of my prefabEntity.

    Hope this will help others.
     
    Last edited: Nov 1, 2021
  12. JayTea173

    JayTea173

    Joined:
    May 25, 2020
    Posts:
    1
    This issue wasn't addressed in the release of Entities 0.50. I've found a workaround that works for me. It's in the comments of tertle's repro case.
     
    Ni2Be and guozefei like this.