Search Unity

Question Trouble with Image Recognition in MARS when restarting the scene

Discussion in 'Unity MARS' started by michaelnyffenegger, Apr 16, 2021.

  1. michaelnyffenegger

    michaelnyffenegger

    Joined:
    Jun 24, 2018
    Posts:
    4
    Hey guys
    I have the following problem. My project has a main menu scene and a scanning scene with image regocnition in MARS. Everything works perfectly. But when I jump from the scanning scene to the main menu and back to the scanning scene, the image recognition does not work anymore. As I said, I work with Unity MARS. The scene has 4 different images that have to be recognized, but as soon as one is recognized, it does not work anymore when restarting the scene.

    https://michaelnyffenegger.com/Unity/ImageRegoUnityError.MP4

    Thanks for helping out!
     
    michaelnyffenegger, Apr 16, 2021
    #1

  2. jmunozarUTech

    jmunozarUTech

    Unity Technologies

    Joined:
    Jun 1, 2020
    Posts:
    297
    hello @michaelnyffenegger,

    Thanks for sharing that video!; quick question. Do you get any errors on your device build log?, could you share with us your device log to see if there is anything out of order?
     
    jmunozarUTech, Apr 16, 2021
    #2

  3. michaelnyffenegger

    michaelnyffenegger

    Joined:
    Jun 24, 2018
    Posts:
    4
    Hey @jmunozarUTech

    Thanks for the fast response!
    Yes sure:

    Code (CSharp):
    1. 2021-04-16 16:09:58.605805+0200 ECOAR[695:91025] Built from '2020.3/staging' branch, Version '2020.3.0f1 (c7b5465681fb)', Build type 'Release', Scripting Backend 'il2cpp'
    2. 2021-04-16 16:09:58.606633+0200 ECOAR[695:91025] MemoryManager: Using 'Default' Allocator.
    3. -> applicationDidFinishLaunching()
    4. -> applicationDidBecomeActive()
    5. [Subsystems] Discovering subsystems at path /private/var/containers/Bundle/Application/6D3DAE48-C04C-43D3-9D55-B0946BDF75C5/ECOAR.app/Data/UnitySubsystems
    6. [Subsystems] No descriptors matched for  examples in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json.
    7. [Subsystems] 1 'inputs' descriptors matched in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json
    8. [Subsystems] No descriptors matched for  displays in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json.
    9. [Subsystems] 1 'meshings' descriptors matched in UnitySubsystems/UnityARKit/UnitySubsystemsManifest.json
    10. GfxDevice: creating device client; threaded=1
    11. Initializing Metal device caps: Apple A13 GPU
    12. Initialize engine version: 2020.3.0f1 (c7b5465681fb)
    13. XRGeneral Settings awakening...
    14. UnityEngine.XR.Management.XRGeneralSettings:Awake()
    15.  
    16. 2021-04-16 16:09:59.776537+0200 ECOAR[695:91025]
    17.     // 2D joint skeleton
    18.     enum JointIndices
    19.     {
    20.         Invalid = -1,
    21.         Head = 0, // parent: Neck1 [1]
    22.         Neck1 = 1, // parent: Root [16]
    23.         RightShoulder1 = 2, // parent: Neck1 [1]
    24.         RightForearm = 3, // parent: RightShoulder1 [2]
    25.         RightHand = 4, // parent: RightForearm [3]
    26.         LeftShoulder1 = 5, // parent: Neck1 [1]
    27.         LeftForearm = 6, // parent: LeftShoulder1 [5]
    28.         LeftHand = 7, // parent: LeftForearm [6]
    29.         RightUpLeg = 8, // parent: Root [16]
    30.         RightLeg = 9, // parent: RightUpLeg [8]
    31.         RightFoot = 10, // parent: RightLeg [9]
    32.         LeftUpLeg = 11, // parent: Root [16]
    33.         LeftLeg = 12, // parent: LeftUpLeg [11]
    34.         LeftFoot = 13, // parent: LeftLeg [12]
    35.         RightEye = 14, // parent: Head [0]
    36.         LeftEye = 15, // parent: Head [0]
    37.         Root = 16, // parent: <none> [-1]
    38.     }
    39. 2021-04-16 16:09:59.780443+0200 ECOAR[695:91025]
    40.     // 3D joint skeleton
    41.     enum JointIndices
    42.     {
    43.         Invalid = -1,
    44.         Root = 0, // parent: <none> [-1]
    45.         Hips = 1, // parent: Root [0]
    46.         LeftUpLeg = 2, // parent: Hips [1]
    47.         LeftLeg = 3, // parent: LeftUpLeg [2]
    48.         LeftFoot = 4, // parent: LeftLeg [3]
    49.         LeftToes = 5, // parent: LeftFoot [4]
    50.         LeftToesEnd = 6, // parent: LeftToes [5]
    51.         RightUpLeg = 7, // parent: Hips [1]
    52.         RightLeg = 8, // parent: RightUpLeg [7]
    53.         RightFoot = 9, // parent: RightLeg [8]
    54.         RightToes = 10, // parent: RightFoot [9]
    55.         RightToesEnd = 11, // parent: RightToes [10]
    56.         Spine1 = 12, // parent: Hips [1]
    57.         Spine2 = 13, // parent: Spine1 [12]
    58.         Spine3 = 14, // parent: Spine2 [13]
    59.         Spine4 = 15, // parent: Spine3 [14]
    60.         Spine5 = 16, // parent: Spine4 [15]
    61.         Spine6 = 17, // parent: Spine5 [16]
    62.         Spine7 = 18, // parent: Spine6 [17]
    63.         LeftShoulder1 = 19, // parent: Spine7 [18]
    64.         LeftArm = 20, // parent: LeftShoulder1 [19]
    65.         LeftForearm = 21, // parent: LeftArm [20]
    66.         LeftHand = 22, // parent: LeftForearm [21]
    67.         LeftHandIndexStart = 23, // parent: LeftHand [22]
    68.         LeftHandIndex1 = 24, // parent: LeftHandIndexStart [23]
    69.         LeftHandIndex2 = 25, // parent: LeftHandIndex1 [24]
    70.         LeftHandIndex3 = 26, // parent: LeftHandIndex2 [25]
    71.         LeftHandIndexEnd = 27, // parent: LeftHandIndex3 [26]
    72.         LeftHandMidStart = 28, // parent: LeftHand [22]
    73.         LeftHandMid1 = 29, // parent: LeftHandMidStart [28]
    74.         LeftHandMid2 = 30, // parent: LeftHandMid1 [29]
    75.         LeftHandMid3 = 31, // parent: LeftHandMid2 [30]
    76.         LeftHandMidEnd = 32, // parent: LeftHandMid3 [31]
    77.         LeftHandPinkyStart = 33, // parent: LeftHand [22]
    78.         LeftHandPinky1 = 34, // parent: LeftHandPinkyStart [33]
    79.         LeftHandPinky2 = 35, // parent: LeftHandPinky1 [34]
    80.         LeftHandPinky3 = 36, // parent: LeftHandPinky2 [35]
    81.         LeftHandPinkyEnd = 37, // parent: LeftHandPinky3 [36]
    82.         LeftHandRingStart = 38, // parent: LeftHand [22]
    83.         LeftHandRing1 = 39, // parent: LeftHandRingStart [38]
    84.         LeftHandRing2 = 40, // parent: LeftHandRing1 [39]
    85.         LeftHandRing3 = 41, // parent: LeftHandRing2 [40]
    86.         LeftHandRingEnd = 42, // parent: LeftHandRing3 [41]
    87.         LeftHandThumbStart = 43, // parent: LeftHand [22]
    88.         LeftHandThumb1 = 44, // parent: LeftHandThumbStart [43]
    89.         LeftHandThumb2 = 45, // parent: LeftHandThumb1 [44]
    90.         LeftHandThumbEnd = 46, // parent: LeftHandThumb2 [45]
    91.         Neck1 = 47, // parent: Spine7 [18]
    92.         Neck2 = 48, // parent: Neck1 [47]
    93.         Neck3 = 49, // parent: Neck2 [48]
    94.         Neck4 = 50, // parent: Neck3 [49]
    95.         Head = 51, // parent: Neck4 [50]
    96.         Jaw = 52, // parent: Head [51]
    97.         Chin = 53, // parent: Jaw [52]
    98.         LeftEye = 54, // parent: Head [51]
    99.         LeftEyeLowerLid = 55, // parent: LeftEye [54]
    100.         LeftEyeUpperLid = 56, // parent: LeftEye [54]
    101.         LeftEyeball = 57, // parent: LeftEye [54]
    102.         Nose = 58, // parent: Head [51]
    103.         RightEye = 59, // parent: Head [51]
    104.         RightEyeLowerLid = 60, // parent: RightEye [59]
    105.         RightEyeUpperLid = 61, // parent: RightEye [59]
    106.         RightEyeball = 62, // parent: RightEye [59]
    107.         RightShoulder1 = 63, // parent: Spine7 [18]
    108.         RightArm = 64, // parent: RightShoulder1 [63]
    109.         RightForearm = 65, // parent: RightArm [64]
    110.         RightHand = 66, // parent: RightForearm [65]
    111.         RightHandIndexStart = 67, // parent: RightHand [66]
    112.         RightHandIndex1 = 68, // parent: RightHandIndexStart [67]
    113.         RightHandIndex2 = 69, // parent: RightHandIndex1 [68]
    114.         RightHandIndex3 = 70, // parent: RightHandIndex2 [69]
    115.         RightHandIndexEnd = 71, // parent: RightHandIndex3 [70]
    116.         RightHandMidStart = 72, // parent: RightHand [66]
    117.         RightHandMid1 = 73, // parent: RightHandMidStart [72]
    118.         RightHandMid2 = 74, // parent: RightHandMid1 [73]
    119.         RightHandMid3 = 75, // parent: RightHandMid2 [74]
    120.         RightHandMidEnd = 76, // parent: RightHandMid3 [75]
    121.         RightHandPinkyStart = 77, // parent: RightHand [66]
    122.         RightHandPinky1 = 78, // parent: RightHandPinkyStart [77]
    123.         RightHandPinky2 = 79, // parent: RightHandPinky1 [78]
    124.         RightHandPinky3 = 80, // parent: RightHandPinky2 [79]
    125.         RightHandPinkyEnd = 81, // parent: RightHandPinky3 [80]
    126.         RightHandRingStart = 82, // parent: RightHand [66]
    127.         RightHandRing1 = 83, // parent: RightHandRingStart [82]
    128.         RightHandRing2 = 84, // parent: RightHandRing1 [83]
    129.         RightHandRing3 = 85, // parent: RightHandRing2 [84]
    130.         RightHandRingEnd = 86, // parent: RightHandRing3 [85]
    131.         RightHandThumbStart = 87, // parent: RightHand [66]
    132.         RightHandThumb1 = 88, // parent: RightHandThumbStart [87]
    133.         RightHandThumb2 = 89, // parent: RightHandThumb1 [88]
    134.         RightHandThumbEnd = 90, // parent: RightHandThumb2 [89]
    135.     }
    136. [Subsystems] Loading plugin UnityARKit for subsystem ARKit-Input...
    137. [Subsystems] UnityARKit successfully registered Provider for ARKit-Input
    138. [Subsystems] Loading plugin UnityARKit for subsystem ARKit-Meshing...
    139. [Subsystems] Failed to initialize subsystem ARKit-Meshing [error: 1]
    140. 2021-04-16 16:10:00.060851+0200 ECOAR[695:91025] Unbalanced calls to begin/end appearance transitions for <UnityViewControllerStoryboard: 0x119579bb0>.
    141. 2021-04-16 16:10:00.082613+0200 ECOAR[695:91025] [assertion] Error acquiring assertion: <Error Domain=RBSAssertionErrorDomain Code=3 "Target is not running or required target entitlement is missing" UserInfo={RBSAssertionAttribute=<RBSDomainAttribute| domain:"com.apple.webkit" name:"Background" sourceEnvironment:"(null)">, NSLocalizedFailureReason=Target is not running or required target entitlement is missing}>
    142. 2021-04-16 16:10:00.082665+0200 ECOAR[695:91025] [ProcessSuspension] 0x10fdb8b40 - ProcessAssertion: Failed to acquire RBS Background assertion 'WebProcess Background Assertion' for process with PID 698, error: Error Domain=RBSAssertionErrorDomain Code=3 "Target is not running or required target entitlement is missing" UserInfo={RBSAssertionAttribute=<RBSDomainAttribute| domain:"com.apple.webkit" name:"Background" sourceEnvironment:"(null)">, NSLocalizedFailureReason=Target is not running or required target entitlement is missing}
    143. UnloadTime: 1.151542 ms
    144. Unloading 6 Unused Serialized files (Serialized files now loaded: 0)
    145. UnloadTime: 1.309542 ms
    146. MARS GeoLocation is not configured to request location service permission for this project. See 'GeoLocationModule' asset.
    147. Unity.MARS.Providers.GeoLocationModule:Unity.XRTools.ModuleLoader.IModule.LoadModule()
    148. Unity.XRTools.ModuleLoader.ModuleLoaderCore:LoadModulesWithTypes(List`1)
    149. Unity.MARS.MARSSession:Awake()
    150.  
    151.  
    152. Unloading 19 unused Assets to reduce memory usage. Loaded Objects now: 3464.
    153. Total: 6.401208 ms (FindLiveObjects: 0.108708 ms CreateObjectMapping: 0.041375 ms MarkObjects: 6.211917 ms  DeleteObjects: 0.038833 ms)
    154.  
    155. 2021-04-16 16:10:18.873581+0200 ECOAR[695:91025] UnityARKit: Updating ARSession configuration with <ARWorldTrackingConfiguration: 0x2819f94a0 worldAlignment=Gravity lightEstimation=Disabled frameSemantics=None videoFormat=<ARVideoFormat: 0x280dc0d20 imageResolution=(1920, 1440) framesPerSecond=(60)> autoFocus=Enabled environmentTexturing=None wantsHDREnvironmentTextures=Enabled planeDetection=Horizontal|Vertical collaboration=Disabled userFaceTracking=Disabled sceneReconstruction=None detectionImages=[<ARReferenceImage: 0x280ae4600 name="overview_66F5A6DF-EDC2-3A42-A73B-14C88648442F" physicalSize=(0.167, 0.300)>, <ARReferenceImage: 0x280a95b60 name="spareParts_304C5387-2756-6245-AC43-3380ED81C489" physicalSize=(0.161, 0.300)>, <ARReferenceImage: 0x280aaa700 name="usage_864B07C1-90F8-3A4A-BBC7-DAE8E6353319" physicalSize=(0.154, 0.310)>, <ARReferenceImage: 0x280ab2940 name="usage02_F8D3E0AC-21ED-C84B-914A-2EB44AD322A9"'GeoLocationModule' asset.
    156. Unity.MARS.Providers.GeoLocationModule:Unity.XRTools.ModuleLoader.IModule.LoadModule()
    157. Unity.XRTools.ModuleLoader.ModuleLoaderCore:LoadModulesWithTypes(List`1)
    158. Unity.MARS.MARSSession:Awake()
    159.  
    160. 2021-04-16 16:10:25.749286+0200 ECOAR[695:91025] UnityARKit: Updating ARSession configuration with <ARWorldTrackingConfiguration: 0x2819fc160 worldAlignment=Gravity lightEstimation=Disabled frameSemantics=None videoFormat=<ARVideoFormat: 0x280dc0d20 imageResolution=(1920, 1440) framesPerSecond=(60)> autoFocus=Enabled environmentTexturing=None wantsHDREnvironmentTextures=Enabled planeDetection=Horizontal|Vertical collaboration=Disabled userFaceTracking=Disabled sceneReconstruction=None detectionImages=[<ARReferenceImage: 0x280ae4600 name="overview_66F5A6DF-EDC2-3A42-A73B-14C88648442F" physicalSize=(0.167, 0.300)>, <ARReferenceImage: 0x280a95b60 name="spareParts_304C5387-2756-6245-AC43-3380ED81C489" physicalSize=(0.161, 0.300)>, <ARReferenceImage: 0x280aaa700 name="usage_864B07C1-90F8-3A4A-BBC7-DAE8E6353319" physicalSize=(0.154, 0.310)>, <ARReferenceImage: 0x280ab2940 name="usage02_F8D3E0AC-21ED-C84B-914A-2EB44AD322A9""The connection to service on pid 0 named com.apple.UIKit.KeyboardManagement.hosted was invalidated from this process." UserInfo={NSDebugDescription=The connection to service on pid 0 named com.apple.UIKit.KeyboardManagement.hosted was invalidated from this process.}
    161. -> applicationWillTerminate()
    162. Setting up 1 worker threads for Enlighten.
    163.   Thread -> id: 16d96f000 -> priority: 1
    164.  
     
    michaelnyffenegger, Apr 16, 2021
    #3

  4. jmunozarUTech

    jmunozarUTech

    Unity Technologies

    Joined:
    Jun 1, 2020
    Posts:
    297
    Th
    Thanks for the log!, on this log did you do the same as you did on the video? (Track the image, go to the menu, try to track the image again).

    Also, which version of MARS, Unity, ARFoundation and ARkit / face tracking are you using?
     
    jmunozarUTech, Apr 16, 2021
    #4

  5. michaelnyffenegger

    michaelnyffenegger

    Joined:
    Jun 24, 2018
    Posts:
    4

    Yes the process was the same. I use MARS 1.3.0. AR Foundation 4.1.5 and Unity 2020.3.0. face tracking I actually do not use, but installed is version 4.0.12
     
    michaelnyffenegger, Apr 16, 2021
    #5

  6. CiaranWills

    CiaranWills

    Unity Technologies

    Joined:
    Apr 24, 2020
    Posts:
    199
    I was able to reproduce this with ARKit (but not ARCore). A solution is to reset the ARSession when you go back to the tracking scene by calling

    Code (CSharp):
    1. ARFoundationSessionProvider.currentSession?.Reset();
     
    CiaranWills, Apr 20, 2021
    #6

  7. michaelnyffenegger

    michaelnyffenegger

    Joined:
    Jun 24, 2018
    Posts:
    4
    WOW Amazing it works! Thanks a lot for your help CiaranWills! I hope they will fix this issue in a later version of MARS.
     
    michaelnyffenegger, Apr 23, 2021
    #7
    mtschoen likes this.

  8. CiaranWills

    CiaranWills

    Unity Technologies

    Joined:
    Apr 24, 2020
    Posts:
    199
    Following up - actually a better thing to do is to call AR Foundation's LoaderUtility Deinitialize() then Initialize() methods when switching scenes. This will reset all the AR subsystems to ensure you are starting with a clean slate.

    (From this forum thread)
     
    CiaranWills, May 26, 2021
    #8