Search Unity

Bug MARS misplacing elements

Discussion in 'Unity MARS' started by Ricardotlpz, Jun 7, 2021.

  1. Ricardotlpz

    Ricardotlpz

    Joined:
    May 17, 2021
    Posts:
    63
    Hey there!

    I'm using 2020.3.7f1 with the following packages:

    upload_2021-6-6_18-11-59.png

    I'm facing a very weird problem since many days ago. When I launch the app I made, some times when scanning an image marker, the 3D asset doesn't get anchored to the image marker, but gets "anchored" to the real world position when I opened the app (Lets say I open the app while holding my phone over my PC keyboard, move somewhere else, scan the image marker, and the 3D asset is over my keyboard).

    I'm doing this with MARS and Android.

    upload_2021-6-6_18-14-46.png
     
  2. jmunozarUTech

    jmunozarUTech

    Unity Technologies

    Joined:
    Jun 1, 2020
    Posts:
    297
    hello @ricardotipz,

    Strange, Can you also add a
    TrackingStateCondition
    to the image marker and set it to "
    Tracking
    " to be able to remove the gameobjects when lost and see if the objects get anchored correctly?
     
  3. Ricardotlpz

    Ricardotlpz

    Joined:
    May 17, 2021
    Posts:
    63
    Hey there! Thanks for your reply. I tried what you suggested and the 3D object did not render at all.
     
  4. jmunozarUTech

    jmunozarUTech

    Unity Technologies

    Joined:
    Jun 1, 2020
    Posts:
    297
    hello @ricardotipz,

    Even more strange; Does it work on the simulator?; could you share with us your stack trace to see whats going on?.

    Lets try something simpler; an image with a cube/sphere/any mesh. If you follow this: https://docs.unity3d.com/Packages/com.unity.mars@1.3/manual/Markers.html can you get a basic image marker track?
     
  5. Ricardotlpz

    Ricardotlpz

    Joined:
    May 17, 2021
    Posts:
    63
    Weirdly enough I made these changes with the basic image and now I couldn't build and run. Ended with "successful" deployment but didn't actually succeed. Manually installed and the problem persisted but with something new.

    I used this hierachy and the AR-common is still being rendered even tho the image marker hasn't been scanned, but when I scan the image, the sphere gets rendered where I opened the app (like described before)
    upload_2021-6-7_15-6-27.png

    Also I have another image markers with other 3D assets and those are working properly.

    Not sure how to send you the stack trace. Could you please guide me on that?
     
    Last edited: Jun 9, 2021
  6. Ricardotlpz

    Ricardotlpz

    Joined:
    May 17, 2021
    Posts:
    63
    Forgot to mention. On simulation view the sphere works properly but the "AR-common" is still floating (even tho it's not under the image marker) where the Synthetic Image Marker spawned (similar to the spawning over the keyboard issue I mentioned before)
     
  7. jmunozarUTech

    jmunozarUTech

    Unity Technologies

    Joined:
    Jun 1, 2020
    Posts:
    297
    Well since the ARCommon gameobject is not attached to anything it will appear in the space, depending on where SLAM starts to track and sets the initial position.

    With regards to stack trace on android, you will have to do it with ADB Logcat https://answers.unity.com/questions/492681/how-to-use-adb-logcat.html
     
  8. Ricardotlpz

    Ricardotlpz

    Joined:
    May 17, 2021
    Posts:
    63
  9. jmunozarUTech

    jmunozarUTech

    Unity Technologies

    Joined:
    Jun 1, 2020
    Posts:
    297
    hello @ricardotipz,

    If you checked your logs, seems you are having a null reference exception

    Code (CSharp):
    1. 06-07 16:11:52.808 20215 20324 E Unity   : ArgumentException: Index out of bounds.
    2. 06-07 16:11:52.808 20215 20324 E Unity   :   at (wrapper managed-to-native) UnityEngine.Input.GetTouch_Injected(int,UnityEngine.Touch&)
    3. 06-07 16:11:52.808 20215 20324 E Unity   :   at UnityEngine.Input.GetTouch (System.Int32 index) [0x00000] in <e77348f842b441a984e2e95edba2a91d>:0
    4. 06-07 16:11:52.808 20215 20324 E Unity   :   at OnTouchScreen.Update () [0x00001] in D:\UnityProjects\EpixARURPv2\Assets\Scripts\OnTouchScreen.cs:26
    This might be preventing the logic of the image markers work properly. We are trying to diagnose if you have a problem with Image Markers; could you create a new empty project with just an image marker proxy and a cube for it like its mentioned in the docs?

    This is to diagnose for errors on the ImageMarker's side.
     
  10. Ricardotlpz

    Ricardotlpz

    Joined:
    May 17, 2021
    Posts:
    63
    While creating the new project (URP) I saw some weird errors and got the prompt to restart Unity to try and fix it. Everything went fine with that and the new project works correctly with the image marker and cube.
     
  11. Ricardotlpz

    Ricardotlpz

    Joined:
    May 17, 2021
    Posts:
    63
    On further investigation, I removed the OnTouchScreen.cs and misplacement problem persists on the original project.
     
    Last edited: Jun 8, 2021
  12. Ricardotlpz

    Ricardotlpz

    Joined:
    May 17, 2021
    Posts:
    63
    Hey @jmunozarUTech, I have just noticed something that may be interferring. I remember when doing the project (the bugged one) I duplicated the ImageMarker and just changed the name, image, etc. I noticed all my image markers have a blue diamond on its center.

    Now, I made this new test project and decided to rebuild everything from scratch (At least I got time to do that) and when placing ImageMarkers, I didn't duplicate it but click three times the ImageMarker button on the MARS panel and these diamonds now have different colors.

    Could it be that Unity is messing up because of bad references since I duplicated the markers?
     
  13. mtschoen

    mtschoen

    Unity Technologies

    Joined:
    Aug 16, 2016
    Posts:
    194
    Hi @Ricardotlpz! Sorry to hear that you're having trouble.

    What you just described is normal behavior. The color of the diamond is based on a serialized field in the Proxy class, which means that if you duplicate a proxy, it will have the same color diamond. When you create a _new_ proxy with the Image Marker button, we assign a random color to each one for convenience.

    Looking at the log you shared, I see the following:

    ARTrackedImageManager 'AR Session' was enabled but no reference image library is specified. To enable, set a valid reference image library and then re-enable this component.


    This seems to indicate that something is wrong setting up the reference image library. Do you have one set up on the MARSSession in the scene you are loading? I think this should be required for simulation, but it may not be.

    If you _do_ have a library set up, this is a bug. If you are able to do so, please submit your project through the Unity Bug Reporter (in the menu, Help > Report a Bug...) and share your case number. This will help us replicate the issue on our end and figure out what's going wrong.
     
    Ricardotlpz and jmunozarUTech like this.
  14. Ricardotlpz

    Ricardotlpz

    Joined:
    May 17, 2021
    Posts:
    63
    Hey guys! First of all, yeah I did have a library set up. I did a new project trying to rebuild everything and it keeps getting bugged. Thank y'all for your help.

    I sent the bug report last night (like 10 hours ago) but didn't get a case number neither on the Bug report window or my email. I gave it the title "MARS misplacing" if I recall correctly.
     
    jmunozarUTech likes this.
  15. mtschoen

    mtschoen

    Unity Technologies

    Joined:
    Aug 16, 2016
    Posts:
    194
    Yep! I see it. Case # is 1341952.

    I'm downloading your project now and hopefully we'll be able to come back with a fix or a workaround.
     
    Ricardotlpz likes this.
  16. Ricardotlpz

    Ricardotlpz

    Joined:
    May 17, 2021
    Posts:
    63
    Thanks man! I have just downloaded 2020.3.11f hoping it was going to work and I had a corrupt engine, but still the same :p
     
  17. mtschoen

    mtschoen

    Unity Technologies

    Joined:
    Aug 16, 2016
    Posts:
    194
    Hi there. I was able to build your project, and I think I see what you were seeing, but I did eventually get the image to track. I'm sorry to say it, but I think the images you are using for tracking just don't have enough valid fiducial markers for good image tracking. But! This is relatively easily fixed. You can keep the content you have as long as you build out more high-variance detail along the border, or in "chunks" within the image. Something like a QR code is the quintessential image marker because it is entirely black-and-white corners. These make great fiducial markers, and give you good, reliable tracking. I have seen apps get creative with blocky patterns, different shapes like triangles or diamonds, etc.

    Edit: My bad... According to Google's advice, a QR code is not a good image marker, but I think the logic still applies--you need some more areas with high-frequency detail to get a better tracking score.

    In order to make sure we're on the right track, please try making a build using the included image markers in the MARS sample content under Assets/MARS/Content/Shared/Textures. I tried a build with these in the marker library instead of your custom images, and the tracking worked much better. Let us know if this works for you, and whether you are able to get better tracking out of your custom image markers with added fiducials. Google has some advice for how to choose good images, and instructions for how to use their `arcoreimg` utility to get a score for your images as you iterate on them.

    Hope this helps!
     
    Last edited: Jun 10, 2021
    Ricardotlpz likes this.
  18. Ricardotlpz

    Ricardotlpz

    Joined:
    May 17, 2021
    Posts:
    63
    Hey, don't wanna ruin the party but I actually used those image markers because I ran them through the arcoreimg tool and scored the Google's recommendation. I did this when working on AR with Unreal Engine and since both use ARcore, there shouldn't be a problem with that. Worked flawless and butter smooth on UE4 with the same image markers (image candidates on UE4).

    upload_2021-6-9_20-6-36.png

    I'm trying not to use QR codes for image markers since I'll be using QR for other features inside the app. Besides, the issue kinda improved, but it's still present (ignore the image is blurry)(This QR also scored 100).

    upload_2021-6-9_20-19-54.png
     
    Last edited: Jun 10, 2021
  19. Ricardotlpz

    Ricardotlpz

    Joined:
    May 17, 2021
    Posts:
    63
    Let me just add that when I started this project on Unity, the image markers were working properly (the same ones you used) and pretty smooth, but days passed and every day the tracking was deteriorating to the point of no tracking at all. I'm gonna suspect again on reference issues.
     
  20. Ricardotlpz

    Ricardotlpz

    Joined:
    May 17, 2021
    Posts:
    63
    Update: Tried the image markers in the MARS sample content under Assets/MARS/Content/Shared/Textures and it seems to work "Ok". Not as good as I'd expect and not as good as when I started the project, but it works. I think I can keep on building with some new image markers, but I still think there's something funky going on on the background.
     
  21. mtschoen

    mtschoen

    Unity Technologies

    Joined:
    Aug 16, 2016
    Posts:
    194
    That's pretty strange. Can you describe when you started and what may have changed? When you say "the same ones you used" are you referring to the MARS sample images or the ones that you were using in the project you shared? Were you using MARS the whole time, or did you start with AR Foundation and then try MARS? Did you upgrade your Unity version and/or package versions? When you compare with UE4, do you see any configuration differences like the maximum number of moving images? Something I tried when I was troubleshooting was to check "specify size" for the images in the library, which I think helps improve tracking quality.

    I can sympathize. Computer vision adds a lot of uncertainty to what is going on, making it harder to identify and fix bugs. It could be things like time of day affecting the lighting or introducing motion blur to the camera feed, or a huge number of subtle differences from one test to the next. In order for us to try and fix the issue we need to know as much as we can about what might have changed between earlier, when thing were working well, and now.

    We will continue to investigate the issue on our end. It is especially helpful to have a real-world example, so thank you for sharing. We'll share any insights we find and if there's a bug on our end we'll fix it! :)
     
    Ricardotlpz likes this.
  22. Ricardotlpz

    Ricardotlpz

    Joined:
    May 17, 2021
    Posts:
    63
    I'm referring to the ones I were using. I didn't use MARS sample images since they don't work really well. The ARcore CNN extracts similar features from the MARS sample images, so it constantly gets confused when using multiple image markers. Those sample images I consider good only if using one image marker. I'd suggest you guys updating them with more complex features such as more curves and not many repetitive patterns.

    While in desperation I switched between AR frameworks using AR Foundation and MARS, however I ended up making a fresh new project to discard that being a problem. I even reinstalled Unity engine. No package upgrading.

    Not really. Configuration is very much the same between UE4 and MARS. Note that UE4 uses C++ and Unity C# which may perform different since CNN seems to enjoy more C++. Not sure about this statement tho, I code CNNs with Python but I've heard C++ and JavaScript is cool too, never heard of C# on computer vision being cool stuff. Also, not sure how Unity handles ARcore.

    Really appreciate your support!
     
  23. Ricardotlpz

    Ricardotlpz

    Joined:
    May 17, 2021
    Posts:
    63
    Now, if anyone comes to stumble to bad score images and can't find some, here's three images I'm using with 100 score according to the arcoreimg tool.

    dog.jpeg Earth.jpg

    tars.png
     
  24. mtschoen

    mtschoen

    Unity Technologies

    Joined:
    Aug 16, 2016
    Posts:
    194
    Thanks for the info! I'll keep you posted as we explore this issue further.
     
    Ricardotlpz likes this.