Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Resolved ARKit support for iOS via Unity-ARKit-Plugin

Discussion in 'AR' started by jimmya, Jun 5, 2017.

Thread Status:
Not open for further replies.
  1. jimmya

    jimmya

    Joined:
    Nov 15, 2016
    Posts:
    793
    Do you have more than one camera? That might be messing with the rendering. The "clear to video frame" is done by UnityARVideo.cs script on whatever camera it is attached on. If another camera is rendering before or after that, your results may vary.
     
  2. jimmya

    jimmya

    Joined:
    Nov 15, 2016
    Posts:
    793
    Don't know how you got into that state - as I said, we have a UnityARBuildPostProcessor script that will add the ARKit framework to your XCode project when you do a build. Maybe you deleted the UnityARBuildPostProcessor script?
     
  3. stevencoull

    stevencoull

    Joined:
    Sep 15, 2016
    Posts:
    16
    I've found an issue where the objects in my scene jitter around by a few centimeters every know and then. Is there any reason why this would happen? From a performance perspective everything looks okay, and the fps isn't dropping.
     
  4. mirostraka

    mirostraka

    Joined:
    Feb 17, 2014
    Posts:
    41
    hey jimmya,
    first of all, amazing work, everything works handsomely, even the remote!
    I however experience occasional video feed interruptions - everything works, but instead of camera feed I see only skybox - using Unity 5.6.3 and iPhone 6s. This only happens at certain angle of the device, tilting it to different angle fixes this - and tilting back causes this again.
    Now, I haven't found a correlation yet, but one thing that I am doing is multiplying camera localPosition by 10 in LateUpdate, so scale camera up (I can't scale the objects down). Not sure if related...
    Did you ever experience smth like this?
    thanks!
     
  5. skdev3

    skdev3

    Joined:
    Jul 15, 2015
    Posts:
    64
    Hi. We have a problem. ARkit + cardboard = Works badly.

    Sorry for the small video...
    ARkit rotation camera bug. "scene is staggering, when changings position and rotation in the device"

    How I'm set up ARkit + Cardboard:
    1. Added Virtual Reality Supported.

    2. My code to Switching to Virtual Reality mode:
    Code (csharp):
    1.  
    2.     IEnumerator LoadDevice(bool isVR)
    3.     {
    4.         // VRDeviceName none
    5.         string newDevice = VRSettings.supportedDevices.Length>0 ? VRSettings.supportedDevices[0] : "none";
    6.         if (isVR)
    7.         {
    8.             //VRDeviceName Cardboard or Split
    9. #if UNITY_EDITOR
    10.             newDevice = "Split";
    11. #else
    12.             newDevice = VRSettings.supportedDevices.Length > 1 ? VRSettings.supportedDevices[1] : "none";
    13. #endif
    14.         }
    15.         Debug.Log("VRDeviceName " + newDevice);
    16.         VRSettings.LoadDeviceByName(newDevice);
    17.         yield return null;
    18.         VRSettings.enabled = isVR;
    19.     }
    20.  
     
    Last edited: Sep 1, 2017
    Arvin6 likes this.
  6. Slaymantis

    Slaymantis

    Joined:
    May 12, 2014
    Posts:
    11
    Hi

    Thanks for this plugin so far.
    When I try build on Xcode 9 beta I see this error

    Interface Builder XIB Compiler Error Group
    /Users/pixel/AugmentedReality/ARKIT_UNITY/builds/iOSARkit/LaunchScreen-iPhone.xib: Internal error. Please file a bug at bugreport.apple.com and attach "/var/folders/7j/j15v5zpd6ld2nz34wpm682jc0000gn/T/IB-agent-diagnostics_2017-09-01_11-14-53_521000".
     
  7. ZYSHAQ

    ZYSHAQ

    Joined:
    Jun 26, 2015
    Posts:
    76
    I am having the same problem:

    Attempting to connect to player ip: 127.0.0.1 with a 1 ms timeout
    UnityEditorInternal.ProfilerDriver:DirectURLConnect(String)
    UnityEditor.AttachProfilerUI:SelectProfilerClick(Object, String[], Int32)

    Could you please elaborate?
     
  8. dlazares

    dlazares

    Joined:
    Jul 24, 2017
    Posts:
    3
    My app was previously working great and then I updated iOS 11 versions. Now when I build I get a lot of errors.

    I tested the UnityARKitScene that comes in the bitbucket repo and that also produced the following errors.
    Are my errors coming from the fact that Unity ARKit Plugin doesn't support iOS 11 Beta 9 and if so, when should we expect a fix?

    - iPhone 7 Plus, iOS 11 Beta 9
    - Unity 2017.1.0f3
    - XCode 9 Beta 3
    - Latest ARKit Plugin from BitBucket (as of 10am 9/1/2017)

    Screen Shot 2017-09-01 at 10.44.18 AM.png
     
  9. Kowciany

    Kowciany

    Joined:
    Feb 24, 2015
    Posts:
    6
    Hello @christophergoy ! I used your scaled-content-test approach (bitbucket branch) and it's really working good. One issue here I've noticed I cannot set much more than 30 on far clipping plane "ContentCamera". I know the solution of "UnityARCameraNearFar" and as in your example it's still there on MainCamera (tracking camera). Messed up with attaching this to both, first and second etc. with of course proper values in inspector. Tried LateUpdate also. Nothing worked... no more than 30 on far clipping... please help !

    Unity 2017.1.0p1 / p5 (tried both) + remote ARkit (ipad 2017) + latest ARKit Plugin from assetstore
     
    Last edited: Sep 1, 2017
  10. shazamsoyeux

    shazamsoyeux

    Joined:
    Jun 14, 2016
    Posts:
    3
    You might need to upgrade to Xcode Beta 6. If the download is too slow on the Apple website, I advice downloading from this stackoverflow answer.
     
  11. dlazares

    dlazares

    Joined:
    Jul 24, 2017
    Posts:
    3
    Thanks shazamsoyeux, that solved my problem!
     
  12. ZYSHAQ

    ZYSHAQ

    Joined:
    Jun 26, 2015
    Posts:
    76
    I had same issues:
    1. update xCode to iOS11.0 Beta 6
    2. update unity to what was released just today (2017.2.0 beta)
    3. reimport all your assets
     
    phits likes this.
  13. ccmos

    ccmos

    Joined:
    May 3, 2017
    Posts:
    1
    try
    https://bitbucket.org/Unity-Technol...ixed-crashing-when-scenemanagerloadscene/diff
     
    jimmya and unityuse-a-gts like this.
  14. ZYSHAQ

    ZYSHAQ

    Joined:
    Jun 26, 2015
    Posts:
    76
    in UnityARCameraManager.cs, go to the update routine (at the very bottom and adjust line 76)
    example, I added *20.00f as you can see
    if (m_camera != null)
    {
    // JUST WORKS!
    Matrix4x4 matrix = m_session.GetCameraPose();
    m_camera.transform.localPosition = UnityARMatrixOps.GetPosition(matrix)*20.00f;
    m_camera.transform.localRotation = UnityARMatrixOps.GetRotation (matrix);

    m_camera.projectionMatrix = m_session.GetCameraProjection ();
    }
     
  15. massivebacon

    massivebacon

    Joined:
    Apr 24, 2014
    Posts:
    27
    Using only a single camera, UnityARVideo.cs is attached to the Main Camera. I'm wondering if it has something to do with the scale of the models? My whole scene exists between -1 and 1 — would that cause any issues?
     
  16. ZYSHAQ

    ZYSHAQ

    Joined:
    Jun 26, 2015
    Posts:
    76
    Check my example above, just multiply by 0.1f instead
     
  17. jimmya

    jimmya

    Joined:
    Nov 15, 2016
    Posts:
    793
    You should not mess with main camera's transform. If you want to show scaled content, see the branch that @christophergoy made (using a separate camera for scaled content).
     
    mirostraka likes this.
  18. jimmya

    jimmya

    Joined:
    Nov 15, 2016
    Posts:
    793
    See this.
     
  19. jimmya

    jimmya

    Joined:
    Nov 15, 2016
    Posts:
    793
    You should follow those instructions.
     
  20. massivebacon

    massivebacon

    Joined:
    Apr 24, 2014
    Posts:
    27
    This doesn't work. It also throws the tracking planes out of alignment and makes the camera not properly rotate around the scene.
     
  21. jimmya

    jimmya

    Joined:
    Nov 15, 2016
    Posts:
    793
    Read FAQ.
     
  22. jimmya

    jimmya

    Joined:
    Nov 15, 2016
    Posts:
    793
    It shouldn't be problematic, unless you're giving some models a negative scale, which would make them render inside out (i.e. it might be backface culling your polygons). BTW, you should try the scaled-content branch for your models.
     
  23. ZYSHAQ

    ZYSHAQ

    Joined:
    Jun 26, 2015
    Posts:
    76
    do not multiply the rotation, only the translation.
     
  24. jimmya

    jimmya

    Joined:
    Nov 15, 2016
    Posts:
    793
  25. massivebacon

    massivebacon

    Joined:
    Apr 24, 2014
    Posts:
    27
    Nah no negative scale. To be clear it also works pretty consistently, but then will sometimes just start ghosting like that. The framerate stays around 60, but it looks like the camera isn't properly clearing the scene. Could that be inadvertently triggered? Is it waiting on something before it clears?
     
  26. jimmya

    jimmya

    Joined:
    Nov 15, 2016
    Posts:
    793
    When the cube is made bigger it works as expected, because the center of the cube is the one that is placed at the hittest location. That means it will look like the bottom is moving, because the bottom of the cube is actually below the ground plane. To test the behavior you want, you can translate the cube up by half of your scale and it should look like it has been placed on the floor.
    Similarly, you will have to adjust the height of your model so that its bottom/floor is placed at the hit test position to make it appear correct.
     
  27. stuartlangfield

    stuartlangfield

    Joined:
    Jul 21, 2017
    Posts:
    13
  28. stuartlangfield

    stuartlangfield

    Joined:
    Jul 21, 2017
    Posts:
    13
    Never mind, installed the latest beta of Unity and the package installs fine now.
     
  29. jimmya

    jimmya

    Joined:
    Nov 15, 2016
    Posts:
    793
    Some changes have gone into BitBucket that people were waiting for/ had PRs for:
    - a change from earlier (post)
    - a fix for UnityARAmbient not cleaning up when destroyed(commit)
    - added config options to ARCameraManager (commit)
    - expose lightEstimate.ambientColorTemperature that was opened up in former ARKit Beta (commit)
    - a version of HitTest that will work in Editor for use with ARKit Remote (commit) inspired by @Inboxninja

    Asset Store update will have to wait till Tuesday at least.
     
    KwahuNashoba, 4NDDYYYY and ZYSHAQ like this.
  30. ZYSHAQ

    ZYSHAQ

    Joined:
    Jun 26, 2015
    Posts:
    76
    thanks for the update!
     
  31. yuliwei

    yuliwei

    Joined:
    Aug 19, 2017
    Posts:
    22
    Thanks, I have found out the reason, my model is too big and the distance is too far away, so visually it looks normal, but actually the distance in the model world is further than the real world, so you have to walk more to get there. I scaled it down and set it closer, now it works fine.
     
  32. joeld42

    joeld42

    Joined:
    Oct 3, 2016
    Posts:
    31
    Yeah, I'm having this same problem. You can set the camera to "clear flags" to solid color and at least you'll get a solid background instead of drawing over the last frame.

    So far I've narrowed it down to something in UnityARVideo component or the shader, tracking is still fine and it's actually getting good video frames (you can see them in the metal debugger), it's just not displaying them for some reason.

    It's weird, in my case it fails when my object is onscreen, but works when it's offscreen or partially offscreen. But I can't see how that would break things since the Video draw happen well before the object draw.


     
  33. joeld42

    joeld42

    Joined:
    Oct 3, 2016
    Posts:
    31
    Hey I fixed the Video Clearing problem that I was having (and looks to be similar to massivebacon's problem).

    Image-1_sm.jpg

    It was not drawing the video background "sometimes", depending on what else was in view, resulting in drawing over the last frame which caused weird stuff like this.

    The problem was that the Video quad was getting backface culled sometimes (though I'm not sure why). To fix this, I added "Cull off" as the first thing in the SubShader block, at line 12 in YUVShader.shader and it seems to behave now.

    This was driving me crazy. Hope this helps someone else.

    Joel
     
    DavidErosa likes this.
  34. jimmya

    jimmya

    Joined:
    Nov 15, 2016
    Posts:
    793
    As mentioned before, try not to attach screenshots, but text files. You should look through your XCode console logs when it runs to see if it says/shows something about why it doesn't work. Most likely its a situation that is covered in the FAQ, or do a search about that text string in the forums. If you are still unable to figure it out, post the text of your console log here.[/QUOTE]
     
  35. stuartlangfield

    stuartlangfield

    Joined:
    Jul 21, 2017
    Posts:
    13
    [/QUOTE]

    My apologies for the images, and I'd missed the FAQ, so thanks for linking to that.

    I reverted back to 10.12 which has immediately fixed any issues I was having when building the the example scenes. Looking forward to testing with this now.
     
  36. MythicalCity

    MythicalCity

    Joined:
    Feb 10, 2011
    Posts:
    420
    Hi, I'm trying out ARkit with an older project and was wondering how to setup 2 cameras properly. I have one main camera that has the ARkit near far script and the arkit video script on it and a 2nd camera that is a child of this one that renders a specific layer over the main camera. Both cameras have field of view set to 60 and the same settings all around but it looks like either their field of view or positions are different as the the objects they render don't have the same positions (they should line up). Do I need to put AR scripts onto the child camera as well?
     
  37. JohannChristoph

    JohannChristoph

    Joined:
    Jun 21, 2009
    Posts:
    141
    Thanks you helped me a lot! It drove me almost crazy too and I thought I was the only one on planet having this problem. Being not familiar with shader coding I wrote it wrong the first time. The right way is:
    In YUVShader.shader add after the line 13: LOD100 a new line 14: Cull Off.
     
    jimmya likes this.
  38. John1515

    John1515

    Joined:
    Nov 29, 2012
    Posts:
    248
    How to stop UnityARGeneratePlane? I calling
    Code (CSharp):
    1. unityARAnchorManager.Destroy ();
    but it doesn't stop planes from being generated :(
     
  39. jimmya

    jimmya

    Joined:
    Nov 15, 2016
    Posts:
    793
    You will need to setup the projection matrix on the second camera similar to what is done on UnityARCameraManager: m_camera.projectionMatrix = m_session.GetCameraProjection ();
     
  40. jimmya

    jimmya

    Joined:
    Nov 15, 2016
    Posts:
    793
    You can either make ARKit stop generating planes (see above), or set GeneratePlanes GO to inactive (only stops generating planes in unity).
     
    John1515 likes this.
  41. KarelA

    KarelA

    Joined:
    Dec 30, 2008
    Posts:
    422

    Hi.

    Could you elaborate by what you meant by "messed up with attaching to both"?

    Maybe i just misunderstood your comment but just in case wanted to let you know that i set ContentCamera script on on both cameras. Also for for testing purposes i put far clip on both cameras 100k. Not sure if this was needed to set these extreme values in the first place. But the clipping on 30 was gone. I could scale my scene to absurdly small and walk many meters away and i could still see the content on screen. It was just few pixels, but it was there.
     
  42. sama-van

    sama-van

    Joined:
    Jun 2, 2009
    Posts:
    1,734
    @christophergoy @jimmya
    I just would like to double quote it to be sure it gets an answer ^_^...

    I would add the following : does the Android AR implementation will be the same as the current ARKIT then we do not have to code 2 times the same thing for both platform?

    Thank you! ^_^

     
  43. Kowciany

    Kowciany

    Joined:
    Feb 24, 2015
    Posts:
    6
    First of all thx for answer ! I mean that I've attached UnityARCameraNearFar to both cameras. I don't understand your sollution what script do you mean when you wrote "ContentCamera script". My oryginal hierarchy is:

    -> Directional Light
    -> ContentCameraParent < ContentScaleManager >
    |-> Content Camera < Camera>
    |-> ContentCameraManager < UnityARCameraManager >
    -> TrackingDataCameraParent
    |-> Main Camera < Camera, UnityARVideo, UnityARCameraNearFar >
    |-> ARCameraManager < UnityARCameraManager >
    -> PlaneManager
    -> UI
    -> GameField < ScaledContentPlacer >

    Additional info:
    TrackingDataCameraParent (with children) - set Layer ARKitTrackingData
    Main Camera - set CullingMask ARKitTrackingData
    Content Camera - set Culling Mask to exclude ARKitTrackingData layer


    So setup is almost copy from scale-content-test branch from ARKit repo.
     
  44. flyingdeutschman

    flyingdeutschman

    Joined:
    Jul 26, 2013
    Posts:
    8
    Once again, amazing work! This plugin is fantastic.
    I'm curious - if I'm building a game that instantiates object in the place that you tap them, is it unexpected if they instantiate off of a plane (i.e., the user taps in the sky)? Plz see attached.
    I don't remember that behavior, but am seeing it now after updating. I haven't modified the code that's delivered in the latest version of the plugin. Thanks!
    Image-1 (4).jpg

     

    Attached Files:

  45. elhongo

    elhongo

    Joined:
    Aug 13, 2015
    Posts:
    47
    Has anyone ran into issue when tracking very large objects? If I start with a small object and start scaling it in real time in the ARKit at one point the object looses tracking is stuck at same relative position to the camera. Has anyone ran into this issue?
     
  46. jimmya

    jimmya

    Joined:
    Nov 15, 2016
    Posts:
    793
    Are you using UnityARHitTestExample.cs? AFAIK, this always had the HitTest set to test against: existing planes, immediate planes, and nearest feature point; so it should almost always find something.
     
  47. jimmya

    jimmya

    Joined:
    Nov 15, 2016
    Posts:
    793
    Are you sure its not a loss of perspective? See discussion starting here: https://forum.unity3d.com/threads/a...unity-arkit-plugin.474385/page-5#post-3109616
     
  48. John1515

    John1515

    Joined:
    Nov 29, 2012
    Posts:
    248
    I have done an object of 200 meter on a distance of approx 500m, went fine.. :) but I didn’t scale it up at runtime like you
     
  49. John1515

    John1515

    Joined:
    Nov 29, 2012
    Posts:
    248
    I had a few times that I left a debugplane prefab in the scene that caused some unexpected behavior
     
  50. craig_cerceo

    craig_cerceo

    Joined:
    Apr 3, 2012
    Posts:
    3
    Sorry if this has been asked before - this thread is getting pretty big.

    I'm trying to get ARKit Remote working. I'm running the EditorTestScene in Unity Editor with my iPad Pro connected, and running the UnityARKitRemote scene. In Unity, I see the Start Remote ARKit Session button. When I click it, the camera on my iPad turns on (albeit very laggy), but the video is not transmitted to the Unity editor.

    Does anyone know why Unity would not be receiving the video from my iPad's camera? Thanks in advance for any help!

    My setup:
    Unity 2017.1.0f3
    Xcode 9.0 beta 6
    iOS 11.0 (15A5370a)
    iPad Pro 2017
    Latest from bitbucket as of today (commit hash f0f51db15f50c5e676e2d1f25d42abae93e8d208)

    It seems none of the handlers registered in ARKitRemoteConnection.Start are ever invoked. Here is the log from Xcode console after I connect via Unity:

     
    Last edited: Sep 4, 2017
Thread Status:
Not open for further replies.