Search Unity

  1. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Resolved ARKit support for iOS via Unity-ARKit-Plugin

Discussion in 'AR' started by jimmya, Jun 5, 2017.

Thread Status:
Not open for further replies.
  1. mimminito

    mimminito

    Joined:
    Feb 10, 2010
    Posts:
    780
    Has anyone done an implementation of the FocusSquare from the "Placing Objects" demo Apple did, and willing to open source it?
     
    Aestial likes this.
  2. PdrMnts

    PdrMnts

    Joined:
    Dec 12, 2016
    Posts:
    13
    I upgraded the version today (1.0.5, I downloaded it from the Asset Store) and I am getting weird error that says: Screen position is out of the view frustum. And also the main camera rotated 90 degrees in the y axis and z axis automatically.... and the worst part is that the camera is rendering nothing (its just green), just the canvas.

    I removed everything and installed the previous package of ARKit ( I download 1 month ago from this post, I think it is the 1.0.2 version) and works well... but I am having issues with Xcode9 beta1

    Anyone with these errors with the new version??

    using 5.6.1p2 Unity.
     
    Last edited: Jul 27, 2017
  3. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    Hello,
    The only way to add your own anchors manually is through the NativeSessionInterface with the AddAnchor AddAnchorForGameObject methods. These methods will add them natively to ARKit. The UnityUserAnchor is limited because it's more of a placeholder than an object that we wanted to define for you. The thought process behind it is the you would have your own mapping from the UnityUserAnchor and your GameObjects are associated with it.

    The other thing is that if you add your own anchors, they are not actually tracked/updated by the ARSession. They are literally placeholders for the position of virtual objects in the real world. This API is very limited in it's functionality at the moment and I'm not sure what Apple's plans are for it. Let me know if you have any other questions.
    Cheers,
    Chris
     
    lourd and ipstatic like this.
  4. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    I see your bug, but I don't see an Xcode project file associated with it.
     
  5. 4NDDYYYY

    4NDDYYYY

    Joined:
    Jan 28, 2016
    Posts:
    13
    thanks x 1000000
     
    jmunozar likes this.
  6. mmortall

    mmortall

    Joined:
    Dec 28, 2010
    Posts:
    89
    How to enable Plane Detection in ARKit ? My plane is just in the center of the camera view. Thanks
     
  7. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    If you take a look in the ARKitScene, there is a game object named GeneratePlanes. This game object has a component that listens to the ARSession anchor events (AnchorAdded AnchorUpdated AnchorRemoved) and generates anchors as appropriate. That component takes a prefab in and that's what it uses to create the game objects in the scene. Let me know if you have further questions.
    Cheers,
    Chris
     
  8. GMM

    GMM

    Joined:
    Sep 24, 2012
    Posts:
    301
    What is the best way to stop the ARKit Camera? Calling UnityARSessionNativeInterface.GetARSessionNativeInterface ().Pause (); and then deleting the components does not seems to work, always causes my application to crash.

    More specifically:
    Code (CSharp):
    1. NullReferenceException: GetRef
    2.   at UnityEngine.XR.iOS.UnityARVideo.OnDestroy () [0x00000] in <filename unknown>:0
    It's not a huge problem for now as i can just keep the ARKit camera turned on in the background, but in the interest of optimization i would like to be able to turn on/off the camera at will.
     
  9. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    hmm, I wonder if the camera component is already gone at this point and that's why you're seeing the NullReferenceException. It might be a bug on our side. I'll investigate. Thanks for bringing it up.
     
  10. GMM

    GMM

    Joined:
    Sep 24, 2012
    Posts:
    301
    Specifically i am deleting a prefab containing the ARCameraManager and CameraParent as found in the examples.

    I really like the ARKit integration and despite having a few issue here and there, i cannot wait to show off what i have been building with it :)
     
    christophergoy likes this.
  11. sama-van

    sama-van

    Joined:
    Jun 2, 2009
    Posts:
    1,733
    Would be nice to make a single folder and drop the entire project in then would be super easy to everyones to import in any of out project (even the package is actually entire project...).

    That my first feedback... now will start...
    Cheers :)
     

    Attached Files:

    Last edited: Jul 28, 2017
  12. petey

    petey

    Joined:
    May 20, 2009
    Posts:
    1,806
    Thanks Christopher, that would be an awesome solution! :)
    Good luck, i’ll keep an eye out.
    Also give me a yell if there’s anything I can help out with, I’d be more than happy.
     
    christophergoy and mikejohnstn like this.
  13. ipstatic

    ipstatic

    Joined:
    Jul 27, 2017
    Posts:
    2
    Thank you for your response, guess it's gonna be complicated hehe.
     
  14. idspe

    idspe

    Joined:
    Apr 12, 2016
    Posts:
    3
    Blarp, christophergoy and sama-van like this.
  15. sama-van

    sama-van

    Joined:
    Jun 2, 2009
    Posts:
    1,733
    Unity remote works a charm, but I have this message "Failed to connect to Player" and scene is green.
    (Unity 2017.1.0b5)

    Also checked out the iphone 6 wifi's settings to get the Ip adress and it fails after 3 attempts.
    (iOS 11.0 beta)

    Any idea?... :(


    EDIT :

    Oh nooooo..... iphone 6 doesn't support it T__T....
    >>> http://appleinsider.com/articles/17...compatibility-to-iphone-6s-ipad-pro-and-newer
     

    Attached Files:

    Last edited: Jul 28, 2017
  16. Blarp

    Blarp

    Joined:
    May 13, 2014
    Posts:
    269
  17. idorurez

    idorurez

    Joined:
    Dec 28, 2015
    Posts:
    13
    Hi @jimmya, any update on this?
     
  18. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    This has been rectified with a function on UnityARSessionNativeInterface.SetCameraNearFar
    Let us know if you still have issues.
    Cheers,
    Chris
     
  19. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    Stay tuned...
     
  20. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    Awesome stuff! This looks like a native integration. It could probably be achieved through plugin if you are feeling froggy :)
     
  21. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    This has been done for the bickbucket repo as of yesterday, and I submit the package with this directory restructure as well.
    Cheers,
    Chris
     
  22. idorurez

    idorurez

    Joined:
    Dec 28, 2015
    Posts:
    13
    Hi Chris,

    That's great, but has this been implemented in the asset store version? I must not be finding an accessible implementation in UnityARSessionNativeInterface.cs

    Alfred
     
  23. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    Hi Alfred,
    I believe it was released with version 1.0.5, which was the 25th I believe. If not, there should be a new version out today.
    Cheers,
    Chris
     
    idorurez likes this.
  24. jrhowa

    jrhowa

    Joined:
    Feb 3, 2015
    Posts:
    5
    What is the link for this update?
     
  25. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
  26. idorurez

    idorurez

    Joined:
    Dec 28, 2015
    Posts:
    13
    Hey all,

    I looked back through this thread on collision issues with the collisionPlanePrefab, but there was only one instance where someone had the same issue as I do. The response was to reread the manual on colliders and rigidbody. However, I believe the problem is deeper than that:

    GeneratePlanes creates the prefabs but seem to lack the collider component.
    If I instantiate the same prefab with a button, it's fine: my falling objects collide without an issue.

    Can anyone confirm that they have the same issue? My attempts at using the ARKit remote to check the instantiated prefab planes didn't work so well since there is incredible lag.
     
  27. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    I just tried, but was able to have success. The debugPlane prefab has a Plane object as a child and it that has the boxCollider (and strangely, a MeshCollider) attached to it along with the Plane Physical Material. Do you see that in your project?
     
  28. idorurez

    idorurez

    Joined:
    Dec 28, 2015
    Posts:
    13
    Thanks for verifying Chris. Yes, they're all there as noted, and yes I noticed the additional collider.
    I'll keep debugging, I appreciate the confirmation and your time!
     
    christophergoy likes this.
  29. MassiveTchnologies

    MassiveTchnologies

    Joined:
    Jul 5, 2016
    Posts:
    87
    I had some luck with the world scaling issue by having the AR Camera under a parent and then scaling that parent instead, tracking worked fine, and now I can change world scale dynamically on the fly by scaling the AR camera parent instead of scaling the entire scene (the scene is located at position 0,0,0 and has too many meshes, static objects, complex particle effects etc). I also was able to position the camera parent when doing hit testing instead of moving the scene objects (so the user can tap on the surface in front of him and have the scene framed in view correctly). I'm still having some weird issue with quaternions, though. Currently when doing the hit testing I move the camera parent with the following logic:

    Code (CSharp):
    1.                 m_HitTransform.position = UnityARMatrixOps.GetPosition (hitResult.worldTransform);
    2.                 m_HitTransform.rotation = UnityARMatrixOps.GetRotation (hitResult.worldTransform);
    3.  
    4.                 ARSystem.instance.ARCamRoot.transform.position = -m_HitTransform.position * ARSystem.instance.ARCamRoot.transform.localScale.x;
    5.                 ARSystem.instance.ARCamRoot.transform.rotation = Quaternion.Inverse (m_HitTransform.rotation);
    6.  
    Anyone has a better approach would? Suggestions?
     
    Last edited: Jul 28, 2017
    christophergoy likes this.
  30. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    Awesome stuff! I'm still experimenting on my side, would you be willing to share a smaller version of your scene for me to mess around with?
     
  31. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    I tried this approach. While the tracking of the camera looks accurate, the rendering of the tracked data is also scaled down. So planes/point cloud/etc. are rendered at 1/scale size. Maybe there can be a two camera approach where we render the user made content with one camera, and the tracked data with. I'll let you know how it goes.
     
  32. MassiveTchnologies

    MassiveTchnologies

    Joined:
    Jul 5, 2016
    Posts:
    87
    Thanks! I suspected tracked data would also need to be scaled accordingly. But I think the effort to incorporate world scaling in Unity ARKit plugin would be worth it. Then we could incorporate complex scenes without having to worry about moving/scaling each single scene element to frame the scene and instead only scale and move the camera parent in relation to the scene which should make development for complex scenes easier.
     
  33. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    I'm working on an example scene now. I don't think I'll be done with it today, but maybe over the weekend. It is the two camera approach I mentioned above, where one camera renders the ARKit tracking data (which is optional anyway) and the other renders your content from a scaled camera. Getting close to a solution I think.
     
  34. MassiveTchnologies

    MassiveTchnologies

    Joined:
    Jul 5, 2016
    Posts:
    87
    Ok. I attached a test unity project with the basic world scaling + camera based framing for anyone who wants to toy with it, two main advantages (ARKit tracking data rendering is missing though):

    - When framing the scene, I'm moving the camera parent instead of moving the AR scene objects. This allows for better handling of complex scenes that have a lot static objects that can't be moved in play time, particle effects etc...For that I'm using reverse Quaternions and subtracting the scene object's root position (which is vector3.zero) from the hitTest position and then multiplying that by the camera root scaling (See ARManager script lines 38 and 39):

    Code (CSharp):
    1.                 m_HitTransform.position = UnityARMatrixOps.GetPosition (hitResult.worldTransform);
    2.                 m_HitTransform.rotation = UnityARMatrixOps.GetRotation (hitResult.worldTransform);
    3.  
    4.                 CameraParent.transform.localPosition = 0 - m_HitTransform.position * WorldScaleSlider.value;
    5.                 CameraParent.transform.localRotation = Quaternion.Inverse (m_HitTransform.rotation);
    6.  
    - You can change world scaling on the fly by moving the slider (and that simply scales the camera parent), you will need to tap the screen again to do another hit test with the new "world" scaling for the cube to appear again.

    Scene name is "ARKitWorldScaling"
     

    Attached Files:

  35. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    Wooooo! Thanks!
     
    MassiveTchnologies likes this.
  36. SkaZonic

    SkaZonic

    Joined:
    Jun 20, 2017
    Posts:
    23
    Hey guys so I tried building and running this project on unity to Xcode(https://github.com/MatthewHallberg/ARkitMinecraft) and it was successful. However once I launched the application on my iPhone 7 it crashes and closes. Afterwards I got these in the console in Unity...
    Screen Shot 2017-07-29 at 01.02.05.png
    Anyone know whats wrong with the project maybe or something I've done wrong/ not setup?? Any help will be greatly appreciated!

    p.s My setup is all updated -> iOS 11 beta 4, Xcode 9 beta 3, Unity 2017.1
    Thanks
     
  37. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    Hey,
    Do you have any output from the Xcode console?
     
  38. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    Hey,
    With the inspiration from @MassiveTchnologies, I've got a preliminary scene working with content scaling. I need to fix up the hit test stuff to account for the content scale. But once I get my example scene up and running I'll upload it to bitbucket and the asset store and notify y'all here.
     
  39. SkaZonic

    SkaZonic

    Joined:
    Jun 20, 2017
    Posts:
    23
    Screen Shot 2017-07-29 at 02.03.02.png
    Ok this is pretty much all I get on Xcode (I just built and ran it again, getting the usual thing(I know its generic iOS device, dw I know how to build arkit projects lol))
     
  40. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    Do you have a device to build to? You can’t run with “Generic Device” selected. You can click that and select the device you want to build to
     
  41. SkaZonic

    SkaZonic

    Joined:
    Jun 20, 2017
    Posts:
    23
    Ok no problem I just made a new project. P.s it wasn't the generic Device thing I just took a screenshot with that set and couldn't be bothered to take another screenshot. But again Thanks for the quick response! Also I really would want to see the CoreML implementation in Unity, that would be awesome! Keep up the great work over at Unity!
     
  42. User340

    User340

    Joined:
    Feb 28, 2007
    Posts:
    3,001
    Anyone having luck getting MobileARShadow shader to work? It's showing as a solid white plane for me.

    Edit: it is actually working fine on device, it is just glitchy in the editor. Works great!
     
    Last edited: Jul 30, 2017
  43. sama-van

    sama-van

    Joined:
    Jun 2, 2009
    Posts:
    1,733
    Having the following error importing the AR Kit asset from store :

    Unity 2017.1.0b5

     

    Attached Files:

  44. sama-van

    sama-van

    Joined:
    Jun 2, 2009
    Posts:
    1,733
    Finally fixed from code the 2 errors to the top, made a straight build and working a charm on iphone 7!

    Any possibility to drop every files from the Unity AR Kit plugin file into a single "_UnityARKit" folder to the next update?
    Would be super convenient to use this plugin into an existing project ^_^!...
     

    Attached Files:

  45. TimNedvyga

    TimNedvyga

    Joined:
    May 18, 2015
    Posts:
    95
    Have same problem with Unity 2017.1.0f3, AR Kit 1.0.5 and iPhone 7 Plus via cable.
    Any ideas to solve this?
     
  46. Arvin6

    Arvin6

    Joined:
    Jun 2, 2017
    Posts:
    27
    @christophergoy I'm trying to fine tune with with stereoscopic view, also posted here a while back. I believe mixed reality is possible with ARkit now (even if it has a little tracking issue when on a headset). Can you suggest a better way to add Stereo to current ARkit projects? I tried GVR plugin and it works but I'm looking for a solution that works on the camera level rather than rendering the main camera to a quad and displaying it with GVR camera. I also tried the Viewport mod on the main camera by adding two more cameras, the problem is one eye (camera) shows the ARTexture and other shows the virtual world. Is it possible to make this work or am I asking too much for now? Any input from your side would be very valuable for me.
     
  47. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    Hey, I believe that in order to use the ARRemote you need to use 2017.1. We should update the package to reflect that. My apologies.
    Cheers,
    Chris
     
    sama-van likes this.
  48. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    Hey,
    Sounds like a really cool idea. I don’t have an answer for you right now. I will inquire and see what we can come up with for you. In mean time, keep at it! Maybe your solution will be the one that’s used.
    Cheers,
    Chris
     
    Arvin6 likes this.
  49. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    Glad you got it working :). The directory restructure has been done if you look back through this thread. We are waiting on approval from the asset store, the bit bucket repo already reflects this.
    Cheers,
    Chris
     
    sama-van likes this.
  50. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    One issue that might crop up is that there is no way to render the video texture in a stereo way. I guess you could fake it some how, but your mileage may vary.
     
Thread Status:
Not open for further replies.