Search Unity

  1. Engage, network and learn at Unite Austin 2017, Oct 3 - 5. Get your ticket today!
    Dismiss Notice
  2. Introducing the Unity Essentials Packs! Find out more.
    Dismiss Notice
  3. Check out all the fixes for 5.6 on the patch releases page.
    Dismiss Notice
  4. Unity 2017.1 is now released.
    Dismiss Notice
  5. Unity 2017.2 beta is now available for download.
    Dismiss Notice

ARKit support for iOS via Unity-ARKit-Plugin

Discussion in 'ARKit Preview' started by jimmya, Jun 5, 2017.

  1. DigitalBeach

    DigitalBeach

    Joined:
    Jan 17, 2015
    Posts:
    8
    I think that was it. Adding the tag and reloading the scene made it behave more like I would expect. For anyone else following along, I integrated the plugin code changes listed above. Copied over the project from branch, and added the tag in from the branch.
     
    christophergoy likes this.
  2. dreamplanet

    dreamplanet

    Joined:
    Jun 17, 2017
    Posts:
    7
    Chris is right on this one, Zyshaq .. I couldn't get the above to work. It seemed to just scale the particles (rather than the terrain and all the models) and then everything would just rapidly swim left (or right) with subsequent zoom. His multicam approach is the right one .. it's a bit confusing, but to anyone out there facing this same scaling challenge for tabletop dev - Christopher's multicam branch is the way to go for this problem.
    Christopher, I downloaded your branch and have spent many hours with it .. thank you so much for tackling the problem. Yours is the best approach to solving it, I believe .. so I'm hoping the branch will find its way into the main build soon, as this will be a very common need/problem with so many devs trying to scale cities down into tabletop experiences.

    Still, I haven't been able to get it to work perfectly (yet). When highly scaled, I can get a lot of drift, which only gets worse and starts to swim. I'm scaling at 0.001x so perhaps moving the virtual cam so far out is what makes it shaky? I know this is one's experience with binoculars - would the same hold true for virtual distance causing increasing levels of inaccuracy?

    I am also getting two renderings from time to time, one slightly beneath the other, which only really become noticeable after scaling up and down a bunch. If you were turning off the camera in your example, I wasn't able to see where. I tried using clipping planes on one camera and then tried removing a light source for one camera, but no joy. The primary difference between the two cameras in your example seems to be that one has a Depth of 1, while the other has a Depth of -1. If this is why I am seeing one rendering behind the other, I suppose I can change it to a smaller gap between, like 0.1 and -0.1. Would that work? Is there a more elegant (and less CPU-heavy) way to do this, like having one camera not render at all? Thanks again, your code has been hugely helpful.
     
  3. Arvin6

    Arvin6

    Joined:
    Jun 2, 2017
    Posts:
    27
    Oh that was the cardboard profile. Thanks for your input man :) Keep us posted if you get to solve the drift issue. I'm looking into it too. It would be better if VRSettings has api to access FOV and other settings.
     
  4. mibande12

    mibande12

    Joined:
    Sep 14, 2017
    Posts:
    1
    I encountered this problem too, got it working for me. Go to Unity/ARTest/ARDemo2/Libraries/Plugins/iOS/UnityARKit/NativeInterface/ARSessionNative.mm in your Xcode Project, copy and paste the code from this repository 'https://bitbucket.org/Unity-Technol...NativeInterface/ARSessionNative.mm?at=default', Go into build settings and make Swift use version 4.0 if it isn't already, as this is Swift Code you're adding into your project. Then run your project, and hopefully it'll come out with no errors.
     
    Last edited: Sep 14, 2017
  5. mm_sto

    mm_sto

    Joined:
    Aug 28, 2017
    Posts:
    5
    Has anyone tried applying the YUVShader to some sort of cubemap or reflection probe?

    Kinda want to try it for custom "real" reflections
     
  6. StefanDjordjevic

    StefanDjordjevic

    Joined:
    Sep 14, 2017
    Posts:
    1
    Hello!
    I'm wondering if there is any progress in investigating this issue. I'm currently working with the ARKit and I managed to switch scenes with the dontdestroyonload() and setting them inactive during the other scene (put all the AR objects in a parent object to make it easier to activate and deactivate). I can't find anything to help me with my problem. I found this post but could not find any follow up posts.
    thanks! :D

    [EDIT]
    Nevermind me! I'm an idiot! Thanks anyways!
     
    Last edited: Sep 14, 2017
  7. Arvin6

    Arvin6

    Joined:
    Jun 2, 2017
    Posts:
    27
    Hi @christophergoy , If you have seen the earlier posts. We have used Unity native VR to get it working with ARKit but there's a lot of drift on the detected planes and the object. It seems to move around with the user if you see the attached video. Is this a bug on ARkit / GVR? Why is the plane and object moving ? It doesn't seem to be a FOV issue to me, is the GVR messing with ARkit camera? Why is it not possible to render the camera feed in a stereo way? For now, I'm sticking to rendertexture but the quality is too bad that way.
     
    Last edited: Sep 14, 2017
  8. skdev3

    skdev3

    Joined:
    Jul 15, 2015
    Posts:
    19
    I have the same problem. Arkit/Unity team, how to solve this problem? Without creating an additional camera and render to texture...
     
  9. aas395

    aas395

    Joined:
    Aug 26, 2017
    Posts:
    4
    I have "Development Build" checked... is that what you mean?
     
  10. WorrellAR

    WorrellAR

    Joined:
    Jul 20, 2017
    Posts:
    2
    I'm trying to build any example scene from Unity ARKit. I've already redownloaded everything and rebuilt it as suggested from previous posts. However, I'm still getting the same errors:

    Screen Shot 2017-09-14 at 10.27.21 AM.png

    From console, we have:

    Undefined symbols for architecture arm64:

    "_OBJC_CLASS_$_ARWorldTrackingConfiguration", referenced from:

    objc-class-ref in ARSessionNative.o

    "_OBJC_CLASS_$_AROrientationTrackingConfiguration", referenced from:

    objc-class-ref in ARSessionNative.o

    "_OBJC_CLASS_$_ARAnchor", referenced from:

    objc-class-ref in ARSessionNative.o

    "_OBJC_CLASS_$_ARPlaneAnchor", referenced from:

    objc-class-ref in ARSessionNative.o

    "_OBJC_CLASS_$_ARSession", referenced from:

    objc-class-ref in ARSessionNative.o

    ld: symbol(s) not found for architecture arm64

    clang: error: linker command failed with exit code 1 (use -v to see invocation)

    Using:
    • Unity ARKit - Latest imported from Asset Store
    • Unity version 2017.1.0f3
    • Build architecture: Universal
    • Xcode: Version 9.0 beta 6 (9M214v)
    • Deployment Target: 8.0
    Any suggestions would be appreciated. Thanks!
     
  11. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    461
    Make sure the ARKit.framework is linked in your projected. You can find it by clicking on the project (the blue file) in the project navigator, going to the general tab, and looking for "Linked Libraries".
     
  12. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    461
    Are you messing with the projection matrix of the cameras? If yes, then this would definitely cause drift. I have no idea what your code looks like so I can't give you a solution. Do you have anything you can share?
     
    Arvin6 likes this.
  13. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    461
    Yes
     
  14. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    461
    The errors you are seeing mean you have an old version of the Xcode beta and need to update.
     
  15. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    461
    Hey, thanks for your detailed post. I would love to tinker with you if you'd be willing to share you project. I am making a more elaborate example in that branch as we speak, though the code will remain the same. If you could direct message me, I'd be interested in collaborating on this solution with you.

    I think some of your intuition is correct about the shakiness of the camera. Though that normally happens when you zoom in, as opposed to out. Is the shakiness from ARKit or just from rendering? This is where sharing your project would help :)

    The depth of the cameras determine the rendering order. I'm not sure making the gap smaller would change much of anything.

    If you don't care about rendering the tracking data you can turn that off.

    I agree that the technique used is not the most intuitive, and on top of that, it has it's own drawbacks. Maybe we could start a thread to come up with a solution to scaling content. Either tweaking the 2 camera approach, or coming up with a new one.
    Cheers,
    Chris
     
  16. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    461
    If you read the answer from Apple staff on the first forum post, you see that this is by design.
     
    UCS likes this.
  17. DigitalBeach

    DigitalBeach

    Joined:
    Jan 17, 2015
    Posts:
    8
    Christopher, I got the scale working well. But one thing that is broken is that raycasts from the camera don't hit the scene now. When attached to the Tracking Camera, the scale was off. So attached to the content camera and it works if I don't move the camera orientation from the initial spot where I spawn the item but not if it has moved much. No idea why that would be the case.
     
  18. aas395

    aas395

    Joined:
    Aug 26, 2017
    Posts:
    4
    Thank you for clarifying. I double-checked that this option was checked and then re-built the project and played the scene in the editor. In the game view in Unity, clicking "Start Remote ARKit Session" still does nothing.
     
  19. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    461
    Yes, you need to recast from the content camera. I'm not sure I understand why it wouldn't work after moving around though. Can you elaborate more?
     
  20. WorrellAR

    WorrellAR

    Joined:
    Jul 20, 2017
    Posts:
    2
    It's already linked. Still same errors.
     

    Attached Files:

  21. jameshurlbut

    jameshurlbut

    Joined:
    Nov 11, 2016
    Posts:
    1
    Is there a way to get the postprocessing stack working with ARKit? When I try to combine them I get a stuttering of the background video. thanks
     
  22. DigitalBeach

    DigitalBeach

    Joined:
    Jan 17, 2015
    Posts:
    8
    I figured it out. It works for the ones I am doing manually. But we were using some GraphicRaycaster components to handle UI. That defaults to MainCamera and I don't see how to override since the eventCamera is private. I guess doing that stuff differently is the best idea.
     
    christophergoy likes this.
  23. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    461
    hmm, would you be willing to share your xcode project? I'm not sure why this would be happening based on the information you are giving me. Could you try to unlink, and relink the framework? Do you have multiple versions of Xcode on your computer?
     
  24. bharath989

    bharath989

    Joined:
    Dec 1, 2016
    Posts:
    1
    @christophergoy . When switching between ARkit Scenes in Unity. I face the following error and the app is crashed. I am struck here from many days . To my knowledge I have not missed any UI references in the scene..Thank you in advance
    .i

    SPECS - Unity 2017.1.0f3 , XCode 9- Beta 6 , IOS 11 (Iphone 6S)


    Insode

    UnityEngine.XR.iOS.UnityARHitTestExample:Update()



    (Filename: /Users/builduser/buildslave/unity/build/artifacts/generated/common/runtime/DebugBindings.gen.cpp Line: 51)



    Insode

    UnityEngine.XR.iOS.UnityARHitTestExample:Update()



    (Filename: /Users/builduser/buildslave/unity/build/artifacts/generated/common/runtime/DebugBindings.gen.cpp Line: 51)



    Insode

    UnityEngine.XR.iOS.UnityARHitTestExample:Update()



    (Filename: /Users/builduser/buildslave/unity/build/artifacts/generated/common/runtime/DebugBindings.gen.cpp Line: 51)



    Insode

    UnityEngine.XR.iOS.UnityARHitTestExample:Update()



    (Filename: /Users/builduser/buildslave/unity/build/artifacts/generated/common/runtime/DebugBindings.gen.cpp Line: 51)



    Insode

    UnityEngine.XR.iOS.UnityARHitTestExample:Update()



    (Filename: /Users/builduser/buildslave/unity/build/artifacts/generated/common/runtime/DebugBindings.gen.cpp Line: 51)



    Unloading 6 Unused Serialized files (Serialized files now loaded: 0)

    UnloadTime: 2.588333 ms



    Unloading 3770 unused Assets to reduce memory usage. Loaded Objects now: 797.

    Total: 3.914958 ms (FindLiveObjects: 0.308666 ms CreateObjectMapping: 0.681666 ms MarkObjects: 0.660416 ms DeleteObjects: 2.263291 ms)



    libc++abi.dylib: terminating with uncaught exception of type Il2CppExceptionWrapper

    (lldb)
     
    Last edited: Sep 15, 2017
  25. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    461
    Without knowing your code I can’t help you with the information you have given. Please attach Xcode logs and device crash logs as text files. Please also read the FAQ thread to see how to properly form a question if you are having an issue. There is a lot of helpful information there.
     
  26. Arvin6

    Arvin6

    Joined:
    Jun 2, 2017
    Posts:
    27
    I'm not changing anything. That was just the demo UnityARkitScene with Unity native VR enabled. The only coding there is to switch to VR mode with the VRSettings api (`VRSettings.enabled = true`). This seems to be a bug with ARKit / GVR which messes with the projection matrix on stereo mode. When I switch it back from VR mode i.e, `VRSettings.enabled=false`, it is back to normal. I can attach the project if you want me to, but it is just the demoscene with Virtual Reality enabled. Should I file a bug report and send you the reference ID?
     
  27. skdev3

    skdev3

    Joined:
    Jul 15, 2015
    Posts:
    19
    Not changing anything, use Unity ARKit Plugin and just added:
    1. Virtual Reality Supported (UnityEngine.VR).
    To enable native VR support for your game builds and the Editor, open the Player Settings (menu: Edit > Project Settings > Player). Select Settings and check the Virtual Reality Supported checkbox. Set this for each build target.
    First VR device - None ( Standart camera mode )
    Second VR device - Cardboard ( Unity native Cardboard camera mode )
    [​IMG]
    2. This code for enable VR mode (Add to any object on scene),
    Code (csharp):
    1.  
    2. using UnityEngine;
    3. using System.Collections;
    4. using System;
    5. using UnityEngine.VR;
    6.  
    7. public class VREnable : MonoBehaviour {
    8.     bool isEnableVR = false;
    9.     bool isVRinProgres = false;
    10.     void OnGUI()
    11.     {
    12.         if (!isVRinProgres && GUI.Button(new Rect((Screen.width) * 0.1f, (Screen.height) * 0.1f, (Screen.width) * 0.1f, (Screen.height) * 0.1f), "VR:" + isEnableVR.ToString()))
    13.         {
    14.             isVRinProgres = true;
    15.             isEnableVR = !isEnableVR;
    16.             SetupVR(isEnableVR);
    17.         }        
    18.     }
    19.  
    20.     /// ////////////////  VR CODE here
    21.  
    22.     public void SetupVR(bool isVR)
    23.     {
    24.         Camera[] camers = (Camera[])GameObject.FindObjectsOfType<Camera>();
    25.         StartCoroutine(LoadDevice(isVR, camers));
    26.     }
    27.  
    28.     IEnumerator LoadDevice(bool isVR, Camera[] camers)
    29.     {
    30.         // VRDeviceName none
    31.         string newDevice = VRSettings.supportedDevices.Length > 0 ? VRSettings.supportedDevices[0] : "none";
    32.         if (isVR)
    33.         {
    34.             //VRDeviceName Cardboard or Split
    35. #if UNITY_EDITOR
    36.             newDevice = "Split";
    37. #else
    38.             newDevice = VRSettings.supportedDevices.Length > 1 ? VRSettings.supportedDevices[1] : "none";
    39. #endif
    40.         }
    41.         Debug.Log("VRDeviceName " + newDevice);
    42.         VRSettings.LoadDeviceByName(newDevice);
    43.         yield return null;
    44.         Debug.Log(VRSettings.loadedDeviceName + " is loaded");
    45. #if !UNITY_EDITOR
    46.         foreach (Camera cam in camers)
    47.             {
    48.                 if (cam != null)
    49.                 {
    50.                     VRDevice.DisableAutoVRCameraTracking(cam, isVR);
    51.                 }
    52.             }
    53.         if(Camera.main)
    54.         VRDevice.DisableAutoVRCameraTracking(Camera.main, isVR);
    55. #endif
    56.         VRSettings.enabled = isVR;
    57.         isVRinProgres = false;
    58.     }
    59.     /// ////////////////  VR CODE End
    60. }
    61.  
    62.  
    And I have This ARkit bug to ((( :


    Native Unity VR support does not Work with ARkit camera. When this bug fix in Unity ARKit Plugin?
     
    Last edited: Sep 15, 2017
  28. Elliott-Mitchell

    Elliott-Mitchell

    Joined:
    Oct 8, 2015
    Posts:
    21
    I offset the camera parent in z to deal with this. works well for me.
     
  29. Elliott-Mitchell

    Elliott-Mitchell

    Joined:
    Oct 8, 2015
    Posts:
    21
    Deployment target should be 11 (right?)
     
  30. Dambusters

    Dambusters

    Joined:
    Jan 6, 2012
    Posts:
    44
    Hi @christophergoy,

    (I previously asked this on the new "ARKit Preview" thread but it looks like the discussions are continuing here for the moment so I'm reposting it - sorry about the double post if the threads get merged)

    Updated to the latest iOS11, ARKit (from bitbucket), Xcode 9 & Unity 2017.1.1f.
    The video in the ARKit now appears to be cropped and repeated when using an iPad Pro - both in a build and in the remote. Up until now it's all been working fine. Any thoughts on how to fix it?

    Bruce

    [​IMG]
     
  31. Elliott-Mitchell

    Elliott-Mitchell

    Joined:
    Oct 8, 2015
    Posts:
    21
    I got it all working!

    It was a bit challenging because the project was vanilla iOS and crazy SteamVR (which now supports ARKit) - a lot of moving parts :p

    The repo's example was a good start but it took a bit more work to figure out how to implement in my case.

    Thanks!
     
  32. skdev3

    skdev3

    Joined:
    Jul 15, 2015
    Posts:
    19
    You use lastest version ARkit?
    https://bitbucket.org/Unity-Technol...mits/0d2d903745741fad9c5ce49ef2dcc12df19ae9c7
    this bug fixed yesterday on bitbucket repo.
     
  33. Elliott-Mitchell

    Elliott-Mitchell

    Joined:
    Oct 8, 2015
    Posts:
    21
    I personally merged the ScaledContent branch on my end. it wasn't difficult to do. Hopefully someone officially does the merge :)

    Result: A 3 story hive scaled to a ~less than a meter tall; technically dozens of meters away, but now appears to be less than a meter away! Previously, the hive did look small and one could rotate away from it. The problem was, it never appeared to change scale if one walked towards or away from it. One could never walk around the hive.

    [​IMG]
     
    Last edited: Sep 15, 2017
    jimmya and christophergoy like this.
  34. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    461
    I've merged default into the scaled-content-test branch and pushed it.
    Glad to see you've got your content to work with the scaled content branch :). Were the changes you made specific to your use case, or could be a general change that everyone benefits from?
     
    Elliott-Mitchell likes this.
  35. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    461
    I'll take a look. We made some changes to make it work with the iPhone X, but maybe it broke other devices. If you go to the previous version of the YUVShader.shader, it should work for your iPad pro again. We will have the universal fix soon.
     
  36. Elliott-Mitchell

    Elliott-Mitchell

    Joined:
    Oct 8, 2015
    Posts:
    21
    Thanks for the merge! Some changes I made were specific to my case.

    I added a controller to the parent of the scaled content camera to offset the camera's parent tranforms to account for differences in rendering for iPad Pro and iPhone; it also accounts for if the scene is rotated 180 from the world.

    I added to the UnityARCameraManager.cs to reset camera orientation at the app start, and at any time the player needs to.

    If people want to see my code, I'll gladly share :)
     
  37. raoul_95

    raoul_95

    Joined:
    Jul 4, 2017
    Posts:
    1
    how to get the AR texture ?
     
  38. Elliott-Mitchell

    Elliott-Mitchell

    Joined:
    Oct 8, 2015
    Posts:
    21
    We submitted our game to the App Store and were accepted!

    My concern is: I didn't set UIRequiredDeviceCapabilities in Xcode to require ARKit (so non-ARKit supported devices may run the game). Will this prevent us from being listed as an ARKit game on the App Store? If so, I should probably resubmit!
     
  39. AxelMG

    AxelMG

    Joined:
    Jul 4, 2012
    Posts:
    8
    Hey Stepan! I saw u were very interested on this topic, so am I :) Do you know if there has been any progress on that regard? Bests!
     
  40. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    461
    It has ARKit so you might still get featured, but users might give you 1 star if their device doesn’t support it and they download your game. I’d recommend a resubmit, but it’s your call. Congrats!
     
    DerekLerner likes this.
  41. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    461
    I don’t understand your question
     
  42. Elliott-Mitchell

    Elliott-Mitchell

    Joined:
    Oct 8, 2015
    Posts:
    21
    OH! I never thought of that! GOOD POINT! I'm going to resubmit.

    Thanks! Without you all, we would not be on the App Store at all <3!

    FWIW, we'll be demoing at the Unite Austin Showcase. Stop by, I'd love to meet all of you on this forum! I sense celebratory beers at Unite as well!
     
    jimmya and christophergoy like this.
  43. RentificUnity

    RentificUnity

    Joined:
    Aug 11, 2017
    Posts:
    3
    Quick question..

    What was changed from the earlier UnityARKitPlugincode.. say early to mid July?

    When I downloaded the plugin in mid July it ran out of the box on my iPad Pro no problems.
    When I download and try to build the UnityARKit Scene in a fresh Unity Project (2017.1.1) I get the blue screen and "ARKit unable to run the session, configuration not supported on this device" feedback on Xcode. Shouldn't it work out of the box on an iPad pro?

    Earlier ARkit plugin worked perfect and ran perfect on my iPad. With Xcode 9 final seed I started to get a ton of errors with the ARSessionNative settings..

    Many thanks in advance!
     
  44. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    461
    What errors are you seeing? What ar configuration are you using? Which generation of iPad Pro do you have? What iOS version is installed on your device and which beta is it? All of this information helps us help you. Please include it. Otherwise I have no idea what your problem is based on the information you’ve posted.
     
  45. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    461
    I wish I could but I won’t be there. Have one for me! Cheers!
     
  46. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    377
    Also, if you have an app that supports both AR and non-AR mode for older devices, you should be using the IsSupported on a ARWorldTrackingSessionConfiguration to find out whether that configuration will be supported on that device at runtime.

    I'll look you up at the showcase for sure. Good luck!
     
    DerekLerner likes this.
  47. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    377
    You have some problem with your XCode setup. Check that you're building for a device and not simulator. You can try getting XCode 9 GM version.
     
  48. haydex

    haydex

    Joined:
    Jun 16, 2017
    Posts:
    6
    I have a green screen only, I have done all the steps required, the remote app build is for development, I have XCode 9 GM and iOS 11 GM, and the latest build of unity arkit plugin
     
  49. zacware

    zacware

    Joined:
    Mar 3, 2014
    Posts:
    5
    Did you drag the ARRemoteConnection prefab into your scene?

     
  50. haydex

    haydex

    Joined:
    Jun 16, 2017
    Posts:
    6
    I am using the EditorTestScene.unity and it has ARRemoteConnection