Search Unity

  1. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice
  2. Ever participated in one our Game Jams? Want pointers on your project? Our Evangelists will be available on Friday to give feedback. Come share your games with us!
    Dismiss Notice

ARKit support for iOS via Unity-ARKit-Plugin

Discussion in 'ARKit' started by jimmya, Jun 5, 2017.

  1. pgeorges-dpt

    pgeorges-dpt

    Joined:
    Apr 7, 2016
    Posts:
    43
    Found my problem. Integrated ARKit into an existing project that was using Natcam for camera support. Initializing Natcam was screwing up the ARKit camera matrix. Heads up.
     
  2. geoffa

    geoffa

    Joined:
    Apr 27, 2015
    Posts:
    2
    These are great! Would love to get insight around your scene and physics set up for the "minigame" with the catapults. Have been trying to get the physics system to work with smaller objects and haven't quite nailed the right approach there.
     
  3. joukev

    joukev

    Joined:
    Apr 22, 2013
    Posts:
    10
    Great to have ARKit working within Unity3D!
    I am trying to optimize out rendering performance, but the standard FPS prefab always indicates 60FPS when I run the example App on iPad (2017, no pro). Even when I notice the difference (lag time) when focusing on a large (>4M triangles) mesh.
     
  4. waldgeist

    waldgeist

    Joined:
    May 6, 2017
    Posts:
    70
    I am trying to get the samples running on my iPad. I'm using the latest Xcode build 9 beta 5, iOS 11 and Unity 2017.1.0f3. I built the Unity project for iOS successfully and was able to open it in Xcode. When I try to run the app on the device, however, both the app and Xcode crash. If I re-try, sometimes Xcode does not crash itself and shows a SIGABRT in line 419 of ARSessionNative.mm, in the method sessionDidUpdateFrame(). There's also an warning saying 'displayTransformwithViewportSize:eek:rientation' is deprecated on the same line. Any ideas how to get the sample running? I tried the same with Xcode 9 beta 3 a while ago, and there it worked. Unfortunately, Apple does only offer beta 5 now.
     

    Attached Files:

    jessevan likes this.
  5. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    556
    Hi,
    We need to update the plugin for Xcode/iOS Beta 5. This will be done later today. Sorry for the inconvenience.
    Cheers,
    Chris
     
  6. theiajsanchez

    theiajsanchez

    Joined:
    Feb 2, 2017
    Posts:
    18
    Hi everyone,

    Does someone know a way to reduce the quality of the image returned by the camera with ARKit?

    Thanks in advance
     
  7. GMM

    GMM

    Joined:
    Sep 24, 2012
    Posts:
    296
    Why exactly would you want to do that?
     
  8. JoeBibby

    JoeBibby

    Joined:
    Jun 13, 2017
    Posts:
    8
    I don't know if this is happening for everyone, but on my two dev phones the Unity ARKit plugin crashes on load on the new iOS 11 Beta 5. Happens even from the standard scene from the plugin in the asset store. About to try the bitbucket repo to see if it has a fix.


    Code (Boo):
    1. UnloadTime: 2.501125 ms
    2. 2017-08-07 18:01:18.861614-0500 arkitscene[315:13501] [MC] Reading from public effective user settings.
    3. 2017-08-07 18:01:19.371961-0500 arkitscene[315:13345] -[ARFrame displayTransformWithViewportSize:orientation:]: unrecognized selector sent to instance 0x1c02e9200
    4. 2017-08-07 18:01:19.377308-0500 arkitscene[315:13345] Uncaught exception: NSInvalidArgumentException: -[ARFrame displayTransformWithViewportSize:orientation:]: unrecognized selector sent to instance 0x1c02e9200
    5. (
    6.     0   CoreFoundation                      0x0000000185584d74 <redacted> + 148
    7.     1   libobjc.A.dylib                     0x0000000184a98528 objc_exception_throw + 56
    8.     2   CoreFoundation                      0x0000000185592218 <redacted> + 0
    9.     3   CoreFoundation                      0x000000018558a708 <redacted> + 1380
    10.     4   CoreFoundation                      0x000000018547035c _CF_forwarding_prep_0 + 92
    11.     5   arkitscene                          0x0000000103e9f980 -[UnityARSession session:didUpdateFrame:] + 756
    12.     6   ARKit                               0x00000001a00c7b54 <redacted> + 132
    13.     7   libdispatch.dylib                   0x0000000184f0946c <redacted> + 24
    14.     8   libdispatch.dylib                   0x0000000184f0942c <redacted> + 16
    15.     9   libdispatch.dylib                   0x0000000184f15e70 <redacted> + 1016
    16.     10  CoreFoundation                      0x000000018552d138 <redacted> + 12
    17.     11  CoreFoundation                      0x000000018552ad14 <redacted> + 2012
    18.     12  CoreFoundation                      0x000000018544b558 CFRunLoopRunSpecific + 436
    19.     13  GraphicsServices                    0x00000001872cef84 GSEventRunModal + 100
    20.     14  UIKit                               0x000000018f116984 UIApplicationMain + 208
    21.     15  arkitscene                          0x0000000103b051cc main + 172
    22.     16  libdyld.dylib                       0x0000000184f6ddb4 <redacted> + 4
    23. )
    24. 2017-08-07 18:01:19.381619-0500 arkitscene[315:13345] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[ARFrame displayTransformWithViewportSize:orientation:]: unrecognized selector sent to instance 0x1c02e9200'
    25. *** First throw call stack:
    26. (0x185584d5c 0x184a98528 0x185592218 0x18558a708 0x18547035c 0x103e9f980 0x1a00c7b54 0x184f0946c 0x184f0942c 0x184f15e70 0x18552d138 0x18552ad14 0x18544b558 0x1872cef84 0x18f116984 0x103b051cc 0x184f6ddb4)
    27. libc++abi.dylib: terminating with uncaught exception of type NSException
     
  9. JoeBibby

    JoeBibby

    Joined:
    Jun 13, 2017
    Posts:
    8
    Same issue using the version from the BitBucket repo. Time to roll back to beta 4...

     
  10. GMM

    GMM

    Joined:
    Sep 24, 2012
    Posts:
    296
    Thought i would post a little AR Kit showcase thingie since i got reminded about the use of realtime pseudo reflections in AR using the camera feed, so here is a very metallic statue in dark conditions and brighter conditions. As you can see the metallic surface adjusts nicely to the change in brightness, though i find materials like these extreme and the results would likely be better with a lesser reflective surface.




    Edit - Bonus image using relatively shiny wood:
    http://abload.de/img/img_012185s9u.png
     
    Last edited: Aug 8, 2017
    JoeBibby likes this.
  11. JoeBibby

    JoeBibby

    Joined:
    Jun 13, 2017
    Posts:
    8
    If I had read above I would have seen this is already a known issue. Sorry about that.
     
  12. Bird_LYKKE

    Bird_LYKKE

    Joined:
    Jul 26, 2017
    Posts:
    15
    I see some of demo when app cannot detect plane show custom message like "The room is to dark", "Find more space" it's a random message or UnityARKit can detect that?
     
  13. immersion

    immersion

    Joined:
    Oct 23, 2013
    Posts:
    2
    How do I access the timestamp in ARFrame in C#? I'm having issues figuring this out.
     
  14. smallbit

    smallbit

    Joined:
    Oct 3, 2013
    Posts:
    54
    Originally each brick was 2x1x1 meter, but I later scale the entire thing to 2%. The final mass is 0.2 kg. the ball is 8 kg. and the gravity for the project is set to -0.2 to give it this slow motion feeling
     
    Last edited: Aug 8, 2017
    geoffa likes this.
  15. Scale1Portal

    Scale1Portal

    Joined:
    Jan 27, 2015
    Posts:
    17
    This is a damn good plugin ! We are working on a ghost hunting device, see it in action here:



    I'm a bit stuck now with beta 5 - do you know any easy way to rollback to beta 4 on the device ? I can't find this release anymore in the iOS dev center.
     
    phits, mimminito, lourd and 1 other person like this.
  16. Scale1Portal

    Scale1Portal

    Joined:
    Jan 27, 2015
    Posts:
    17
    got a fix for this : in the file "ARSessionNative.mm" located in the Plugins/iOS/UnityARKit/NativeInterface, replace:

    Code (CSharp):
    1. s_CurAffineTransform = [frame displayTransformWithViewportSize:nativeSize orientation:[[UIApplication sharedApplication] statusBarOrientation]];
    by

    Code (CSharp):
    1. s_CurAffineTransform = [frame displayTransformForOrientation:[[UIApplication sharedApplication] statusBarOrientation] viewportSize:nativeSize];
    Works here at last, and will help while the official fix gets in !
     
    smallbit, phits, ina and 5 others like this.
  17. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    For the first, they are probably using https://developer.apple.com/documentation/arkit/arcamera.trackingstate.reason , which is returned via Unity ARKit Plugin, while for the second, they are probably checking the extents of the plane that is found and making sure its large enough for their content (again can be obtained by checking plane extents, which are received by plugin).
     
  18. JelmerV

    JelmerV

    Joined:
    Nov 29, 2012
    Posts:
    197
    For anyone having trouble with remote: for me it did not work with the asset store version, but it works with the bitbucket repo.

    Update on the reflections: this is a plane and a block hoovering above it.


    The main Camera is used to render the reflections, but still shows the solid background color instead of unity AR Video in reflections.

    I should probably change the 4th line to something else?
    Code (CSharp):
    1.            Camera reflectCamera = go.GetComponent<Camera>();
    2.  
    3.             reflectCamera.backgroundColor = clearColor;
    4.             reflectCamera.clearFlags = reflectSkybox ? CameraClearFlags.Skybox : CameraClearFlags.SolidColor;
     
    Last edited: Aug 8, 2017
  19. SoerenL

    SoerenL

    Joined:
    Mar 24, 2014
    Posts:
    12
    Hi, first good job on implementing this!

    2 questions:

    1. I want to place large GameObjects physically quite far from each other, think 20 meter large objects, 50 meters apart. Should I place an anchor at each of the GameObjects ? Or is 1 anchor enough ?

    2. Is there a way to lock those GameObjects in place, so if the user picks up the app a few days later, the GameObjects will be in the same spot ?

    Many thanks in advance!
     
  20. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    556
    If you copy the link for beta 5 but switch the 5s to 4s you should be able to download Xcode again
     
  21. theiajsanchez

    theiajsanchez

    Joined:
    Feb 2, 2017
    Posts:
    18
    I'm using the image returned by the camera for sending it through a high performance websocket and I need to find the best transfer/performance relation.

    Am I able to change this setting?

    Thanks
     
  22. JoeBibby

    JoeBibby

    Joined:
    Jun 13, 2017
    Posts:
    8
    This works for us as well. My partner found the same fix last night/early this morning. Thanks for posting!
     
    DerekLerner likes this.
  23. Grislymanor

    Grislymanor

    Joined:
    Aug 15, 2014
    Posts:
    23
    Did you ever figure out the problem with this? I have experienced something similar with additive loading of another scene - where the ARKit Scene's "Random cube" is the only thing properly synched with Pointcloud tracking, while the other objects were only bound by IMU and loosely stayed where they were originally placed. Also, I noticed that larger scenes needed more time for ARKit plugin to localize...
     
  24. DerekLerner

    DerekLerner

    Joined:
    Jun 12, 2017
    Posts:
    19
    Thank you! This worked for me as well.
     
  25. theiajsanchez

    theiajsanchez

    Joined:
    Feb 2, 2017
    Posts:
    18
    Answering my self: this could not be done by now as the following post says https://forums.developer.apple.com/thread/81301
     
  26. biscito

    biscito

    Joined:
    Apr 3, 2013
    Posts:
    86
    hi, maybe we've updated our iPhones to quit, but it's not working anymore... ios 11 beta 5
    any news
     
  27. septinite

    septinite

    Joined:
    Jul 18, 2017
    Posts:
    1
    Running into a small problem with the ARKitRemoteConnection on 2015 iPad Pro.

    Everything is as would be expected through launching the app and the connection, however, while the video feed shows up in Unity (albeit very laggy), the display on the device is still black with 'waiting for editor to connect'. Any ideas as to where I've gone wrong?
     
  28. waldgeist

    waldgeist

    Joined:
    May 6, 2017
    Posts:
    70
    @christophergoy:
    Thanks a lot for your very fast reply! Looking forward to the new version.
     
  29. DerekLerner

    DerekLerner

    Joined:
    Jun 12, 2017
    Posts:
    19
  30. czater

    czater

    Joined:
    Aug 15, 2015
    Posts:
    4
  31. JoeBibby

    JoeBibby

    Joined:
    Jun 13, 2017
    Posts:
    8
    What is the proper way to prepare the ARKit plugin before switching scenes? I need to be able to switch to and from an scene using the ARKit plugin. I can load the ARKit scene fine using SceneManager.LoadScene, but when I switch away from it the app crashes. Also, this only when on an iOS device, scene switching works fine in the editor. I can replicate this in my custom scenes as well as in the example scenes from the ARKit plugin.
     
  32. seon

    seon

    Joined:
    Jan 10, 2007
    Posts:
    1,441
    Hey Chris,
    Any word on this? Kinda stuck atm and don't want to downgrade back to iOS 11 beta 4.
    Thanks
     
  33. GMM

    GMM

    Joined:
    Sep 24, 2012
    Posts:
    296
    Likely an error where the UnityARVideo script can no longer reference the camera, you probably have to fix that yourself.
     
  34. phits

    phits

    Joined:
    Aug 31, 2010
    Posts:
    41
    Yes, that worked for me as well. Thank you!
     
  35. jessevan

    jessevan

    Joined:
    Jan 3, 2017
    Posts:
    35
    I just got word from Apple that they believe they have fixed it in beta 5. I'm in the process of upgrading all the things . . . so I haven't confirmed the fix yet.
     
    Last edited: Aug 9, 2017
    jimmya likes this.
  36. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    You should make all ARKit related gameobjects (most of the gameobjects in the original example scene) be "DontDestroyOnLoad", since these need to persist even when you change scenes.
     
  37. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    There is a temporary fix above: link.
     
    biscito likes this.
  38. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    If you look at how we send the video across from Unity ARKit Remote, you can see how we capture the video at full resolution into byte arrays. You can compress and/or scale it down those byte arrays (we currently don't do it, but it's probably something we could so to make the transfer faster).
     
    theiajsanchez likes this.
  39. esverin

    esverin

    Joined:
    Jul 3, 2017
    Posts:
    11
    Hey guys!

    Just out of curiosity. I'm making edits to the UnityARHitTestExample class and I'm trying to find a way that will rotate my object along the Y axis so it's always facing the camera, but i find no appropriate way to "hijack" the methods to place my rotation scripts. I've been playing around with Debug.Log but i can't figure out how to rotate the object at the same time as it's being translated on finger press.

    How did you guys approach it, if you did?
     
  40. biscito

    biscito

    Joined:
    Apr 3, 2013
    Posts:
    86
    thx
     
  41. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    In that script, we translate and rotate based on hittest result. You could create an gameobject at the hittest location that is scripted to always look at the camera using one of the solutions here. i.e. instead of the cube that exists in the scene, create a prefab that you can instantiate which has your rotation scripts on it.
     
  42. esverin

    esverin

    Joined:
    Jul 3, 2017
    Posts:
    11
    It´s not possible to make the rotation to be based on the camera´s position?

    The effect im kinda aiming for is to rotate the object only when i press the screen so a the object transforms to the hit test. Currently, it seems to only rotate the object to the origin position of when the original track was initiated
     
    Last edited: Aug 9, 2017
  43. PdrMnts

    PdrMnts

    Joined:
    Dec 12, 2016
    Posts:
    13
    I replaced that but I having other error...






    Using last version of Unity, Xcode Beta 3 and the last version of iOs11 (beta 5)
     
    sohamsardar and ina like this.
  44. Scale1Portal

    Scale1Portal

    Joined:
    Jan 27, 2015
    Posts:
    17
    You need XCode beta 5, which includes the latest ARKit SDK.
     
    ina likes this.
  45. PdrMnts

    PdrMnts

    Joined:
    Dec 12, 2016
    Posts:
    13
    Fixed. That was the problem. Thank you so much
     
  46. MSFX

    MSFX

    Joined:
    Sep 3, 2009
    Posts:
    115
    Other than obviously a potential higher rendering cost is there any reason to not set the camera far clipping plane to say 500?
     
  47. GMM

    GMM

    Joined:
    Sep 24, 2012
    Posts:
    296
    Shadow quality drops the wider the clipping range is.
     
  48. SkaZonic

    SkaZonic

    Joined:
    Jun 20, 2017
    Posts:
    23
    Hi all, so I read a post on anchoring GameObjects and to be able to see the game objects in the same position when you reopen the app. However your device has to be at the exact location you closed the app, this leads me onto my question I realised that if you completely close the app so its not running anymore, is it possible to save this data and the next time you open the app in the same position you will be able to see those GameObjects anchored? And if so this is possible how would you implement to have this saved data in the next session?
     
  49. Grislymanor

    Grislymanor

    Joined:
    Aug 15, 2014
    Posts:
    23
    Hello Guys,

    We love Unity's ARKit implementation! That being said, we anticipate releasing a game this September/October and wanted to ask/bring up the device heating up issue...

    We are concerned that our characters (2500 polis/tris) which will eventually become fairly numerous (up to 30) will limit game play time - i.e. older phones will heat upend must be shut down. We've talked to other teams that (using different AR methods the ARKit) melted processors doing SLAM (3 years ago on a pre iPad Air 2). We obviously don't want to damage players devices (Yikes)! I am willing to add a method that times out the game with a friendly UI (unlike the shut down I've experienced with other IOS products) based on sustained performance or, better yet, high temperature readings.

    (BTW: we are testing on iPhone 6s')

    Question #1: Does anybody know of such a method that we can access in Unity to queue player to "take a break" and let their device cool down?

    Question #2: We should be going with Unity's mobile shaders all the way right (this is our first metal project) or are their more optimized "Metal Unity Shaders"?

    I realize this is a little early in ARKit's development... Thanks!
     
  50. Elliott-Mitchell

    Elliott-Mitchell

    Joined:
    Oct 8, 2015
    Posts:
    58
    Looking for quick sanity check; does this look like it's including all compatible iOS devices which support ARKit?
    Code (CSharp):
    1.     public void CheckDeviceType(){
    2.  
    3.         if(Device.generation == DeviceGeneration.iPhoneSE1Gen){
    4.             DeviceIsCompatible = true;
    5.         }
    6.  
    7.         else if(Device.generation == DeviceGeneration.iPhone6S){
    8.             DeviceIsCompatible = true;
    9.         }
    10.  
    11.         else if(Device.generation == DeviceGeneration.iPhone6SPlus){
    12.             DeviceIsCompatible = true;
    13.         }
    14.  
    15.         else if(Device.generation == DeviceGeneration.iPhone7){
    16.             DeviceIsCompatible = true;
    17.         }
    18.  
    19.         else if(Device.generation == DeviceGeneration.iPhone7Plus){
    20.             DeviceIsCompatible = true;
    21.         }
    22.  
    23.         else if(Device.generation == DeviceGeneration.iPadPro1Gen){
    24.             DeviceIsCompatible = true;
    25.         }
    26.  
    27.         else if(Device.generation == DeviceGeneration.iPadPro10Inch1Gen){
    28.             DeviceIsCompatible = true;
    29.         }
    30.  
    31.         else if(Device.generation == DeviceGeneration.iPad5Gen){
    32.             DeviceIsCompatible = true;
    33.         }
    34.  
    35.         else {
    36.             DeviceIsCompatible = false;
    37.         }
    38.  
    39.         Debug.Log ("iOS Device = " + Device.generation.ToString());
    40.         iOSSystem = Device.generation.ToString();
    41.     }
    42.  
     
    JelmerV and mimminito like this.
unityunity