Search Unity

  1. Read here to learn more about our latest preview of the XR Interaction Toolkit.
    Dismiss Notice
  2. Click here to receive a gift with your purchase of Unity Pro or Unity Enterprise.
    Dismiss Notice
  3. Read here for Unity's latest plans on OpenXR.
    Dismiss Notice

ARKit support for iOS via Unity-ARKit-Plugin

Discussion in 'AR' started by jimmya, Jun 5, 2017.

  1. JoMaHo

    JoMaHo

    Joined:
    Apr 2, 2017
    Posts:
    94
    Been working on an ARKit app for a few months, and suddenly (after updating to iOS12 and building to Xcode10) the AR camera renders some of the bottom at the top, and parts of the top at the bottom of the screen as seen on the picture below.

    The UI is positioned just fine, but the AR camera feed from has som weird issues.

    Any suggestions?
     
    unnanego likes this.
  2. todds_unity

    todds_unity

    Unity Technologies

    Joined:
    Aug 1, 2018
    Posts:
    168
    Hi @JoMaHo ,

    I have not seen this behavior. Do you have a Unity project that you can share that reproduces this issue?

    Also, on what type of device are you running?

    Todd
     
    Last edited: Sep 24, 2018
  3. JoMaHo

    JoMaHo

    Joined:
    Apr 2, 2017
    Posts:
    94
    Hi!
    Unity 2017.2.2, Xcode 10.1 betal, iOS 12.1 on an iPhone 7.

    I could just send you a stripped project in an PM for you to test?

    best
     
  4. JoMaHo

    JoMaHo

    Joined:
    Apr 2, 2017
    Posts:
    94
    Sent you a pm with download link for test file. Hope you can find anything,.

    best
     
  5. todds_unity

    todds_unity

    Unity Technologies

    Joined:
    Aug 1, 2018
    Posts:
    168
    Hi @JoMaHo ,

    I was able to find the source of your video issue and the proper fixes that address the problem.

    I will start off by saying that your version of the Unity ARKit plugin code is quite old (11-12 months). Ideally, you should update this to the latest code in the "default" branch of the repository. Though doing so would require some time, it could be beneficial with numerous other fixes as well as additional ARKit functionality. The latest in "default" exposes ARKit 2.0 functionality into Unity.

    That said ... to fix your camera issue specifically, you need to merge in at least the following 4 changesets. The changes in this 4 changesets will address your video issue.

    https://bitbucket.org/Unity-Technol...mits/83d9dfe1c211bae9eaaf537b08976d56fec820e8
    https://bitbucket.org/Unity-Technol...mits/3f2853dfcdbbec044545617780d5237c114baec4
    https://bitbucket.org/Unity-Technol...mits/44be34bfd07a9dab138b5d43834858d11a499f42
    https://bitbucket.org/Unity-Technol...mits/b0355f702c426cb285be4d0d55c1bca8f485479c


    Todd
     
    adammpolak likes this.
  6. unnanego

    unnanego

    Joined:
    May 8, 2018
    Posts:
    154
    I'm using the latest unity and the latest plugin from the default branch and the remote app is very very slow. Is there a fix? Device is 8+
     
  7. JoMaHo

    JoMaHo

    Joined:
    Apr 2, 2017
    Posts:
    94
    Hi!
    That was a massive thing; is all this because of iOS12?

    Thank you for solving it!

    best
    Johnny
     
    Last edited: Sep 28, 2018
  8. bluemode

    bluemode

    Joined:
    Jun 1, 2017
    Posts:
    2
    Hi, I'm using the ARKit 2 plugin for image tracking, however when the image is lost, the prefab that is generated stays on screen instead of disappearing. Is this the default behaviour? And if so, is there a way to make the model disappear once the image is lost?

    Thank you
     
  9. todds_unity

    todds_unity

    Unity Technologies

    Joined:
    Aug 1, 2018
    Posts:
    168
    Hi @bluemode ,

    Take a look at the changes that are in the "image-anchors/adding-istracked-field-to-image-anchors" branch.
    https://bitbucket.org/Unity-Technologies/unity-arkit-plugin/src/default/

    Todd
     
  10. nrorn

    nrorn

    Joined:
    Sep 23, 2016
    Posts:
    5
    Does anyone know if the autofocus speed can be increased?

    I am using Vision's native barcode recognition in combination with ARKit, but autofocus speed is relatively slow.
    Any thought on how to increase it is welcome.
     
  11. JelmerV

    JelmerV

    Joined:
    Nov 29, 2012
    Posts:
    219
    Can someone describe how to work with Environment Probes?
    I'm want one probe, that updates itself.
    The DebugReflectionProbePrefab doesn't upate itself, correct? (reflections stay grey on my iphone8)
     
  12. todds_unity

    todds_unity

    Unity Technologies

    Joined:
    Aug 1, 2018
    Posts:
    168
    Hi @JelmerV ,

    ARKit environment probe functionality is only enabled in iOS 12. What version of iOS are you running?

    Have you tried the scene in Assets/UnityARKitPlugin/Examples/ARKit2.0/UnityAREnvironmentTexture/UnityAREnvironmentProbeAnchorManual.unity ?

    When I run this example scene, I am able to tap on planes in the AR scene to place environment probe anchors represented by spheres. See the attached image.

    Todd
    arkit-plugin-env-probe.png
     
  13. pacepiro

    pacepiro

    Joined:
    Jun 1, 2016
    Posts:
    4
  14. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    Just copy the UnityARKitPlugin folder under Assets in the repo to your Assets folder in your project. You may have to adjust your ProjectSettings for some of the advanced features, if so, check the differences between your ProjectSettings folder and the one in the repo.
     
  15. djgrahamj

    djgrahamj

    Joined:
    Oct 15, 2018
    Posts:
    3
    Well that doesn't sound good.
     
  16. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    What doesn't? Having to mess with the Project settings? If you didn't change the ProjectSettings in your project, you can just take the one that is distributed with the plugin. Otherwise you will need to merge in your changes from the plugin, whether you got it from Bitbucket or Asset Store or anywhere else.
     
  17. f-win

    f-win

    Joined:
    Jul 28, 2016
    Posts:
    3
     
  18. f-win

    f-win

    Joined:
    Jul 28, 2016
    Posts:
    3
    I am seeing the same exception. Does anybody have a suggestion how to debug this? There are no call stacks. Thanks!
     
  19. jdgauchat

    jdgauchat

    Joined:
    Jun 24, 2017
    Posts:
    18
    Hi, guys. I have two questions.

    Why the asset was removed from the asset store? Bitbucket is now the only way to install it?
    Why the camera on iPad is zoomed in? Is this expected or is it a bug?

    I posted a message about the camera on iPad and I opened a support ticket but I didn't get any answer. I guess the support guy thought I was asking how to fix my code.

    I posted some pictures already if you want to see what the camera looks like on an iPad 2017 in ARKit.
    https://forum.unity.com/threads/ipad-camera-zoomed-in.569497/

    Thanks
    JD
     
  20. djgrahamj

    djgrahamj

    Joined:
    Oct 15, 2018
    Posts:
    3
    I meant that it wasn't available in the Unity Store anymore. At that time I happened to not be able to get it from BitBucket either so I thought maybe something had happened and you had to cancel the project or something.

    I'm glad that's not the case!
     
    lumeneo likes this.
  21. djgrahamj

    djgrahamj

    Joined:
    Oct 15, 2018
    Posts:
    3
    While I'm here, I have a question too. I'm finding that all anchors created by ARUnityGeneratePlane have the same Y value of zero. If I scan the device over the floor and then a table I would expect that planes would be generated with different Y values. Am I wrong about that?
     
  22. charutak

    charutak

    Joined:
    Jan 6, 2016
    Posts:
    1
    Hey, though it is a bit late, it might help someone. So, This worked perfectly for me for the scale issue.
    https://blogs.unity3d.com/2017/11/16/dealing-with-scale-in-ar/

    There is a video explaining this here:


    Since, I use the mixture of scale method and you never really change the scale of the objects, the physics works great!
     
  23. Patrk

    Patrk

    Joined:
    Apr 9, 2017
    Posts:
    48
    Why is the ARKit plugin gone from the Unity assets store? will it still be supported on bitbucket in the future? or is it to get people to use AR Foundation?
     
    Last edited: Oct 18, 2018
    robhennigar likes this.
  24. JelmerV

    JelmerV

    Joined:
    Nov 29, 2012
    Posts:
    219
    Hey,

    I'm trying to render a second ortographic camera to show some 3D spheres in my UI. Whether I pick clearflags Don't clear or Depth only, I get really weird artifacts in the part that should let through the ARKit video feed.
    Any ideas what to do?
    IMG_6246.PNG

    Camera settings:
    Screenshot 2018-10-21 at 21.04.44.png
     
    Burglecut likes this.
  25. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    Try setting the depth to 1.
     
    Burglecut likes this.
  26. mp-paramlabs

    mp-paramlabs

    Joined:
    Oct 31, 2017
    Posts:
    9
    hello @jimmya and @todds_unity
    I am facing freeze issue while using image anchors. After the app starts I can see camera feed only for a second. UI is completely working fine. I am using sample scene provided with the plugin and performed all the checks about image size in Xcode and tested with jpg also but the issue remains same.
    Here are development details.
    • plugin copied from bitbucket default branch
    • Xcode stable10.0 (10A255)
    • Unity 2018.1.0f2
    • device iPad, ios 12.0.1(16A404)
    I've found many persons having this issue. Also found on StackOverflow that issue is caused by XCode stable and suggested to use Xcode beta 3.
    Please let me know if there is any quick fix available. we are really stuck with this from a couple of days.
     
  27. JelmerV

    JelmerV

    Joined:
    Nov 29, 2012
    Posts:
    219
    Hey! Turned out it was the HDR flag. The main camera is already HDR so I guess 2 cameras with HDR is too much :)

    But thanks, it works now, quite happy with the result!
    IMG_6269.PNG
    Off topic, but It is a little hard to align the UI elements with the 3D rendered spheres. I only get it to work for one aspect ratio. When changing to iPhone X it is all misaligned again.
    If anyone has tips..?
     
    Burglecut and jimmya like this.
  28. salmi700r

    salmi700r

    Joined:
    Jun 13, 2017
    Posts:
    2
    It says on the asset store the ArKit Plugin is deprecated? Did I miss something.. :(
     

    Attached Files:

  29. JelmerV

    JelmerV

    Joined:
    Nov 29, 2012
    Posts:
    219
    I think you have to use the package manager and install it as a package rather than an asset :)
     
    salmi700r likes this.
  30. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    edhjones likes this.
  31. edhjones

    edhjones

    Joined:
    Oct 23, 2018
    Posts:
    6
    What's the "cleanest" way to get the plugin included into a new project? I mean; import the minimal amount of stuff, without the example scenes and whatnot.
     
  32. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    There is no minimal set separated out well, but all the files live under the UnityARKitPlugin folder, so it should not pollute any of your project files. And extra files in the project should not affect your final build if you are not referencing it.
     
    edhjones likes this.
  33. JoMaHo

    JoMaHo

    Joined:
    Apr 2, 2017
    Posts:
    94
    Hi!
    I can not record at 1080p on my iPhone 7. I've tried using the provided ARVideoFormats example in ARKIT1.5, but it seems that my phone will only record video in 720p, I need 1080p. Using Unity 2018.2.13, iPhone7, Xcode10.1, iOS beta5 of 12.1

    Any suggestions?
     
  34. Mal_Duffin

    Mal_Duffin

    Joined:
    Jan 22, 2015
    Posts:
    51
  35. Chadobado

    Chadobado

    Joined:
    Oct 19, 2016
    Posts:
    12
    Am writing some custom shaders for use w/ ARKit. One of my shaders applies the camera's transpose model view transformation matrix to the object's vert positions to create a camera-facing billboard of the gameobject. Unfortunately when I use it w/ ARKit I'm getting unexpected repositioning of my verts after the matrix is applied. Especially noticeable w/ up/down panning and rotation.

    Any feedback for what might be going on under the hood? Has been painful trying to debug what's happening.

    Not at all married to the transpose matrix, just need to get the verts into static positions in front of my camera to capture a rendertexture from the correct position/FOV.

    Thanks in advance!
     
    Last edited: Nov 8, 2018
  36. Kunukunuk

    Kunukunuk

    Joined:
    Jun 21, 2017
    Posts:
    1
    Hi, I am learning Arkit through the Udacity course and the remote is confusing me.
    The remote works, but it's stuck on the first frame. I can move it around only on the editor and not on my iPhone X.
    When I open it on the unity remote app, it freezes at first frame.
    The project I download is this one https://github.com/udacity/arkit-unity-course-assets
     
  37. Numa

    Numa

    Joined:
    Oct 7, 2014
    Posts:
    84
    Hi there,

    I'm getting weird jumps in the reflections when using the environment probes. Using Unity 2018.2.16


    At first I thought the object was getting past the probe's box but it still happens even after making the object smaller, or the box really big.
    It's not due to the probe being updated either, since it goes back and forth even when I'm not moving, and doesn't happen if I walk around the object, only when spinning the object.
    The reflection seems somehow accurate, I can discern parts of the room in both textures.

    It looks like the probe swaps between different faces of the cube map?

    I'm using the default code and prefabs from the ARKit plugin examples.

    Any help appreciated!
     
  38. yjcnbnbnb200

    yjcnbnbnb200

    Joined:
    Nov 1, 2017
    Posts:
    25
    Any plan when we can have iPad 2018 supported by ARKitRemote? Thanks jimmy
     
    JellePossiblyPixels likes this.
  39. ZK0116

    ZK0116

    Joined:
    Mar 10, 2017
    Posts:
    2
  40. ferretnt

    ferretnt

    Joined:
    Apr 10, 2012
    Posts:
    336
    I also cannot connect over USB (iPhone XS, iOS12). Is there a way we can lower the resolution of the frames sent over wifi? I tried naively reducing the size of the textures (currently CbCR is downscaled by 2, so I made that for a downscaled luminance by 2) but this crashed in frame decompression, so probably needs a bit more debugging. Before I go down that rabbit hole I'd love to know if there's a better fix (I only just started using ARKit so am assuming remote performance was fine for eberyone over usb)

    Unity 2017.4.15
    XCode 10.1
    iPhone XS, ios 12.1

    Edit to add: Even if I use an iPhone 6S, which must be connected via USB because I've switched it to airplane mode before connecting, performance of ARKit remote still isn't anything like in the videos (or as I'm sure it was when I tried it ages ago). Perf is more like 1FPS in the editor using the standard ARTestScene.
     
    Last edited: Nov 21, 2018
  41. dilipMobisharnam

    dilipMobisharnam

    Joined:
    Nov 3, 2018
    Posts:
    1
    Set Unity Remote ->

    device to Any iOS device or Any Android Device

    from

    Edit>Project Setting>Editor before make build for ARRemoteScene for device
     
  42. ZK0116

    ZK0116

    Joined:
    Mar 10, 2017
    Posts:
    2
    Tried, still not working on XR :(
    iPhone X works fine as usual
     
  43. daviner

    daviner

    Joined:
    Feb 14, 2018
    Posts:
    10
    Hi guys anyone been able to MULTIPLE save and load serialized maps. My aim is to create a new map use hit testing and create cube (in each new map I create I create new coloured cube so I can tell each map is different).

    Now I can create a new map by running a new config with worldMap as null then hittest and create a cube on planes. I save this map ie MapOne to a persistent path with name as MapOne.worldmap using WorldMapManager.SaveSerialized(). Then I add this to my mapsList in MapHandler.cs. Once the map is saved I load it using WorldMapManager.LoadSerialized(ARWorldMap map) giving it the map and path so it knows the right map to load which it does perfectly.

    The problem is I save the map and the cube I created doesn't destroy after saving the map. If I destroy the cube after saving the map it destroys but when I load the map it doesn't display the cube maybe because I destroy it after saving the map. Bear in mind when calling WorldMapManager.LoadSerialized it attempts to relocalise and runs with a new config. Config.map = mySavedMap and with options such asUnityARSessionRunOption.ARSessionRunOptionRemoveExistingAnchors | UnityARSessionRunOption.ARSessionRunOptionResetTracking.

    I understand my cube is not destroyed by default as of the example in UnityARWorldMapSerialized.scene but how can I destroy my cube after saving my map and still be able to load it when I select load scene, because now if I destroy my cube after saving the map it will not appear when I load the very same map. Any ideas??????
     
  44. JellePossiblyPixels

    JellePossiblyPixels

    Joined:
    Feb 11, 2018
    Posts:
    6
    Is it known or listed that this device is not supported yet?
    Im running into an in issue where, when I try to connect the new iPad pro 2018 with Remote, I get ApplicationException: Couldn't create device API for device.
     
    StefanoCecere likes this.
  45. StefanoCecere

    StefanoCecere

    Joined:
    Jun 10, 2011
    Posts:
    186
    really? the ipad 2018 isn't supported? where did you read that?
    i'm having problems, too, with my ipad 2018 (but works with iPhone SE)... sigh
     
  46. dorukeker

    dorukeker

    Joined:
    Dec 6, 2016
    Posts:
    37
    ARKit Camera Feed Very Dark / Low Exposure

    Hello All,

    I am building an ARKit application via the Unity ARKit plugin (Bitbucket latest version)
    However the camera feed is very dark compared to other ARKit applications. (e.g. Measure from Apple).

    Is there a way to make the camera feed brighter / more exposure?
    Thanks!
    Doruk
     
  47. Numa

    Numa

    Joined:
    Oct 7, 2014
    Posts:
    84
  48. Pilltech101

    Pilltech101

    Joined:
    Jul 25, 2014
    Posts:
    15
    Would somebody be able to help me find out the ISO speed ratings with the current AR Camrea? I know how to get this data using Av Capture Through the Metadata . But every time I try to use Av Capture In my unity seen it completely messes up the ar kit scene. I have tried grabbing the meta-data from the pixel buffer But the method that I’m using does not seem to produce the ISO speed. If you do not know how to grab the iSo speed Would anybody know how to run both the Av Capture class using Web textures and the unity ARC and at the same time? Without them interfering with each other.
     
  49. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    You can do one of two things - see @Numa suggestion below, or you can change the Clear Material on the UnityARVideo component on the camera to YUVMaterialLinear (and leave ColorSpace as Linear).
     
  50. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    ARWorldMap is an ARKit data structure that saves the map of the real world around you, so that you can start from the same coordinate system as last time. The cube you create is a virtual object and does not get saved with ARWorldMap. If you wanted to save its position, you could place a user anchor in a specific spot and that will get saved with the world map. Then when you loaded the map, you would get a user anchor with the transform of where you need to create your cube.
     
    daviner likes this.
unityunity