Search Unity

  1. We've closed the job boards. If you're looking for work, or looking to hire check out Unity Connect. You can see more information here.
    Dismiss Notice
  2. Unity 2017.3 has arrived! Read about it here.
    Dismiss Notice
  3. Unity’s Demo Team is excited to share their upcoming interactive demo, Book of the Dead. See more information here.
    Dismiss Notice
  4. Want to see the most recent patch releases? Take a peek at the patch release page.
    Dismiss Notice

ARKit support for iOS via Unity-ARKit-Plugin

Discussion in 'ARKit Preview' started by jimmya, Jun 5, 2017.

  1. Snaxz

    Snaxz

    Joined:
    May 13, 2013
    Posts:
    7
    Awesome, thanks for the quick reply!
    - For Crash on iPhone 1 - under "General/Software Update" it says "iOS 11 Your software is up to date"...is there someway to force a updated build of iOS11?
    - For Crash 2, i'm updating the ARKit plugin now. Fingers crossed

    Thanks again!
    J
     
  2. jessevan

    jessevan

    Joined:
    Jan 3, 2017
    Posts:
    26
    Yeah, pretty sure I had it set to something very large, like 5000. I'll triple check though and get back to you.
     
  3. TJUnityBuilder

    TJUnityBuilder

    Joined:
    Jul 19, 2017
    Posts:
    2
    Hi! The iOS version is 10.3.2 on my iPhoen 5 (on my iPhone 6 as well). Do I have to update the software on my phone? Thanks.
     
  4. Snaxz

    Snaxz

    Joined:
    May 13, 2013
    Posts:
    7
    Oy, yep. plugin update worked. Thanks for that. Also an iOS11 build update hit the other phone this evening, all good there as well.

    Thanks again all.
     
  5. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    547
    Just released on bitbucket repo due to popular demand: Unity ARKit Remote! Also released on Unity Asset Store.

    Here's a demo:


    And here's usage instructions:

    Unity ARKit Remote

    This is a two part solution for game developers who want to iterate on ARKit apps (made with Unity ARKit Plugin) from within the Unity Editor. It consists of an iOS app (UnityARKitRemote) that provides the ARKit data from a compatible device that it is installed on, working in conjunction with a GameObject called ARKitRemoteConnection in your Unity scene for your ARKit project. This GameObject simulates ARKit working in the editor by passing data via UnityARSessionNativeInterface to the other GameObjects in the editor. The UnityARKitRemote app on the device will forward the following information to Unity scene in editor:

    -The video feed coming from the device camera (separated into Y and UV textures) as provided by ARKit.
    -The camera translation and rotation based on device movement
    -The ARPlaneAnchor addition, removal and update events along with data about the plane affected
    -Point cloud data

    Requirements:
    It has all the same requirements as the Unity ARKit Plugin, with an additional recommendation of using Unity 2017.1 or later, as the PlayerConnection works way better and has better UI support in that version.

    Future work on this:
    -HitTest API
    -AR Session Configuration

    Steps to using remote:

    1. First download the latest Unity ARKit Plugin code that includes the Unity ARKit Remote from either bitbucket or Unity Asset Store.
    2. Build the scene called UnityARKitRemote out to your compatible iOS device. You should use “Development Build” in your Build Settings. This is the only build time to iOS you will need to endure - the rest of the iterations on your project can happen in the editor. When you build out to iOS, you should change the product name and bundle identifier in the PlayerSettings to signify that it is the Unity ARKit Remote.
    3. Open the scene which contains your app that uses the Unity ARKit Plugin. From the ARKitRemote folder, add the ARKitRemoteConnection prefab into the root of your scene. If you want to test it out, it has already been added to EditorTestScene in the same folder.
    4. Run the UnityARKitRemote app from step 2 on your device. It should display a black screen with “Waiting for editor connection..”
    5. Press play in the editor: your game window should have a green screen with “Please connect to player in the console menu” near the bottom.
    6. In this step we need to connect the editor to the Unity ARKit Remote app on the device. This is where Unity 2017.1 comes in handy: it has a menu item in the Console window to “Connected player” with a dropdown of all the available players to connect to. Select the one that corresponds to your device. In Unity 5.6 variants, you have to create a Profiler window via Window/Profiler menu. And then at the top of the Profiler window, there is a dropdown “Active Profiler” from which you select your device.
    7. If you were successful in step 6, you should have a button on the top part of your Game window labelled “Start Remote ARKit Session” in editor that when pressed will start the ARKit session on the device and start transmitting data to the editor. The editor should be displaying the same video as the device, as well as navigating through your scene, and it will momentarily start showing the point cloud data as well as the planes found.

    Let us know how you like it, and what improvements you'd like.
    Thanks.

    @ina @SweatyChair @Alex_F @DerekLerner @TheTaj

    [edit: it has landed on Asset Store]
     
    Last edited: Jul 22, 2017
    durants, victorreis777, lourd and 9 others like this.
  6. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    547
    Have a look at this announcement: https://forum.unity3d.com/threads/a...nity-arkit-plugin.474385/page-13#post-3153415

    Then look at the latest source on bitbucket repo, as the Unity ARKit Remote does exactly what you need (sending Y and UV textures over the network). See UnityRemoteVideo.cs and helpers on the ARKit side, and ARKitRemoteConnection.cs and UnityARVideo.cs (uses same shader as ARKit apps do to render RGB color) on the destination.
     
    theiajsanchez likes this.
  7. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    547
    Please check the requirements: you need at least an iPhone 6S (or some other iOS device with A9 or better processor), and you need to install ios 11 beta on that device.
     
    noahmilam likes this.
  8. _Daniel_

    _Daniel_

    Joined:
    Feb 28, 2007
    Posts:
    2,506
    @jimmya : Great idea! Any plans to integrate this with Unity Remote?
     
    sama-van likes this.
  9. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    547
    None at this time - Unity Remote data is received in editor native code and not in script, so there are some complications. The communication here is handled strictly by scripts (PlayerConnection and EditorConnection APIs). Is there something that Unity Remote provides that you would like to have?
     
  10. chenditc_hyfield

    chenditc_hyfield

    Joined:
    Dec 28, 2016
    Posts:
    2
    Last edited: Jul 21, 2017
    christophergoy likes this.
  11. _Daniel_

    _Daniel_

    Joined:
    Feb 28, 2007
    Posts:
    2,506
    No, it just seems like it would be a nice enhancement of Unity remote. We would have 1 remote app instead of 2.
     
  12. mmortall

    mmortall

    Joined:
    Dec 28, 2010
    Posts:
    58
    When would it be possible to test AR in Editor Mode?

    For example some debug version of this events in Editor Mode:

    UnityARSessionNativeInterface.ARAnchorAddedEvent
    UnityARSessionNativeInterface.ARAnchorUpdatedEvent
    UnityARSessionNativeInterface.ARAnchorRemovedEvent

    plus some AR camera feedback would be nice to have too.
     
    Last edited: Jul 21, 2017
  13. mimminito

    mimminito

    Joined:
    Feb 10, 2010
    Posts:
    752
    Really interesting to see you working on this. I have been thinking about localisation and how this could "easily" be achieved from a users perspective. My thoughts are to align something in the real world with a known marker, so your users can always reference that. Looks like your doing something similar but with 3 markers instead of 1.
     
  14. chaselortie

    chaselortie

    Joined:
    Jun 27, 2017
    Posts:
    1
    Hi there! I am currently working on a demo for scanning your environment and whenever a plane is detected (that's not the floor) a cube is instantiated that extends to the floor. I did this by replacing the debugPlaneprefab to a cube that has a script attached that changes its position and scale. I got the offset height of the cube correct, but I don't know how to scale the x and z coordinates (length and width of the cube). I want the cube to match the length and width of the planeAnchor that is found so that the cube is the same size as whatever obstacle in the way. Is this possible? If so, how can I get the plane extent and have the cube rescale?
     
    Last edited: Jul 21, 2017
  15. theiajsanchez

    theiajsanchez

    Joined:
    Feb 2, 2017
    Posts:
    13
    Hi @christophergoy this is an entire new code in which we are trying to do what you mentioned before. How would you recommend to copy the render texture into a texture2d in a good performance way?

    Thanks
     
  16. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    547
    Did you not read https://forum.unity3d.com/threads/a...ity-arkit-plugin.474385/page-13#post-3153415? Doesn't that do what you're looking for?
     
  17. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    547
    When a plane is reported by the ARAnchorAddedEvent, it contains all the information in the C# equivalent of https://developer.apple.com/documentation/arkit/arplaneanchor . That has all the info you need?
     
    JasperMeijering likes this.
  18. tomasdgarcia

    tomasdgarcia

    Joined:
    Nov 2, 2014
    Posts:
    1
    Hi @jimmya, I updated the script package from the asset store (updated yesterday) and now I'm receiving these error from Xcode building the scene UnityARKitScene without touching anything.

    "Use of undeclared identifier 'ARTrackingStateReasonInitializing'; did you mean 'UnityARTrackingState"
    "Duplicate case value: 'UnityARTrackingStateReasonInitializing' and 'ARTrackingStateReasonExcessiveMotion' both equal '1'"

    I build a lot of demos with the previous version and I didn't have any issue.

    Thanks,
     
  19. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    547
    Upgrade to XCode Beta 3
     
    tomasdgarcia likes this.
  20. tannerhearne

    tannerhearne

    Joined:
    Feb 13, 2015
    Posts:
    9
    Hey @jimmya — I have placed the ARKitRemote in my app. When I attempt to connect via the console I see the following error. Any pointers for where the problem may lie?

    [​IMG]
     
  21. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    547
    If you're using Unity 5.6.x, I would recommend upgrading to 2017.1 - the player connection works way better there. Make sure you've built the remote app using Development Build. After all these steps, if you still have problems, you should try force-closing the remote app, reload your Unity project (which effectively shuts down and restarts Unity), open the remote app on the device and try the connection again.

    Unity ARKit Remote uses the same connection as the Profiler, so if you can make Profiler work with your iOS device, then it should be able to work for this.
     
    Last edited: Jul 21, 2017
    tannerhearne likes this.
  22. chicolima

    chicolima

    Joined:
    Jul 12, 2017
    Posts:
    1
    Hello, Will the ARKit plugin work with:
    Version Released
    2017.2.0 Beta 4?

    Cheers!
     
  23. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    547
    Yes. All versions of 2017 past 2017.1.0b5 should work.
     
    Last edited: Jul 21, 2017
  24. cWe1028167

    cWe1028167

    Joined:
    May 26, 2014
    Posts:
    4
    Been playing around with Unity and ARKit the past couple days. Its easy to get up and running.
    here is a video grab of the experiment:

     
  25. tannerhearne

    tannerhearne

    Joined:
    Feb 13, 2015
    Posts:
    9
    Hey @jimmya — I believe it was the "Development Build" checkbox that fixed it for me. So now I am having another issue. Instead of seeing video within Unity and on the actual iOS build on my iPhone I am seeing pink painted everywhere. I can see video behind the pink for half a second and then it gets painted over with pink. I am on 2017.1 and I have been updating the ARKit Unity Plugin about once per day when y'all release new stuff.

    See screenshot:
    [​IMG]

    Thanks,
    Tanner
     
  26. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    547
    Looks like the shader compile step may have failed - that color signifies a shader error. Are you using OpenGLES or Metal? You should be using Metal.
     
  27. tannerhearne

    tannerhearne

    Joined:
    Feb 13, 2015
    Posts:
    9
    Hey @jimmya under Player Settings > Other Settings > Graphic APIs it shows "Metal" by itself. Is that where I should be looking? See image...
    [​IMG]
     
  28. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    547
    The graphics api looks correct, but you should probably have the experimental graphics jobs turned off. BTW, are you trying with the test scene that we've included? That might be a good first test to make sure there are no other conflicts.
     
  29. tannerhearne

    tannerhearne

    Joined:
    Feb 13, 2015
    Posts:
    9
    Hey @jimmya — this started occurring a few days ago before adding in the AR Remote app to the workflow. Also I tried turning Experimental Graphics off and that did not solve the issue. Do you have any additional suggestions?
     
  30. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    482
    Do you see any errors in the console related to this?
     
  31. Staus

    Staus

    Joined:
    Jul 7, 2014
    Posts:
    3
    While it would of course be great if ARkit had something like this (vision looks way to complicated for me honestly) that could be extended to unity I was just thinking for now it would be nice if ARkit first of all didn't overrule Vuforia (might be a mistake on their part) and second if their was support for them working together (like Vuforia have with the Hololens).

    So a "Vuforia world" could live inside the ARkit world without messing with the ARkit's overall tracking.
    If you skim a few lines in this tutorial on integrating Vuforia with Hololens you'll get what I mean. Their implementation is very dev friendly: (read from Configuring a Vuforia App for HoloLens)
    https://library.vuforia.com/articles/Training/Developing-Vuforia-Apps-for-HoloLens
     
  32. tannerhearne

    tannerhearne

    Joined:
    Feb 13, 2015
    Posts:
    9
    Hi @christophergoy — I figured out what was causing the issue. I was trying to use a Trail Renderer on an object.
     
  33. SkaZonic

    SkaZonic

    Joined:
    Jun 20, 2017
    Posts:
    22
    Ok so I have a iPhone 7 updated to iOS 11, Xcode 9 beta 3 and Unity 5.6.2p2 - After trying to load my Arkit Scene it failed so I tried out the example scene and as i opened the folder on Xcode - Xcode crashed. Whats up with this?

    P.S These are the problems i was experiencing with my own ArKit Scene
    Screen Shot 2017-07-22 at 14.16.31.png
     
    Last edited: Jul 22, 2017
  34. imaginethepoet

    imaginethepoet

    Joined:
    Aug 23, 2016
    Posts:
    11
    I have also run into those errors in the past
    1. The only way i got around this was to switch to my master profile in IOS
    2. You need to change the name of the project to something other than com.unity - change it to com.myproject.something
    3.You need to specify ios 11

    You can also configure these under player settings before building in unity
     
  35. yaffa

    yaffa

    Joined:
    Nov 21, 2016
    Posts:
    21
    Does the Camera in ARkit have world location ?

    Like If I have two apps running the same app , and I get the position and rotation of the camreas using the camera.Transform.LocalPosition , Unity co-ordinates, or are they relative to the Latitude and longtude in the real world or is it or are all camera at zero ?
     
  36. imaginethepoet

    imaginethepoet

    Joined:
    Aug 23, 2016
    Posts:
    11
    Still running into these two errors even in the latest unity arkit plugin when building in xcode

    The namespace is incorrect and the other seems to be duplicate. So I just changed the namespace to UnityART... and comment out the other one.

    switch (trackingReason)

    {

    case ARTrackingStateReasonNone:

    return UnityARTrackingStateReasonNone;

    case ARTrackingStateReasonInitializing:

    return UnityARTrackingStateReasonInitializing;

    case ARTrackingStateReasonExcessiveMotion:

    return UnityARTrackingStateReasonExcessiveMotion;

    case ARTrackingStateReasonInsufficientFeatures:

    return UnityARTrackingStateReasonInsufficientFeatures;

    default:

    [NSException raise:mad:"UnrecognizedARTrackingStateReason" format:mad:"Unrecognized ARTrackingStateReason: %ld", (long)trackingReason];

    break;
     
  37. SkaZonic

    SkaZonic

    Joined:
    Jun 20, 2017
    Posts:
    22
    May i ask, (Sorry if I sound like a complete noob in advance, came from Android) how do I switch to my master profile in iOS? Also my project is already set up like that with "com.SkaZonic.arkitscene" as the bundle identifier. I guess I specify iOS 11 with my build settings set at a minimum of iOS 11 what else can I do to specify iOS 11??
    So these are the two problems I get - (they're signing problems).
    Screen Shot 2017-07-22 at 14.45.03.png Screen Shot 2017-07-22 at 14.45.03.png
     
  38. imaginethepoet

    imaginethepoet

    Joined:
    Aug 23, 2016
    Posts:
    11
    So for some reason personal team is not working i discovered. You need to actually have a regular apple account setup. When I tried to use personal team it didnt work. I could only do it under my company profile. I also got that error with maximum never registered unitll i switched the the profile to my company. This could be a bug in xcode beta.


     
  39. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    547
    Get latest of the plugin code, and install the latest XCode Beta 3
     
  40. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    547
    Whenever you start an ARKit session on a device, the camera on that device will be at (0,0,0)
     
  41. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    547
    This is really an XCode issue, so Apple Developer Support might know better, but I'll try and answer from what I think I know. The error message seems to indicate that you're trying to build to a device that you have not built to before, but your free personal license only allows you to build to 3 devices. There is no automated way to remove devices from your personal team, so your solutions seem to be: 1) build it to a device you have built to before 2) ask Apple support to remove old devices from your account 3) create a new apple developer account and use that one
     
    SkaZonic likes this.
  42. SkaZonic

    SkaZonic

    Joined:
    Jun 20, 2017
    Posts:
    22
    Thanks solved the issue! Just made a new account. Again thank you Unity team for this awesome plugin!
     
  43. SkaZonic

    SkaZonic

    Joined:
    Jun 20, 2017
    Posts:
    22
    Does anyone know how I would implement a sort of waiting screen for the plane to be detected and then tell the user it is ok to place an object? Also Has Someone said this already?
     
  44. LRavenwolf

    LRavenwolf

    Joined:
    Dec 26, 2013
    Posts:
    2
    Hi all,
    After updating to the latest ARKit plugin, I've begun getting an error on play in the editor whenever my mouse is anywhere inside the game window:

    Screen position out of view frustum (screen pos 715.000000, 511.000000) (Camera rect 0 0 1325 520)
    UnityEngine.SendMouseEvents: DoSendMouseEvents(Int32)

    Can anyone help with this? I'm getting this error anytime the Unity AR Camera Manager component is enabled. Which is pretty vital for things to work.

    Thanks!
     

    Attached Files:

    ina and AmieD like this.
  45. AmieD

    AmieD

    Joined:
    Jan 15, 2015
    Posts:
    3
    Hey I'm having the same issues. I'm using Unity 2017.1, and I have updated the ARKit.

    Assertion failed: Screen position out of view frustum (screen pos 676.374695, 141.260361) (Camera rect 0 0 701 162)
    UnityEngine.SendMouseEvents:DoSendMouseEvents(Int32)
     
    ina and LRavenwolf like this.
  46. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    547
    This usually happens if your camera does not get initialized to some good value. I've got some lines at the top of ARKitRemoteConnection that is supposed to remedy this:

    Code (CSharp):
    1.             //put some defaults so that it doesnt complain
    2.             UnityARCamera scamera = new UnityARCamera ();
    3.             scamera.worldTransform = new UnityARMatrix4x4 (new Vector4 (1, 0, 0, 0), new Vector4 (0, 1, 0, 0), new Vector4 (0, 0, 1, 0), new Vector4 (0, 0, 0, 1));
    4.             Matrix4x4 projMat = Matrix4x4.Perspective (60.0f, 1.33f, 0.1f, 30.0f);
    5.             scamera.projectionMatrix = new UnityARMatrix4x4 (projMat.GetColumn(0),projMat.GetColumn(1),projMat.GetColumn(2),projMat.GetColumn(3));
    6.  
    But I am not really sure how you got your project in such a state - the scenes in my project do not have that problem.
     
    Last edited: Jul 22, 2017
    AmieD likes this.
  47. atreyukun

    atreyukun

    Joined:
    Jan 22, 2017
    Posts:
    1
    I'm having a small problem with ARKit. I'm working on an app that will let me place objects down on a surface and then change those objects with UI buttons. But the only way I can do that is to make the HitCubeParent a child of an object controller.
    Right now, I think it's breaking ARKit's functionality. I can view the objects, change them, and resize them, but I can't place them nor can I walk around them.
    So I guess my question is does parenting the HitCubeParent kill the functionality?
     
  48. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    547
    Doh! I realized why - I didn't account for scenes that do not have ARKitRemoteConnection gameobject. I'll fix it, but in the meantime, you could add that gameobject to your scene to make it work.
     
    LRavenwolf likes this.
  49. jessevan

    jessevan

    Joined:
    Jan 3, 2017
    Posts:
    26
    I've confirmed that my far clipping plane is set to 1000. I haven't measured exactly where the object clips on device, but I'm pretty sure its 30 meters still. Have you had a chance to test this, does it work for you?