Search Unity

  1. Welcome to the Unity Forums! Please take the time to read our Code of Conduct to familiarize yourself with the forum rules and how to post constructively.
  2. Join us on March 30, 2023, between 5 am & 1 pm EST, in the Performance Profiling Dev Blitz Day 2023 - Q&A forum and Discord where you can connect with our teams behind the Memory and CPU Profilers.
    Dismiss Notice
New Forum User Notice Performance Profiling Dev Blitz Day 2023

Resolved ARKit support for iOS via Unity-ARKit-Plugin

Discussion in 'AR' started by jimmya, Jun 5, 2017.

Thread Status:
Not open for further replies.
Page 13 of 59

  1. Snaxz

    Snaxz

    Joined:
    May 13, 2013
    Posts:
    13
    Awesome, thanks for the quick reply!
    - For Crash on iPhone 1 - under "General/Software Update" it says "iOS 11 Your software is up to date"...is there someway to force a updated build of iOS11?
    - For Crash 2, i'm updating the ARKit plugin now. Fingers crossed

    Thanks again!
    J
     
    Snaxz, Jul 20, 2017
    #601

  2. jessevan

    jessevan

    Joined:
    Jan 3, 2017
    Posts:
    35
    Yeah, pretty sure I had it set to something very large, like 5000. I'll triple check though and get back to you.
     
    jessevan, Jul 21, 2017
    #602

  3. TJUnityBuilder

    TJUnityBuilder

    Joined:
    Jul 19, 2017
    Posts:
    2
    Hi! The iOS version is 10.3.2 on my iPhoen 5 (on my iPhone 6 as well). Do I have to update the software on my phone? Thanks.
     
    TJUnityBuilder, Jul 21, 2017
    #603

  4. Snaxz

    Snaxz

    Joined:
    May 13, 2013
    Posts:
    13
    Oy, yep. plugin update worked. Thanks for that. Also an iOS11 build update hit the other phone this evening, all good there as well.

    Thanks again all.
     
    Snaxz, Jul 21, 2017
    #604
    amkissler likes this.

  5. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    Just released on bitbucket repo due to popular demand: Unity ARKit Remote! Also released on Unity Asset Store.

    Here's a demo:


    And here's usage instructions:

    Unity ARKit Remote

    This is a two part solution for game developers who want to iterate on ARKit apps (made with Unity ARKit Plugin) from within the Unity Editor. It consists of an iOS app (UnityARKitRemote) that provides the ARKit data from a compatible device that it is installed on, working in conjunction with a GameObject called ARKitRemoteConnection in your Unity scene for your ARKit project. This GameObject simulates ARKit working in the editor by passing data via UnityARSessionNativeInterface to the other GameObjects in the editor. The UnityARKitRemote app on the device will forward the following information to Unity scene in editor:

    -The video feed coming from the device camera (separated into Y and UV textures) as provided by ARKit.
    -The camera translation and rotation based on device movement
    -The ARPlaneAnchor addition, removal and update events along with data about the plane affected
    -Point cloud data

    Requirements:
    It has all the same requirements as the Unity ARKit Plugin, with an additional recommendation of using Unity 2017.1 or later, as the PlayerConnection works way better and has better UI support in that version.

    Future work on this:
    -HitTest API
    -AR Session Configuration

    Steps to using remote:

    1. First download the latest Unity ARKit Plugin code that includes the Unity ARKit Remote from either bitbucket or Unity Asset Store.
    2. Build the scene called UnityARKitRemote out to your compatible iOS device. You should use “Development Build” in your Build Settings. This is the only build time to iOS you will need to endure - the rest of the iterations on your project can happen in the editor. When you build out to iOS, you should change the product name and bundle identifier in the PlayerSettings to signify that it is the Unity ARKit Remote.
    3. Open the scene which contains your app that uses the Unity ARKit Plugin. From the ARKitRemote folder, add the ARKitRemoteConnection prefab into the root of your scene. If you want to test it out, it has already been added to EditorTestScene in the same folder.
    4. Run the UnityARKitRemote app from step 2 on your device. It should display a black screen with “Waiting for editor connection..”
    5. Press play in the editor: your game window should have a green screen with “Please connect to player in the console menu” near the bottom.
    6. In this step we need to connect the editor to the Unity ARKit Remote app on the device. This is where Unity 2017.1 comes in handy: it has a menu item in the Console window to “Connected player” with a dropdown of all the available players to connect to. Select the one that corresponds to your device. In Unity 5.6 variants, you have to create a Profiler window via Window/Profiler menu. And then at the top of the Profiler window, there is a dropdown “Active Profiler” from which you select your device.
    7. If you were successful in step 6, you should have a button on the top part of your Game window labelled “Start Remote ARKit Session” in editor that when pressed will start the ARKit session on the device and start transmitting data to the editor. The editor should be displaying the same video as the device, as well as navigating through your scene, and it will momentarily start showing the point cloud data as well as the planes found.

    Let us know how you like it, and what improvements you'd like.
    Thanks.

    @ina @SweatyChair @Alex_F @DerekLerner @TheTaj

    [edit: it has landed on Asset Store]
     
    Last edited: Jul 22, 2017
    jimmya, Jul 21, 2017
    #605
    wunder-ilmari, stomachmonkey, durants and 11 others like this.

  6. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    Have a look at this announcement: https://forum.unity3d.com/threads/a...nity-arkit-plugin.474385/page-13#post-3153415

    Then look at the latest source on bitbucket repo, as the Unity ARKit Remote does exactly what you need (sending Y and UV textures over the network). See UnityRemoteVideo.cs and helpers on the ARKit side, and ARKitRemoteConnection.cs and UnityARVideo.cs (uses same shader as ARKit apps do to render RGB color) on the destination.
     
    jimmya, Jul 21, 2017
    #606
    theiajsanchez likes this.

  7. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    Please check the requirements: you need at least an iPhone 6S (or some other iOS device with A9 or better processor), and you need to install ios 11 beta on that device.
     
    jimmya, Jul 21, 2017
    #607
    noahmilam likes this.

  8. User340

    User340

    Joined:
    Feb 28, 2007
    Posts:
    3,001
    @jimmya : Great idea! Any plans to integrate this with Unity Remote?
     
    User340, Jul 21, 2017
    #608
    sama-van likes this.

  9. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    None at this time - Unity Remote data is received in editor native code and not in script, so there are some complications. The communication here is handled strictly by scripts (PlayerConnection and EditorConnection APIs). Is there something that Unity Remote provides that you would like to have?
     
    jimmya, Jul 21, 2017
    #609

  10. chenditc_hyfield

    chenditc_hyfield

    Joined:
    Dec 28, 2016
    Posts:
    3
    Last edited: Jul 21, 2017
    chenditc_hyfield, Jul 21, 2017
    #610
    christophergoy likes this.

  11. User340

    User340

    Joined:
    Feb 28, 2007
    Posts:
    3,001
    No, it just seems like it would be a nice enhancement of Unity remote. We would have 1 remote app instead of 2.
     
    User340, Jul 21, 2017
    #611

  12. mmortall

    mmortall

    Joined:
    Dec 28, 2010
    Posts:
    89
    When would it be possible to test AR in Editor Mode?

    For example some debug version of this events in Editor Mode:

    UnityARSessionNativeInterface.ARAnchorAddedEvent
    UnityARSessionNativeInterface.ARAnchorUpdatedEvent
    UnityARSessionNativeInterface.ARAnchorRemovedEvent

    plus some AR camera feedback would be nice to have too.
     
    Last edited: Jul 21, 2017
    mmortall, Jul 21, 2017
    #612
    AJEMadden likes this.

  13. mimminito

    mimminito

    Joined:
    Feb 10, 2010
    Posts:
    780
    Really interesting to see you working on this. I have been thinking about localisation and how this could "easily" be achieved from a users perspective. My thoughts are to align something in the real world with a known marker, so your users can always reference that. Looks like your doing something similar but with 3 markers instead of 1.
     
    mimminito, Jul 21, 2017
    #613
    Burglecut likes this.

  14. chaselortie

    chaselortie

    Joined:
    Jun 27, 2017
    Posts:
    1
    Hi there! I am currently working on a demo for scanning your environment and whenever a plane is detected (that's not the floor) a cube is instantiated that extends to the floor. I did this by replacing the debugPlaneprefab to a cube that has a script attached that changes its position and scale. I got the offset height of the cube correct, but I don't know how to scale the x and z coordinates (length and width of the cube). I want the cube to match the length and width of the planeAnchor that is found so that the cube is the same size as whatever obstacle in the way. Is this possible? If so, how can I get the plane extent and have the cube rescale?
     
    Last edited: Jul 21, 2017
    chaselortie, Jul 21, 2017
    #614

  15. theiajsanchez

    theiajsanchez

    Joined:
    Feb 2, 2017
    Posts:
    18
    Hi @christophergoy this is an entire new code in which we are trying to do what you mentioned before. How would you recommend to copy the render texture into a texture2d in a good performance way?

    Thanks
     
    theiajsanchez, Jul 21, 2017
    #615

  16. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    Did you not read https://forum.unity3d.com/threads/a...ity-arkit-plugin.474385/page-13#post-3153415? Doesn't that do what you're looking for?
     
    jimmya, Jul 21, 2017
    #616

  17. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    When a plane is reported by the ARAnchorAddedEvent, it contains all the information in the C# equivalent of https://developer.apple.com/documentation/arkit/arplaneanchor . That has all the info you need?
     
    jimmya, Jul 21, 2017
    #617
    JasperMeijering likes this.

  18. tomasdgarcia

    tomasdgarcia

    Joined:
    Nov 2, 2014
    Posts:
    2
    Hi @jimmya, I updated the script package from the asset store (updated yesterday) and now I'm receiving these error from Xcode building the scene UnityARKitScene without touching anything.

    "Use of undeclared identifier 'ARTrackingStateReasonInitializing'; did you mean 'UnityARTrackingState"
    "Duplicate case value: 'UnityARTrackingStateReasonInitializing' and 'ARTrackingStateReasonExcessiveMotion' both equal '1'"

    I build a lot of demos with the previous version and I didn't have any issue.

    Thanks,
     
    tomasdgarcia, Jul 21, 2017
    #618

  19. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    Upgrade to XCode Beta 3
     
    jimmya, Jul 21, 2017
    #619
    tomasdgarcia likes this.

  20. tannerhearne

    tannerhearne

    Joined:
    Feb 13, 2015
    Posts:
    9
    Hey @jimmya — I have placed the ARKitRemote in my app. When I attempt to connect via the console I see the following error. Any pointers for where the problem may lie?

     
    tannerhearne, Jul 21, 2017
    #620

  21. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    If you're using Unity 5.6.x, I would recommend upgrading to 2017.1 - the player connection works way better there. Make sure you've built the remote app using Development Build. After all these steps, if you still have problems, you should try force-closing the remote app, reload your Unity project (which effectively shuts down and restarts Unity), open the remote app on the device and try the connection again.

    Unity ARKit Remote uses the same connection as the Profiler, so if you can make Profiler work with your iOS device, then it should be able to work for this.
     
    Last edited: Jul 21, 2017
    jimmya, Jul 21, 2017
    #621
    TechDoodles and tannerhearne like this.

  22. chicolima

    chicolima

    Joined:
    Jul 12, 2017
    Posts:
    1
    Hello, Will the ARKit plugin work with:
    Version Released
    2017.2.0 Beta 4?

    Cheers!
     
    chicolima, Jul 21, 2017
    #622

  23. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    Yes. All versions of 2017 past 2017.1.0b5 should work.
     
    Last edited: Jul 21, 2017
    jimmya, Jul 21, 2017
    #623

  24. cWe1028167

    cWe1028167

    Joined:
    May 26, 2014
    Posts:
    5
    Been playing around with Unity and ARKit the past couple days. Its easy to get up and running.
    here is a video grab of the experiment:

     
    cWe1028167, Jul 21, 2017
    #625
    dpindrys, phil-harvey, yaffa and 3 others like this.

  25. tannerhearne

    tannerhearne

    Joined:
    Feb 13, 2015
    Posts:
    9
    Hey @jimmya — I believe it was the "Development Build" checkbox that fixed it for me. So now I am having another issue. Instead of seeing video within Unity and on the actual iOS build on my iPhone I am seeing pink painted everywhere. I can see video behind the pink for half a second and then it gets painted over with pink. I am on 2017.1 and I have been updating the ARKit Unity Plugin about once per day when y'all release new stuff.

    See screenshot:


    Thanks,
    Tanner
     
    tannerhearne, Jul 21, 2017
    #626
    TechDoodles likes this.

  26. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    Looks like the shader compile step may have failed - that color signifies a shader error. Are you using OpenGLES or Metal? You should be using Metal.
     
    jimmya, Jul 21, 2017
    #627

  27. tannerhearne

    tannerhearne

    Joined:
    Feb 13, 2015
    Posts:
    9
    Hey @jimmya under Player Settings > Other Settings > Graphic APIs it shows "Metal" by itself. Is that where I should be looking? See image...
     
    tannerhearne, Jul 21, 2017
    #628

  28. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    The graphics api looks correct, but you should probably have the experimental graphics jobs turned off. BTW, are you trying with the test scene that we've included? That might be a good first test to make sure there are no other conflicts.
     
    jimmya, Jul 21, 2017
    #629

  29. tannerhearne

    tannerhearne

    Joined:
    Feb 13, 2015
    Posts:
    9
    Hey @jimmya — this started occurring a few days ago before adding in the AR Remote app to the workflow. Also I tried turning Experimental Graphics off and that did not solve the issue. Do you have any additional suggestions?
     
    tannerhearne, Jul 21, 2017
    #630

  30. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    735
    Do you see any errors in the console related to this?
     
    christophergoy, Jul 22, 2017
    #631

  31. Staus

    Staus

    Joined:
    Jul 7, 2014
    Posts:
    13
    While it would of course be great if ARkit had something like this (vision looks way to complicated for me honestly) that could be extended to unity I was just thinking for now it would be nice if ARkit first of all didn't overrule Vuforia (might be a mistake on their part) and second if their was support for them working together (like Vuforia have with the Hololens).

    So a "Vuforia world" could live inside the ARkit world without messing with the ARkit's overall tracking.
    If you skim a few lines in this tutorial on integrating Vuforia with Hololens you'll get what I mean. Their implementation is very dev friendly: (read from Configuring a Vuforia App for HoloLens)
    https://library.vuforia.com/articles/Training/Developing-Vuforia-Apps-for-HoloLens
     
    Staus, Jul 22, 2017
    #632

  32. tannerhearne

    tannerhearne

    Joined:
    Feb 13, 2015
    Posts:
    9
    Hi @christophergoy — I figured out what was causing the issue. I was trying to use a Trail Renderer on an object.
     
    tannerhearne, Jul 22, 2017
    #633

  33. SkaZonic

    SkaZonic

    Joined:
    Jun 20, 2017
    Posts:
    23
    Ok so I have a iPhone 7 updated to iOS 11, Xcode 9 beta 3 and Unity 5.6.2p2 - After trying to load my Arkit Scene it failed so I tried out the example scene and as i opened the folder on Xcode - Xcode crashed. Whats up with this?

    P.S These are the problems i was experiencing with my own ArKit Scene
    Screen Shot 2017-07-22 at 14.16.31.png
     
    Last edited: Jul 22, 2017
    SkaZonic, Jul 22, 2017
    #634

  34. imaginethepoet

    imaginethepoet

    Joined:
    Aug 23, 2016
    Posts:
    44
    I have also run into those errors in the past
    1. The only way i got around this was to switch to my master profile in IOS
    2. You need to change the name of the project to something other than com.unity - change it to com.myproject.something
    3.You need to specify ios 11

    You can also configure these under player settings before building in unity
     
    imaginethepoet, Jul 22, 2017
    #635

  35. yaffa

    yaffa

    Joined:
    Nov 21, 2016
    Posts:
    41
    Does the Camera in ARkit have world location ?

    Like If I have two apps running the same app , and I get the position and rotation of the camreas using the camera.Transform.LocalPosition , Unity co-ordinates, or are they relative to the Latitude and longtude in the real world or is it or are all camera at zero ?
     
    yaffa, Jul 22, 2017
    #636

  36. imaginethepoet

    imaginethepoet

    Joined:
    Aug 23, 2016
    Posts:
    44
    Still running into these two errors even in the latest unity arkit plugin when building in xcode

    The namespace is incorrect and the other seems to be duplicate. So I just changed the namespace to UnityART... and comment out the other one.

    switch (trackingReason)

    {

    case ARTrackingStateReasonNone:

    return UnityARTrackingStateReasonNone;

    case ARTrackingStateReasonInitializing:

    return UnityARTrackingStateReasonInitializing;

    case ARTrackingStateReasonExcessiveMotion:

    return UnityARTrackingStateReasonExcessiveMotion;

    case ARTrackingStateReasonInsufficientFeatures:

    return UnityARTrackingStateReasonInsufficientFeatures;

    default:

    [NSException raise:mad:"UnrecognizedARTrackingStateReason" format:mad:"Unrecognized ARTrackingStateReason: %ld", (long)trackingReason];

    break;
     
    imaginethepoet, Jul 22, 2017
    #637

  37. SkaZonic

    SkaZonic

    Joined:
    Jun 20, 2017
    Posts:
    23
    May i ask, (Sorry if I sound like a complete noob in advance, came from Android) how do I switch to my master profile in iOS? Also my project is already set up like that with "com.SkaZonic.arkitscene" as the bundle identifier. I guess I specify iOS 11 with my build settings set at a minimum of iOS 11 what else can I do to specify iOS 11??
    So these are the two problems I get - (they're signing problems).
    Screen Shot 2017-07-22 at 14.45.03.png Screen Shot 2017-07-22 at 14.45.03.png
     
    SkaZonic, Jul 22, 2017
    #638

  38. imaginethepoet

    imaginethepoet

    Joined:
    Aug 23, 2016
    Posts:
    44
    So for some reason personal team is not working i discovered. You need to actually have a regular apple account setup. When I tried to use personal team it didnt work. I could only do it under my company profile. I also got that error with maximum never registered unitll i switched the the profile to my company. This could be a bug in xcode beta.


     
    imaginethepoet, Jul 22, 2017
    #639

  39. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    Get latest of the plugin code, and install the latest XCode Beta 3
     
    jimmya, Jul 22, 2017
    #640

  40. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    Whenever you start an ARKit session on a device, the camera on that device will be at (0,0,0)
     
    jimmya, Jul 22, 2017
    #641

  41. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    This is really an XCode issue, so Apple Developer Support might know better, but I'll try and answer from what I think I know. The error message seems to indicate that you're trying to build to a device that you have not built to before, but your free personal license only allows you to build to 3 devices. There is no automated way to remove devices from your personal team, so your solutions seem to be: 1) build it to a device you have built to before 2) ask Apple support to remove old devices from your account 3) create a new apple developer account and use that one
     
    jimmya, Jul 22, 2017
    #642
    SkaZonic likes this.

  42. SkaZonic

    SkaZonic

    Joined:
    Jun 20, 2017
    Posts:
    23
    Thanks solved the issue! Just made a new account. Again thank you Unity team for this awesome plugin!
     
    SkaZonic, Jul 22, 2017
    #643

  43. SkaZonic

    SkaZonic

    Joined:
    Jun 20, 2017
    Posts:
    23
    Does anyone know how I would implement a sort of waiting screen for the plane to be detected and then tell the user it is ok to place an object? Also Has Someone said this already?
     
    SkaZonic, Jul 22, 2017
    #644

  44. LRavenwolf

    LRavenwolf

    Joined:
    Dec 26, 2013
    Posts:
    2
    Hi all,
    After updating to the latest ARKit plugin, I've begun getting an error on play in the editor whenever my mouse is anywhere inside the game window:

    Screen position out of view frustum (screen pos 715.000000, 511.000000) (Camera rect 0 0 1325 520)
    UnityEngine.SendMouseEvents: DoSendMouseEvents(Int32)

    Can anyone help with this? I'm getting this error anytime the Unity AR Camera Manager component is enabled. Which is pretty vital for things to work.

    Thanks!
     

    Attached Files:

    LRavenwolf, Jul 22, 2017
    #645
    ina and AmieD like this.

  45. AmieD

    AmieD

    Joined:
    Jan 15, 2015
    Posts:
    3
    Hey I'm having the same issues. I'm using Unity 2017.1, and I have updated the ARKit.

    Assertion failed: Screen position out of view frustum (screen pos 676.374695, 141.260361) (Camera rect 0 0 701 162)
    UnityEngine.SendMouseEvents:DoSendMouseEvents(Int32)
     
    AmieD, Jul 22, 2017
    #646
    ina and LRavenwolf like this.

  46. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    This usually happens if your camera does not get initialized to some good value. I've got some lines at the top of ARKitRemoteConnection that is supposed to remedy this:

    Code (CSharp):
    1.             //put some defaults so that it doesnt complain
    2.             UnityARCamera scamera = new UnityARCamera ();
    3.             scamera.worldTransform = new UnityARMatrix4x4 (new Vector4 (1, 0, 0, 0), new Vector4 (0, 1, 0, 0), new Vector4 (0, 0, 1, 0), new Vector4 (0, 0, 0, 1));
    4.             Matrix4x4 projMat = Matrix4x4.Perspective (60.0f, 1.33f, 0.1f, 30.0f);
    5.             scamera.projectionMatrix = new UnityARMatrix4x4 (projMat.GetColumn(0),projMat.GetColumn(1),projMat.GetColumn(2),projMat.GetColumn(3));
    6.  
    But I am not really sure how you got your project in such a state - the scenes in my project do not have that problem.
     
    Last edited: Jul 22, 2017
    jimmya, Jul 22, 2017
    #647
    AmieD likes this.

  47. atreyukun

    atreyukun

    Joined:
    Jan 22, 2017
    Posts:
    1
    I'm having a small problem with ARKit. I'm working on an app that will let me place objects down on a surface and then change those objects with UI buttons. But the only way I can do that is to make the HitCubeParent a child of an object controller.
    Right now, I think it's breaking ARKit's functionality. I can view the objects, change them, and resize them, but I can't place them nor can I walk around them.
    So I guess my question is does parenting the HitCubeParent kill the functionality?
     
    atreyukun, Jul 22, 2017
    #648

  48. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    Doh! I realized why - I didn't account for scenes that do not have ARKitRemoteConnection gameobject. I'll fix it, but in the meantime, you could add that gameobject to your scene to make it work.
     
    jimmya, Jul 23, 2017
    #649
    LRavenwolf likes this.

  49. jessevan

    jessevan

    Joined:
    Jan 3, 2017
    Posts:
    35
    I've confirmed that my far clipping plane is set to 1000. I haven't measured exactly where the object clips on device, but I'm pretty sure its 30 meters still. Have you had a chance to test this, does it work for you?
     
    jessevan, Jul 23, 2017
    #650
Page 13 of 59
Thread Status:
Not open for further replies.