Search Unity

ARKit support for iOS via Unity-ARKit-Plugin

Discussion in 'ARKit' started by jimmya, Jun 5, 2017.

  1. domdev

    domdev

    Joined:
    Feb 2, 2015
    Posts:
    326
    Hi, is it possible to change size of the 3d object base from the size of the plane? that will not streched the 3d object? I was having hard time with the proper size of the 3d object.. I got multiple object it like a small city..
     
  2. historysa

    historysa

    Joined:
    Jun 26, 2015
    Posts:
    3
    Hi

    I have managed to connect to the device using 'Start AR Remote Session', and the camera fires up on the device, however nothing changes on the Unity editor screen. Hopefully there is an easy fix without upgrading to a 2017 version as I have a fixed license

    I am using Unity 5.6.4

    Cheers
    Oliver
     
  3. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    https://blogs.unity3d.com/2017/11/16/dealing-with-scale-in-ar/
     
  4. arturmandas

    arturmandas

    Joined:
    Sep 29, 2012
    Posts:
    197
    Hi,
    thanks for this, will be testing it, looks like you might have saved me couple of hours of cursing and unnecessary work.

    Best!
     
  5. arturmandas

    arturmandas

    Joined:
    Sep 29, 2012
    Posts:
    197
    Tested, doesn't work, it fails the same way. Any idea? Anyone managed to solve this some other way.

    Thanks!
     
  6. natwales

    natwales

    Joined:
    Dec 5, 2012
    Posts:
    10
    FYI- I saw the pull request for intrinsics in the repo and manually patched the latest build so I could access arkits intrinsics property in the unity plugin. I'm using this for the purposes of marker detection in opencv. I notice that arkits intrinsic parameters deviate slightly from what I am getting when I calibrate manually off a checker board. My manual measurements work spot on. Using arkit intrinsics everything is just a little bit off. I'm not sure if it's my error or if its related to this here . I'd be curious to know if anyone has had this working well in Unity.
     
  7. mikeyt0078

    mikeyt0078

    Joined:
    Nov 20, 2017
    Posts:
    8
    bump? anyone have any advice on this?
     
  8. hanskwork

    hanskwork

    Joined:
    Sep 12, 2017
    Posts:
    1
    Hi All -

    Here's a sample "review" application for construction that we created using Unity and the ARKit plugin.

    It allows a user to create pushpin issues, localized to the model, and synced with a server. Other users can create and sync issues. It's very basic in its functionality, but still generated a lot of excitement for a real world AR use case.



    We plan to publish a set of blog articles describing how it was created along with releasing the code. Here's the first of the articles:

    https://forge.autodesk.com/blog/conxtech-rethinking-job-site-apples-arkit-forge

    Thanks again for the ARKit plugin and support.
     
  9. RyanYN

    RyanYN

    Joined:
    Aug 13, 2013
    Posts:
    18
     
  10. RyanYN

    RyanYN

    Joined:
    Aug 13, 2013
    Posts:
    18
    Thank you RoboticEyes, you are so cool!!
     
  11. arturmandas

    arturmandas

    Joined:
    Sep 29, 2012
    Posts:
    197

    Thanks! Precious tip. Could be in a FAQ section.
     
  12. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    Thats fantastic! - thanks for sharing.
     
  13. Kholeslaw

    Kholeslaw

    Joined:
    May 7, 2013
    Posts:
    11
    I'd like to get ARKit Remote working. I had it working on my new iPad a couple of weeks ago, but I rebuilt the Remote App after updating ARKit Plugin to the latest version and it will not connect to Unity.

    I am running Windows 10 and Unity 2017.2.0f3
    I am running an iPad that supports ARKit (I can build and run my AR game just fine)
    I have "Development Build" turned on. (see attached image)
    I run the Remote App before pressing play in the Editor.
    I have reset/restarted Unity, the iPad, and the App in different combinations.
    I have ensured I am using the same version of ARKit Plugin for both the Remote App and Editor.
    I have tried building and connecting with the latest Asset Store version as well as the latest Bitbucket version.
    I have tried building and connecting with a new, clean project, with both the Asset Store and Bitbucket versions.
    I have tried using USB-2 and USB-3 ports.
    I have tried connecting via the Wired connection and the Wireless (IP) connection via the Editor options.
    I have read the README, as well as watched the tutoral videos and read through previous posts in this thread.


    When I try and connect to the editor via the USB option, I get the following connection error:

    Attempting to connect to player ip: 127.0.0.1 with a 1 ms timeout
    UnityEditorInternal.ProfilerDriver:DirectURLConnect(String)
    UnityEditor.AttachProfilerUI:SelectProfilerClick(Object, String[], Int32)


    I don't know why it's trying to find Localhost IP for a USB connection? Is there another setting in Unity, or something blocking USB connectivity?

    I am running out of things to try. Like I mentioned, it has worked for me before and I don't know what I have done to break it. It is probably something simple that I am overlooking. Hopefully someone knows what it is. :)

    Thanks!
     

    Attached Files:

  14. 8bitRick

    8bitRick

    Joined:
    Nov 18, 2013
    Posts:
    11
    For anyone else having this problem of far clipping plane not working - I only have one camera in my scene but the far clipping plane doesn't appear to update (even with the special script: " UnityARCameraNearFar.cs"). Maybe it should be working but I'm doing something dumb. Anyway, I've found if I just hack "UnityARCameraManager.cs" in the start where Matrix4x4.Perspective is set to 30.0f, just change this to whatever you want and it will stick. That's my hack for anyone else who runs into this problem ;) ... I'll follow up if I ever figure out what other dumb thing I may have did to require this but I'm happy to say I can look beyond 30 meters now.
     
  15. aaron_l

    aaron_l

    Joined:
    Nov 20, 2017
    Posts:
    1
    Hi, I've noticed that there's a LightDataType.DirectionalLightEstimate datatype which contains the direction of the light. However, the light data that unity receives never seems to be of this type.

    Is there any way to access this information? I'm looking at the files UnityARAmbient.cs and ARLightEstimate.cs.
     
  16. kushG

    kushG

    Joined:
    May 25, 2017
    Posts:
    18

    New

    Were you able to resolve this ? I'm getting the same error

    @christophergoy @pixelshader
     
    Last edited: Dec 1, 2017
  17. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    This only gets returned when you use face tracking on iPhone X. See https://developer.apple.com/documentation/arkit/ardirectionallightestimate
     
  18. JelmerV

    JelmerV

    Joined:
    Nov 29, 2012
    Posts:
    188
    I'm trying to spawn some objects randomly on the generated debugplanes. But I can't get a hold of those planes in my code (don't want to raycast, prefabs should just pop up here and there)
    I was hoping there's some sort of event that my function can subscribe to?
     
  19. Aiursrage2k

    Aiursrage2k

    Joined:
    Nov 1, 2009
    Posts:
    4,832
    I spent a stupid amount of time trying to get it to work but eventuallyI got the markerless and marker working using vuforia 7 in the 2017.3 version of unity. It actually works pretty great, all you need to do is add the image-target, and add the planefinder.

    I was watching a vuforia video where they were giving a talk. Basically there are 40k+ android devices so only a fraction of them will work, but only 63 ios devices and yuforia said 93% work. So if you have a device can use ARKit or ArCore(? didnt test that) it will otherwise I think it will try to do it using there own implementation
     
    Last edited: Dec 4, 2017
  20. ywj7931

    ywj7931

    Joined:
    Jun 24, 2015
    Posts:
    41
    Is there a way to get the mesh model of ARkit facial tracking? I want to map the arkit's blendhshape into mine. Is there detail about the blendshape?
     
  21. ina

    ina

    Joined:
    Nov 15, 2010
    Posts:
    845
    Project Settings - minimum SDK required should be 11.0?

    I noticed that the Unity asset store download requires importing complete project. Should the project settings for this asset be set to minimum sdk 11.0? (I am assuing ARKit requires SDK11.0+ or does it work with an earlier?)
     
  22. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    Indeed - you can look at how UnityARAnchorManager does it - hook UnityARSessionNativeInterface.ARAnchorAddedEvent etc. If you just wanted a list of currently existing planes, UnityAnchorManager.GetCurrentPlaneAnchors() would do it.
     
    JelmerV likes this.
  23. shawnblais

    shawnblais

    Joined:
    Oct 11, 2012
    Posts:
    215
    Is there a way to delete all existing planes, short of creating a new AR session? Sometimes the plane results from ARKit can get a little messed up, and it would be nice to just wipe the data and try again.

    Also, is there a working example of the single-camera approach we can look at? I've tried to get it working for a cpl hrs, and can't seem to get it right. Would be really nice to just see a functioning example.
     
    Last edited: Dec 5, 2017
  24. piotrtekien

    piotrtekien

    Joined:
    Feb 16, 2015
    Posts:
    7
    We tried sending the TextureFormat as R8 using the TextureY but that didn’t work out as you have suggested this in a comment before, but this results in an unusable texture for our OpenCV Markers.

    We also tried to convert the IntPtr buffer to Texture2D but that results in a black screen.

    We have put the renderTexture from the camera to a Texture2D using ReadPixels and that works, but it scans the entire screen and that's practically unusable on bigger resolutions like iPads have.

    If there was a way to send a texture natively from Xcode to Unity or doing something with the texture in Xcode natively then that would be a beautiful solution. Any ideas? We'd love to hear from you!
     
  25. Kholeslaw

    Kholeslaw

    Joined:
    May 7, 2013
    Posts:
    11
    Nobody knows what this problem is? I have seen a number of posts from other developers experiencing this same error message.

    When I try and connect Remote to the editor, I get the following connection error:

    Attempting to connect to player ip: 127.0.0.1 with a 1 ms timeout
    UnityEditorInternal.ProfilerDriver: DirectURLConnect(String)
    UnityEditor.AttachProfilerUI:SelectProfilerClick(Object, String[], Int32)

    I don't know why it's trying to find Localhost IP for a USB connection?

    thanks
     
  26. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    There is a runoption for removing existing planes.
     
  27. shawnblais

    shawnblais

    Joined:
    Oct 11, 2012
    Posts:
    215
    Cheers, here's the code if anyone else needs it. You can just call it over and over, each time you want to re-init the tracking session:
    Code (CSharp):
    1.     public void InitARSession() {
    2.         bool getLightData = true; bool getPointCloud = true;
    3.         var sessionConfig = new ARKitWorldTrackingSessionConfiguration(
    4.             UnityARAlignment.UnityARAlignmentGravity,
    5.             UnityARPlaneDetection.Horizontal, getLightData, getPointCloud);
    6.         UnityARSessionNativeInterface.GetARSessionNativeInterface().
    7.            RunWithConfigAndOptions(sessionConfig,
    8.             UnityARSessionRunOption.ARSessionRunOptionRemoveExistingAnchors |
    9.             UnityARSessionRunOption.ARSessionRunOptionResetTracking);
    10.     }
    I tested it and it seem to work well.

    Any word on a working example of single camera scaling?
     
  28. shawnblais

    shawnblais

    Joined:
    Oct 11, 2012
    Posts:
    215
    Also I'm a bit confused how the 2-camera scaling works when you have multiple pieces of content on different planes, and you're not necessarily looking directly at any of them, meaning you don't really have an "area of interest" per se, but would still expect them to be visible. Or even a single piece of content, when you're not looking directly at the plane. Does this technique only work when focusing on a single content plane, and only when looking directly at the plane?

    I can see in the example that you toggle visiblity of the content when there's no valid HitResult, so I'm guessing yes? If so, then I guess single camera scaling is the only way to go for us, since we're working on a tower which is 3-4ft tall, and our players will not typically be looking down at the planeAnchor itself, meaning no hitTest result.
     
  29. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    Doesn't the one in github.com/Unity-Technologies/experimental-ARInterface work?
     
  30. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    If you remove the "| UnityARSessionRunOption.ARSessionRunOptionResetTracking" part of it, it will not reset the tracking - i.e. you will be in the same world coordinates as before resetting.
     
  31. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    HitTestResult is only used to figure out the point of placement.
    The point of interest just means where on the scene you want to point the camera at - should not matter where you look, the scene will appear as if the point of interest was placed on the point of placement. Try out the castle scene in the github project. And watch the Unite Austin video.
     
  32. shawnblais

    shawnblais

    Joined:
    Oct 11, 2012
    Posts:
    215
    Thanks Jimmy, after downloading and going throug that repo, I realize that much of my frustration was because I was working with the old code over on bit bucket.

    The new examples and components in the AR-Interface repo are mucho better :) Great choice to just simplify hit tests using RayCast against your generated planes, using the same code for editor and device.
     
    Last edited: Dec 6, 2017
  33. hawken

    hawken

    Joined:
    Aug 22, 2013
    Posts:
    514
    I have the same issue here, using the asset store version of the code drop, the remote shows up black and can't tap
     
  34. hawken

    hawken

    Joined:
    Aug 22, 2013
    Posts:
    514
    Unity-ARKit-Plugin (bitbucket)

    or

    experimental-ARInterface (github)

    ARKit-Plugin
    • can get from bitbucket
    • same as asset store
    • face tracking access
    • incompatible with experimental-ARInterface
    • no ARCore
    experimental-ARInterface
    • can get from github
    • no face tracking
    • ARCore
    • incompatible with bitbucket version of ARKit-Plugin (has own version)
    • has scaling and other scenes
    • has different approach to scene (AR Root)
    Which to get?

    The experimental version lacks facetracking but has Scaling and ARCore. The experimental version cannot be added to a project with the bitbucket / asset store version of ARKIt (has it's own slightly different version)

    Both contain the ARKitRemote but must be built to device separately for each (not transferrable between scenes run from different repos)
     
    Last edited: Dec 6, 2017
  35. shawnblais

    shawnblais

    Joined:
    Oct 11, 2012
    Posts:
    215
    Even if you don’t need Android support, you should grab the ARInterface repo. The examples are much cleaner, the components are better thought out, and it has working scaling. Basically it seems like the unity guys have moved on from the ARKit sdk without ever really cleaning it up or refactoring it, instead putting all that work into ar-interface.

    They've also added controls to the camera so you can move it around in the editor, and a nice EditorInterface class so you can basically simulate the entire lifecycle of the ARKit app inside the editor. Which makes the Remote a little redundant. At this point if it works in the editor it probably works on the device.

    Easiest way to do this for me was just to toast the ARKit folder in my existing project, then import the entire AR-Interfact repo. Then tweak your components to talk to ARInterface rather than ARKit.
     
    Last edited: Dec 6, 2017
  36. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    If you read the README.md for that repo, you will realize that the ARInterface is just experimental, and builds on top of the Unity ARKit Plugin and Google's ARCore plugins. The intent of it was to provide an experimental cross-platform layer that could work with both platforms. You should still be able to use the latest Unity ARKit Plugin with the ARInterface by deleting the one that is in there, and copying the latest UnityARKitPlugin folder over. Does that not work for you?

    Also, Unity ARKit plugin has not been abandoned at all as you claim - all the new features of ARKit SDK have shown up there - check the commits on the bitbucket repo.
     
    Last edited: Dec 6, 2017
    KwahuNashoba likes this.
  37. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    i just tried it here, and it seems to work fine for me. You will have to provide more information if it's not working for you. Check to make sure you have read the FAQ to eliminate the obvious. If it still does not work, let me know your Unity version used to build ARKit Remote (is it same as used to try to connect to it?), which device and what iOS version is on it, which scene you're using to connect, any console log messages if you run the remote under XCode and try to connect to it?

    As for tapping on device screen, we don't support that. We have an EditorHitTest script that allows you to use mouse in the editor to "tap" on a plane to simulate a hittest on a plane.

    [Update] : Ok, I seem to see the issue on Unity 2017.2.0f3, while it was fine on 5.6.2. Investigating further.
     
    Last edited: Dec 7, 2017
    hawken likes this.
  38. JelmerV

    JelmerV

    Joined:
    Nov 29, 2012
    Posts:
    188
    I'm trying to update my app but I just don't understand it anymore! I'm using the alt-scaled-content branch (is that one good?) .
    I cannot match the FocusSquare's position with the position of a scaled GameObject..

    I thought something like this would just work:
    ScaledContentGO.transform.position*scaleFactor = FocusSquare.transform.position;

    I've been on many detours now so hope there's a solution :)
     
  39. shawnblais

    shawnblais

    Joined:
    Oct 11, 2012
    Posts:
    215
    I was mainly referring to the examples and components, not functionality. I didn't say you abandoned it, I said you guys moved on without a "clean up and refactor". In ARInterface the new examples are clean, the components are de-coupled and lean. Whereas the ARKit examples are clearly from another point in time when everything is in a much more half-baked state and the examples are very cluttered and redundant.

    Worth pointing out that if you accidentally start with the ARKit examples, you will be wasting a lot of time, as everything has been refined in ARInterface. You guys might want to consider back-porting the new architecture and cleaning up the ARKit examples if you intend for that to stand on it's own as a repo, and not have people rely on ARInterface.

    I wasted a solid day and half refactoring components from the ARKit examples, and trying to figure out scaling, only to find that you guys had already done all the work over in ARInterface. Which I originally ignored, as I had no interest in supporting Android.
     
    Last edited: Dec 6, 2017
    hawken likes this.
  40. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    Fair enough, though the way it ended up happened to be a side-effect of building ARInterface on top of the existing plugins, and having to make it cross-platform, so it was easier to figure out something cleaner. We have in fact been trying to port some of the examples from ARKit plugin over to ARInterface so that we can have those available across platforms, as well as on remote interfaces and editor interface.
     
    shawnblais likes this.
  41. hawken

    hawken

    Joined:
    Aug 22, 2013
    Posts:
    514
    I'm moving forward with the ARInterface version as a base, mostly for the scale function.
     
  42. SandeepRajbhar

    SandeepRajbhar

    Joined:
    Dec 6, 2017
    Posts:
    1
    I Am getting this error . Please help

    Assets/Plugins/iOS/UnityARKit/Editor/UnityARBuildPostprocessor.cs(22,17): error CS0246: The type or namespace name `UnityARKitPluginSettings' could not be found. Are you missing an assembly reference?
     
    cherrypicks likes this.
  43. terop

    terop

    Joined:
    Nov 10, 2017
    Posts:
    2
    Jimmy, what is the polycount of the Mr. Sloth character used in your iPhoneX face tracking demo? Or what do you think is the maximum polycount if we want to use all the 51 blendshapes?

    Thanks!
     
  44. Parallax-hk

    Parallax-hk

    Joined:
    Mar 28, 2017
    Posts:
    4
    I have two scene call tutorial and AR,while i go to the scene tutorial during playing in Scene AR,and get back to Scene AR, after scanned the floor, the planes will generated but will suddenly disappear. the planes still work but sometimes can not see it.
     
  45. jimmya

    jimmya

    Unity Technologies

    Joined:
    Nov 15, 2016
    Posts:
    793
    Alright I've put in a fix for this (video on remote device not showing up when using ARKit Remote). The Camera in the ARKitRemote scene had "Dont Clear" in the Clear flags instead of "Depth Only". "Don't Clear" should still work as it does on Unity versions earlier than 2017.2, but there seems to be a change to the interpretation of that flag. Anyway, if you get latest on bitbucket (commit) it should fix this issue.
     
    Dev_Sebas, mikeyt0078 and ColdPants like this.
  46. mikeyt0078

    mikeyt0078

    Joined:
    Nov 20, 2017
    Posts:
    8
    Perfect this works. Thanks @jimmya
     
  47. Numa

    Numa

    Joined:
    Oct 7, 2014
    Posts:
    54
    What's the correct way to update the plugin once it's being used in a project? Is it safe to update via the asset store? (it gives a warning about replacing the project settings, which I don't want). How do you guys do it?
     
  48. shawnblais

    shawnblais

    Joined:
    Oct 11, 2012
    Posts:
    215
    ARInterface has a snapshot of the latest Android/iOS repos. So you can just download that, and then grab the ARKit folder and past manually into your project.
     
  49. sama-van

    sama-van

    Joined:
    Jun 2, 2009
    Posts:
    1,533
  50. sama-van

    sama-van

    Joined:
    Jun 2, 2009
    Posts:
    1,533
    Another very small request due to every revert on my project on each plugin update...
    Could we have an easy way to disable/enable the debug mode from PointCloudParticleExample.cs

    I understand that is an "Example" script but using it as it is, is not that bad too.
    Just need to be a bit more flexible, that would be super nice! :)

    Thanks by advance!
    Screen Shot 2017-12-10 at 11.36.13.png
     
unityunity