Resolved ARKit support for iOS via Unity-ARKit-Plugin

I created "iTunes Connect TestFlight "missing Marketing Icon" - Unity 5.6.1p ARkit iOS11 Xcode 9 beta 2"
Should I now delete my original reply here? I'm new to the forum obviously. Thanks for your help.

 

UI objects on the main camera would not interfere with anything ARKit does. In any case, see the UnityParticlePainter example on adding UI (e.g. ColorPicker) to your ARKit project.

 

Would be nice Others have asked for this as well - We're looking into it.

 

Hi guys,
First thing : it's a really cool plugin we have here.
I have a small question : let's say I have this cube in the demo scene (the one with the checkboard texture). I've seen that if I walk around and can't detect many feature points, it would make the cube move away. It's like it's trying to compute new positions for the cube. From what I've read, it's the anchors which are updated.
So if I want to have a fixed cube which will not move even if I don't detect feature points (like a real world anchor which was mentioned a few posts ago) is it possible ?
It seems like every video about the ARKit are examples of non moving devices (I mean that they won't really move away from the original location of when they've started the app) and are in environments whith a lot of feature points.

I also know that at the moment there is no solution to store the state of the app to use it on a restart of anything similar.

Thanks !

 

ARKit works by using both the camera and motion sensors on the phone to do world tracking - see this article. If the data it acquires via camera (the feature point detection) is not there, the world tracking will not work properly. That's why in low light or other such situations, the TrackingState returned will be low quality, and it will revert to something like 3DOF (which is like rotations only).

 

Hi, I was wondering how to get the size of the current detected plane? Thanks

 

ARKit + Face Tracking

Has anyone seen the plugin for Unity ?

 

Scale Is a very troublesome problem. I can scale models and adjust size of my scene,but particle effect cant scale currently.

 

Yes they can. You just have to enable it in the inspector.

 

I am trying to position objects at selected coordinates within an ARPlaneAnchor object. i.e., how do I convert a point in the ARPlaneAnchor x-y coordinates into unity 3d world coordinates?

Thanks!

 

That demo is example. Its made by an israeli company that is going to turn it into a product though.

Itd be amazing if someone could create an example of that and open source. Not sure how youd do facial rec w/ arkit without using opencv as well.

 

Hey Everyone! This tool is fantastic! The only issue I am having is when trying to implement a "pinch to scale" feature for run-time dynamic model scaling. This seems to cause some issues as ARKit is trying to "take care of" position, rotation and scale. So in a way I need to step in when user input (touch) is detected and tell ARkit to listen to user input for scale over itself. Any Guidance

 

I don't think the ARkit actually changes object's (other than the camera's) position, rotation, or scale. It just moves the camera around.

 

Has anyone gone ahead and done this and is willing to share?

I haven't learned how to make shaders yet, normally I'd put the time into learning it myself but I don't have much free time at the moment and I want to get a small project done before school starts and the iOS update hits.

 

First of all, this plugin is awesome! for someone with limited unity experience and development, super badass to play around with and try out the examples and everything!

I am having a bit of some issues, when I test a scene build of the example unity particle system, the painting works, but on this project -

, I get controls and everything, but no particles. These are the error codes i am getting. Any ideas?

Code (CSharp):

1.  
2.  
3. 2017-07-03 10:49:57.847534-0500 arkitscene[501:105853] [DYMTLInitPlatform] platform initialization successful
4.  
5. 2017-07-03 10:49:57.903025-0500 arkitscene[501:105749] -> registered mono modules 0x102c01030
6. 2017-07-03 10:49:58.018817-0500 arkitscene[501:105749] You've implemented -[<UIApplicationDelegate> application:didReceiveRemoteNotification:fetchCompletionHandler:], but you still need to add "remote-notification" to the list of your supported UIBackgroundModes in your Info.plist.
7. -> applicationDidFinishLaunching()
8. Player data archive not found at `/var/containers/Bundle/Application/C4ED552F-2056-4444-9A75-7D6441FE05DB/arkitscene.app/Data/data.unity3d`, using local filesystem2017-07-03 10:49:58.050515-0500 arkitscene[501:105749] Metal GPU Frame Capture Enabled
9. 2017-07-03 10:49:58.051012-0500 arkitscene[501:105749] Metal API Validation Disabled
10. 2017-07-03 10:49:58.128376-0500 arkitscene[501:105749] [Snapshotting] Snapshotting a view (0x105e387d0, _UIButtonBarStackView) that has not been rendered at least once requires afterScreenUpdates:YES.
11. 2017-07-03 10:49:58.129202-0500 arkitscene[501:105749] [Snapshotting] Snapshotting a view (0x105e397b0, _UIButtonBarStackView) that has not been rendered at least once requires afterScreenUpdates:YES.
12. -> applicationDidBecomeActive()
13. GfxDevice: creating device client; threaded=1
14. Initializing Metal device caps: Apple A9 GPU
15. Initialize engine version: 5.6.2f1 (a2913c821e27)
16. =================================================================
17. Main Thread Checker: UI API called on a background thread: -[UIApplication delegate]
18. PID: 501, TID: 105842, Thread name: (none), Queue name: NSOperationQueue 0x1c402be40 (QOS: UNSPECIFIED), QoS: 0
19. Backtrace:
20. 4   arkitscene                          0x0000000101f73788 UnityCurrentOrientation + 60
21. 5   arkitscene                          0x0000000101f6140c __UnityCoreMotionStart_block_invoke + 108
22. 6   Foundation                          0x00000001838562c0 <redacted> + 16
23. 7   Foundation                          0x0000000183796b70 <redacted> + 72
24. 8   Foundation                          0x0000000183786828 <redacted> + 868
25. 9   libdispatch.dylib                   0x000000018192e418 <redacted> + 16
26. 10  libdispatch.dylib                   0x0000000181935c34 <redacted> + 260
27. 11  libdispatch.dylib                   0x000000018192e418 <redacted> + 16
28. 12  libdispatch.dylib                   0x0000000181935c34 <redacted> + 260
29. 13  libdispatch.dylib                   0x0000000181935afc <redacted> + 104
30. 14  Foundation                          0x0000000183857f98 <redacted> + 376
31. 15  libdispatch.dylib                   0x000000018192e418 <redacted> + 16
32. 16  libdispatch.dylib                   0x0000000181936cd4 <redacted> + 456
33. 17  libdispatch.dylib                   0x00000001819351d4 <redacted> + 660
34. 18  libdispatch.dylib                   0x000000018193bf28 <redacted> + 596
35. 19  libdispatch.dylib                   0x000000018193bc74 <redacted> + 56
36. 20  libsystem_pthread.dylib             0x0000000181bcf338 _pthread_wqthread + 1260
37. 21  libsystem_pthread.dylib             0x0000000181bcee40 start_wqthread + 4
38. 2017-07-03 10:49:58.518680-0500 arkitscene[501:105842] [reports] Main Thread Checker: UI API called on a background thread: -[UIApplication delegate]
39. PID: 501, TID: 105842, Thread name: (none), Queue name: NSOperationQueue 0x1c402be40 (QOS: UNSPECIFIED), QoS: 0
40. Backtrace:
41. 4   arkitscene                          0x0000000101f73788 UnityCurrentOrientation + 60
42. 5   arkitscene                          0x0000000101f6140c __UnityCoreMotionStart_block_invoke + 108
43. 6   Foundation                          0x00000001838562c0 <redacted> + 16
44. 7   Foundation                          0x0000000183796b70 <redacted> + 72
45. 8   Foundation                          0x0000000183786828 <redacted> + 868
46. 9   libdispatch.dylib                   0x000000018192e418 <redacted> + 16
47. 10  libdispatch.dylib                   0x0000000181935c34 <redacted> + 260
48. 11  libdispatch.dylib                   0x000000018192e418 <redacted> + 16
49. 12  libdispatch.dylib                   0x0000000181935c34 <redacted> + 260
50. 13  libdispatch.dylib                   0x0000000181935afc <redacted> + 104
51. 14  Foundation                          0x0000000183857f98 <redacted> + 376
52. 15  libdispatch.dylib                   0x000000018192e418 <redacted> + 16
53. 16  libdispatch.dylib                   0x0000000181936cd4 <redacted> + 456
54. 17  libdispatch.dylib                   0x00000001819351d4 <redacted> + 660
55. 18  libdispatch.dylib                   0x000000018193bf28 <redacted> + 596
56. 19  libdispatch.dylib                   0x000000018193bc74 <redacted> + 56
57. 20  libsystem_pthread.dylib             0x0000000181bcf338 _pthread_wqthread + 1260
58. 21  libsystem_pthread.dylib             0x0000000181bcee40 start_wqthread + 4
59. UnloadTime: 3.798500 ms
60. 2017-07-03 10:50:00.578848-0500 arkitscene[501:105841] [MC] System group container for systemgroup.com.apple.configurationprofiles path is /private/var/containers/Shared/SystemGroup/systemgroup.com.apple.configurationprofiles
61. 2017-07-03 10:50:00.579397-0500 arkitscene[501:105841] [MC] Reading from public effective user settings.
62. 2017-07-03 10:50:00.597295-0500 arkitscene[501:105922] [] network_config_register_boringssl_log_debug_updates Failed to register for BoringSSL log debug updates
63. Setting up 1 worker threads for Enlighten.
64.  Thread -> id: 17f50f000 -> priority: 1
65. 2017-07-03 10:50:00.613904-0500 arkitscene[501:105865] [] network_config_register_boringssl_log_debug_updates Failed to register for BoringSSL log debug updates
66. 2017-07-03 10:50:00.778326-0500 arkitscene[501:105842] [] network_config_register_boringssl_log_debug_updates Failed to register for BoringSSL log debug updates
67. 2017-07-03 10:50:10.709472-0500 arkitscene[501:105749] AR ANCHOR ADDED
68. 2017-07-03 10:50:10.809855-0500 arkitscene[501:105749] AR ANCHOR UPDATE
69. 2017-07-03 10:50:10.909961-0500 arkitscene[501:105749] AR ANCHOR UPDATE
70. 2017-07-03 10:50:11.009868-0500 arkitscene[501:105749] AR ANCHOR UPDATE
71. 2017-07-03 10:50:20.402950-0500 arkitscene[501:105749] AR ANCHOR UPDATE
72. 2017-07-03 10:50:34.076416-0500 arkitscene[501:105749] AR ANCHOR UPDATE
73. 2017-07-03 10:50:34.593541-0500 arkitscene[501:105749] AR ANCHOR UPDATE
74. 2017-07-03 10:50:34.893297-0500 arkitscene[501:105749] AR ANCHOR UPDATE
75. 2017-07-03 10:50:46.189883-0500 arkitscene[501:105951] [Technique] World tracking performance is being affected by resource constraints [1]
76. 2017-07-03 10:50:47.827756-0500 arkitscene[501:105951] [Technique] World tracking performance is being affected by resource constraints [1]
77. 2017-07-03 10:50:48.759698-0500 arkitscene[501:105951] [Technique] World tracking performance is being affected by resource constraints [1]
78. 2017-07-03 10:50:56.960704-0500 arkitscene[501:105749] AR ANCHOR UPDATE
79. 2017-07-03 10:50:57.060835-0500 arkitscene[501:105749] AR ANCHOR UPDATE
80. 2017-07-03 10:50:57.161012-0500 arkitscene[501:105749] AR ANCHOR UPDATE
81. 2017-07-03 10:50:57.277568-0500 arkitscene[501:105749] AR ANCHOR UPDATE
82. 2017-07-03 10:50:57.377385-0500 arkitscene[501:105749] AR ANCHOR UPDATE
83. 2017-07-03 10:50:57.461038-0500 arkitscene[501:105749] AR ANCHOR UPDATE
84. 2017-07-03 10:50:57.677713-0500 arkitscene[501:105749] AR ANCHOR UPDATE
85. 2017-07-03 10:50:57.777625-0500 arkitscene[501:105749] AR ANCHOR UPDATE
86. 2017-07-03 10:50:57.877873-0500 arkitscene[501:105749] AR ANCHOR UPDATE
87. 2017-07-03 10:50:57.978434-0500 arkitscene[501:105749] AR ANCHOR UPDATE
88. 2017-07-03 10:50:58.078388-0500 arkitscene[501:105749] AR ANCHOR UPDATE
89. 2017-07-03 10:50:58.178450-0500 arkitscene[501:105749] AR ANCHOR UPDATE
90. 2017-07-03 10:50:58.278562-0500 arkitscene[501:105749] AR ANCHOR UPDATE
91. 2017-07-03 10:50:58.378328-0500 arkitscene[501:105749] AR ANCHOR UPDATE
92. 2017-07-03 10:50:58.479721-0500 arkitscene[501:105749] AR ANCHOR UPDATE
93. 2017-07-03 10:50:58.645463-0500 arkitscene[501:105749] AR ANCHOR UPDATE
94. 2017-07-03 10:50:58.763199-0500 arkitscene[501:105749] AR ANCHOR UPDATE
95. 2017-07-03 10:50:58.862432-0500 arkitscene[501:105749] AR ANCHOR UPDATE
96. 2017-07-03 10:50:58.927064-0500 arkitscene[501:105749] AR ANCHOR UPDATE
97. 2017-07-03 10:50:58.959916-0500 arkitscene[501:105749] AR ANCHOR UPDATE
98. 2017-07-03 10:50:59.259581-0500 arkitscene[501:105749] AR ANCHOR UPDATE
99. 2017-07-03 10:50:59.659740-0500 arkitscene[501:105749] AR ANCHOR UPDATE
100. 2017-07-03 10:50:59.759978-0500 arkitscene[501:105749] AR ANCHOR UPDATE
101. 2017-07-03 10:50:59.876362-0500 arkitscene[501:105749] AR ANCHOR UPDATE
102. 2017-07-03 10:51:01.994426-0500 arkitscene[501:105749] AR ANCHOR UPDATE
103. 2017-07-03 10:51:02.094397-0500 arkitscene[501:105749] AR ANCHOR UPDATE
104. 2017-07-03 10:51:06.007833-0500 arkitscene[501:105749] AR ANCHOR UPDATE
105. 2017-07-03 10:51:06.073460-0500 arkitscene[501:105749] AR ANCHOR UPDATE
106. 2017-07-03 10:51:06.140302-0500 arkitscene[501:105749] AR ANCHOR UPDATE
107. 2017-07-03 10:51:06.190403-0500 arkitscene[501:105749] AR ANCHOR UPDATE
108. 2017-07-03 10:51:06.390658-0500 arkitscene[501:105749] AR ANCHOR UPDATE
109. 2017-07-03 10:51:06.490517-0500 arkitscene[501:105749] AR ANCHOR UPDATE
110. 2017-07-03 10:51:06.590401-0500 arkitscene[501:105749] AR ANCHOR UPDATE
111. 2017-07-03 10:51:06.773457-0500 arkitscene[501:105749] AR ANCHOR UPDATE
112. 2017-07-03 10:51:06.890419-0500 arkitscene[501:105749] AR ANCHOR UPDATE

 

I'm curious why you would want to do this? Maybe there is a simpler way to do what you want?

In any case, if this is actually what you want, we do that in UnityARUtility::UpdatePlaneWithAnchorTransform. In that case, we only use the center, but for yours can add your x-y plane coords to the center when you set the localPosition.

 

Nothing obvious in the log - check that you have not disabled the particlesystem in your scene.

 

Haven't quite got one working yet, but you can maybe get a headstart with the "SpatialMappingOcclusion.shader" that is in Unity built-in shader source.

 

ARPlaneAnchor.extent

 

Hi,
Thanks for the answer.
I have a new question : is it possible to have real world object and use it as a real world anchor ? For example a iBeacon giving the (0,0,0) position for objects in our Unity scene and then we could track it to further improve the results of the tracking of ARKit, is that something you see as possible at the moment ?

 

Thanks for the info. I basically just want to find a random point on that plane to spawn a gameobject. It seems that using a helper object via this utility function might be the easiest way to do this.

 

Hmm, just running into the next problem.

To acually retrieve a plane anchor, I would like to go through the list of all available plane anchors in the scene and pick a suitable plane in relation to the user position/orientation. However,

List<ARPlaneAnchorGameObject> arpags = unityARAnchorManager.GetCurrentPlaneAnchors ();

always returns a count of 0, even if plane objects are visible in the scene.

Is it not possible to retrieve the plane anchors directly? Or is there an easy way to retrieve all the transforms of the debugplane objects?

 

That call should work. But possibly a simpler way:

You could use the HitTest API, making sure you're looking for hits on existing planes within extents, and it will return the world position of the hit in the results. You could use the center of the screen as your origin for hit test.

 

Something like the Vuforia DepthMask shader? Take a look, its pretty simple and I think there is one on the wiki as well.

 

Has this been tested to work with 5.6.2

 

How is the "rectangular" plane created? (I assume this occurs when there are enough points to be certain that it is a flat surface? What conditions are generally required?)

 

I support his bringing this up here - working with IOS/Xcode for several years now - there are always issues with new Xcode and OS versions when attempting to submit to store, test flight, or even merely compile (like the missing camera privacy declaration we've all encountered).

The great work your team has put into rapid implementation of ARKit is pushing Unity developers to the very cutting edge frontier of IOS development - fraught with beta glitches and bugs. I personally like when others identify issues I will run into with this setup.

 

Hi!
I have a randomly crash with following error https://gyazo.com/488fe32ac021760ed5dd4d83d8f9d2ee any idea ?

 

Hi, Really basic question but probably alot of people are wondering.
How can we spawn a new instance of an object everytime a "hit" is pressed?
If you could supply a script that would work with the HitCube example ie.generate new cubes every hit that would be most helpfullCheers

 

Hi, Is there a way to run it in Unity? Or Do i Have to deploy it into the iphone to test it?
What is the best way to work with it?

Thanks!

 

Hi, I need to track an image, it can be black or white, but something it can change the color or hide to track it and do something. Is it possible?
I couldn't find anything related to this

Thanks

 

see https://developer.apple.com/videos/play/wwdc2017/602/

 

Have a look at this page: https://docs.unity3d.com/Manual/InstantiatingPrefabs.html, or also look at the UnityARBallz scene on how the balls are generated.

 

It should work.

 

Could you capture the text in the console log and post here or send a bug.

 

I don't quite understand this question. Also, please read previous entries in this forum as your questions may already be answered.

 

Hi

ARKit plugin is really awesome! I have some doubts about it:
- Is it possible to track a face using this plugin like the Snapchat filters? (if so, could you share an example or tutorial about this feature?)
- It works with iPhone 6? (if not, why?)
- Where can I find some tutorials about this plugin? Any Unity official tutorial?

Thanks in advance

 

I think you should take a look at the iOS documentation and release notes on ARKit. Here is the link: https://developer.apple.com/arkit/
Ill answer your questions below though:
1) No it does not track faces. It tracks environmental features but does not have the ability to tell you what an object is within the environment. It can estimate horizontal planes for you to place objects onto.
2) This works with iPhone 6S and newer. The reason is they have specified the A9 chipset as a minimum requirement for processing power.
3) If you take a look at the plugin or on Bitbucket there are a bunch of examples and also a tutorial text which outlines the process of getting this setup. I would also advise checking back through this thread as a YouTube tutorial was posted for ARKit.

Hope this helps.

 

Absolutely! Thank you.

 

isn't Vuforia getting integrated natively in Unity - if so, it would be great if ARKit works seamlessly

 

Is there a Unity plugin for the Vision framework you mentioned

 

Hi, I was able to install ios11 on my ipad mini 2 model A1489... when I run an example of arkit the camera doesn't work. I only see a blue screen with the ui button
Any idea? thanks!

using Unity 5.6.2 and XCode 9

 

Is Frontal Camera Supported?

 

Updated bitbucket with occlusion shader and example scene. Also updated the scenes.txt to describe the new scenes added. Asset store asset will be updated later today. @Gharry @mimminito @lusas @sam598 etc

 

This question was answeres before. Up to now it is not possible

 

If anyone has the time to help me out it would be much appreciated...

I have the newest plugin from the Asset store and was able to export a logo from Photoshop to unity as well as download a model from the asset store and view them in AR just fine. I am getting confused as to how I add a realtime shadow which I saw you have a prefab for in the plugin. How do I have a shadow appear in AR under my model? I am new to both ARKit and Unity but have been sneaking around this thread's shadows for quite some time.

If I am looking at my model in unity what would be the quickest/best way to make this happen? Thanks!

 

I have just updated on bitbucket the scenes.txt file that might help you out. Part of the update (the indented portion might be applicable): "
UnityARShadows.unity: This scene shows how to use a material that can render shadows on real objects (mainly planes in this example) by using a shader that combines shadowmapping with blending.

The main change is the GeneratePlane GO creates instances of the shadowPlanePrefab that uses the shadowPlaneMaterial, which in turn uses MobileARShadow.shader. You could also use this material on a plane that is attached to the foot of the object you want to create the shadow. When running the scene, detect a plane, and use the hittest to place a "player" object on the plane. The player object will display a shadow on the plane.
"

 

Is the shadow able to determine direction? As I understand ARKit only gives a float or light intensity in lumens but does not tell you which direction it's from