Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Resolved ARKit support for iOS via Unity-ARKit-Plugin

Discussion in 'AR' started by jimmya, Jun 5, 2017.

Thread Status:
Not open for further replies.
  1. jimmya

    jimmya

    Joined:
    Nov 15, 2016
    Posts:
    793
    One other thing that causes this might be the "Camera Usage Description" on the player settings (also affecting some people earlier in this forum). If these are not working, please submit a bug with your project or xcode project included and put the bug number in a post in this forum and i will take a look.
     
  2. Aiursrage2k

    Aiursrage2k

    Joined:
    Nov 1, 2009
    Posts:
    4,835
    Well I guess the only way to do that would be to somehow it live on a cloud. Basically we want to be able to do things like find your way to store x in a mall, or navigate your way through a hospital. So being able to walk to the nodes at any point rather then having to always start at a location
     
  3. KwahuNashoba

    KwahuNashoba

    Joined:
    Mar 30, 2015
    Posts:
    110
    I'm working on a very similar problem. The idea, I came up so far with, is to have markers at some positions that will sinc your position in augmented and real world, practicaly offset the whole world or players avatar in augmented world once you scan marker. But for that I need to find a nice (if any) way to merge vuforia with ARKit. Since I have a bit of time I will start working on it now.
     
  4. cybertron86

    cybertron86

    Joined:
    Jun 12, 2017
    Posts:
    2
    Thank you for the nice post. I run into some issues when replacing the HitCube with my own model. The new model just floats in the air instead of sitting on a surface like HitCube does. Please help. This is what I did: in the UnityARKitScene, I replace the 'HitCube' with another model and placed new model under 'HitCubeParent'. The Transform of the new model is set to X=0, Y=3.2, Z=0. Y is set to 3.2 because my model is larger than HitCube. I copied the 'Unity AR Hit Test Example' script to the new model. The Hit Transform is kept as HitCubeParent. Thank you!
     
  5. glenrhodes

    glenrhodes

    Joined:
    Oct 25, 2013
    Posts:
    56
    Thanks for this. Got it up and running without issues. The tracking is amazing for something coming straight from the camera. Playing around with building a low poly city on my sofa:



    As the planes resolve, their local axes of course rotate to an arbitrary Y rotation, so the buildings align to the rotation of the plane.
     
  6. jimmya

    jimmya

    Joined:
    Nov 15, 2016
    Posts:
    793
    For those that haven't had a chance to, to learn more about the underlying framework of this plugin, please watch this introductory WWDC session: https://developer.apple.com/videos/play/wwdc2017/602/

    BTW, many of the apps shown at during that talk were made with unity plugin. And we worked with other developers too to integrate this plugin with their titles.
     
    Last edited: Jun 12, 2017
    KwahuNashoba and nickmc1210 like this.
  7. nickmc1210

    nickmc1210

    Joined:
    Jun 9, 2017
    Posts:
    5
    Hello again

    Got another issue. I’ve got an ARPlaneAnchorGameObject and am attempting to add a new GameObject to it using a script. The object gets added to the scene appropriately, but it is always created and anchored to (0,0,0) corresponding to the ARSession’s initial position. I deliberately set the position and rotation of the new GameObject’s transform to match the ARPlaneAnchorGameObject’s transform, the one to which the script belongs:

    NewGameObject.transform.position = ARPlaneAnchorGameObject.transform.position.... same for rotation.

    Yet the new game object remains at (0,0,0).

    Is there some specific function or transform action I’m missing? The new GameObject is built from a prefab in the script, and the prefab’s original coordinates are indeed (0,0,0)... but that shouldn’t matter after I’ve set the transform to match the “parent” transform, should it?
     
  8. Joyceawt

    Joyceawt

    Joined:
    Feb 23, 2017
    Posts:
    11
    is there a way to link a cloud database or some sort of database to have different set of animations that pop up? say i had a collection of different animations made with ARkit and unity, is there a way to select a particular one?
     
  9. jimmya

    jimmya

    Joined:
    Nov 15, 2016
    Posts:
    793
    The center of your object will be 3.2 meters above the plane your hittest result gives you. When you are in the editor scene view, change the transform on the object until it is resting on the ground (x-z) plane.
     
  10. cybertron86

    cybertron86

    Joined:
    Jun 12, 2017
    Posts:
    2

    Thanks jimmya.

    Would you please help address the following question? Many Thanks.

    I run into some issues when replacing the HitCube with my own model. The new model just floats in the air instead of sitting on a surface like HitCube does. Please help. This is what I did: in the UnityARKitScene, I replace the 'HitCube' with another model and placed new model under 'HitCubeParent'. The Transform of the new model is set to X=0, Y=3.2, Z=0. Y is set to 3.2 because my model is larger than HitCube. I copied the 'Unity AR Hit Test Example' script to the new model. The Hit Transform is kept as HitCubeParent. Thank you!
     
  11. onchekwu

    onchekwu

    Joined:
    Aug 18, 2013
    Posts:
    2
    Removing "Camera Usage Description" in player settings worked. The app runs now! Thank you!

    FYI, this still appers in the logs : "Main Thread Checker: UI API called on a background thread: -[UIApplication delegate]" with the backtrace following. For those of you encountering this problem, it seems unrelated.
     
  12. Joyceawt

    Joyceawt

    Joined:
    Feb 23, 2017
    Posts:
    11
    hi, i need some help. i downloaded unity and the plugins but all my assets are disappearing once i launch unity. even when i manually import the asset it says that the assets are already in the folder and it appears to be empty. just to add i think this only happened after i updated my mac's ios version.
     
  13. kaiyiwong

    kaiyiwong

    Joined:
    May 1, 2017
    Posts:
    2
    Hi, thanks for the plugin! Been testing out on different things. I do have one question after seeing some other examples though, how can I cast shadow on the 'ground' in real environment? I see that's when it makes any example even more interesting.
     
  14. rockstarsaad

    rockstarsaad

    Joined:
    Sep 26, 2015
    Posts:
    7
    Hi Jimmya, I been working on something big since the release of this plugin but only major issue I am having is I don't know how to create these plane anchors manually. Is it possible to do that and how? Please let me know I really need your help on this one. Thanks mate & appreciate all the support.
     
    Last edited: Jun 13, 2017
  15. SweatyChair

    SweatyChair

    Joined:
    Feb 15, 2016
    Posts:
    140
    Is there any way to get it to work in Unity Remote? Or what is the best practice for fast debugging? :)
     
  16. imaginethepoet

    imaginethepoet

    Joined:
    Aug 23, 2016
    Posts:
    44
    So this is really working well in ios. One challenge I'm having is lighting. Any tips on how to improve that? Perhaps I am in just too dark of an environment but most of my models show up really dark.
    Thanks
     
  17. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    Hey @Joyceawt,
    Check out this thread for info about your assets disappearing. https://forum.unity3d.com/threads/unity-and-macos-10-13-high-sierra.474527/

    There is a new preview build of unity to fix this issue, but it's risky upgrading to High Sierra, or any Beta OS.
    Cheers,
    Chris
     
  18. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    Hi @kaiyiwong,
    What other people have been doing is adding a drop shadow to their assets to make it look like it's casting a shadow. If this is done correctly, it can add a subtle, but noticeable, effect to your scene. I hope this helps.

    Cheers,
    Chris
     
  19. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    Hey @rockstarsaad,
    If you take a look at the UnityARAnchorManager there is code in there that shows how to subscribe to Anchor Events within ARKit, and how to create these planes from the data you receive in those events. Let me know if this helps you.
    Cheers,
    Chris
     
  20. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    Hey @SweatyChair (cool name and picture btw),
    Unity Remote is an interesting Debugging tool we have considered for long term support. Please leave feedback at https://feedback.unity3d.com/ so it can be tracked. I believe having and editor iteration loop would be ideal, but we only have so many people with so much time :).
    Cheers,
    Chris
     
  21. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    Hey @imaginethepoet,
    I would check a few things:
    • Make sure you have a light in your scene and see if it is hooked up to the ambient light that comes back from ARKit
    • Make sure your shaders on your assets are using said light
    • tweak the ambient lighting value we get back from ARKit to best fit the rendering of your assets with light/shader combination
    If you could share more information about your scene privately with me, I'd be happy to answer your questions.
    Cheers,
    Chris
     
    tonOnWu likes this.
  22. Joyceawt

    Joyceawt

    Joined:
    Feb 23, 2017
    Posts:
    11
    @christophergoy thanks so much for the tip! i downgraded my OS back to the pre-beta version. I have one question though - I tried test building the scene, but it doesn't completely stay fix and the cube keeps 'running away' how do i fix this problem? also would you happen to know how i could upload the 3d objects to a database and then have them be downloaded depending on the user's choice?
     
  23. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    Hey @Joyceawt,
    What device are you running on? Apple Recommends that you run ARKit on the iPhone 6s or later. If you run it on older devices, this "running away" problem is expected.
    Cheers,
    Chris

    edit: Sorry I forgot about your other question. I believe the way developers normally do this is with asset bundles. This is a way, within unity, to distribute assets to users from a server. This is a pretty large topic to cover here, and I'm sure it's been spoken about many times if you search through the forums. Here is a link to the manual for it: https://docs.unity3d.com/Manual/AssetBundlesIntro.html
     
    Last edited: Jun 13, 2017
  24. Joyceawt

    Joyceawt

    Joined:
    Feb 23, 2017
    Posts:
    11
    @christophergoy hey! im actually using an iphone 6s. that' why im a little confused as to why this is happening, the cube doesnt need to be anchored by planes like the ball example right?
     
  25. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    Hey @Joyceawt,
    Can you create a repro project for us, file a bug report, and post the case number here?
    Cheers,
    Chris
     
  26. jimmya

    jimmya

    Joined:
    Nov 15, 2016
    Posts:
    793
    The cube just exists in the world at around 0,0,1. Another reason this could be happening is that you're using the AR3DOFCameraManager for some reason (its normally disabled in the scene). When you rotate the phone, does the cube seem to rotate? Then this is most likely the reason. Also, if you have low light or ARKit is otherwise unable to track properly in your environment, it might be doing that. Another possibility is that you've scaled the cube to be really large? The basic project without any changes should work properly out of the box in normal lighting conditions.
     
  27. kaiyiwong

    kaiyiwong

    Joined:
    May 1, 2017
    Posts:
    2
    Thanks Chris, I'll try to figure it out then.

    Another question, I'm trying to create a toggle to show the prefab planes/point cloud and hide them in within an app. Is there already a way to do it right now? It seems like the planes are being generated and added to the scene right now and not tied to any parent object that we can show/hide? Also where can I find the trigger when an anchor is detected and the location of it? Thanks!
     
  28. Joyceawt

    Joyceawt

    Joined:
    Feb 23, 2017
    Posts:
    11
    Thanks for your helps! I think I figured it out. I've also been trying to figure out the lighting/shadows for an object I've placed in the view. How do I configure the shadows/lighting to adjust based on the user's angle?
     
  29. nickmc1210

    nickmc1210

    Joined:
    Jun 9, 2017
    Posts:
    5
    UnityARAnchorManager, under the Utiliy directory, is in charge of converting ARPlaneAnchors to ARPlaneAnchorGameObjects. It also used the UnityARSessionNativeInterface class, you may want to look at that as well.
     
    jimmya and Blarp like this.
  30. SunnyChow

    SunnyChow

    Joined:
    Jun 6, 2013
    Posts:
    360
    i know in unity 1 unit length is 1 meter, is ARKit on scale?
     
  31. John1515

    John1515

    Joined:
    Nov 29, 2012
    Posts:
    248
    Yes! See my video :) Though I have no idea how they do that with just visual data? Perhaps they compare the gyroscope movement with visual movement? And/or they use the camera´s focal length?

    Small video of my experience with AR Kit. Skip to 0.25 for AR kit.

    Sorry for the weird voiceover, first time i made a sort of videoblog..
    Mostly impressed with the scale of the tracking. Even when walking away and pointing back at the object, it was in the same location. Wondering when we'll see some sort of SLAM. Vertical planes tracking would be a welcome addition.
     
    jimmya, christophergoy and petey like this.
  32. rockstarsaad

    rockstarsaad

    Joined:
    Sep 26, 2015
    Posts:
    7
    Hey Chris Thanks for the answer but that was not what I mean. I meant i want to create anchor as for now it is detected and created automatically by ARKIT but i want to create it manually and disable automatic creation.
     
  33. jimmya

    jimmya

    Joined:
    Nov 15, 2016
    Posts:
    793
    Creating planes in the world in unity is straightforward, and the UnityARUtility functions do that for the planes that ARKit detects. I don't quite understand how/why you are going to "create" anchors - you would have to figure out where your anchors are based on the video? Could you give us more details on what you're trying to do?

    [UPDATE: After Chris's post I realized you might be talking about the addAnchor and removeAnchor functionality. ]
     
    Last edited: Jun 14, 2017
  34. jimmya

    jimmya

    Joined:
    Nov 15, 2016
    Posts:
    793
    Yes, you can parent the planes generated to a GameObject if you wish to control them that way. The "PointCloudParticleExample" GameObject in the scene can be used to control the point cloud particles. See the tutorial (step 10) for information on how to hook into the anchor detection and update events.
     
  35. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    Hey @rockstarsaad,
    Sorry I misunderstood your post. There is an API to manually add anchors to ARKit that was added a bit later that we didn't add to the native interface yet. We will add that support (of if you're feeling froggy you can follow our example and add it yourself too :)). Here is their API on ARSession

    Code (CSharp):
    1. - (void)addAnchor:(ARAnchor *)anchor;
    2. - (void)removeAnchor:(ARAnchor *)anchor;
    These functions allow you to add and remove your own anchors to the ARSession.

    There is no support for turning off ARKit's detection and creation of Anchors internally, You can update the ARKitSessionConfiguration to turn off plane generation in ARKit. There is code in the UnityARCameraManager.cs you can look at to see how to do this. You can definitely remove the game object creation from the example scene that we have provided. I hope this answers your question.
    Cheers,
    Chris

    edit: updated to correct myself :p
     
    Last edited: Jun 14, 2017
  36. christophergoy

    christophergoy

    Joined:
    Sep 16, 2015
    Posts:
    735
    Hey @Joyceawt,
    I'm not sure I understand the question. Unless your source of light is attached to the camera, the shadows should remain relatively static relative to the light source in your scene. Can you please clarify?
    Cheers,
    Chris
     
  37. jimmya

    jimmya

    Joined:
    Nov 15, 2016
    Posts:
    793
    Updated the repo with a "MobileARShadow.shader" that can display shadows on "real" surfaces behind it by using multiplicative blending. To demonstrate how to use it, have included a mesh (the player from survival shooter) and setup into a new scene UnityARShadows.unity. It is similar to the first scene, except GeneratePlanes GameObject now points to shadowPlane prefab which has a material that uses the new shader. And the HitCubeParent now uses a "Player" GameObject as the target for the hit test, and that will display a shadow on the plane that you have hit.

    Credit to https://alastaira.wordpress.com/201...-unity-vertexfragment-shader-in-7-easy-steps/ for showing me the way.

    [UPDATE: updated the linked unitypackage to this version as well]
     
    Last edited: Jun 14, 2017
    Griffo and kaiyiwong like this.
  38. jimmya

    jimmya

    Joined:
    Nov 15, 2016
    Posts:
    793
    See this post for a possible solution.
     
    kaiyiwong likes this.
  39. Method-Studios

    Method-Studios

    Joined:
    Feb 5, 2015
    Posts:
    2
    Hi jimmiya,

    What exactly UnityARAlignment enum references to?
     
    Last edited: Jun 15, 2017
  40. jimmya

    jimmya

    Joined:
    Nov 15, 2016
    Posts:
    793
  41. digitald

    digitald

    Joined:
    Jun 14, 2017
    Posts:
    3
    Anyone have any tips on how to get a frame from the camera? WebCamTexture seems to break motion tracking.
     
  42. jimmya

    jimmya

    Joined:
    Nov 15, 2016
    Posts:
    793
    The best way to get it depends on what you want to use it for. UnityARVideo.cs does this for the examples to render the video as the background of the camera. ARKit provides the video frame as two textures in the YUV format: one texture contains the Y component, the other contains the U and V components. Right before we render anything in the scene using the camera, we use a command buffer to combine the two textures using YUVShader to get the resulting video on the camera rendertarget (think of it as a "clear to video").

    You can either get the data from the camera rendertexture, or you could do the same operation the command buffer does to create a separate texture for your needs. If you need to access the pixels in either case, that is going to be an expensive operation (copy from gpu to cpu).
     
    elhongo and rocket5tim like this.
  43. rockstarsaad

    rockstarsaad

    Joined:
    Sep 26, 2015
    Posts:
    7
    Hi and Thanks Chris for explanation. Can you please add the API properly and update the repo please. And Jim I am trying to add anchor myself and disable automatic detection because it keep on changing anchor position thus moving the object referenced. Apple advised to disable automatic detection after you detect the plane but what if we could be able to ask users to create the anchor themselves and disable auto-detection altogether? Here is Apple solution in their documentation:

    "Allow time for plane detection to produce clear results, and disable plane detection when you have the results you need. Plane detection results vary over time—when a plane is first detected, its position and extent may be inaccurate. As the plane remains in the scene over time, ARKit refines its estimate of position and extent. When a large flat surface is in the scene, ARKit may continue changing the plane anchor’s position, extent, and transform after you’ve already used the plane to place content."
    Source: https://developer.apple.com/documen...ed_reality?changes=latest_minor&language=objc

    Please let me know if I am wrong or misunderstanding something.
    Thanks guys
     
    Last edited: Jun 15, 2017
  44. Blarp

    Blarp

    Joined:
    May 13, 2014
    Posts:
    269
  45. mimminito

    mimminito

    Joined:
    Feb 10, 2010
    Posts:
    780
    So, I know that ARKit does not support re-localisation right now which is a shame. I am thinking about the concepts for how this all works and wondered if it was possible to implement our own re-localisation? So the idea being you save AR frames and the point cloud data into a file. When you launch the app again you read back this data and compare it with what your device is currently showing you. If its matches fairly closely (obviously you would need some degree of inaccuracy) then you can relocalise and place content back down in your new session based on where it was placed in its last session?

    There is probably an underlying reason this will not work (origin of tracking?) but its a thought and wanted to gauge peoples interest here and see if anyone could chime in on its possibilities?

    Thanks,
    Adam
     
    Rickert0_o likes this.
  46. mimminito

    mimminito

    Joined:
    Feb 10, 2010
    Posts:
    780
    Also, any chance this could be placed onto the Asset Store at some point and the updated? Using the asset store is great for many reasons. I see you now have a Unity Package available but still, Asset Store would be neat :)
     
  47. Arvin6

    Arvin6

    Joined:
    Jun 2, 2017
    Posts:
    27
    I'm now facing a similar issue. Was your phone having a blue screen like the screenshot I have attached below? I have also provided a camera description, still facing the issue.

    P.S. I'm running it on a iPhone 6s with iOS 11 beta.

    https://forum.unity3d.com/conversat...ac-10-11-5-and-unity-5-6-1p1-playmode.483854/
     
  48. OvidiuGhe

    OvidiuGhe

    Joined:
    Oct 1, 2012
    Posts:
    12
    norahx3x, christophergoy and jimmya like this.
  49. jimmya

    jimmya

    Joined:
    Nov 15, 2016
    Posts:
    793
    How are your users going to figure out where in the virtual world to put the anchor - use a hittest? (that will be inaccurate unless a plane anchor has already been found)? Instead of pinnng virtual objects to an anchor, I would instead use a hit test against existing planes, figure out where on the plane they object should be placed, and place it in the world in that position (not as a child of the planeanchor). Then the object will not move around even if the anchor that was found does.
     
  50. jimmya

    jimmya

    Joined:
    Nov 15, 2016
    Posts:
    793
    ARKit is super accurate once you have the origin position and rotation. So one idea might be to have a stand or holder at the point at which you started creating the content, so that when you later want to consume the content, you just have to restart tracking from that position and rotation by putting your phone on the holder/stand and pressing a button on screen. Then all you have to save is the unity scene elements at their positions and all should work as expected.

    Note that this kind of solution could also work to sync up two or more devices into the same space in the real world, for example if you wanted to play a multiplayer game.
     
Thread Status:
Not open for further replies.