Search Unity

  1. Good news ✨ We have more Unite Now videos available for you to watch on-demand! Come check them out and ask our experts any questions!
    Dismiss Notice

ARKit support for iOS via Unity-ARKit-Plugin

Discussion in 'ARKit' started by jimmya, Jun 5, 2017.

  1. jachana

    jachana

    Joined:
    Nov 3, 2016
    Posts:
    5
    I am trying to get the RGB background image ( from the YUV material) on a texture of better on a Color32[] , i have being trying even modifying the shader to make it create an additional texture with the RGB info ... because i can't use readPixels on the mat, and i can't use readpixels on the y Material and the CbCr Material, as they are R8 and RG16 any idea how to obtain the image (in RGB or YCBCR) in a Color32[]?
     
  2. yaffa

    yaffa

    Joined:
    Nov 21, 2016
    Posts:
    34
    ANY solution there?
     
  3. yaffa

    yaffa

    Joined:
    Nov 21, 2016
    Posts:
    34
    I tried to build the Unity sameple scenefor IOs11 on Iphone 7s , with beta 5 , with Xcode 9 beta and Unity 2017.
    THe app keeps crashing on starting.
     
  4. arronpoon

    arronpoon

    Joined:
    Jul 25, 2017
    Posts:
    5
    Just updated an iPhone 7 to iOS 11 beta, rebuilt my current project with Unity 2017.1.0f3, ARKit Plugin 1.0.7 and Xcode 9 beta 5, everything is working as on beta 5;)
     
  5. esverin

    esverin

    Joined:
    Jul 3, 2017
    Posts:
    11
    Heyo guys!

    Curious on what everyone's approaches are on table edge detection! For instance in the Wingnut AR demo (Unreal Engine yeah), it seems to detect the edges of the table since people are falling off the table!
     
    phits and sama-van like this.
  6. Spooklight

    Spooklight

    Joined:
    Dec 22, 2015
    Posts:
    3
    Hello, where should I report bug about ARKit plugin ?

    I have downloaded XCode Beta 5 and UnityARKitPlugin from the asset store and on two different computers (MacBookPro & MacMini) I got the same error without adding anything else.

    upload_2017-8-15_16-26-40.png
     
    Vakyla and Elliott-Mitchell like this.
  7. mimminito

    mimminito

    Joined:
    Feb 10, 2010
    Posts:
    780
    This is just the standard plane detection that ARKit provides. Looking at the video (multiple times) I can see some of the experience must have been canned for the demo. The tracking of that table is near perfect and it gets it size so quick. Anyways, the plugin will return the plane it has found and then you can use that as a collider. When objects reach the edge, they naturally fall off.
     
  8. esverin

    esverin

    Joined:
    Jul 3, 2017
    Posts:
    11
    Huh! I've been playing around with the plane generator prefab but it sometimes causes overflowing on the edges. I wonder how they solved it! Or if the demo even had that "live".
     
  9. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    556
    It is required
     
    HonorableDaniel likes this.
  10. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    556
    which version of ios are you using? Can you give us all the information you have about your development environment?
     
  11. Aaron-Meyers

    Aaron-Meyers

    Joined:
    Dec 8, 2009
    Posts:
    283
    Apologies if this has already been covered in the thread, but I'm finding that the performance of plane detection using ARKitRemote with the editor is almost unusable. Planes sometimes pop in and out in one frame and otherwise rarely show up in what are very good tracking conditions.

    When I make a build of the same code and put it on a device, the plane detection works rather nicely as I would expect. This makes it a bit difficult to work out bits that make use of the plane detection. I've spoken to another dev who has had the same experience. Does anyone who has had the same experience know a way around it?
     
  12. janla

    janla

    Joined:
    May 8, 2013
    Posts:
    13
    I can confirm the fix. Everything stable in beta 5.
     
  13. holliebuckets

    holliebuckets

    Moderator

    Joined:
    Oct 23, 2014
    Posts:
    496
    Can confirm that was 100% live :)
     
  14. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    556
    Your mileage will definitely vary with using the ARKit Remote. The CPU load on the remote app is probably much more taxing than a normal CPU load for a Unity Game, this actually can have an effect on the performance on the tracking that ARKit provides. This is something to keep in mind while building your apps.
    Cheers,
    Chris
     
    Aaron-Meyers likes this.
  15. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    556
    At SIGGRAPH this scene was brought up as a great "best practices" use of jumping off the edge. If you look closely, the NPC actually takes quite a big leap to make up for any error that could be on either side of the edge of the table. And the effect is great. I think that was my favorite part of the demo.
     
    mimminito and JelmerV like this.
  16. sama-van

    sama-van

    Joined:
    Jun 2, 2009
    Posts:
    1,619
    @christophergoy
    Hi,

    Any news concerning the scale issue previously mentioned about the Navigation Mesh?
    Having a presentation on 20th for an rts game I wrote using the ARKit but really if the navmesh isn't working it is useless feature yet :( ....

    Would be super helpful,

    Thank you by advance! :)
     
  17. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    556
    Yes, I have a branch that I will push up tonight. I just made it up and it seems like kind of a hack, so if you are feeling froggy, you can pull it and try it out. Give me a couple of hours and I’ll post it up.
     
    Last edited: Aug 16, 2017
    sama-van likes this.
  18. sama-van

    sama-van

    Joined:
    Jun 2, 2009
    Posts:
    1,619
    I'll be there!!
     
    christophergoy likes this.
  19. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    556
    Hey all,
    So there is a branch on the bit bucket repo here that does this two camera trick to get content to scale without you having to scale your content :p This means that your to scale content can stay to scale (physics and all) and you can still render it as a teeny weeny little thing. There are some issues with clipping if you get really small, but you can adjust your far camera plane accordingly if this happens.

    The way it works is that instead of scaling your content, we are scaling a transform that is the parent of a camera. This has the effect of your content being scaled. Here is a video depicting the effect:



    One camera draws the ARKit tracking data at normal scale. These objects live on the ARKitTrackingData layer. This camera only renders things on that layer.
    The other camera renders everything but the ARKitTracking data stuff. Simple.

    This is on a separate branch so people can experiment with it. If it gains traction, we can merge it into the main line and release it in the package. Let me know if you have feedback or questions.
    Cheers,
    Chris

    edit: @petey @sama-van
     
    Last edited: Aug 16, 2017
  20. Elliott-Mitchell

    Elliott-Mitchell

    Joined:
    Oct 8, 2015
    Posts:
    58
    I'm getting the same error.
    Unity 5.6.2p4, Xcode 9 beta 5, iOS 11 beta 5, OSX 10.12.6
     
  21. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    556
    Which devices are you using? What are the logs from Xcode? Please post more information about your dev environment, otherwise it’s hard to diagnose what the issue is.
     
  22. Elliott-Mitchell

    Elliott-Mitchell

    Joined:
    Oct 8, 2015
    Posts:
    58
    I updated iOS to 11 beta 6 and it works now!!
     
  23. JelmerV

    JelmerV

    Joined:
    Nov 29, 2012
    Posts:
    198
    The camerastream with ARKit in Unity (perhaps also without Unity), is of a lower resolution than then the actual camera in the phone/tablet. Is there any way to have the full videoresolution (I believe this would be 1080p)?
    Also it seems that the camera doesn't do autofocus? Stuff in the distance is always a bit unsharp I think (I hope I can hotlink to twitter):



    This is ARKit in Unity with a daylight system based on time, date, GPS and compass =) Gives cool results I think :)
    - Any way to improve the size estimation? Maybe there is a way to tell arkit to "scan" the scene for a longer time? The car sometimes appears about 20% larger thatn it is hehe
     
    Last edited: Aug 16, 2017
    jimmya likes this.
  24. oliver-jones

    oliver-jones

    Joined:
    Nov 8, 2010
    Posts:
    25
    Im using a project that uses both Vuforia and ARKit.
    When I switch between the two, and try and start ARKit, I get 'Unsupported pixel format: 875704438', which prevents the ARKit from starting up correctly - and I'm presented with a frozen video image, with no tracking.


    Xcode Error Log;

    017-08-15 19:48:52.356520+0100 MY_APP[512:113836] [Sensor] Unsupported pixel format: 875704438

    2017-08-15 19:48:52.357276+0100 MY_APP[512:113836] [Session] Session did fail with error: Error Domain=com.apple.arkit.error Code=104 "Unsupported capture session configuration." UserInfo={

    NSLocalizedRecoverySuggestion=Make sure that the correct device and format are being used for capture.,

    NSLocalizedDescription=Unsupported capture session configuration.,

    NSLocalizedFailureReason=Input device and/or format of the provided capture session are not supported for the given configuration.}

    It's extremely vital that I get ARKit working after Vuforia has been used. Does anyone have an idea how I can fix this? I have a feeling Vuforia is changing the Camera Device formatting, and that's being cached somewhere; thus when ARKit boots up, it ends up using the cached values. That is just a complete guess, I don't actually know.

    I would appreciate any help in the matter. Thanks

    PS. This worked fine on Beta-2, but not on any recent Betas after 2.
    PPS. When switching from Vuforia to ARKit; it works. Then when switching BACK to Vuforia, and then back to ARKit; it fails.
     
    Last edited: Aug 16, 2017
  25. mimminito

    mimminito

    Joined:
    Feb 10, 2010
    Posts:
    780
    That looks amazing, great work. I would be interested to know which packages you used to achieve the final result.

    As for Auto-Focus, I dont think this is going to be supported any time soon. ARKit uses a fixed focus system so that the camera matrix calculations stay correct. If the focus had to change then the calculations would constantly have to be updated, resulting in performance issues. This is my understanding, I could be wrong though :)
     
  26. mimminito

    mimminito

    Joined:
    Feb 10, 2010
    Posts:
    780
    Yeah, makes sense. They managed to line up the content on that table pretty well, probably due to the height of the table vs the backdrop and the floor, they could determine its location easier. It was an impressive demo. Do you have a link to the video on this at SIGGRAPH?
     
  27. MSFX

    MSFX

    Joined:
    Sep 3, 2009
    Posts:
    115
    pretty sure you can't build to a generic device... you need to select your actual device!
     
  28. MSFX

    MSFX

    Joined:
    Sep 3, 2009
    Posts:
    115
    looks awesome, great effort! That the standard shader or something else?
     
  29. JelmerV

    JelmerV

    Joined:
    Nov 29, 2012
    Posts:
    198
    Thanks, I mostly used standard components (direct light, unity skybox and standard shader) + a script to set the position of the direct light. But right now (not shown in the photo) I have found a way to combine unity's skybox with a reflection map, best of both worlds.
    Pity about the auto focus, hope they will find a way to include it for scenes that are larger than table top (like a car on 1:1 scale)
     
  30. Bird_LYKKE

    Bird_LYKKE

    Joined:
    Jul 26, 2017
    Posts:
    15
    Can I adjust camera resolution?
     
  31. Trupacz

    Trupacz

    Joined:
    Aug 16, 2017
    Posts:
    3
    Hi
    Which elements i can't destroy when swaping scenes from arkit scene to something without it? I'm asking cuz in my current project have problem when swaping from scene using ARKit plugin to scene without, it only happens when testing app on my device.
     
  32. esverin

    esverin

    Joined:
    Jul 3, 2017
    Posts:
    11
    Out of curiosity! What's everyones experience with vertical planar tracking? Reading the second paragraph on https://developer.apple.com/arkit/ , they only mention horizontal surfaces. But what if i wanted to preview artwork on my walls?
     
  33. JelmerV

    JelmerV

    Joined:
    Nov 29, 2012
    Posts:
    198
    not possible (yet). Some other frameworks allow for it I think (saw a video here earlier), but the tracking is less good than ARKit.
     
  34. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    556
    We scale the video texture to fit the aspect ratio of the device. This is by the recommendation of the ARKit team. This question was already answered. Cool daytime effect!
     
  35. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    556
    Sounds like they aren’t playing together nicely. ARKit might be trying to grab the texture from the camera in a different format than what Victoria is requesting. At least that is what it seems like from your error log
     
  36. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    556
    No, unless you buy a different device :p
     
  37. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    556
    This isn’t in yet so you’d have to write a plane detection algorithm to process the point cloud data and estimate a vertical plane.
     
  38. esverin

    esverin

    Joined:
    Jul 3, 2017
    Posts:
    11
    Does that mean that it will be included? ;)
     
  39. oliver-jones

    oliver-jones

    Joined:
    Nov 8, 2010
    Posts:
    25
    Is there any way to always force the correct format every time ARKit Starts?
    Thanks.

    -- update --

    I find it weird that if I use Vuforia first, and then ARKit after; it works just fine. Only fails when I switch back to Vuforia, and then back to ARKit. :/
     
    Last edited: Aug 16, 2017
  40. sama-van

    sama-van

    Joined:
    Jun 2, 2009
    Posts:
    1,619
    @christophergoy
    That looks great!
    A bit late in Japan now....
    We’ll be on it once to the office tomorrow for sure!
    Thank you so much!! ^_^
     
    christophergoy likes this.
  41. harperAtustwo

    harperAtustwo

    Joined:
    Nov 15, 2016
    Posts:
    21
    I read (past tense) about 7 pages on this forum but there are 23 so forgive me if this has been asked before. How do I get real world scale data into Unity from ARK kit. Like if I wanted to know the real world distance between 2 points. Like in this ruler demo.
     
  42. mimminito

    mimminito

    Joined:
    Feb 10, 2010
    Posts:
    780
    Its already supported out of the box. 1 unit in Unity is 1 metre in reality. So you just need to use something like Vector3.Distance to calculate the distance between two Vectors inside your scene and you should have your solution!
     
    jimmya likes this.
  43. mimminito

    mimminito

    Joined:
    Feb 10, 2010
    Posts:
    780
    Amazing work! Thanks for doing this. This question popped up for our game recently and it was quickly dismissed, but I think we will revisit the idea now :)
     
    jimmya and christophergoy like this.
  44. harperAtustwo

    harperAtustwo

    Joined:
    Nov 15, 2016
    Posts:
    21
    That is awesome. Thank you.
     
  45. Spooklight

    Spooklight

    Joined:
    Dec 22, 2015
    Posts:
    3
    Update about errors.

    Software: Unity 5.6.2p4, XCode beta5, ARKit plugin from Asset Store.
    Error when launching the app after XCode compilation.

    Here is the log:
    2017-08-16 17:57:24.960792+0200 arkitscene[322:40831] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[ARFrame displayTransformWithViewportSize:eek:rientation:]: unrecognized selector sent to instance 0x1c02ef780'

    *** First throw call stack:

    (0x185a3bd38 0x184f50528 0x185a491f8 0x185a416e4 0x1859270dc 0x10049d734 0x19fe42ff8 0x1853c0fec 0x1853c0fac 0x1853cdad8 0x1859e3f20 0x1859e1afc 0x1859022d8 0x187788f84 0x18ee34c90 0x100123040 0x18542656c)

    libc++abi.dylib: terminating with uncaught exception of type NSException

    (lldb)

    The problem is probably easy, but I have really low understanding of the ARKit library. Does anyone have encountered this error ?
     
    Aiursrage2k likes this.
  46. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    556
    This has been posted at least a handful of times. displayTransformWithViewportSize is deprecated and you need to update the ARKit plugin. Make sure you are at least on Unity 5.6.2.p2 or later. And make sure you have the latest plugin code. Please search through the forums before posting questions, as this has been answered many, many times.
     
  47. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    556
    Hey all,
    A bug fix was just pushed up to the BitBucket Repo that fixes a crash bug when importing the ARKit Plugin. The Asset Store Package is Pending review. If you were hitting a crash during import, I suggest you update to this latest version.
    Cheers,
    Chris
     
  48. nickfourtimes

    nickfourtimes

    Joined:
    Oct 13, 2010
    Posts:
    180
    I've created a session using
    Code (CSharp):
    1. m_session = UnityARSessionNativeInterface.GetARSessionNativeInterface();
    We want to give players the option of turning AR off; is this enough?
    Code (CSharp):
    1. m_session = null;
    Or is there an equivalent Destroy() method for a UnityARSessionNativeInterface?
     
  49. CManzione

    CManzione

    Joined:
    May 11, 2015
    Posts:
    7
  50. christophergoy

    christophergoy

    Unity Technologies

    Joined:
    Sep 16, 2015
    Posts:
    556
    Your Xcode is out of date. Please refer to the many posts about having the latest xcode/ios/arkit plugin combination in order to keep up to date with the development. And please search the forums before asking questions. You will find that your answer is somewhere here most of the time.
     
unityunity