Resolved ARKit support for iOS via Unity-ARKit-Plugin

Been working on an ARKit app for a few months, and suddenly (after updating to iOS12 and building to Xcode10) the AR camera renders some of the bottom at the top, and parts of the top at the bottom of the screen as seen on the picture below.

The UI is positioned just fine, but the AR camera feed from has som weird issues.

Any suggestions?

 

Hi @JoMaHo ,

I have not seen this behavior. Do you have a Unity project that you can share that reproduces this issue?

Also, on what type of device are you running?

Todd

 

Hi!
Unity 2017.2.2, Xcode 10.1 betal, iOS 12.1 on an iPhone 7.

I could just send you a stripped project in an PM for you to test?

best

 

Sent you a pm with download link for test file. Hope you can find anything,.

best

 

Hi @JoMaHo ,

I was able to find the source of your video issue and the proper fixes that address the problem.

I will start off by saying that your version of the Unity ARKit plugin code is quite old (11-12 months). Ideally, you should update this to the latest code in the "default" branch of the repository. Though doing so would require some time, it could be beneficial with numerous other fixes as well as additional ARKit functionality. The latest in "default" exposes ARKit 2.0 functionality into Unity.

That said ... to fix your camera issue specifically, you need to merge in at least the following 4 changesets. The changes in this 4 changesets will address your video issue.

https://bitbucket.org/Unity-Technol...mits/83d9dfe1c211bae9eaaf537b08976d56fec820e8
https://bitbucket.org/Unity-Technol...mits/3f2853dfcdbbec044545617780d5237c114baec4
https://bitbucket.org/Unity-Technol...mits/44be34bfd07a9dab138b5d43834858d11a499f42
https://bitbucket.org/Unity-Technol...mits/b0355f702c426cb285be4d0d55c1bca8f485479c


Todd

 

I'm using the latest unity and the latest plugin from the default branch and the remote app is very very slow. Is there a fix? Device is 8+

 

Hi!
That was a massive thing; is all this because of iOS12?

Thank you for solving it!

best
Johnny

 

Hi, I'm using the ARKit 2 plugin for image tracking, however when the image is lost, the prefab that is generated stays on screen instead of disappearing. Is this the default behaviour? And if so, is there a way to make the model disappear once the image is lost?

Thank you

 

Hi @bluemode ,

Take a look at the changes that are in the "image-anchors/adding-istracked-field-to-image-anchors" branch.
https://bitbucket.org/Unity-Technologies/unity-arkit-plugin/src/default/

Todd

 

Does anyone know if the autofocus speed can be increased?

I am using Vision's native barcode recognition in combination with ARKit, but autofocus speed is relatively slow.
Any thought on how to increase it is welcome.

 

Can someone describe how to work with Environment Probes?
I'm want one probe, that updates itself.
The DebugReflectionProbePrefab doesn't upate itself, correct? (reflections stay grey on my iphone8)

 

Hi @JelmerV ,

ARKit environment probe functionality is only enabled in iOS 12. What version of iOS are you running?

Have you tried the scene in Assets/UnityARKitPlugin/Examples/ARKit2.0/UnityAREnvironmentTexture/UnityAREnvironmentProbeAnchorManual.unity ?

When I run this example scene, I am able to tap on planes in the AR scene to place environment probe anchors represented by spheres. See the attached image.

Todd

 

Why is the ARKit plugin not available anymore on the asset store?
https://assetstore.unity.com/packages/essentials/tutorial-projects/unity-arkit-plugin-92515

I downloaded the Bitbucket repository but I actually need to import the plugin in an already existing project. What's the best way to proceed? Thanks.

 

Just copy the UnityARKitPlugin folder under Assets in the repo to your Assets folder in your project. You may have to adjust your ProjectSettings for some of the advanced features, if so, check the differences between your ProjectSettings folder and the one in the repo.

 

Well that doesn't sound good.

 

What doesn't? Having to mess with the Project settings? If you didn't change the ProjectSettings in your project, you can just take the one that is distributed with the plugin. Otherwise you will need to merge in your changes from the plugin, whether you got it from Bitbucket or Asset Store or anywhere else.

 

I am seeing the same exception. Does anybody have a suggestion how to debug this? There are no call stacks. Thanks!

 

Hi, guys. I have two questions.

Why the asset was removed from the asset store? Bitbucket is now the only way to install it?
Why the camera on iPad is zoomed in? Is this expected or is it a bug?

I posted a message about the camera on iPad and I opened a support ticket but I didn't get any answer. I guess the support guy thought I was asking how to fix my code.

I posted some pictures already if you want to see what the camera looks like on an iPad 2017 in ARKit.
https://forum.unity.com/threads/ipad-camera-zoomed-in.569497/

Thanks
JD

 

I meant that it wasn't available in the Unity Store anymore. At that time I happened to not be able to get it from BitBucket either so I thought maybe something had happened and you had to cancel the project or something.

I'm glad that's not the case!

 

While I'm here, I have a question too. I'm finding that all anchors created by ARUnityGeneratePlane have the same Y value of zero. If I scan the device over the floor and then a table I would expect that planes would be generated with different Y values. Am I wrong about that?

 

Hey, though it is a bit late, it might help someone. So, This worked perfectly for me for the scale issue.
https://blogs.unity3d.com/2017/11/16/dealing-with-scale-in-ar/

There is a video explaining this here:

Since, I use the mixture of scale method and you never really change the scale of the objects, the physics works great!

 

Why is the ARKit plugin gone from the Unity assets store? will it still be supported on bitbucket in the future? or is it to get people to use AR Foundation?

 

Hey,

I'm trying to render a second ortographic camera to show some 3D spheres in my UI. Whether I pick clearflags Don't clear or Depth only, I get really weird artifacts in the part that should let through the ARKit video feed.
Any ideas what to do?


Camera settings:

 

Try setting the depth to 1.

 

hello @jimmya and @todds_unity
I am facing freeze issue while using image anchors. After the app starts I can see camera feed only for a second. UI is completely working fine. I am using sample scene provided with the plugin and performed all the checks about image size in Xcode and tested with jpg also but the issue remains same.
Here are development details.

• plugin copied from bitbucket default branch
• Xcode stable10.0 (10A255)
• Unity 2018.1.0f2
• device iPad, ios 12.0.1(16A404)

I've found many persons having this issue. Also found on StackOverflow that issue is caused by XCode stable and suggested to use Xcode beta 3.
Please let me know if there is any quick fix available. we are really stuck with this from a couple of days.

 

Hey! Turned out it was the HDR flag. The main camera is already HDR so I guess 2 cameras with HDR is too much

But thanks, it works now, quite happy with the result!

Off topic, but It is a little hard to align the UI elements with the 3D rendered spheres. I only get it to work for one aspect ratio. When changing to iPhone X it is all misaligned again.
If anyone has tips..?

 

It says on the asset store the ArKit Plugin is deprecated? Did I miss something..

 

I think you have to use the package manager and install it as a package rather than an asset

 

Not quite - Unity ARKit Plugin is now available as a download from BitBucket: https://bitbucket.org/Unity-Technologies/unity-arkit-plugin/downloads/?tab=tags

 

What's the "cleanest" way to get the plugin included into a new project? I mean; import the minimal amount of stuff, without the example scenes and whatnot.

 

There is no minimal set separated out well, but all the files live under the UnityARKitPlugin folder, so it should not pollute any of your project files. And extra files in the project should not affect your final build if you are not referencing it.

 

Hi!
I can not record at 1080p on my iPhone 7. I've tried using the provided ARVideoFormats example in ARKIT1.5, but it seems that my phone will only record video in 720p, I need 1080p. Using Unity 2018.2.13, iPhone7, Xcode10.1, iOS beta5 of 12.1

Any suggestions?

 

This change has made quite a difference for me, regarding the speed of the ARKit Remote.

https://bitbucket.org/Unity-Technol...in/issues/78/remote-very-low-fps-and-repeated

 

Am writing some custom shaders for use w/ ARKit. One of my shaders applies the camera's transpose model view transformation matrix to the object's vert positions to create a camera-facing billboard of the gameobject. Unfortunately when I use it w/ ARKit I'm getting unexpected repositioning of my verts after the matrix is applied. Especially noticeable w/ up/down panning and rotation.

Any feedback for what might be going on under the hood? Has been painful trying to debug what's happening.

Not at all married to the transpose matrix, just need to get the verts into static positions in front of my camera to capture a rendertexture from the correct position/FOV.

Thanks in advance!

 

Hi, I am learning Arkit through the Udacity course and the remote is confusing me.
The remote works, but it's stuck on the first frame. I can move it around only on the editor and not on my iPhone X.
When I open it on the unity remote app, it freezes at first frame.
The project I download is this one https://github.com/udacity/arkit-unity-course-assets

 

Hi there,

I'm getting weird jumps in the reflections when using the environment probes. Using Unity 2018.2.16

At first I thought the object was getting past the probe's box but it still happens even after making the object smaller, or the box really big.
It's not due to the probe being updated either, since it goes back and forth even when I'm not moving, and doesn't happen if I walk around the object, only when spinning the object.
The reflection seems somehow accurate, I can discern parts of the room in both textures.

It looks like the probe swaps between different faces of the cube map?

I'm using the default code and prefabs from the ARKit plugin examples.

Any help appreciated!

 

Any plan when we can have iPad 2018 supported by ARKitRemote? Thanks jimmy

 

I can't connect my iPhone XR to Unity via USB(works fine with wifi), could this be a bug of Unity?
https://bitbucket.org/Unity-Technol...73/when-trying-to-use-arkitremote-with-iphone

 

I also cannot connect over USB (iPhone XS, iOS12). Is there a way we can lower the resolution of the frames sent over wifi? I tried naively reducing the size of the textures (currently CbCR is downscaled by 2, so I made that for a downscaled luminance by 2) but this crashed in frame decompression, so probably needs a bit more debugging. Before I go down that rabbit hole I'd love to know if there's a better fix (I only just started using ARKit so am assuming remote performance was fine for eberyone over usb)

Unity 2017.4.15
XCode 10.1
iPhone XS, ios 12.1

Edit to add: Even if I use an iPhone 6S, which must be connected via USB because I've switched it to airplane mode before connecting, performance of ARKit remote still isn't anything like in the videos (or as I'm sure it was when I tried it ages ago). Perf is more like 1FPS in the editor using the standard ARTestScene.

 

Set Unity Remote ->

device to Any iOS device or Any Android Device

from

Edit>Project Setting>Editor before make build for ARRemoteScene for device

 

Tried, still not working on XR
iPhone X works fine as usual

 

Hi guys anyone been able to MULTIPLE save and load serialized maps. My aim is to create a new map use hit testing and create cube (in each new map I create I create new coloured cube so I can tell each map is different).

Now I can create a new map by running a new config with worldMap as null then hittest and create a cube on planes. I save this map ie MapOne to a persistent path with name as MapOne.worldmap using WorldMapManager.SaveSerialized(). Then I add this to my mapsList in MapHandler.cs. Once the map is saved I load it using WorldMapManager.LoadSerialized(ARWorldMap map) giving it the map and path so it knows the right map to load which it does perfectly.

The problem is I save the map and the cube I created doesn't destroy after saving the map. If I destroy the cube after saving the map it destroys but when I load the map it doesn't display the cube maybe because I destroy it after saving the map. Bear in mind when calling WorldMapManager.LoadSerialized it attempts to relocalise and runs with a new config. Config.map = mySavedMap and with options such asUnityARSessionRunOption.ARSessionRunOptionRemoveExistingAnchors | UnityARSessionRunOption.ARSessionRunOptionResetTracking.

I understand my cube is not destroyed by default as of the example in UnityARWorldMapSerialized.scene but how can I destroy my cube after saving my map and still be able to load it when I select load scene, because now if I destroy my cube after saving the map it will not appear when I load the very same map. Any ideas??????

 

Is it known or listed that this device is not supported yet?
Im running into an in issue where, when I try to connect the new iPad pro 2018 with Remote, I get ApplicationException: Couldn't create device API for device.

 

really? the ipad 2018 isn't supported? where did you read that?
i'm having problems, too, with my ipad 2018 (but works with iPhone SE)... sigh

 

ARKit Camera Feed Very Dark / Low Exposure

Hello All,

I am building an ARKit application via the Unity ARKit plugin (Bitbucket latest version)
However the camera feed is very dark compared to other ARKit applications. (e.g. Measure from Apple).

Is there a way to make the camera feed brighter / more exposure?
Thanks!
Doruk

 

@dorukeker Maybe try using a different color space? In Player settings -> others -> rendering ->color gamut (or color space), try to switch from srgb to gamma, it will make everything really bright, but might not be the look you're after. Doc here: https://docs.unity3d.com/Manual/LinearRendering-LinearOrGammaWorkflow.html

 

Would somebody be able to help me find out the ISO speed ratings with the current AR Camrea? I know how to get this data using Av Capture Through the Metadata . But every time I try to use Av Capture In my unity seen it completely messes up the ar kit scene. I have tried grabbing the meta-data from the pixel buffer But the method that I’m using does not seem to produce the ISO speed. If you do not know how to grab the iSo speed Would anybody know how to run both the Av Capture class using Web textures and the unity ARC and at the same time? Without them interfering with each other.

 

You can do one of two things - see @Numa suggestion below, or you can change the Clear Material on the UnityARVideo component on the camera to YUVMaterialLinear (and leave ColorSpace as Linear).

 

ARWorldMap is an ARKit data structure that saves the map of the real world around you, so that you can start from the same coordinate system as last time. The cube you create is a virtual object and does not get saved with ARWorldMap. If you wanted to save its position, you could place a user anchor in a specific spot and that will get saved with the world map. Then when you loaded the map, you would get a user anchor with the transform of where you need to create your cube.