EXC_BAD_ACCESS when getting pinned array handle using UnityARKitPlugin

I'm trying to read a YUV camera frame using Unity ARKit Plugin, into a pair of pinned byte arrays using SetCapturePixelData with a double-buffer. To do this, I need to first pin the array in managed memory to stop the GC changing its address, then pass the address of that array into the native plugin code, which will then write it. To prevent write/read violations I'm double buffering, so each new frame we switch the write between two pairs of byte arrays and read from the last one that was written.

On almost all compatible iOS devices, this works fine (iPhone 7, 7+, X & the iPad 9.7 2017/5th Gen); however on the iPad 2018/6th Gen, I get an EXC_BAD_ACCESS when trying to read back the address from the pinned handle, after pinning the array.

I've attached a minimally viable Monobehaviour that should show what I'm trying to do. I don't think there's anything particularly weird with the code attached, but would be keen to know if anyone can spot anything I've done wrong. Essentially, this is a stripped down version of the code used in Unity's experimental AR Interface to retrieve the camera frame.

Since this only occurs on a single device, I suspect it's a bug in the underlying native code (probably an extra free somewhere), so I've also logged this as an issue on the UnityARKitPlugin issue tracker; if I get a response, I'll update or delete this question accordingly.

(We are looking at moving over to AR Foundation, but for our current timescales, this probably isn't an immediate option for us).

 

Can you just use UnityRemoteVideo.cs and just change the connectToEditor.SendToEditor to instead do whatever you want to do with it?

 

Hi jimmya, thanks for your response, sorry for my slow reply. That code is pretty much the same as what we're using. Unfortunately, when we pin the array to push it into `SetCapturePixelData`, we get the issue.

 

The access to the pixelbuffer is time sensitive - it only lasts for a frame- and then the pointer will be pointing to garbage. See how the access works in that other source file and try to conform to its timing. You can see how that works by using the ARKit Remote which uses that code, and should not have the same issue.

 

I can confirm the same issue occurs when using the provided Unity ARKit Remote scene on the 6th gen IPad. EXC_BAD_ACCESS during the read back of the pinned address.

 

One thing to try is to find lower resolution video formats and use them - you might be spending too much time copying all the data across for the higher resolutions. (See the "Video Formats" section of https://blogs.unity3d.com/2018/02/16/developing-for-arkit-1-5-update-using-unity-arkit-plugin)