Resolved App gets rejected because of TrueDepth API, but we need to use the front camera

We have an AR app in the stores that allows people to drop virtual "time-capsules" in the real world so you can place any kind of digital content at any location.

The AR part of the app uses world-tracking only, so we're using the rear-facing camera for this. However, to let people shoot selfies (and place them as memories), we also need to switch to the front camera temporarily.

Currently, the only way to do that with AR Foundation is to enable face-tracking (Face Manager). However, enabling this package also enables the TrueDepth APIs, which we don't actually need.

Because of this, Apple keeps rejecting our app. Their argument is: You're not using TrueDepth features, so you have to disable that API. However, there is no other way to switch the cameras in AR Foundation, so we're pretty stuck. We thought about implementing a face filter (like a butterfly on your nose every time you shoot a selfie), just to please Apple, but that would be quite ridiculous to the users.

Does anybody know another way to switch cameras if in AR mode, or how to convince Apple that this is a limitation of Unity we cannot resolve?

 

The only way to get ARKit to switch to the selfie camera is to use an ARFaceTrackingConfiguration, which requires the TrueDepth camera. There's nothing ARFoundation can do about that with the current ARKit API.

If all you want is the selfie camera, you can use the WebCamTexture API to open the front camera. However, in your example of "butterfly on your nose", you wouldn't know where the face was without some form of face detection.

 

@tdmowrer Thanks for your feedback. I had tried the WebCamTexture API, but that did not work in combination with AR. The camera did not switch. So, if I get you right, that restriction is actually in ARKit, not in AR Foundation? That is good to know.

For the butterfly: that was just a weird fallback idea if Apple should insist in having some AR in the front-facing mode, too. We don't need any face-tracking. We only need to switch to the front camera to be able to shoot a selfie. And the WebCamTexture API does not work in this case.

 

@tdmowrer Since my app got rejected again, I tried the WebCamTexture API approach again. However, as soon as I try to "play" the camera texture, the AR camera (which still shows the back facing camera) freezes and thus the whole app freezes. There is no output in the Xcode logs what happens here. This confirms my first attempts.

Are you sure that it is possible to switch to the front camera using the WebCamTexture API when using ARKit at the same time?

 

@mfuad: Thanks for the confirmation.

But then I am pretty much stuck. Apple wants me to remove the TrueDepth API in order to get the app into the app store, but I need access to the front-camera (yet without using any face recognition features). I even appealed to the app review board, but they denied accepting the app if it isn't using any TrueDepth features.

I could implement a silly "butterfly-on-your-nose" feature, just to justify front-camera access, but IMHO that's stupid.

Any another idea how to resolve this deadlock?

(Not sure why this thread has been marked as "Resolved". My problem is still there.)