Question Objects either "floating" or "locked" at night-time

I'm half convinced this happens somewhere between the hardware and ARCore, so there's nothing I can do here, but just in case I'm missing something, here's what happens:

1) when the mobile phone camera with AR is pointed to a feature-less view, it can be a single-color wall, or during night-time, a blackness which naturally comes with the lack of light, the objects tend to be "stuck", rendering always at the same screen position. At first, it puzzled me because I expected the gyro and the accelerometer would be used to determine which way the user looks, but it turns out that camera image processing has a huge impact there.

2) at dusk or low visibility conditions, the objects tend to "float" in a random direction. They still behave as AR objects, they have perspective, they look three-dimensional, it's just like they are moving away. Here's an example (in the first couple of seconds of the video):

https://photos.app.goo.gl/CVC1HVBS3kMZnqNo7

This example is actually a good case, usually the movement doesn't stop and I need to restart the app. This is filmed on Pixel 4.

Since the objects actually are drawn, and since VR doesn't (AFAIK) use image processing to determine the device orientation and can work in the dark, I'm frustrated that the same thing doesn't happen for AR.

As I said, I'm not sure anything can be done, but just in case anyone has a good idea how to work around those problems, I'd be very happy to try it out!

 

ARCore uses so-called feature points to determine camera location. The rotation indeed comes from a gyroscope.
In dark environments or when your surroundings don't have enough feature points (solid-color wall, for example), ARCore can't figure out the exact position so drifting occurs.
There is not much you can do except check the ARSession.notTrackingReason and present a message to the user.

 

Do you solve ?

 

No, it is like KirillKuzyk said.

 

Yeah, it is normal for Android. At the same time, iOS devices without LiDAR (e.g., XS Max) have the similar, but a little bit better behavior at the dark time.

 

I also assumed that the gyroscope and accelerometer would be used when there was a lack of features to track. I no longer believe this is the case and I suspect it's because of power consumption and thermal issues when using both the camera and the orientation sensors for tracking. I experimented using the gyroscope for orientation tracking and letting ARCore manage the positional tracking and my phone definitely got hot much quicker. The ARCore developers probably decided it was best to go all in on visual feature tracking.

In my app I can just destroy everything when tracking is lost and then respawn the stuff when tracking recovers. Obviously this can't work for all apps.