Hi I am working on an augmented reality game, which has an unity scene as an overlay to the iPhone camera view. What I want to achieve is that when the phone is rotated up with the camera facing the celling or down towards the ground, the unity camera will follow realistically making it look like the object in the scene is where it would be expected. So far I am unity the iPhone gyroscope for the rotation. Code (csharp): newAngles.x += (Input.gyro.rotationRate.y * (180 / Mathf.PI)) * Time.deltaTime; However the rotation gets offset after a while due to the relative calculation of the rotation. What would be the best way to recalibrate the gyroscope? I tried to experiment with the attitude of the gyroscope Code (csharp): Gyroscope.attitude It gives a quaternion which I cant assign directly to the camera angles as the values seem a bit strange to me. As far as I know the values are suppose to be the yaw, pitch and roll. But it seems like some of the angles are dependent on each other. For instance when rotating around the X axis the value of the Y rotation suddenly switches so its inverted when using the euler angles representation. Im not sure why this happens?

Okay I am looking for the exact same thing and will look into it but the place I am stuck is accessing iPhone camera from unity! :S How did you do that?

Im using the Qualcomm AR plug in for unity. It includes an AR camera which uses the iPhone camera. https://developer.qualcomm.com/press/qualcomm-extends-augmented-reality-platform-unity-technologies

There's a gyro cam script somewhere on the forums that does exactly that. I'm using it for an augmented reality game myself

Is this the one you are referring to? http://forum.unity3d.com/threads/98...-camera-on-iPhone-4?highlight=gyro+cam+script

The gyro drifts over time, but it's great for detecting sharp movements. I made a script last week that smoothly corrects it based on compass data, which works quite well - it's a challenge because the compass itself is a bit naff. Still, I was happy with the result. On my Android tablet the compass is absolutely awful, though. Code-wise, the gist is to use gyro.attitude as a basic orientation with good high-frequency response, and also calculate the corresponding orientation based on the compass reading. Then calculate the difference, i.e. compassOrientation * Quaternion.Inverse(gyroOrientation), and maintain a correction quaternion that interpolates towards this difference (various algorithms to choose from depending on the accuracy-stability trade-off you're happy with). Finally, premultiply this smoothed correction factor to the gyro orientation, and you should get something which matches the compass when the device is stationary, but is more responsive when the device is rotated. I had to post-multiply gyro.attitude by a 90 degree rotation, too, before doing anything else - I think this is because the device was initialised in landscape mode. Here's the code, anyway. Code (csharp): using UnityEngine; public class MoveWithCompass : MonoBehaviour { private double _lastCompassUpdateTime = 0; private Quaternion _correction = Quaternion.identity; private Quaternion _targetCorrection = Quaternion.identity; private Quaternion _compassOrientation = Quaternion.identity; void Start() { Input.gyro.enabled = true; Input.compass.enabled = true; } void Update() { // The gyro is very effective for high frequency movements, but drifts its // orientation over longer periods, so we want to use the compass to correct it. // The iPad's compass has low time resolution, however, so we let the gyro be // mostly in charge here. // First we take the gyro's orientation and make a change of basis so it better // represents the orientation we'd like it to have Quaternion gyroOrientation = Quaternion.Euler (90, 0, 0) * Input.gyro.attitude * Quaternion.Euler(0, 0, 90); // See if the compass has new data if (Input.compass.timestamp > _lastCompassUpdateTime) { _lastCompassUpdateTime = Input.compass.timestamp; // Work out an orientation based primarily on the compass Vector3 gravity = Input.gyro.gravity.normalized; Vector3 flatNorth = Input.compass.rawVector - Vector3.Dot(gravity, Input.compass.rawVector) * gravity; _compassOrientation = Quaternion.Euler (180, 0, 0) * Quaternion.Inverse(Quaternion.LookRotation(flatNorth, -gravity)) * Quaternion.Euler (0, 0, 90); // Calculate the target correction factor _targetCorrection = _compassOrientation * Quaternion.Inverse(gyroOrientation); } // Jump straight to the target correction if it's a long way; otherwise, slerp towards it very slowly if (Quaternion.Angle(_correction, _targetCorrection) > 45) _correction = _targetCorrection; else _correction = Quaternion.Slerp(_correction, _targetCorrection, 0.02f); // Easy bit :) transform.rotation = _correction * gyroOrientation; } }

Ah cool I was looking for this as well thanks for posting! Could you explain this part tho: Code (csharp): // Work out an orientation based primarily on the compass Vector3 gravity = Input.gyro.gravity.normalized; Vector3 flatNorth = Input.compass.rawVector - Vector3.Dot(gravity, Input.compass.rawVector) * gravity; _compassOrientation = Quaternion.Euler (180, 0, 0) * Quaternion.Inverse(Quaternion.LookRotation(flatNorth, -gravity)) * Quaternion.Euler (0, 0, 90); // Calculate the target correction factor _targetCorrection = _compassOrientation * Quaternion.Inverse(gyroOrientation); How is the compass orientation calculated?

Just a note: One of the reasons the various AR libraries exist (and they are pretty much all based on visual image marker tracking), is that gyroscope is just not accurate enough to keep the unity camera and real camera in sync. I know there is a QR code tracking libary somewhere on the forums, which @jasperstocker used to do an AR driving game in Unity a while back. Also, definitely worth looking at QCAR/Vuforia (free) or String (+$).

You can use the magnetic or true heading fields of Input.compass, but I found calculating my own values worked better. So, I take the gravity vector from the gyro, and use it to flatten the magnetic flux vector from the compass sensor, giving me a horizontal "north" vector, relative to the device. This whole calculation is in portrait-device-relative coordinates. Then I make an orientation quaternion from that. Quaternion.LookRotation gives a rotation from relative-to-north-vector to relative-to-portrait-device - but I want the opposite, so I run it through Quaternion.Inverse. Then, I post-multiply by a Z rotation, because I told Unity in the project settings that the device would be landscape and it has aligned the main camera that way. I think this would not be necessary if I told Unity to use portrait mode. I also pre-multiply by a 180-degree X rotation, just because that seemed to be necessary too... I didn't think about it much, I just observed what was going on and added the rotations that seemed necessary to make the camera behave in a stable way. Once you've got _compassOrientation, you can assign it straight into transform.rotation if you like and experiment. It only updates infrequently, so things will be jerky, but you can still see whether it's giving the right orientation. This is how I determined whether the device responded to rotations in the right axes (if not, change the post-multiplied quaternion) and after that whether, overall, "north" and "up" were in the right direction (if not, change the pre-multiplied quaternion). Just put distinctive objects in the Unity scene above, below, and in the four cardinal directions from the camera, then see if they're in the right place in "real world" space. _targetCorrection is calculated as the thing I need to premultiply by in order to exactly turn gyroOrientation into _compassOrientation. I want X such that X * G = C. Post-multilpy by Ginv, and you get X = C * Ginv, so that's what I calculate here.

Sure - Vuforia is very nice. It's not really true that the gyroscope is not accurate enough though - I was very impressed with the result you can get from it, but you need to be aware of the limitiations - you only get directional information. As soon as you want to do anything translational, you hit serious roadblocks. GPS is not accurate, many devices don't have it, and the accelerometers are not accurate enough to integrate out into real world movements, even in a crude fashion. It was fun to try though. So with this code you could fairly easily do something like Nintendo's 3DS face shooting game, as it's all directional, but if you want to tie game objects to real world objects and let the user move around them, you need image processing. Vuforia has several options here, and looks really powerful. I haven't looked at the others.

Im trying to get the compass correction by using the trueHeading. But cant seem to get the rotations right, I tried several things but without luck. This is the current solution: Code (csharp): gyroOrientation = Quaternion.Euler (90, 0, 0) * currentAttitude * Quaternion.Euler (0, 0, 90); Quaternion yDiff = Quaternion.Euler(0, heading, 0) * Quaternion.Inverse(gyroOrientation); transform.localRotation = yDiff * gyroOrientation; Any suggestions?

I used to derive compassOrientation from magneticHeading - here's the code: Code (csharp): _compassOrientation = Quaternion.Euler( Mathf.Rad2Deg * Mathf.Asin(-Input.gyro.gravity.z), Input.compass.magneticHeading, Mathf.Rad2Deg * Mathf.Atan2(Input.gyro.gravity.y, -Input.gyro.gravity.x)); You should probably use this, but switch Input.compass.magneticHeading with Input.compass.trueHeading, or whatever it's called. This then slots in to the rest of the code I posted the other day, in place of the flatNorth stuff. Oh how I wish Unity used radians for angles, like sane people do! Or at least make the whole UnityEngine.Mathf library use degrees, for consistency.

Thanks for quick reply. But I get a similar problem as before, when holding the phone in landscape mode and tilting on the Z axis. Thats is down to the left or right, the whole scene rotates by a huge amount. Did that also happen to you?

Post-multiply by whatever I was multiplying by for the gyro orientation - Quaternion.Euler(0, 0, 90) or something like that. I should really have moved that to the end of the function, rather than having it get mixed up in the rest of the calculation. The earlier thread that was linked before has a more extensive list of corrections for different orientations: http://forum.unity3d.com/threads/98...-camera-on-iPhone-4?highlight=gyro+cam+script

Hey all A very interesting thread I must say! This is also something I am looking for in my current project. I am trying to just use "Input.compass.trueHeading" as MikeWG is as well. George Foot, to calculate the rotation you basically do this (when disregarding your check if the targetcorrection is more than 45 degrees away from the correction of last frame): _targetCorrection = _compassOrientation * Quaternion.Inverse(gyroOrientation); transform.rotation = _targetCorrection * gyroOrientation; I am just wondering why you need to multiply _compassOrientation with the inverse gyroOrientation? I am having trouble visualizing what _targetCorrection is other than a composite rotation of the Compass rotation and the inverse gyro rotation (which is its conjugate [-g_rotX, -g_rotY, -g_rotZ, scalar] because of its unit length). How does this composite rotation help us? If you rewrite the above code to this: transform.rotation = _compassOrientation * Quaternion.Inverse(gyroOrientation) * gyroOrientation; Don't you just end up with this?: transform.rotation = _compassOrientation * 1; Or have I got it wrong (which I may well have - quaternions are not my strong suit ;-) ) Many thanks beforehand

That's exactly right, but the whole point is that we don't actually multiply the correction factor by the gyro orientation - we multiply a smoothed version of the correction factor. So in the long term, if you hold the device still then it does end up just returning the compass orientation, but in the short term it trusts the gyro more closely than the compass. One reason for this is that the compass updates are low frequency; another is that they're not particularly accurate. The degree of smoothing I ended up using was very large - about 10 seconds I think to completely reach equilibrium. Also note that the correction target is only updated when the compass has changed. So "Quaternion.Inverse(gyroOrientation)" is really "Quaternion.Inverse(gyroOrientationLastTimeTheCompassWasUpdated)". It cancels out on that particular frame, but on subsequent frames the gyro will have moved, and you can read the whole thing as "_compassOrientation * changeInGyroOrientation" which might be more intuitive? That is still ignoring the smoothing.

In order to fix the rotations when tilting I tried applying the tilt amount to the heading. So when the device goes from landscape to portrait mode it would subtract 90 degrees from the original heading. This works, but sometimes still gives some jitter. Code (csharp): float zAngle = Mathf.Rad2Deg * Mathf.Atan2 (Input.gyro.gravity.y, -Input.gyro.gravity.x); usedHeading + zAngle; I guess the most optimal calculations would be to calculate your own heading vector like above?

I'm running this on an Android device (Galaxy Tab) but i notice when the device flips towards facedown (looking upwards at a building) my compass flips north to south and everything rotates 180 degrees... how can i prevent this? btw fixed the application to landscape mode.

Just wanted to help out anyone who lands here. I made this script to control the rotation of the camera with a gyroscope. Attach this script to the camera you want to control with the gyroscope: Code (CSharp): using UnityEngine; using System.Collections; public class GyroCamera : MonoBehaviour { Quaternion initialRotation; Quaternion gyroInitialRotation; bool gyroEnabled; void Start () { initialRotation = transform.rotation; Input.gyro.enabled = true; gyroInitialRotation = Input.gyro.attitude; } void Update() { if(gyroEnabled){ #if !UNITY_EDITOR Quaternion offsetRotation = ConvertRotation(Quaternion.Inverse(gyroInitialRotation) * Input.gyro.attitude); transform.rotation = initialRotation * offsetRotation; #else //for unity editor contorl float speed = 2.0f; transform.Rotate(Input.GetAxis("Mouse Y") * speed, Input.GetAxis("Mouse X") * speed, 0); #endif } } public void AlignGyro() { gyroEnabled = false; transform.rotation = Quaternion.identity; } public void StartGyro() { initialRotation = transform.rotation; gyroInitialRotation = Input.gyro.attitude; gyroEnabled = true; } private static Quaternion ConvertRotation(Quaternion q) { return new Quaternion(q.x, q.y, -q.z, -q.w); } }

@George First of all, wow, amazing job, this works like charm! There's one thing I'm struggling with though - I want to align the game object with the real world, that is - invert the orientation and point north. An example would be a compass needle tilted along the x axis in the opposite direction than the phone, so when I look at my phone from the top - is see the needle from the top, when I tilt the phone and have it in front of me, positioned vertically - I see the back of it. How should I approach that?

@George Foot really nice code, one of the smoothest ways of doing this that i have seen, but i am struggling trying to get the correct rotations for using this in portrait mode on a ipad. Does anyone have any ideas.