New buildin Tango api crashes in 2017.2.0b4

trying to get camera image but it crashes, im doing it right?

Code (CSharp):

1.  
2. void Start()
3.     {
4.         Debug.Log ("Start");
5.         TangoDevice.Connect(new TangoConfig() { enableColorCamera = true, enableDepth = true, enableMotionTracking = true });
6.         line = Instantiate(m_lineRenderer);
7.  
8.         var br = new ARBackgroundRenderer ();
9.         br.camera = GameObject.Find("CameraImage").GetComponent<Camera>();
10.         mat.mainTexture = new Texture2D(0, 0, TextureFormat.RGBA32, false);
11.         br.backgroundMaterial = mat;
12.         br.backgroundRendererChanged += BrOnBackgroundRendererChanged;
13.         br.mode = ARRenderMode.MaterialAsBackground;
14.         TangoDevice.backgroundRenderer = br;
15.     }
16.  
17.     private void BrOnBackgroundRendererChanged()
18.     {
19.         Debug.Log("BrOnBackgroundRendererChanged");
20.     }

error
<i>AndroidPlayer(ADB@127.0.0.1:34999)</i> Could not get a pose for the latest image buffer at time 161744.275238. Error code 0-2
UnityEngine.XR.Tango.TangoDevice:TryGetLatestPointCloudInternal(Object, UInt32&, Double&)
UnityEngine.XR.Tango.TangoDevice:TryGetLatestPointCloud(PointCloudData&) (at /Users/builduser/buildslave/unity/build/artifacts/generated/common/modules/AR/TangoBindings.gen.cs:184)
NewBehaviourScript:Update() (at D:\Users\Igor\Documents\New Unity Project 17\Assets\NewBehaviourScript.cs:54)

[./Runtime/AR/Tango/TangoDevice.cpp line 206]
(Filename: /Users/builduser/buildslave/unity/build/artifacts/generated/common/modules/AR/TangoBindings.gen.cs Line: 184)

 

Hi @friuns -

Thanks for reporting the crash! Your usage of the ARBackground renderer is not quite correct. You will need to create a Resources directory in Assets and create a material there that has its shader set to AR/TangoARRender. No need to attach a texture to this material as the background texture is created in an optimized way by Unity. Here is an example MonoBehavior you can use for the renderer setup:

public class ARRenderSetup : MonoBehaviour {

private UnityEngine.XR.ARBackgroundRenderer backgroundRenderer = new UnityEngine.XR.ARBackgroundRenderer();
private Material mat;

void Awake()
{
// First parameter is the name of your material in the Resources folder
mat = Resources.Load("TangoARBackground", typeof(Material)) as Material;
}

// Use this for initialization
void Start ()
{
// Associate this background renderer object with Tango
UnityEngine.XR.Tango.TangoDevice.backgroundRenderer = backgroundRenderer;

// Associate the renderer's background material with the material in your Resources folder
backgroundRenderer.backgroundMaterial = mat;
backgroundRenderer.mode = UnityEngine.XR.ARRenderMode.MaterialAsBackground;
}
}

This MonoBehavior moves the Unity camera in sync with the position of the Tango camera:

public class CameraMovement : MonoBehaviour {
void Update () {
UnityEngine.XR.Tango.PoseData pose;

if (UnityEngine.XR.Tango.TangoInputTracking.TryGetPoseAtTime(out pose, UnityEngine.XR.Tango.CoordinateFrame.StartOfService, UnityEngine.XR.Tango.CoordinateFrame.CameraColor)) {
Camera.main.transform.position = pose.position;
Camera.main.transform.rotation = pose.rotation;
}
}
}

This MonoBehavior properly initializes Tango for a simple AR scene (no depth camera).

public class TangoConnection : MonoBehaviour
{
private TangoConfig tangoAPIConfig = new TangoConfig();

#if UNITY_EDITOR
// A camera field already exists in MonoBehavior for Editor-only,
// so use the new keyword there to silence a warning about it.
public new Camera camera;
#else
// The runtime doesn't have this problem, so don't use the
// new keyword here.
public Camera camera;
#endif

private Nullable<float> m_VerticalFov = null;
private Nullable<float> m_HorizontalFov = null;

void Awake()
{
tangoAPIConfig.enableMotionTracking = true;
tangoAPIConfig.enableColorCamera = true;
tangoAPIConfig.enableDepth = false;

if (TangoDevice.Connect(tangoAPIConfig))
TangoDevice.synchronizeFramerateWithColorCamera = false;
else
Debug.LogWarning("Tango failed to connect!");
}

protected Nullable<float> GetCameraFov(ScreenOrientation screenOrientation)
{
if ((screenOrientation == ScreenOrientation.Portrait) ||
(screenOrientation == ScreenOrientation.PortraitUpsideDown))
{
if (m_HorizontalFov == null)
{
float fov;
if (TangoDevice.TryGetHorizontalFov(out fov))
m_HorizontalFov = fov;
}

return m_HorizontalFov;
}
else
{
if (m_VerticalFov == null)
{
float fov;
if (TangoDevice.TryGetVerticalFov(out fov))
m_VerticalFov = fov;
}

return m_VerticalFov;
}
}

protected virtual void SetCameraFov()
{
if (camera != null)
{
Nullable<float> cameraFov = GetCameraFov(Screen.orientation);

if (cameraFov != null)
camera.fieldOfView = cameraFov.Value;
}
}

void OnDestroy()
{
TangoDevice.Disconnect();
}

void Update()
{
SetCameraFov();
}
}

Please reply to the thread if you have further questions.

-Best,
Mike Durand

 

Hi. Is there any example project for the new built-in Tango API?

 

Hi @mdurand


still getting crash

Code (CSharp):

1. using System;
2. using System.Collections;
3. using System.Collections.Generic;
4. using System.Linq;
5. using UnityEngine;
6. using UnityEngine.UI;
7. using UnityEngine.XR;
8. using UnityEngine.XR.Tango;
9.  
10. public class NewBehaviourScript : MonoBehaviour
11. {
12.     public Camera tmpCam;
13.     // Use this for initialization
14.     private UnityEngine.XR.ARBackgroundRenderer br = new UnityEngine.XR.ARBackgroundRenderer();
15.     void Start()
16.     {
17.         Debug.Log ("Start");
18.         if(TangoDevice.Connect(new TangoConfig() { enableColorCamera = true, enableDepth = true, enableMotionTracking = true }))
19.             TangoDevice.synchronizeFramerateWithColorCamera = false;
20.         line = Instantiate(m_lineRenderer);
21.  
22.      
23.         TangoDevice.backgroundRenderer = br;
24.         br.camera = GameObject.Find("CameraImage").GetComponent<Camera>();
25.         //mat.mainTexture = new Texture2D(0, 0, TextureFormat.RGBA32, false);
26.         mat = Resources.Load("TangoARBackground", typeof(Material)) as Material;
27.         br.backgroundMaterial = mat;
28.         br.backgroundRendererChanged += BrOnBackgroundRendererChanged;
29.         br.mode = ARRenderMode.MaterialAsBackground;
30.     }
31.  
32.     private void BrOnBackgroundRendererChanged()
33.     {
34.         Debug.Log("BrOnBackgroundRendererChanged");
35.     }
36.     public Material mat;
37.     //ARBackgroundRenderer br;
38.     LineRenderer line;
39.     //private Texture2D tx;
40.     // Update is called once per frame
41.     void Update()
42.     {
43.         //var g = TangoDevice.backgroundRenderer;
44.         PoseData data;
45.         float fov;
46.         if (TangoDevice.TryGetVerticalFov(out fov))
47.             tmpCam.fieldOfView = Camera.main.fieldOfView = fov;
48.         if (TangoInputTracking.TryGetPoseAtTime(CoordinateFrame.StartOfService, CoordinateFrame.Device, out data))
49.         {
50.             Camera.main.transform.position = data.position;
51.             Camera.main.transform.rotation = data.rotation;
52.         }
53.         TangoDevice.TryGetLatestPointCloud(ref pointCloudData);
54.         //if (TangoDevice.TryGetLatestImageData(ref img))
55.         //{
56.         //    if (tx != null)
57.         //        DestroyImmediate(tx);
58.  
59.         //    tx = new Texture2D((int)img.width, (int)img.height);
60.         //    var extractArrayFromList = (byte[])Marshal.ExtractArrayFromList(img.data);
61.         //    Debug.Log(extractArrayFromList.Length);
62.         //    Debug.Log(tx.LoadImage(extractArrayFromList));
63.         //    tx.Apply(false);
64.         //}
65.  
66.  
67.         if (Input.GetMouseButton(0))
68.             Bum(Input.mousePosition);
69.  
70.         if (Input.GetMouseButtonUp(0))
71.         {
72.             ls.Add(line);
73.             line = Instantiate(m_lineRenderer);
74.             points = new List<Vector3>();
75.         }
76.         if (Input.GetKeyDown(KeyCode.Escape))
77.         {
78.             foreach (var a in ls)
79.                 Destroy(a.gameObject);
80.             ls.Clear();
81.         }
82.  
83.     }
84.     ImageData img;
85.  
86.     public RawImage m;
87.     private void Bum(Vector2 touchPosition)
88.     {
89.         int pointIndex = FindClosestPoint(touchPosition, 10);
90.         if (pointIndex > -1)
91.         {
92.             var mPoint = pointCloudData.points[pointIndex];
93.             mPoint = Camera.main.transform.TransformPoint(mPoint);
94.             points.Add(mPoint);
95.             line.positionCount = points.Count;
96.             line.SetPositions(points.ToArray());
97.         }
98.     }
99.  
100.     public List<Vector3> points = new List<Vector3>();
101.     List<LineRenderer> ls = new List<LineRenderer>();
102.     public LineRenderer m_lineRenderer;
103.     void OnGUI()
104.     {
105.         GUILayout.Label("p:" + pointCloudData.points.Count);
106.  
107.     }
108.     public int FindClosestPoint(Vector2 pos, int maxDist = 10)
109.     {
110.         int bestIndex = -1;
111.         float bestDistSqr = 0;
112.  
113.         var ray = tmpCam.ScreenPointToRay(pos);
114.         for (int it = 0; it < pointCloudData.points.Count; ++it)
115.         {
116.             float distSqr = Vector3.Cross(ray.direction, (Vector3)pointCloudData.points[it] - ray.origin).sqrMagnitude;
117.             if (distSqr > maxDist * maxDist)
118.                 continue;
119.  
120.             if (bestIndex == -1 || distSqr < bestDistSqr)
121.             {
122.                 bestIndex = it;
123.                 bestDistSqr = distSqr;
124.             }
125.         }
126.  
127.         return bestIndex;
128.     }
129.  
130.  
131.     private PointCloudData pointCloudData;
132. }
133.  

unity 2017.2.0b5

 

and unity 2017.2.0b7 crash, with using scripts you provided

 

I think an example project on GitHub, BitBucket or in other way would be better for us.

 

08-19 15:18:54.132 8813-8825/? W/Unity: Tango failed to connect!

(Filename: ./artifacts/generated/common/runtime/DebugBindings.gen.cpp Line: 51)

It doesn't crash in my case. But it can't connect to the Tango.

 

I updated the core to m21. But it still can't connect to the Tango Core.

 

The title seems to say that Tango is built-in to Unity. Is this the case?

 

It's native integration in 2017.2 instead of the TangoSDK.

 

@mdurand Could you please help?

 

@mdurand It's able to connect Tango now after I check the XR settings.
But it crashes if I use the ARRenderSetup.
And here's crash log.

08-26 16:54:17.659 13562-13603/empty.empty.empty I/tango_client_api: TangoErrorType TangoService_getCameraIntrinsics(TangoCameraId, TangoCameraIntrinsics*): Getting camera intrinsics...
08-26 16:54:17.659 10080-10105/? I/TangoServer.cc: Tango Service: getCameraIntrinsics, internal status 0
08-26 16:54:17.661 13562-13603/empty.empty.empty I/tango_client_api: TangoErrorType TangoService_getCameraIntrinsics(TangoCameraId, TangoCameraIntrinsics*): Done getting camera intrinsics.

--------- beginning of crash

08-26 16:54:17.661 13562-13603/empty.empty.empty A/libc: Fatal signal 11 (SIGSEGV), code 1, fault addr 0x0 in tid 13603 (UnityGfxDeviceW)
08-26 16:54:17.680 447-9679/? W/qdmemalloc: Gralloc using 4K chunk.
08-26 16:54:17.711 447-6208/? W/qdmemalloc: Gralloc using 4K chunk.
08-26 16:54:17.714 1393-5773/? D/PowerManagerService: updateWakeLockWorkSourceInternal: lock=22485117 [AudioMix], ws=null
08-26 16:54:17.745 447-484/? W/qdmemalloc: Gralloc using 4K chunk.
08-26 16:54:17.754 546-3718/? D/APM::AudioPolicyManager: stopOutput() output 6, stream 3, session 51
08-26 16:54:17.763 541-541/? A/DEBUG: *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
08-26 16:54:17.763 541-541/? A/DEBUG: Build fingerprint: 'Lenovo/phinny_prc/PB2PRO:6.0.1/MMB29M/PB2-690N_S200027_161214:user/release-keys'
08-26 16:54:17.763 541-541/? A/DEBUG: Revision: '0'
08-26 16:54:17.763 541-541/? A/DEBUG: ABI: 'arm'
08-26 16:54:17.764 541-541/? A/DEBUG: pid: 13562, tid: 13603, name: UnityGfxDeviceW >>> empty.empty.empty <<<
08-26 16:54:17.764 541-541/? A/DEBUG: signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0
08-26 16:54:17.778 447-482/? W/qdmemalloc: Gralloc using 4K chunk.
08-26 16:54:17.776 541-541/? W/debuggerd: type=1400 audit(0.0:1185): avc: denied { search } for name="com.google.tango" dev="dm-0" ino=508055 scontext=u:r:debuggerd:s0 tcontext=ubject_r:app_data_file:s0:c512,c768 tclass=dir permissive=0
08-26 16:54:17.789 541-541/? A/DEBUG: r0 00000000 r1 00000001 r2 de449a18 r3 de449930
08-26 16:54:17.789 541-541/? A/DEBUG: r4 00000008 r5 ac4b76e0 r6 00000004 r7 00000000
08-26 16:54:17.789 541-541/? A/DEBUG: r8 00000438 r9 007e9000 sl 00000001 fp 00000780
08-26 16:54:17.789 541-541/? A/DEBUG: ip de449a78 sp de449268 lr e1b4be58 pc e177b54c cpsr 400f0010
08-26 16:54:17.810 541-541/? A/DEBUG: backtrace:
08-26 16:54:17.810 541-541/? A/DEBUG: #00 pc 0049b54c /data/app/empty.empty.empty-1/lib/arm/libunity.so (_ZN9Texture2D11InitTextureEii13TextureFormatNS_10EInitFlagsEiii+988)
08-26 16:54:17.810 541-541/? A/DEBUG: #1 pc 00d9bd34 /data/app/empty.empty.empty-1/lib/arm/libunity.so (_ZN5Tango11ARRendering29UpdateExternalTextureCallbackEi+368)
08-26 16:54:17.810 541-541/? A/DEBUG: #2 pc 00874ef4 /data/app/empty.empty.empty-1/lib/arm/libunity.so (_ZN9GfxDevice26InsertCustomMarkerCallbackEPFviEi+36)
08-26 16:54:17.810 541-541/? A/DEBUG: #3 pc 00960978 /data/app/empty.empty.empty-1/lib/arm/libunity.so (_ZN15GfxDeviceWorker10RunCommandER20ThreadedStreamBuffer+29328)
08-26 16:54:17.811 541-541/? A/DEBUG: #4 pc 00961e50 /data/app/empty.empty.empty-1/lib/arm/libunity.so (_ZN15GfxDeviceWorker6RunExtER20ThreadedStreamBuffer+32)
08-26 16:54:17.811 541-541/? A/DEBUG: #5 pc 009596c8 /data/app/empty.empty.empty-1/lib/arm/libunity.so (_ZN15GfxDeviceWorker18RunGfxDeviceWorkerEPv+108)
08-26 16:54:17.811 541-541/? A/DEBUG: #6 pc 00723c50 /data/app/empty.empty.empty-1/lib/arm/libunity.so (_ZN6Thread16RunThreadWrapperEPv+84)
08-26 16:54:17.811 541-541/? A/DEBUG: #7 pc 00041823 /system/lib/libc.so (_ZL15__pthread_startPv+30)
08-26 16:54:17.811 541-541/? A/DEBUG: #8 pc 00019285 /system/lib/libc.so (__start_thread+6)

 

Hi All-

Hopefully you have all seen the announcement on 8/29 about Unity's support for Google's ARCore SDK which has superseded the Tango SDK that was available in earlier betas of 2017.2. As of 2017.2b9 the Tango APIs have been removed.

Please take a look at this forum post for instructions on how to use ARCore with your Pixel and S8 phones: https://forum.unity3d.com/threads/introducing-arcore-an-android-ar-sdk-for-unity.490929/

-Best,
Mike

 

Can you advise what people still interested in developing for Tango do? I can't get the Tango SDK examples to work on 2017.2b9 and I don't own an ARCore-supported device to simply switch SDK's.

 

I'm sitting in the same boat. I found some intesting stuff in a thread on googles Github:
https://github.com/googlesamples/tango-examples-unity/issues/102

With these tricks I was able to prevent the app from crashing in Unity 2017.1 and 2017.2 beta but I'm still not getting any camera data.

I'm trying to go back to Unity 4.6.2 now to see if things work better.

 

The notice on Google's Tango developer site explains that it the legacy Tango SDK for Unity will only be the only path to support the ASUS and Lenovo Tango devices. ARCore does not support these devices.

-Best,
Mike

 

They aren't supported YET.
It would be a very weird decision to not give the Zenfone ARcore support.
Considering the statement that by the end of the year 100 million phones are supposed to be ARcore compatible the list of devices will need to grow quite a bit.

Unity 4.6.2 works fine by the way, I'm happily scripting away for Tango.