Search Unity

  1. Unity 2019.2 is now released.
    Dismiss Notice

How to setup Hololens Spectator view project and Record the video?

Discussion in 'Windows Mixed Reality' started by schetty, Aug 20, 2018.

  1. schetty

    schetty

    Joined:
    Jul 23, 2012
    Posts:
    402
    Hi,
    I am trying to setup the Hololens Spectator project with the shared experience in my own project, but i am unable to setup the spectator view control in my project.
    Couldnt find appropriate dll and as well as the reference for this.
    Can anyone please help me to do this?
    Got stuck from long days?
    Thanks
     
  2. michaelklenk

    michaelklenk

    Joined:
    Mar 9, 2018
    Posts:
    8
    Hy schetty.
    Have you successfully deployed and run the spectator view example from the HoloToolkit-Preview/SpectatorView?

    First you have to build the required dll, which is described on this howto page. Then you can copy the required dll's into your project. Have you build this dll's?

    After that you can build the sample scene for HoloLens (UWP) and iOS.

    After all I was able to successfully run the example and I built a sample testscene by my own where the user can place boxes on the floor and can be observed via an iOS device (I worked with an iPad).
     
    schetty likes this.
  3. AntonieB

    AntonieB

    Joined:
    Apr 9, 2013
    Posts:
    17
    I can't get the spectator client to work on my ios device(s). Using Xcode 10.0 / ios 12.

    I can't build the ios project (I generated from my unity build). Keep getting some strange linker error

    For me it is not exactly clear what is wrong. It is saying 'Linker command failed with exit code 1 (use -v to see invocation)

    and just above that it is saying something like 'libiPhone-lib.a, file was built for archive which is not the architecture being linked (arm64) symbols(s) not found for architecture arm64...

    Is Xcode 10 a problem?
     
  4. michaelklenk

    michaelklenk

    Joined:
    Mar 9, 2018
    Posts:
    8
    On the documentation page of the spectator view you can find the following tip.

    Building for each platform (HoloLens or iOS)
    • When building for iOS, ensure you remove the GLTF component of MRTK as this is not yet compatible with this platform.

    So try the following to build the project for iOS.
    1. Switch to the iPhone target on the spectatorView gameobject in your scene.
    2. Create a zip archive from the folder HoloToolkit/Utilities/Scripts/GLTF.
    3. Delete the GLTF folder.
    4. Build the project for iOS out of unity.
    5. Then open the generated project in Xcode an build the project for your iOS Device
    6. Wenn you switch back to the target platform "HoloLens", you can unzip the GLTF archive and you are ready again to build the project for the HoloLens.
    So Xcode should not be the problem. I recently build my project successful with Xcode 10 für an iOS 12 device.
     
    unity_Jp89DC8cZbKImQ likes this.
  5. AntonieB

    AntonieB

    Joined:
    Apr 9, 2013
    Posts:
    17
    Thanks for helping me out / I currently got the apps running on hololens and my ipad. It still isn't working though but I yet have to take the time to debug into it. A quick look already gave me this message on the ipad side:

    2018-10-02 14:45:09.175661+0200 ProductName[1311:533408] [si_destination_compare] send failed: Invalid argument
    2018-10-02 14:45:09.175687+0200 ProductName[1311:533408] [si_destination_compare] send failed: Invalid argument
    -> applicationWillResignActive()
    Network layer will still work after application suspends
    (Filename: Line: 208)
    ar_session_interrupted
    UnityEngine.XR.iOS.UnityARSessionNativeInterface:_ar_session_interrupted()
    (Filename: /Users/builduser/buildslave/unity/build/artifacts/generated/common/runtime/DebugBindings.gen.cpp Line: 51)
    -> applicationDidEnterBackground()
    -> applicationWillEnterForeground()
    2018-10-02 14:45:12.882403+0200 ProductName[1311:533590] [BoringSSL] nw_protocol_boringssl_error(1584) [C3.1:2][0x10479cef0] Lower protocol stack error: 53
    ar_session_interruption_ended
    UnityEngine.XR.iOS.UnityARSessionNativeInterface:_ar_session_interruption_ended()
    (Filename: /Users/builduser/buildslave/unity/build/artifacts/generated/common/runtime/DebugBindings.gen.cpp Line: 51)
    -> applicationDidBecomeActive()
    Log: cannot connect after {10} attempt address {::ffff:192.168.5.82:7777}
    ClientDisconnected due to error: Timeout
    UnityEngine.Networking.NetworkManager:OnClientDisconnect(NetworkConnection)
    UnityEngine.Networking.NetworkConnection:InvokeHandler(Int16, NetworkReader, Int32)
    UnityEngine.Networking.NetworkClient:Update()
    UnityEngine.Networking.NetworkClient:UpdateClients()
    UnityEngine.Networking.NetworkIdentity:UNetStaticUpdate()
    (Filename: /Users/builduser/buildslave/unity/build/artifacts/generated/common/runtime/DebugBindings.gen.cpp Line: 51)

    I don't think the 'suspend message is of any importance but the fact that he cannot connect after 10 attemps should be part of the problem somehow.

    This is 'AFTER' I scan the QR Code on the IPAD with the Hololens. So lots of things already seems to work. Accept after this the connection to the Hololens from the UPAD does't seem t work.
     
    Last edited: Oct 2, 2018
  6. AntonieB

    AntonieB

    Joined:
    Apr 9, 2013
    Posts:
    17
    I can't understand what the problem is. Only strange thing I guess (but I didn't specify that myself) is the address. the ::ffff: 192.168.5.82 notation I don't understand. whats ::ffff: doing there?

    Anyway I still can't get it to work.
     
  7. AntonieB

    AntonieB

    Joined:
    Apr 9, 2013
    Posts:
    17
    ::ffff: apparantly is something with ipv6 notation and should'nt be a problem. I still can't get it to work / It looks like on my hololens there is nothing listening on port 7777 but I can't be totaly sure about that.

    If I look at the debug information when running the Hololens app I see something like this:


    The following GlobalManagers were stripped from the build (Either because they're not used or not supported on this platform):
    NetworkManager
    MasterServerInterface
    ClusterInputManager
    UnityAnalyticsManager
    CrashReportManager
    PerformanceReportingManager

    Isn't it strange that NetworkManager is in that list? why isn't that supported on Hololens?
     
  8. michaelklenk

    michaelklenk

    Joined:
    Mar 9, 2018
    Posts:
    8
    I now checked out the version 2017.4.1.0 of HoloTookit and run into the same problem that you described here.

    UnityEngine.Networking.NetworkDiscovery:StartAsServer()
    HoloToolkit.Unity.Preview.SpectatorView.NewDeviceDiscovery:ManualStart()

    [ line 485]
    (Filename: Line: 485)

    host id out of bound id {-1} max id should be greater 0 and less than {2}
    UnityEngine.Networking.NetworkDiscovery:Update()


    [ line 541]
    (Filename: Line: 541)

    2018-10-03 08:21:23.524705+0200 ProductName[522:143309] [si_destination_compare] send failed: Invalid argument
    2018-10-03 08:21:23.524772+0200 ProductName[522:143309] [si_destination_compare] send failed: Invalid argument
    Log: cannot connect after {10} attempt address {::ffff:192.168.110.253:7777}
    ClientDisconnected due to error: Timeout


    My running example was build with the version 2017.4.0.0. Sorry but at the moment I also don't know what is exactly going wrong here.
     
  9. schetty

    schetty

    Joined:
    Jul 23, 2012
    Posts:
    402
    Thanks for your answer. I made the dll and copied into my project i can able to see the holograms in my spectator windows.
    but i am getting a new issue here, the hologram which i am seeing that is streaming from the unity editor not from my hololens.
    I have connected the Video capture card and hololens and camera with my system, followed the same from the documentation, but its not streaming my hololens video.

    Can you please tell me what is the exact issue here?
     
  10. michaelklenk

    michaelklenk

    Joined:
    Mar 9, 2018
    Posts:
    8
    What version of the SpectatorView you try to set up.
    I only have worked with the "SpectatorView (Preview)" version vor iOS. So i can give you no advice if you trying to set up the "SpectatorView Pro" version.
     
  11. AntonieB

    AntonieB

    Joined:
    Apr 9, 2013
    Posts:
    17
    I see the original starter of this thread is talking about SpectatorView Pro. Sorry for somewhat hijacking your thread. I'am personaly talking about SpectatorView (Preview) for IOS.

    I will try a 2017.4.0.0 build to see if that is changing things. Thanks again / for your time already Michaelklenk!
     
  12. schetty

    schetty

    Joined:
    Jul 23, 2012
    Posts:
    402
    Yes i am trying a spectator view Pro here :(
     
  13. AntonieB

    AntonieB

    Joined:
    Apr 9, 2013
    Posts:
    17
    I'am not really successfull in building it with older Unity version(s). I'm currently trying with 2017.4.1f but get errors when trying to build the IOS project. UnityInterface.h file not found, Prefix.pch..

    hmm..
     
  14. AntonieB

    AntonieB

    Joined:
    Apr 9, 2013
    Posts:
    17
    2017.4.1f isn't working for me.

    Anybody knows a working combination of Unity version / Spectator view (IOS) and how to get it working? My options are exhausted I can't get it to work.

    Are people able to use 'SpectatorView (Preview)'?
     
    Last edited: Oct 4, 2018
  15. schetty

    schetty

    Joined:
    Jul 23, 2012
    Posts:
    402
    it will work in 2017.4.1f, i think if you place all the dll file it will work.if you brief your issue more that would be great.
     
  16. michaelklenk

    michaelklenk

    Joined:
    Mar 9, 2018
    Posts:
    8
    Sorry for the confusion. For me it worked with Version "HoloToolkit-Unity-Preview-2017.4.0.0" but i was not successful with "HoloToolkit-Unity-Preview-2017.4.1.0".

    The Unity version I use is Unity 2017.4.9f

    Here you can see a short video of my successful test.
     
  17. AntonieB

    AntonieB

    Joined:
    Apr 9, 2013
    Posts:
    17
    Thanks Michaelklenk. I was also debugging / researching some more and I think 'we' have the same problem in the HoloToolkit-Unity-Preview-2017.4.1.0 build.

    See this link:
    https://github.com/Microsoft/MixedRealityToolkit-Unity/issues/2438

    My guess is we have that problem and I am not yet successfull in building the 'htk_development' branche. (so probably it is beter to wait for a 2017.4.2 release but I don't know what the timeline is on that)

    Will try going back to the older Preview release for now to get something working.
     
    michaelklenk likes this.
  18. michaelklenk

    michaelklenk

    Joined:
    Mar 9, 2018
    Posts:
    8
  19. AntonieB

    AntonieB

    Joined:
    Apr 9, 2013
    Posts:
    17
    Nice.. later today I will try the same. Thanks for the information!
     
  20. AntonieB

    AntonieB

    Joined:
    Apr 9, 2013
    Posts:
    17
    I'am using Unity 2017.4.12f1 and version 2017.4.1.0 of the holotoolkit. Applied the code thats in https://github.com/Microsoft/MixedRealityToolkit-Unity/pull/2612/files

    Still not working bummer...

    While debugging the hololens app I see this:

    Failed to capturing image
    (Filename: C:\buildslave\unity\build\artifacts/generated/Metro/runtime/DebugBindings.gen.cpp Line: 51)
    NullReferenceException: Object reference not set to an instance of an object.
    at HoloToolkit.Unity.Preview.SpectatorView.CameraCaptureHololens.OnCapturedPhotoToMemory(PhotoCaptureResult result, PhotoCaptureFrame photoCaptureFrame)
    at UnityEngine.XR.WSA.WebCam.PhotoCapture.InvokeOnCapturedPhotoToMemoryDelegate(OnCapturedToMemoryCallback callback, Int64 hResult, IntPtr photoCaptureFramePtr)
    at UnityEngine.XR.WSA.WebCam.PhotoCapture.$Invoke10(Int64 instance, Int64* args)
    at UnityEngine.Internal.$MethodUtility.InvokeMethod(Int64 instance, Int64* args, IntPtr method)
    (Filename: <Unknown> Line: 0)
    Unable to start photo mode!

    I see for the first time the star object in the hololens so I guess thats progress.

    Any more hints?
     
  21. AntonieB

    AntonieB

    Joined:
    Apr 9, 2013
    Posts:
    17
    UPDATE: I got it working! well partial but it looks good.

    I don't understand exactly what my problem since last post was but I redid everything from start to finish and it connected / shows the hologram on both ipad and hololens.

    The only thing left is the recording isn't working. I can long press to record but after count down it will not record. After second long press I don't get the option to STOP the recording but again the option to START recording. (which tells me it probably never starts)

    I don't see errors yet in logging so no information there.
     
  22. robinstrike8

    robinstrike8

    Joined:
    Mar 20, 2015
    Posts:
    4
    Hi Guys,
    I guess you got the spectator view preview working. I'm able to run the example from 'htk_development' branch and replaced the star with different model. I found that both are not synchronized. It seems as if the iPad had its own version of the app running and anything I do on the Hololens is not being reflected there. I selected the Hololens version as server in the WorldSync game object and iOS version as local player authority. Initially I tried without checking any boxes (Network Identity in Inspector).

    Thanks in advance
     
  23. michaelklenk

    michaelklenk

    Joined:
    Mar 9, 2018
    Posts:
    8
    Yes that is correct. You have to sync all user interaction between the two app instances. The example scene only contains a static object (the star). The only change witch is synchronised is the color change.

    If you would like to add interaction, you have to program the sync between the two apps the same way as a multiplayer game in unity.
     
  24. robinstrike8

    robinstrike8

    Joined:
    Mar 20, 2015
    Posts:
    4
    Thank you Michael. It is working as expected.
     
  25. cjones0828

    cjones0828

    Joined:
    Jan 17, 2019
    Posts:
    1
    I'm really new to this whole process and unity etc but im a quick learner... Can you write up a step by step process to set this up? the one on the main website for spectator view is informative but not prescriptive and I really dont know where to begin. I'm trying to study Mixed Realty for Architectural research at Kennesaw State University and need to document my interactions with holograms and future demonstrations I will propose. Can you guys help a poor soul out?! haha
     
  26. VREasy

    VREasy

    Joined:
    Dec 8, 2016
    Posts:
    10
    Hi,
    Michael's' answer helped me building the dll's, thank you for that. I built the dll's successfully. I built the Hololens app without any problems and going to build the IOS app today. I still have the following questions remaining and hope someone can help me out here please.

    1) Do I have to import the Holotoolkit packages in my own project and then copy the built dll's?
    2) Do I have to drag in anything in the Offline Scene and Offline Scene areas in the Spectator View Network Manager Script?
    3) Do you need to build a new IOS app for each new project?
    4) Are all default values ok to build your apps?

    I have also attached a step by step instructions, especially for beginners and no programming experience:), on how to build the dll's.

    Thanks in advance

    Aydin
     

    Attached Files:

    Last edited: Jan 27, 2019
  27. schetty

    schetty

    Joined:
    Jul 23, 2012
    Posts:
    402
    Has any one tried spectator view pro here?

    That is becoming a challenging for me.
     
  28. schetty

    schetty

    Joined:
    Jul 23, 2012
    Posts:
    402
    This is really a good for beginners.
     
  29. JCIPL

    JCIPL

    Joined:
    Jul 13, 2018
    Posts:
    8
    @michaelklenk.. Do you have done any tutorial for this..?
    Any one have any video tutorial for this
     
  30. schetty

    schetty

    Joined:
    Jul 23, 2012
    Posts:
    402
    till now i am also not getting any tutorial for spectator view pro,