Search Unity

VR3DMediaViewer - View Stereoscopic 3D content in VR

Discussion in 'Assets and Asset Store' started by DarkAkuma, May 9, 2016.

  1. DarkAkuma

    DarkAkuma

    Joined:
    Nov 28, 2013
    Posts:
    46
    ATM I'm working on a update to the asset, mainly to address the common camera setup issue. I see that I considered the Image/RawImage UI component support before as it was already on my list for something to look into and think about.

    After I finish the camera setup procedure update, I'll look into that next.

    I'll look into addressing the null reference error too.
     
  2. DarkAkuma

    DarkAkuma

    Joined:
    Nov 28, 2013
    Posts:
    46
    @rinkusaru

    You will probably be happy to know that I have indeed worked on support for the RawImage component. It required a bit of a script overhaul since my code was designed around Renderer components a bit more then I thought. There was a couple hicups due to oddities with the Canvas UI stuff, but it "should" work fine in the end.

    The Image component however will not be supported. It's just not suitable for this stuff. But the RawImage component should be good enough itself for such cases when you need to work with UI elements.

    While I have no firm date planned for the next update, it will hopefully be soon. After I clean up this code I need to go through my bug list and emails and start knocking off other more minor issues that have built up over the past year. I just figured I'd give you a heads up as early as I can so you can prepare if you want. =)

    EDIT:

    The update is now live! Re-download, re-import and enjoy! =)
     
    Last edited: Oct 4, 2018
  3. rinkusaru

    rinkusaru

    Joined:
    Sep 7, 2017
    Posts:
    14
    Alright Thanks buddy.
     
  4. rinkusaru

    rinkusaru

    Joined:
    Sep 7, 2017
    Posts:
    14
    So I'm having one issue as I'm using the canvas on world space and i need to give the camera reference and as I've two camera there in the scene for 3D video. if i give the left camera ref and then the stereo video like left right work on one eye but remain same on the other eye and vice versa.
    What should I do?
     
  5. rinkusaru

    rinkusaru

    Joined:
    Sep 7, 2017
    Posts:
    14
    any update on this?
     
  6. saru15

    saru15

    Joined:
    Jan 20, 2015
    Posts:
    6
    Hi I've purchase your package and integrated it inside the project I'm seeing the 3D effect but there is blurred video I see just like the 3D movie in theater when i don't wear my glasses.
    Here is the example : https://goo.gl/images/uRj7y4
    is there something that can be done for this to fix?
     
  7. DarkAkuma

    DarkAkuma

    Joined:
    Nov 28, 2013
    Posts:
    46
    I'm very sorry for this late reply. For some reason I was never sent an alert for you're last two messages, and only happened to notice them now.

    I'm having a hard time understanding what you are asking. I suspect you are trying to use this with something else, hence mentioning needing to reference the camera. And somehow that's messing up the 3D for both eyes. I don't know. I would need a better, more detailed explanation to understand and hope to help.

    I can't really say what that is. If you weren't getting the 3D effect, then I may think it was a layer issue. My best guess is its just a issue of the specific video you are using. Maybe its encoding is bad. Perhaps, at best, maybe the shader you are using for the canvas. Or even a post processing effect from your scene.
     
  8. saru15

    saru15

    Joined:
    Jan 20, 2015
    Posts:
    6

    Actually the 3D effect is coming but the part where3D effect is not coming that part is coming blur and its coming with all video I've check.
     
  9. DarkAkuma

    DarkAkuma

    Joined:
    Nov 28, 2013
    Posts:
    46
    I'm having a hard time following you. I don't know what you mean by both having a 3D effect, and not having one in different parts.

    Lets see if I can figure some things out before you respond next, to help this go a little quicker.

    Since you linked a Red/Cyan Anaglyph image before, is that the type of 3D videos you are working with?

    Assuming so... maybe your descriptions of the issue might make some sense. It's possible that due to a color mismatch, that you could get some semblance of a 3D effect with Anaglyph media, while also getting some bleed through that could maybe be described as blurring with a non-3D effect. I've personally seen this with a cheap pair of Red/Not Quite Cyan glasses that I've got, but not with my Asset here.

    If this is the case, I'd suggest adjusting the color in the .Stereoscopic3DImage settings for the video.

    It could maybe be a shader bug. My Assets Anaglyph support is done through a Shader+RenderTexture. While it worked with my own test images/videos, it's not impossible that it has a bug. I'm novice shader coder at best. It could also be a issue with the shader and a different version of Unity than I tested with. That's hard to predict such issues. In this case, I'd be especially interested in getting this figured out and fixed too!

    Beyond these shot in the dark guesses, I don't know without know more/clearer details.
     
  10. saru15

    saru15

    Joined:
    Jan 20, 2015
    Posts:
    6
    Okay Let's just forget everything above you're mixing up the two different issue.

    Now my main issue that i need to fix is when in my headset i try to watch the video I can see the 3d effect but while I can see that 3D effect the video quality I'm having is blurr just like the image I've posted above.
     
  11. DarkAkuma

    DarkAkuma

    Joined:
    Nov 28, 2013
    Posts:
    46
    I'm sorry. That doesn't clarify anything to help me understand the issue to be able to hope to provide an explanation/solution.

    Again, I have to assume you are using Anaglyph 3D videos, without you stating one way or the other. I posted my best theory and solution to that case above. Image Ghosting from a color mismatch could be seen as a blurring to some.

    If I was to further guess at issues beyond that, encoding, shader, and post processing. Maybe it's a black smear issue with the HMD you are using. I've thought that issue was mostly weeded out in modern generations of VR HMDs, but I can't say for sure for everything. Generally you had to combat black smear with making your scenes brighter.

    Hmm...

    If you were to play the video without my asset converting it to 3D, does the burring still seem to happen? If not, then it could be my behind-the-scenes conversion shader. Why that may not be working correctly for you, I'd have to figure out. Last I checked, it of course worked fine enough for me with my Analglyph test videos. It could maybe be some change made since Unity v5.6 (what I dev with to keep this as compatible as possible) and I'd have to update something. It could maybe be quirk with your target platform. I test on PC, with my included custom Cross-eyed Camera Rig and Vive mostly. Sometimes my Oculus Rift CV1.
     
  12. tsanummy

    tsanummy

    Joined:
    Jul 8, 2014
    Posts:
    2
    Is there any easy way to loop the video clips instead of them playing just once?
     
  13. DarkAkuma

    DarkAkuma

    Joined:
    Nov 28, 2013
    Posts:
    46
    If you are using a RenderTexture, it has to be done via scripting. In the "VR3DMediaViewer\Demo\Scripts" folder there is a script called "PlayMovieTextureOnStart.cs" which you can use as an example.

    If you are using the VideoPlayer component, there's a Loop option on the component that you can Check.
     
  14. tsanummy

    tsanummy

    Joined:
    Jul 8, 2014
    Posts:
    2
    I am using the VR3d Media Viewer script, not setting up anything else manually:

    https://i.imgur.com/AcNWjUD.png

    Ignore the warning, I am not displaying 3d content yet.
     
  15. DarkAkuma

    DarkAkuma

    Joined:
    Nov 28, 2013
    Posts:
    46
    On the same GameObject that VR3DMediaViewer script is on, add a VideoPlayer component, and check Loop on it.
     
  16. AlanPT

    AlanPT

    Joined:
    Jul 31, 2013
    Posts:
    21
    Hi, I just used your asset, and it works really well for what I need. 180 Stereoscopic videos (Insta 360 Evo) in a Vive Pro headset. I have two simple questions:

    The texture is moving with the camera - I assume because it is stereoscopic. Can I watch the video but also walk around it (like in a gallery)?

    Does having 180 video wrapped on a curved surface add or detract? Any pointers?

    Thanks
     
  17. DarkAkuma

    DarkAkuma

    Joined:
    Nov 28, 2013
    Posts:
    46
    None of the example scenes I provide are for 180 media (not that my asset cant be used with it). I'll assume you are using the 360 demo, as you suggest it looks fine, and some 180 videos are formatted as 360 with the other 180 degrees blacked out.

    If the canvas is moving with a camera as you move, it sounds like the canvas is parented under the camera/character controller. Dragging it out of the cameras/character controllers hierarchy should stop it from moving.

    180 media should be on a curved surface. The center needs to be in front of the viewer, and the sides need to be beside. So I'd say that it "adds". A flat surface would subtract.

    Since you reminded me... as I said, my examples do not have anything for 180 degree media. So I whipped up a couple meshes just now for such media.

    Since 180 degree media can vary I made 2. The "Cropped" one is for such media thats basically 360, but half of the image is not used for the real content. That other half may have embedded black space and/or logo or something. The Stretched one is for normal true 180 degree media.

    180 Panorama Canvas-HighPoly 64x64 Cropped.fbx
    180 Panorama Canvas-HighPoly 64x64 Stretched.fbx

    From your questions, I'm not really sure if you were using the 360 canvas or a normal flat 2D canvas. But for pointers, or advice I recommend you use one of these instead. Scale it and try to position it largely as makes sense for the media/scene. In general however it should probably be positioned and sized so that the bottom is touching the ground, and the center is at eye level. So for a ballpark... Scale XYZ=+3.2, Position=Y +1.6. You can adjust from there as you need.
     
  18. AlanPT

    AlanPT

    Joined:
    Jul 31, 2013
    Posts:
    21
    Thanks for that. I had attached your script and the video texture to a plane (rotated 90 degrees). I aligned it so that the videos' horizon matched the horizon in Unity. And the stereoscopic works really well. I'm just after attaching it to a slightly curved screen now.

    The plane wasn't a child to the camera and the plane didn't change rotation or move horizontally or vertically with my movements - only when I try walking to it or walking away.
     
  19. DarkAkuma

    DarkAkuma

    Joined:
    Nov 28, 2013
    Posts:
    46
    Then the "Stretched" .fbx I posted above is exactly what you are looking for. Import that into your project, drag it into your scene and use it as the canvas instead of the plane. If you think it looks good without it, wait until you see it with the proper curved canvas! =)

    Then I'm at a loss as to why it would move. There should be nothing in the script that changes its transform at all...

    Did you somehow accidentally add another script to it? From my asset package... maybe "ObjectMotionsOverTime.cs"? But it could be anything that isn't from my package.
     
  20. AlanPT

    AlanPT

    Joined:
    Jul 31, 2013
    Posts:
    21
    Wow that does look better. I rotated the dome so I was looking straight into it and made the horizons match - and it really felt like I was looking through some kind of portal. A lot better than having it be a skydome and completely wrapped around and suffocating.

    It stopped following after re-opening Unity - so it must have been a weird unity glitch.
     
  21. flyingaudio

    flyingaudio

    Joined:
    Dec 3, 2010
    Posts:
    87
    @DarkAkuma, I purchased your asset yesterday, but I am not getting the results I was expecting. Before troubleshooting, let's confirm what is expected.

    You talk about portals and looking into another world, as though you could step into it. That's what I am looking for, a way to look into a picture, as though I am looking through a window. The effect I am seeing is not like that, it's like "3D" postcard, that as you change your viewing perspective it changes orientation. For instance, Big Bunny video, when I get closer to the glass, I can't see what is further to left and right in the world.

    Am I doing something wrong, or is the effect I am describing as you would expect?

    I am running Unity 2019.1.4f1.
     
  22. DarkAkuma

    DarkAkuma

    Joined:
    Nov 28, 2013
    Posts:
    46
    The effect you are describing is correct. This asset is for viewing typical Stereoscopic 3D media, and that's just how Stereoscopic 3D media is. It's static and doesn't change with a change of perspective. While the "portals" line is mainly just marketing, it's still technically correct when it's not take as literally.

    I have actually dealt with the desired effect I think you were expecting, in a old game jam demo. It's certainly possible to get a more flawless window effect were the viewed contents change with your perspective. But that's a costly thing on performance as it's not using a pre-rendered image/video, but basically rendering the contents of 2 scenes at once.

    If you want something in the middle, less jarring than a static image that doesn't change with perspective, but not going as far as rendering a second scene, something can probably be achieved...

    My first idea would be to mix a 360/180 pano image with a square frame. While the image itself is still static, you would be able to see more of it as you change perspective. There's a couple ways to achieve this. The easier but less perfect one being just putting a 180 hemisphere (I linked such canvas objects above) directly behind a square hole.

    A lesser option would be to slide around a standard square 3D image behind a frame based on camera position. It wouldn't be perfect, but might be less jarring if dialed in right.

    I'm not opposed to working on/adding an example of the first idea to my asset. Off hand I'm not sure how I'd want to design such an example, but I guess I can try working on it asap or slap together something for you if that sounds like something you would want.
     
  23. flyingaudio

    flyingaudio

    Joined:
    Dec 3, 2010
    Posts:
    87
    That did it! Thanks for your fast response and suggestions and willingness to dive in.
    It is now like looking into another room. Bravo!

    I used the stretch 180 mesh you referenced. Now, we need to clean up the edge of the 180 if we want people look in that far.
     
  24. DarkAkuma

    DarkAkuma

    Joined:
    Nov 28, 2013
    Posts:
    46
    Awesome! That's great to hear!

    I think the idea can be improved upon over my basic suggestion. A limitation of that method is that the hemisphere has to be behind a wall. It cant be in the middle of a room like with my example scene(s). Or else a player could simply walk/look behind and see parts of an odd dome. For a included example I'd probably use a stencil shader to hide the hemisphere from view unless looking through the frame.

    I believe there's a possibility that's much easier users in the end, using a custom shader on the square canvas that can take a 360/180 pano image. I think I've seen such a thing. But I'll need to write that shader, and I'm not that great at shader code. But I wanted to write it anyway, so more reason to get around to it! =)
     
  25. flyingaudio

    flyingaudio

    Joined:
    Dec 3, 2010
    Posts:
    87
    Right, the first thing I had to do was get rid of the screen and create a frame (not a solid plane) to give it the rectangular look I wanted.

    I am happy to test out any updates you make to the process. Thanks.
     
  26. DarkAkuma

    DarkAkuma

    Joined:
    Nov 28, 2013
    Posts:
    46
    I finished work on the new panoramic shader canvas option. It should be ready in the next release.

    The old Mesh Sphere/Hemi-Sphere method will still be available, this will only be an option as there are pro/cons between the 2 methods.

    The shader option will support Y axis rotation of the texture, and tiling/black space of 180 images based on the textures WrapMode.

    EDIT:

    v2.3 is published now. It contains the new shader method option for panoramic media, and several other new changes/refinements!
     
    Last edited: Jun 10, 2019
  27. AlanPT

    AlanPT

    Joined:
    Jul 31, 2013
    Posts:
    21
    I'm having some problems with using VRTK 4 and your asset. I select the HeadAnchor (with the camera) then click the Make 3D Camera (with preserve.. ticked). It then copies the camera with its settings and assigning left and right (as normal). But then the new camera gets the button and wants it to be converted (copied), and the original camera has nothing to show from it.
    When playing, the video only shows in one eye (obviously) and is upside down. I have had no problem using standard OpenVR without VRTK. Are you able to have a look?

    upload_2019-7-11_14-15-25.png upload_2019-7-11_14-17-6.png

    upload_2019-7-11_14-17-42.png
     
  28. DarkAkuma

    DarkAkuma

    Joined:
    Nov 28, 2013
    Posts:
    46
    I haven't used VRTK in a couple years. I just tried to download and install it, but I didn't have much luck since I don't have a 2018+ build of Unity atm. So I'll have to try to help you solve this more manually atm.

    As best I can tell, I think the one eye part of the issue might be because one of the cameras isnt getting its culling mask layer set during the setup because the default "HeadAnchor" camera does not have the left/right image layers checked by default. That would be a bug in my code. For now just ensure that "HeadAnchor-Left" has LeftImage checked under culling mask, and "HeadAnchor-Right" has RightImage checked. And neither has the other layer checked. The "Make 3D Camera" button shows because those layers arent selected.

    I'm not sure about the upside down thing though. Is Vertical Flip checked in the loaded Stereoscopic3DImage? That option was newly added to the UI in v2.3, and you may have accidentally ticked it being unaware of it. Sometimes textures get generated oddly and may require that option to get flipped right side up, which is why I added it. So perhaps you are even using something now that may require using the option if you are not.
     
    Last edited: Jul 11, 2019
  29. AlanPT

    AlanPT

    Joined:
    Jul 31, 2013
    Posts:
    21
    Thanks! That fixed it - the new camera didn't add itself to the cull. And all I had to do with the other problem was to rotate the gameobject :rolleyes:.

    I'm using VRTK with recent 2019.1.6f1 and it works fine - just need to follow the github instructions carefully.

    Does using dual cameras use up much more resources than the new default way? Is this something I should be concerned about?
     
  30. DarkAkuma

    DarkAkuma

    Joined:
    Nov 28, 2013
    Posts:
    46
    I don't believe so, but I'm not exactly sure. The single pass stuff was added to Unity after I created this asset. I doubt its done in a single pass when cameras are split. But you are only rendering 1 eye each pass. So its a trade off.

    I used to include my own sort of single pass method. It got broke with the update that changed the camera, so I removed it. At the time, updating things to work with the new camera didn't work, but maybe now might be different. I'll have to look into it.

    EDIT:

    I've looked into things a little more to try and give a better answer.

    First, I'll admit that I'm unclear on if 2 different cameras each rendering a different eye to the viewport is done in a single pass. I've assumed it was 2 passes, but now I'm unsure.

    If it is 2 passes, then this split camera setup is slower, naturally. But in what situations its slower and how significantly is project/scene dependent. You can find more detailed information here. But a gist is that in multi pass rendering, the amount of other objects in a scene will play a part as each will have to be rendered twice. This should only be significant if you were making a gallery type of scene, likely with non-panoramic media. For most panoramic media scenes, and some non-panoramic scenes, your scenes object count would probably be quite low.

    Again, this is only if a split camera eye setup exclude it from single pass rendering, which may not be the case.

    Still, I do have the task of implementing support for a more modern shader based rendering method on my agenda to consider for the future. Such a method would bypass these concerns and guarantee that single pass rendering was used.

    I hope this help answer any concerns for now.
     
    Last edited: Jul 12, 2019
    AlanPT likes this.
  31. AlanPT

    AlanPT

    Joined:
    Jul 31, 2013
    Posts:
    21
    Thanks for that, I am creating a real life scene, that the 180 videos are placed into (on video kiosks). I assume a more efficient way would be instead to jump to a new scene with just the video playing.

    I notice that in your notes you need to destroy and reopen the videopPlayer to prevent the black material. If I was to have a trigger which enabled and disabled the GameObject, I should also have it to enable and disable the VideoPlayer? Is there a better method?

    Thanks again.
     
  32. DarkAkuma

    DarkAkuma

    Joined:
    Nov 28, 2013
    Posts:
    46
    Trying to look into what you are referring to, I only see notes regarding destroying a videoPlayer component once. That is done to work around a audio bug, so it has nothing to do with a "black material".

    I may be looking at the wrong notes though. I don't recall every note I write down because that's why I write them down. Well, in part because my memory sucks and its helpful to get clued in on my mindset from when I worked on part of some code, and also for users who need to make edits. If those aren't the notes you are referring to (Line 625 of VR3DMediaViewer.cs), then can you please point me to what you are?

    Regardless, you should probably instead leave the VideoPlayer component enabled when the parent object is disabled and instead just pause/stop/play the video at the same time. Though I'm not sure if that's necessary. it's probably just a good idea to retain control and know exactly what you want/expect to happen to the video when hidden/shown is happening.
     
  33. AlanPT

    AlanPT

    Joined:
    Jul 31, 2013
    Posts:
    21
    Yes, you were talking about audio (Line 625). Sorry for the confusion. When I attempt to stop and the play the video again it only shows up blank. However if its paused it works. I suspect it's my code and nothing to do with you.

    In VR3DMediaViewer I change line 650 to videoPlayer.playOnAwake = false;

    Then I use this code attached alongside the VR3D media viewer script (a collider trigger and a non-gravity rigidbody).:

    Code (CSharp):
    1.     void OnTriggerEnter(Collider player)
    2.  
    3.     {
    4.         var videoPlayer = GetComponent<UnityEngine.Video.VideoPlayer>();
    5.  
    6.         if (player.gameObject.tag == "MainCamera")
    7.         {
    8.             videoPlayer.enabled = true;
    9.             videoPlayer.Prepare();
    10.             videoPlayer.Play();
    11.         }
    12.     }
    13.     void OnTriggerExit(Collider player)
    14.     {
    15.         var videoPlayer = GetComponent<UnityEngine.Video.VideoPlayer>();
    16.  
    17.         if (player.gameObject.tag == "MainCamera")
    18.         {
    19.             videoPlayer.Stop();
    20.             videoPlayer.enabled = false;
    21.  
    22.         }
    23.     }
    24.  
     
  34. DarkAkuma

    DarkAkuma

    Joined:
    Nov 28, 2013
    Posts:
    46
    Looking into your issue myself, I think theres a quirk of the VideoPlayer component. I think it generates a new texture when Play() is called from a full Stop(), regardless if one existed already. New texture means my code would have to know to generate and load a new S3DImage object.

    So hopefully Pause()/Play() would suit your needs enough. Else you would have to just manually generate and load a new S3DImage object yourself each time.
     
    Last edited: Jul 15, 2019
  35. AlanPT

    AlanPT

    Joined:
    Jul 31, 2013
    Posts:
    21
    It works, I just hope that these videos aren't going to use resources when they are paused. My assumption is that a stopped video would clear some needed ram.
     
  36. mikleas

    mikleas

    Joined:
    Dec 31, 2014
    Posts:
    7
    Hi!
    How can it be used on android and other players like EasyMovie Texture or AVPro?
    Thanks!
     
  37. DarkAkuma

    DarkAkuma

    Joined:
    Nov 28, 2013
    Posts:
    46
    There are a couple paths to get the desired result. But the gist for using non-standard textures that are generated at runtime is, you have to have use code.

    The simplest, most direct option is to set a empty Stereoscopic3DImage object in the canvas, get a reference to the canvas in your script, get a reference to the texture after it's generated at runtime and call SetNewImage(texture); on the canvas.

    Code (CSharp):
    1. VR3DMediaViewer _canvas = this.GetComponent<VR3DMediaViewer>();
    2.  
    3. _canvas.SetNewImage(myTexture);
    You could also generate a new Stereoscopic3DImage object at runtime, set the texture to it and use the Stereoscopic3DImage overload to set that to the canvas.

    At least that's how people do it with AVPro. I'd assume EasyMoveTexture is similar.