Search Unity

  1. Megacity Metro Demo now available. Download now.
    Dismiss Notice
  2. Unity support for visionOS is now available. Learn more in our blog post.
    Dismiss Notice

Can shaders do this? Texture/Image Stitching?

Discussion in 'Shaders' started by JasonsFreeTime, Sep 10, 2019.

?

Do you think this is possible?

  1. Yes Absolutely!

  2. Definitely Not!

  3. Maybe, perhaps with a serious amount of time and research.

Results are only viewable after voting.
  1. JasonsFreeTime

    JasonsFreeTime

    Joined:
    May 23, 2015
    Posts:
    36
    This is a general question for shader experts as I don't really know shaders well. In a nutshell I was wondering if I had multiple render textures of real-time camera images (I'm using AVProLive plugin from the asset store), would it be possible to use shaders to find where the images align and overlap and then "stitch" them together before rendering. Basically the idea is to create high resolution 360 surround video from the live real-time input of several fixed fisheye cameras similar to what is being done here: https://www.csie.ntu.edu.tw/~cyy/publications/papers/Lin2018ALC.pdf
    Would any of those algorithms translate to something that could be done in with shader code?
    Sorry if this comes across as ridiculous as if it were possible I assume it would have already been done considering this is a highly sought after solution to 360 video resolution in VR. Thanks.
     
  2. scottunity

    scottunity

    Joined:
    Feb 1, 2014
    Posts:
    72
    Shaders are by and large designed to process vertexes of geometry and pixels being rendered to screen or offscreen.
    Usually thsi is done in a single pass or by layering.
    While you can use shaders to warp and composite images it woudl be pushing it to do the entire feature extraction and stitching process within the shader.

    Thsi is typically done with the aid of different vision libraries to do Slam and other operations. To try to recreate all of that in a shader would be a very big challenge. Much faster and easier to build as it's own app using libraries although you can use the shaders to help render the finished 360.

    Here's a general pipeline for stitching
    https://docs.opencv.org/2.4/modules/stitching/doc/introduction.html
     
  3. mgear

    mgear

    Joined:
    Aug 3, 2010
    Posts:
    9,350