Search Unity

Multi-Res Shading (NVAPI)?

Discussion in 'General Graphics' started by zezba9000, Jan 25, 2016.

  1. zezba9000

    zezba9000

    Joined:
    Sep 28, 2010
    Posts:
    726
    Does Unity support Multi-Res Shading?
    Reference: https://developer.nvidia.com/virtual-reality-development

    QUOTE: "Multi-Res Shading is an innovative new rendering technique for VR whereby each part of an image is rendered at a resolution that better matches the pixel density of the warped image. Multi-Res Shading uses Maxwell’s multi-projection architecture to render multiple scaled viewports in a single pass, delivering substantial performance improvements."

    It would be awesome if we could enable this on Camera's with special RenderTargets or something.
    Something to keep in mind at least.
     
  2. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    7,153
    No.

    It's not something that can "just be enabled on the camera", every post process effect or any shader that reads a full screen texture (grab pass, the entire deferred pipeline, soft particles, etc.) need to be rewritten as well to support Multi-Res Shading. The alternative is doing an initial render using Multi-Res Shading and expand it into a full resolution buffer, but this removes much of the performance benefit it's supposed to be helping with.
     
  3. zezba9000

    zezba9000

    Joined:
    Sep 28, 2010
    Posts:
    726
    This doesn't sound right as it depends on what the shader needs to work. Post process effects that need texel offsets might be effected depending on how it uses them for sampling.

    My understanding is they are rendering into a full resolution "buffer" using multiple viewports but the outer ones process less pixel information. The low rez side buffers must be scaled up at some point. After that its just a texture but maybe I'm missing something.
     
  4. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    7,153
    So the center area is rendered at full resolution, but the corners and sides are squished in such that the rendered image is a lower resolution. Then all of the full screen effects use that lower resolution image but the shaders have to know which pixels are in the squished areas.

    The whole point is that it's never expanded to the full resolution. Rendering at the slightly lower overall resolution has some savings, but also in the full screen effects working on fewer pixels.
     
    Last edited: Jan 27, 2016
  5. zezba9000

    zezba9000

    Joined:
    Sep 28, 2010
    Posts:
    726
    "The whole point is that it's never expanded to the full resolution."
    -- That doesn't make sense as it has to be expanded to the resolution of the Monitor at some point. Not all shaders need to know what the pixel density is of a texture either (depends). Anyway I'll more into it later as the Gameworks VR has an example.
     
  6. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    7,153
    VR has to be rendered to a frame buffer that is a higher resolution than the physical display. The Oculus Rift is a suggested 1.4x multiplier which means you render at 3024x1,680 and not 2160x1200. Multi-Res shading reduces the full frame buffer by squishing the sides where you don't need the resolution, but it's still higher than the physical resolution of the display. When the image gets warped you're actually shrinking it down, not expanding.
     
  7. zezba9000

    zezba9000

    Joined:
    Sep 28, 2010
    Posts:
    726
    I understand that.
     
  8. Binary42

    Binary42

    Joined:
    Aug 15, 2013
    Posts:
    158
    Probably the sides aren't squished, but rendered full res with pixels left out in a checker pattern.
    Check out Alex Vlachos presentation from this years GDC
    http://alex.vlachos.com/graphics/Alex_Vlachos_Advanced_VR_Rendering_Performance_GDC2016.pdf

    Edit:
    As you can read on page 49
    ● Valve is using a custom rendering plugin in Unity for The Lab
    ● The Valve VR Rendering Plugin will be free on the Unity Asset Store soon with full source code

    If we are lucky that plugin will come with Fixed Foveated Rendering and Radial Density Masking. :)
     
    Last edited: Mar 22, 2016
  9. bgolus

    bgolus

    Joined:
    Dec 7, 2012
    Posts:
    7,153
    The radial density mask is a completely different technique. It's a very smart, but dirty hack to try to get some extra performance for low end devices. Rendering with that checkerboard pattern will cause problems for anti-aliasing which means increased temporal aliasing at the periphery. If you look it's only used at the lowest quality setting and only when small detail ledgability (i.e. Text) isn't needed.

    Multi-Res Shading is actually scaling the sides and corners, their whole tech is the trick of splitting up each eye view into 9 parts with out extra draw calls. This usually requires 9 separate draw calls for each object to do manually. Likely this is more similar to the single pass stereo instancing technique that many engines are using now, and which Unity is adding in 5.4, but is handled by Nvidia's drivers.
     
  10. shkar-noori

    shkar-noori

    Joined:
    Jun 10, 2013
    Posts:
    833
    I'm really looking forward to this, not only for VR games, but also for Non-VR games, Shadow Warriors 2 added this functionality, it will be a great boost for games running 4K
     
  11. Bloogson

    Bloogson

    Joined:
    Jan 27, 2015
    Posts:
    12
    NVidia released a Unity package in the asset store to add VRWorks features to your Unity project.

    https://developer.nvidia.com/nvidia-vrworks-and-unity

    I haven't had a chance to try it yet, but from what I've heard, the setup is somewhat unreliable, but if you can get it working you can see massive performance gains in frame rendering time.