Search Unity

  1. How can our website serve you better? Give us your feedback. Take our survey and let us know.
    Dismiss Notice

[RELEASED] Camera Projection Rendering Toolkit (Non-linear perspectives, oblique projections, SSAA)

Discussion in 'Assets and Asset Store' started by Melcx, Jul 14, 2017.

  1. Tryz

    Tryz

    Joined:
    Apr 22, 2013
    Posts:
    3,382
    Hey @Melcx ,

    I'm absolutely loving the tool. It's working great.

    One thing I'm getting in Unity 2017.3.1f1 is a warning when CPRT is enabled:
    OnRenderImage() possibly didn't write anything to the destination texture!

    When I disable CPRT, the warning stops. Any idea how to stop this from generating each frame?

    Thanks!
     
  2. Melcx

    Melcx

    Joined:
    Mar 22, 2016
    Posts:
    42
    Hi @Tryz,

    Sorry for the late reply.
    This warning is a known issue and has no real impact on the released game (it is just logged in the editor and in Debug builds).
    The problem is I don't use a classic rendering method (I'm "redirecting" the rendering from one buffer to another to have access to higher resolution), and Unity think nothing seems to be drawn on the render texture. I've not found a way to remove this warning yet.
    However, it is only logged a few times in my test scenes. I'm not sure how you end up getting this warning every single frame. Could give me more details about your implementation please ? (this might be related to repeated enabling/disabling of a camera or RenderTexture / resolution changes).

    Best regards
     
  3. sasob8

    sasob8

    Joined:
    Jul 11, 2017
    Posts:
    16
    Hey @Melcx,

    Your script is limited to 160 fov? I need a solution for about 240 fov.
    For now i made camera rig with 5 cameras (45 fov, 1920x1200) with overlapping, so final resolution will be 6000x1200 (panorama style - cylindrical). I render to a texture with a recorder, so i don't need real time performance.
    It would be so much easier if i could do it with one camera. Especaially cause of post effect (some of them are repeating, cause of 5 cameras).

    Can it be done with your tool? If not, do you plan to update?

    Thank you
     
  4. Melcx

    Melcx

    Joined:
    Mar 22, 2016
    Posts:
    42
    Hello @sasob8,

    Sorry to disappoint but you wont be able to achieve that with CPRT. It is limited to 160 FOV because you get an anomaly around 180, and the way it works, it can't technically go over 180 with a single camera (it just wouldn't work because that's a limitation of the classic 3D APIs). Trying to go over 180, FOV would require an entirely different technique and it would probably be a terrible idea to base it on CPRT. That would strictly be a different plugin, which couldn't even be used on the Unity rendering engine.

    About your issue at hand, the way you're approaching it is good. Maybe you should try to split your post fx ? You could keep applying the ones not causing issue as you're doing right now, and try to delay the problematic ones to a later stage, on the final composing rendering (ie : bloom can be done that way if you keep the HDR format). I don't know if that's going to help you out but I don't have any other idea at the moment.

    Best of luck with your project,
    Regards
     
    Last edited: Aug 28, 2018
  5. skyecrack

    skyecrack

    Joined:
    Jan 5, 2018
    Posts:
    29
    Hey @Melcx have you taken a look at this?

    Also example

    Just stumbled upon it, looks like it would be really easy to implement, with only drawback being either having to write custom tessellation shaders or just increasing polygon count on all of the meshes from the start, but even with that the performance gain would be massive over what we currently have here. Moreover, combine that with SSAA and voila, you actually benefit from supersampling under 2x :D
     
    AlanMattano likes this.
  6. Melcx

    Melcx

    Joined:
    Mar 22, 2016
    Posts:
    42
    As you said, Vertex Displacement for linear projection is an option and could give pretty good results on screen with not too shabby performances (of course, it would vary depending on the project)

    That said, doing so wouldn't be about helping configure camera and rendering, but about applying vertex shaders for every single 3D object.
    On my side I'd have to create shaders for every custom projection and replace all the corresponding unity standard shaders, and on the user side, if a single object doesn't have that new shader type, the rendering would lose its coherence (that would make using external plugins very tedious). Moreover, you'd have multiple technical issues : ie Surface Shaders and Unity Shadows which aren't meant to work in that way.

    Again, thank you for showing such an interest in the development of this plugin, but this is way out of scope for what I intended to do at the beginning, and would require too much time and effort on my part for it to be worthwhile.
     
  7. Yumby

    Yumby

    Joined:
    Aug 17, 2012
    Posts:
    14
    I'm getting a strange linker error with CPRT when compiling in XCode using Unity 2018.2.14f1. If I don't include CPRT.cs, CPRTFrustrumUpdater.cs and CPRTHelper.cs, everything builds fine. With those files included, everything runs fine in the editor but when trying to compile in XCode, I'm getting:

    ld: symbol(s) not found for architecture arm64
    clang: error: linker command failed with exit code 1 (use -v to see invocation)

    Odd since this plugin is provided as source code.
     
    Last edited: Mar 1, 2019
  8. Melcx

    Melcx

    Joined:
    Mar 22, 2016
    Posts:
    42
    Sorry, I don't have an Apple setting to test that precise use case.
    That said, from what I can see, this error shows up because the compiler doesn't find a "static extern function", which is strange since I didn't use any in my plugin. It must be used indirectly by something outside of CPRT.
    Could you give me more information on this error ? If you look in the log, you should find a line which would tell me which function isn't supported. Something like : << Symbol: _OBJC_CLASS_$_[AnyFunctionName] >> or << "_OBJC_CLASS_$_[AnyFunctionName]" referenced from : >>

    Also, while I was checking online to see if someone else had had a similar issue, I found that some of them solved their problems by upgrading their 3rd party frameworks, so you may want to try this.

    Finally, could you try to test this with an updated version of Unity ? The current version is 2018.3.f6. You could even try 2018.2.20 if you want to stay on the ".2 series" for whatever reason.

    Cheers.
     
  9. philspitlerSF

    philspitlerSF

    Joined:
    Oct 10, 2018
    Posts:
    37
    Hi, does this plugin work on iOS ?

    I have a new iPad and would love to use this plug on my 360 video in Unity

    Cheers

    Phil
     
  10. Melcx

    Melcx

    Joined:
    Mar 22, 2016
    Posts:
    42
    Hello @philspitlerSF

    As mentioned in my previous answer, I don't have any apple device to test this so I can't say for sure. However what I can do is refund you if you find out that the plugin doesn't work at all for you. Just let me know by mail.

    Cheers and a happy new year to all of you :)
     
    Last edited: Jan 6, 2020
  11. philspitlerSF

    philspitlerSF

    Joined:
    Oct 10, 2018
    Posts:
    37
    I will answer my own question here in case others are curious.

    It works great so far in iOS.
     
    AlanMattano likes this.
  12. Zyblade

    Zyblade

    Joined:
    Jul 1, 2014
    Posts:
    136
    So I can trick unity's deferred rendering pipeline, to be orthographic? How much overhead will this be, resolution wise? Theoretically, faking unity's orthographic projection, wouldn't require more resolution, since the perspective shrinks? Not sure about that though ^-^
     
  13. AlanMattano

    AlanMattano

    Joined:
    Aug 22, 2013
    Posts:
    1,383
    I'm a Windows user.
    Does support URP?
    Does support HDRP?
    Does support Build in Standard PBR 2017 to 2020?
    Sorry if this was asked before
     
  14. Melcx

    Melcx

    Joined:
    Mar 22, 2016
    Posts:
    42
    Hello there !

    @Zyblade As long as you stay with orthographic projections (top down, isometric, etc) the plugin won't need anything more than the base resolution you're using.
    If you start using Stereospherical or Pannini though, it will be another story, and SSAA will have to be involved.

    @AlanMattano
    Sorry to disappoint, but this plugin was made before URP and HDRP were added to Unity and it unfortunately uses methods that you don't have access to while using URP/HDRP. I don't plan to adapt CPRT for it to work in these ways.
    That said, I just tested again to be sure, and I can confirm you that the Built-in Render Pipeline still works without any issue in the latest versions of Unity.

    Sorry for the late reply, it's been a while since there was any activity on this thread.

    Cheers!

    Edit : The answer to Zyblade was invalid : orthographic camera in deferred rendering cannot be achieved, even with CPRT.
     
    Last edited: Nov 22, 2020
    Zyblade and AlanMattano like this.
  15. AlanMattano

    AlanMattano

    Joined:
    Aug 22, 2013
    Posts:
    1,383
    That's fine since Unity now supports it in URP and HDRP as a post-process effect.
    From
    https://forum.unity.com/threads/per...lution-new-scriptable-render-pipeline.517617/
    Documentation
    URP https://docs.unity3d.com/Packages/c...manual/Post-Processing-Panini-Projection.html
    HDRP https://docs.unity3d.com/Packages/c...manual/Post-Processing-Panini-Projection.html
     
  16. Melcx

    Melcx

    Joined:
    Mar 22, 2016
    Posts:
    42
    I guess if you're only interested in panini, you could do it without using CPRT, but you may run into some limitations like (correct me if I'm wrong) the lack of SSAA will make the middle of your screen blurry.

    Anyway, CPRT isn't just about panini, so making it work for URP and HDRP would require more than just adapting the methods used for panini.
     
    AlanMattano likes this.
  17. Zyblade

    Zyblade

    Joined:
    Jul 1, 2014
    Posts:
    136
    I just bought the asset and tried it out. Sadly, I can't trick unity's deferred rendering pipeline, to be orthographic with it....
    The Asset says, I need to switch the camera to be orthographic, to use orthographic projection :/ And I can't use deferred rendering with unitys internal orthographic camera mode.

    Correct me if I'm wrong. But if not, I can't use the asset at all :/
     
  18. Melcx

    Melcx

    Joined:
    Mar 22, 2016
    Posts:
    42
    @Zyblade
    Ok, I double checked what I said to you last time and realised I'm an idiot. I misunderstood your question and completely forgot that the specific case of deferred rendering + orthographic views is a no-go in Unity. This might be possible with URP in most recent Unity versions, but as I said to Allan Mattano, CPRT was made before URP so it is not intended for its use either...

    So yeah, CPRT won't be able to help you for what you're trying to achieve. I'll edit my previous answer and the plugin description too, and obviously gonna give you a full refund.
    To do this I'll need you to send me your customer "invoice number". You should find it in the bill Unity sent you after your purchase. (don't post this here obviously, send me a PM). I'll contact Unity as soon as I get this and will ask them to do the refund.
    I'm deeply sorry for the time loss.

    Before you go, I can give you this piece of advice (I owe you that at least) : From what I see, a common "trick" to fix your issue (granted I understand it well !) would be to use a perspective camera, placing it very far from the scene, and with a high "near plane distance" (eg. far plane distance / 2).

    Good luck for your project and have a nice day.
     
    Zyblade likes this.
  19. Zyblade

    Zyblade

    Joined:
    Jul 1, 2014
    Posts:
    136
    That's very kind of you, but you don't have to. All in all, it's a nice asset. Maybe I can use it in another project some day.

    Nah, it's alright. I just imported it, checked "orthographic projection" and saw it didn't work with deferred rendering mode. That was pretty fast ^-^

    Yeah, that should work. But with a very far camera, my shadow quality is pretty low(even on close fit, due to the large bounds from drawmeshInstancedIndirect). I got a workaround, where I set the bounds from my instances to 0 and setting the Camera.cullingMatrix very high. This way, using close fit, I could render them at high quality(my for isometric low resolution needs), independent of the camera distance. But now, the local lights won't affect my instances ones they are out of those 0 bounds >.<
    So yea, my camera is far away from my objects anyway, I could use it. Just need to trick out unitys internal culling for local lights and object bounds : S

    edit: I could reduce my far clipping plane and shrink the bounds a little and it might work now^^
     
    Last edited: Nov 23, 2020
  20. Melcx

    Melcx

    Joined:
    Mar 22, 2016
    Posts:
    42
    @Zyblade
    Well, thank you for your kind words and I hope you will find some use for CPRT :)

    That said this discussion got me thinking, so I took some time early this week to try to find a way to implement all of this in my plugin, and... I found one !

    Basically I'm adding a "pseudo orthographic projection" option which will do what we discussed earlier while keeping light sources and shadows in check :
    - it will be mimicking orthographic behavior by using a very far from the scene perspective
    - deferred rendering will be supported
    - but you won't be able to use this with oblique projections (obviously, since you're simulating orthogonal projection by just getting far away, it won't be able to do the deformations required to get oblique projections).

    Here's an example of what you should be able to achieve :
    upload_2020-11-25_16-32-55.png

    I'll finalize this as soon as I can and submit it to Unity. Expect this to be available next week or the one after. I'll post an update here anyway.

    Cheers !
     
    OdderOtter and Zyblade like this.
  21. Zyblade

    Zyblade

    Joined:
    Jul 1, 2014
    Posts:
    136
    Wow, really nice!
    Can I set a custom "zoom-size", independent from the camera distance?
    I'd need to fake an orthographic "Size" of 13.5 (resolution height->1080 / px per unit->40 / 2).
    I'm gonna find out a way to fake the size on my own. But you're obviously more into the projection term, so you might know it right off the bat.

    Oh, and another question. Is the camera angle a thing, which is baked into the projection or used from the camera directly? Currently I work with 30,0,0 angle for my isometric view (objects are at 45deg so the camera's y angle can stay a 0 for better calculate some movement rounding stuff). If I get this right, the camera angle can be any arbitrary number, because it won't make the projection any "Oblique?
     
    Last edited: Nov 27, 2020
  22. Melcx

    Melcx

    Joined:
    Mar 22, 2016
    Posts:
    42
    I just submited the new version of the plugin to Unity, hopefully it won't take too long to get validated.

    So, there are 2 new parameters (orthographicSize and perspectiveOffset) which allow you to set the orthographic size you're talking about, and the camera distance, independently.
    More in the documentation.

    About camera angle, you can use whatever you want. No constraint here but you won't be able to have oblique perspective obviously (deferred rendering wouldn't accept it).

    Have a nice day !
     
    Last edited: Nov 30, 2020
    Zyblade likes this.
  23. OdderOtter

    OdderOtter

    Joined:
    Aug 16, 2015
    Posts:
    38
    Just wanted to drop in to express my thanks for the pseudo orthographic update!
     
    Melcx likes this.
  24. Zyblade

    Zyblade

    Joined:
    Jul 1, 2014
    Posts:
    136
    Yup, really appreciated you implemented it =)
     
    Melcx likes this.
  25. MrMelSE

    MrMelSE

    Joined:
    Dec 10, 2017
    Posts:
    22
    Hi,

    Purchased this but relay heavy on RayCasting from Screen -> World which does not work with Pannini correction.
    Any way around it?
     
  26. Melcx

    Melcx

    Joined:
    Mar 22, 2016
    Posts:
    42
    Hi @MrMelSE
    I'm very sorry for the late answer, I took a week off and just came back today.

    About your issue, the action you're describing to me seems to work on my end.
    Just to be sure, are you using one of the CPRT.ScreenPointToRay() functions ? If not, that's probably why it doesn't work for you.

    If it's not that, feel free to tell me, and I wouldn't mind a bit more context about what you're trying to achieve (and how).

    Cheers !
     
  27. patrickk2

    patrickk2

    Joined:
    Dec 8, 2019
    Posts:
    61
    Hello everyone!

    We use CPRT on the main camera of a project I am working on, but when I add the CPRT script the image from the second camera (which provides the background) is no longer rendered.

    The main camera has Depth only as clear flags and films planets and spaceshipts in the scene, while the second camera is set to only show objects on the space background layer. The depths of the main and background cameras are -1 and -2 respectively.

    We would like to use CPRT to counter the distortion of the planets when they are placed near an edge of the screen.

    Does anyone have an idea what could cause the rendering of the background camera to disappear? Any hints or tipps that help me tackle this issue would be greatly appreciated!

    Thanks in advance and best regards,
    Patrick
     
  28. Melcx

    Melcx

    Joined:
    Mar 22, 2016
    Posts:
    42
    Hello Patrick !

    Setting up CPRT on 2 different cameras (or more) is what you'd do for a multi-PoV situation, like the one showcased in the documentation.
    In your case, you end up with a single PoV, so you should set up CPRT only once.

    The way you do this with your "stacked" rendering is by having a script that creates an intermediate RenderTexture in which both of your cameras will render (with a resolution matching the one CPRT needs). Then, you'll need a third camera dedicated to post-processing, which will host CPRT.
    This third camera will render a RawImage with the previously mentioned RenderTexture, and will be the one doing any CPRT transformation you want for your scene.
    Note : Every camera must have the same FoV parameter as the one used by CPRT (CPRT.FieldOfViewY).

    I attached a scene to show you an example of how I did all of this (with a script to handle the RenderTexture and the FoV parameter sync)

    Good luck with your project !
     

    Attached Files:

    patrickk2 likes this.
  29. patrickk2

    patrickk2

    Joined:
    Dec 8, 2019
    Posts:
    61
    Hello Melcx!

    Thank you very much for your reply and the example scene! I will try that as soon as I can.

    Best regards,
    Patrick
     
    Melcx likes this.
  30. patrickk2

    patrickk2

    Joined:
    Dec 8, 2019
    Posts:
    61
    Hello again Melcx!

    Thanks again for the example scene, with that I could realize your solution in my project. It looks very good, but now I my click events no longer work. For example I was using
    OnMouseUp()
    to select units and Raycasting from the main camera to an invisible plane to determine where to move a selected unit.

    Is there a way to translate clicks on the image where the CPRT camera renders the final image back to world coordinates?

    Thanks in advance for any thoughts on this, and best regards,
    Patrick
     
  31. Melcx

    Melcx

    Joined:
    Mar 22, 2016
    Posts:
    42
    Hello again!

    OnMouseUp() can't be used directly because it ignores CPRT. You'll need to use CPRT.ScreenPointToRay() or CPRT.ViewportPointToRay() to do a raycast as demonstrated in the "MultiCamera Picking" scene (the script you may want to refer to is "CameraPickingTest.cs"). If you need more information about these methods, you can refer to part 4.1.4 of the documentation.

    Best regards,
    Melcx
     
    Last edited: May 6, 2021
    patrickk2 likes this.
  32. patrickk2

    patrickk2

    Joined:
    Dec 8, 2019
    Posts:
    61
    Hello Melcx!

    Thank you very much for your answer, the demo really helps a lot!

    Best regards,
    Patrick
     
    Melcx likes this.
  33. patrickk2

    patrickk2

    Joined:
    Dec 8, 2019
    Posts:
    61
    Hello again Melcx!

    Thanks to your tipps, I'm making nice progress with CPRT and the stacked cameras. :) I was hoping you could you have an idea for another issue I ran into.

    The units in my scene should have a nametag above them, and I realized that by adding an empty GameObject to the unit models, and use its position to align a text field in a screen overlay canvas. I found CPRT.WorldToViewportPoint() and hoped I could use that on the position of said empty object and Vector3.Scale() the result with (Screen.width, Screen.heit) to get the correct position on my screen.

    But even when the GameObject is on the very left edge of the screen, CPRT.WorldToViewportPoint() returns at least 0.3 for the X coordinate. I also don't get a value near 1 when the object is on the right edge of the screen. Do you have an idea what could be going wrong here, or if I am using WorldToViewportPoint() wrong somehow?

    Thanks in advance for your input, and also thanks again for all your help so far!

    Best regards,
    Patrick
     
  34. Melcx

    Melcx

    Joined:
    Mar 22, 2016
    Posts:
    42
    Hey !

    Just passing by to say that while investigating the problem, I found out that I apparently broke something in the previous version I released. It was missing a radian to degree conversion.
    I'm uploading a fix right now and it should be available soon. You'll find in it a new example script for your specific use case in the scene I previously added.
    (Actually I'm also pming the fix directly to you in case the unity validation would take time)

    Thank you for noticing that, and I hope everything will be fine for you from now on !

    Cheers !
    Melcx
     

    Attached Files:

    patrickk2 likes this.
  35. patrickk2

    patrickk2

    Joined:
    Dec 8, 2019
    Posts:
    61
    Hello Melcx!

    I'm glad I was of help for detecting the problem instead of using CPRT wrong. :D

    Thank you for the quick fix, and the awesome support in general! I will try this as soon as I can.

    Best regards,
    Patrick
     
    Melcx likes this.
  36. patrickk2

    patrickk2

    Joined:
    Dec 8, 2019
    Posts:
    61
    Hello everyone!

    I see a strange ussue in my WebGL builds when using cameras with CPRT and without CPRT in the same scene, but only when switching to fullscreen.

    Here is an example. This is non-fullscreen (the canvas is 1024x768):
    space-canvas.jpg

    and here is what it looks like in fullscreen (1920x1080):

    space-fullscreen.jpg

    The background and the planets are rendered by cameras which use CPRT, and the spaceship is rendered by one without.

    • camera 1 has the culling mask set to the background (stars and nebulas), no CPRT, depth -2
    • camera 2 only films the planets and has its clear mask set to depth only, no CPRT, depth -1
    • camera 3 uses CPRT and a stacked render texture manager which captures cameras 1 and 2, depth 0
    • camera 4 only films the spaceship and has its clear mask set to depth only, no CPRT, depth 1

    I also tried a less complex setup, and could still reproduce this behaviour.

    This is non-fullscreen (1024x768):

    minimal-canvas.jpg

    and this is in fullscreen (1920x1080):

    minimap-fullscreen.jpg

    Here I only have one camera with CPRT (no render textures) filming the red, green and blue spheres, and a second camera without CPRT only filming the black sphere.

    I tried quite a lot of settings on the cameras, but I could not find a fix for this issue. Does anybody have an idea what could be causing this?

    Thanks in advance for any input you guys might have!

    Best regards,
    Patrick
     
  37. Melcx

    Melcx

    Joined:
    Mar 22, 2016
    Posts:
    42
    Hi Patrick,

    I have a few questions about your setup :
    - What is the value of the Render Size Factor in your scenes ? (visible in CPRT component)
    - Have you tried using a render texture with a sufficient resolution ? (as demonstrated in the test scene)

    Kind regards,
    Melcx

     
  38. patrickk2

    patrickk2

    Joined:
    Dec 8, 2019
    Posts:
    61
    Hey Melcx!

    The Render Size Factor on the CPRT component is 1. Concerning the render texture: I thought this was handled by the Stacked Render Texture Manager, which creates the texture on the fly. And this also seems to adapt to resolution changes correctly (at least in the editor and a standalone windows build,the problem only occurs in a WebGL build). I tried to recreate the setup from the example scene you provided, and this worked fine before I added the additional camera.

    In case it helps: this is the camera setup for the postprocessing camera:
    upload_2021-5-14_19-17-10.png

    I also attached the minimal scene from which I took the screenshots with the spheres in my previous post. (Edit: I replaced the scene attachment with a complete minimal project containing the cprt-test scene.)

    Thanks in advance for any more input you can give on this issue!

    Best regards,
    Patrick
     

    Attached Files:

    Last edited: May 26, 2021
  39. Melcx

    Melcx

    Joined:
    Mar 22, 2016
    Posts:
    42
    Hello again and sorry for the long wait.

    Ok so, the last example scene I gave you was for a general desktop app and was not built with WebGL in mind.
    Since WebGL doesn't seem to resize automatically the RenderTargets (this is what's causing the stretching on the edges), you'll have to do a slightly more complex setup in which you'll need to handle a RenderTarget yourself.

    I'll leave you with another example scene attached in this message, which is closer to your use case :
    - the only cam with CPRT is the background one
    - the foreground cam gets the rendered image from the background cam through a RenderTarget
    - the script BackgroundRenderTextureManager creates and updates the background RenderTarget

    This will have a slight impact on performances equal to the drawing of the RawImage displaying the background texture.

    Have a nice day !

    Melcx
     

    Attached Files:

    patrickk2 likes this.
  40. patrickk2

    patrickk2

    Joined:
    Dec 8, 2019
    Posts:
    61
    Hello Melcx!

    Thank you for the explanation and the example scene. I'm sure this will help me a great deal, I will try that ASAP. :)

    Best regards,
    Patrick
     
    Melcx likes this.
unityunity