[RELEASED] Camera Projection Rendering Toolkit (Non-linear perspectives, oblique projections, SSAA)

Hey @Melcx ,

I'm absolutely loving the tool. It's working great.

One thing I'm getting in Unity 2017.3.1f1 is a warning when CPRT is enabled:
OnRenderImage() possibly didn't write anything to the destination texture!

When I disable CPRT, the warning stops. Any idea how to stop this from generating each frame?

Thanks!

 

Hi @Tryz,

Sorry for the late reply.
This warning is a known issue and has no real impact on the released game (it is just logged in the editor and in Debug builds).
The problem is I don't use a classic rendering method (I'm "redirecting" the rendering from one buffer to another to have access to higher resolution), and Unity think nothing seems to be drawn on the render texture. I've not found a way to remove this warning yet.
However, it is only logged a few times in my test scenes. I'm not sure how you end up getting this warning every single frame. Could give me more details about your implementation please ? (this might be related to repeated enabling/disabling of a camera or RenderTexture / resolution changes).

Best regards

 

Hey @Melcx,

Your script is limited to 160 fov? I need a solution for about 240 fov.
For now i made camera rig with 5 cameras (45 fov, 1920x1200) with overlapping, so final resolution will be 6000x1200 (panorama style - cylindrical). I render to a texture with a recorder, so i don't need real time performance.
It would be so much easier if i could do it with one camera. Especaially cause of post effect (some of them are repeating, cause of 5 cameras).

Can it be done with your tool? If not, do you plan to update?

Thank you

 

Hello @sasob8,

Sorry to disappoint but you wont be able to achieve that with CPRT. It is limited to 160 FOV because you get an anomaly around 180, and the way it works, it can't technically go over 180 with a single camera (it just wouldn't work because that's a limitation of the classic 3D APIs). Trying to go over 180, FOV would require an entirely different technique and it would probably be a terrible idea to base it on CPRT. That would strictly be a different plugin, which couldn't even be used on the Unity rendering engine.

About your issue at hand, the way you're approaching it is good. Maybe you should try to split your post fx ? You could keep applying the ones not causing issue as you're doing right now, and try to delay the problematic ones to a later stage, on the final composing rendering (ie : bloom can be done that way if you keep the HDR format). I don't know if that's going to help you out but I don't have any other idea at the moment.

Best of luck with your project,
Regards

 

Hey @Melcx have you taken a look at this?

Also example

Just stumbled upon it, looks like it would be really easy to implement, with only drawback being either having to write custom tessellation shaders or just increasing polygon count on all of the meshes from the start, but even with that the performance gain would be massive over what we currently have here. Moreover, combine that with SSAA and voila, you actually benefit from supersampling under 2x

 

As you said, Vertex Displacement for linear projection is an option and could give pretty good results on screen with not too shabby performances (of course, it would vary depending on the project)

That said, doing so wouldn't be about helping configure camera and rendering, but about applying vertex shaders for every single 3D object.
On my side I'd have to create shaders for every custom projection and replace all the corresponding unity standard shaders, and on the user side, if a single object doesn't have that new shader type, the rendering would lose its coherence (that would make using external plugins very tedious). Moreover, you'd have multiple technical issues : ie Surface Shaders and Unity Shadows which aren't meant to work in that way.

Again, thank you for showing such an interest in the development of this plugin, but this is way out of scope for what I intended to do at the beginning, and would require too much time and effort on my part for it to be worthwhile.

 

I'm getting a strange linker error with CPRT when compiling in XCode using Unity 2018.2.14f1. If I don't include CPRT.cs, CPRTFrustrumUpdater.cs and CPRTHelper.cs, everything builds fine. With those files included, everything runs fine in the editor but when trying to compile in XCode, I'm getting:

ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Odd since this plugin is provided as source code.

 

Sorry, I don't have an Apple setting to test that precise use case.
That said, from what I can see, this error shows up because the compiler doesn't find a "static extern function", which is strange since I didn't use any in my plugin. It must be used indirectly by something outside of CPRT.
Could you give me more information on this error ? If you look in the log, you should find a line which would tell me which function isn't supported. Something like : << Symbol: _OBJC_CLASS_$_[AnyFunctionName] >> or << "_OBJC_CLASS_$_[AnyFunctionName]" referenced from : >>

Also, while I was checking online to see if someone else had had a similar issue, I found that some of them solved their problems by upgrading their 3rd party frameworks, so you may want to try this.

Finally, could you try to test this with an updated version of Unity ? The current version is 2018.3.f6. You could even try 2018.2.20 if you want to stay on the ".2 series" for whatever reason.

Cheers.

 

Hi, does this plugin work on iOS ?

I have a new iPad and would love to use this plug on my 360 video in Unity

Cheers

Phil

 

Hello @philspitlerSF

As mentioned in my previous answer, I don't have any apple device to test this so I can't say for sure. However what I can do is refund you if you find out that the plugin doesn't work at all for you. Just let me know by mail.

Cheers and a happy new year to all of you

 

I will answer my own question here in case others are curious.

It works great so far in iOS.

 

So I can trick unity's deferred rendering pipeline, to be orthographic? How much overhead will this be, resolution wise? Theoretically, faking unity's orthographic projection, wouldn't require more resolution, since the perspective shrinks? Not sure about that though ^-^

 

I'm a Windows user.
Does support URP?
Does support HDRP?
Does support Build in Standard PBR 2017 to 2020?
Sorry if this was asked before

 

Hello there !

@Zyblade As long as you stay with orthographic projections (top down, isometric, etc) the plugin won't need anything more than the base resolution you're using.
If you start using Stereospherical or Pannini though, it will be another story, and SSAA will have to be involved.

@AlanMattano
Sorry to disappoint, but this plugin was made before URP and HDRP were added to Unity and it unfortunately uses methods that you don't have access to while using URP/HDRP. I don't plan to adapt CPRT for it to work in these ways.
That said, I just tested again to be sure, and I can confirm you that the Built-in Render Pipeline still works without any issue in the latest versions of Unity.

Sorry for the late reply, it's been a while since there was any activity on this thread.

Cheers!

Edit : The answer to Zyblade was invalid : orthographic camera in deferred rendering cannot be achieved, even with CPRT.

 

That's fine since Unity now supports it in URP and HDRP as a post-process effect.
From
https://forum.unity.com/threads/per...lution-new-scriptable-render-pipeline.517617/
Documentation
URP https://docs.unity3d.com/Packages/c...manual/Post-Processing-Panini-Projection.html
HDRP https://docs.unity3d.com/Packages/c...manual/Post-Processing-Panini-Projection.html

 

I guess if you're only interested in panini, you could do it without using CPRT, but you may run into some limitations like (correct me if I'm wrong) the lack of SSAA will make the middle of your screen blurry.

Anyway, CPRT isn't just about panini, so making it work for URP and HDRP would require more than just adapting the methods used for panini.

 

I just bought the asset and tried it out. Sadly, I can't trick unity's deferred rendering pipeline, to be orthographic with it....
The Asset says, I need to switch the camera to be orthographic, to use orthographic projection :/ And I can't use deferred rendering with unitys internal orthographic camera mode.

Correct me if I'm wrong. But if not, I can't use the asset at all :/

 

@Zyblade
Ok, I double checked what I said to you last time and realised I'm an idiot. I misunderstood your question and completely forgot that the specific case of deferred rendering + orthographic views is a no-go in Unity. This might be possible with URP in most recent Unity versions, but as I said to Allan Mattano, CPRT was made before URP so it is not intended for its use either...

So yeah, CPRT won't be able to help you for what you're trying to achieve. I'll edit my previous answer and the plugin description too, and obviously gonna give you a full refund.
To do this I'll need you to send me your customer "invoice number". You should find it in the bill Unity sent you after your purchase. (don't post this here obviously, send me a PM). I'll contact Unity as soon as I get this and will ask them to do the refund.
I'm deeply sorry for the time loss.

Before you go, I can give you this piece of advice (I owe you that at least) : From what I see, a common "trick" to fix your issue (granted I understand it well !) would be to use a perspective camera, placing it very far from the scene, and with a high "near plane distance" (eg. far plane distance / 2).

Good luck for your project and have a nice day.

 

That's very kind of you, but you don't have to. All in all, it's a nice asset. Maybe I can use it in another project some day.

Nah, it's alright. I just imported it, checked "orthographic projection" and saw it didn't work with deferred rendering mode. That was pretty fast ^-^

Yeah, that should work. But with a very far camera, my shadow quality is pretty low(even on close fit, due to the large bounds from drawmeshInstancedIndirect). I got a workaround, where I set the bounds from my instances to 0 and setting the Camera.cullingMatrix very high. This way, using close fit, I could render them at high quality(my for isometric low resolution needs), independent of the camera distance. But now, the local lights won't affect my instances ones they are out of those 0 bounds >.<
So yea, my camera is far away from my objects anyway, I could use it. Just need to trick out unitys internal culling for local lights and object bounds : S

edit: I could reduce my far clipping plane and shrink the bounds a little and it might work now^^

 

@Zyblade
Well, thank you for your kind words and I hope you will find some use for CPRT

That said this discussion got me thinking, so I took some time early this week to try to find a way to implement all of this in my plugin, and... I found one !

Basically I'm adding a "pseudo orthographic projection" option which will do what we discussed earlier while keeping light sources and shadows in check :
- it will be mimicking orthographic behavior by using a very far from the scene perspective
- deferred rendering will be supported
- but you won't be able to use this with oblique projections (obviously, since you're simulating orthogonal projection by just getting far away, it won't be able to do the deformations required to get oblique projections).

Here's an example of what you should be able to achieve :


I'll finalize this as soon as I can and submit it to Unity. Expect this to be available next week or the one after. I'll post an update here anyway.

Cheers !

 

Wow, really nice!
Can I set a custom "zoom-size", independent from the camera distance?
I'd need to fake an orthographic "Size" of 13.5 (resolution height->1080 / px per unit->40 / 2).
I'm gonna find out a way to fake the size on my own. But you're obviously more into the projection term, so you might know it right off the bat.

Oh, and another question. Is the camera angle a thing, which is baked into the projection or used from the camera directly? Currently I work with 30,0,0 angle for my isometric view (objects are at 45deg so the camera's y angle can stay a 0 for better calculate some movement rounding stuff). If I get this right, the camera angle can be any arbitrary number, because it won't make the projection any "Oblique?

 

I just submited the new version of the plugin to Unity, hopefully it won't take too long to get validated.

So, there are 2 new parameters (orthographicSize and perspectiveOffset) which allow you to set the orthographic size you're talking about, and the camera distance, independently.
More in the documentation.

About camera angle, you can use whatever you want. No constraint here but you won't be able to have oblique perspective obviously (deferred rendering wouldn't accept it).

Have a nice day !

 

Just wanted to drop in to express my thanks for the pseudo orthographic update!

 

Yup, really appreciated you implemented it =)

 

Hi,

Purchased this but relay heavy on RayCasting from Screen -> World which does not work with Pannini correction.
Any way around it?

 

Hi @MrMelSE
I'm very sorry for the late answer, I took a week off and just came back today.

About your issue, the action you're describing to me seems to work on my end.
Just to be sure, are you using one of the CPRT.ScreenPointToRay() functions ? If not, that's probably why it doesn't work for you.

If it's not that, feel free to tell me, and I wouldn't mind a bit more context about what you're trying to achieve (and how).

Cheers !

 

Hello everyone!

We use CPRT on the main camera of a project I am working on, but when I add the CPRT script the image from the second camera (which provides the background) is no longer rendered.

The main camera has Depth only as clear flags and films planets and spaceshipts in the scene, while the second camera is set to only show objects on the space background layer. The depths of the main and background cameras are -1 and -2 respectively.

We would like to use CPRT to counter the distortion of the planets when they are placed near an edge of the screen.

Does anyone have an idea what could cause the rendering of the background camera to disappear? Any hints or tipps that help me tackle this issue would be greatly appreciated!

Thanks in advance and best regards,
Patrick

 

Hello Patrick !

Setting up CPRT on 2 different cameras (or more) is what you'd do for a multi-PoV situation, like the one showcased in the documentation.
In your case, you end up with a single PoV, so you should set up CPRT only once.

The way you do this with your "stacked" rendering is by having a script that creates an intermediate RenderTexture in which both of your cameras will render (with a resolution matching the one CPRT needs). Then, you'll need a third camera dedicated to post-processing, which will host CPRT.
This third camera will render a RawImage with the previously mentioned RenderTexture, and will be the one doing any CPRT transformation you want for your scene.
Note : Every camera must have the same FoV parameter as the one used by CPRT (CPRT.FieldOfViewY).

I attached a scene to show you an example of how I did all of this (with a script to handle the RenderTexture and the FoV parameter sync)

Good luck with your project !

 

Hello Melcx!

Thank you very much for your reply and the example scene! I will try that as soon as I can.

Best regards,
Patrick

 

Hello again Melcx!

Thanks again for the example scene, with that I could realize your solution in my project. It looks very good, but now I my click events no longer work. For example I was using

OnMouseUp()

to select units and Raycasting from the main camera to an invisible plane to determine where to move a selected unit.

Is there a way to translate clicks on the image where the CPRT camera renders the final image back to world coordinates?

Thanks in advance for any thoughts on this, and best regards,
Patrick

 

Hello again!

OnMouseUp() can't be used directly because it ignores CPRT. You'll need to use CPRT.ScreenPointToRay() or CPRT.ViewportPointToRay() to do a raycast as demonstrated in the "MultiCamera Picking" scene (the script you may want to refer to is "CameraPickingTest.cs"). If you need more information about these methods, you can refer to part 4.1.4 of the documentation.

Best regards,
Melcx

 

Hello Melcx!

Thank you very much for your answer, the demo really helps a lot!

Best regards,
Patrick

 

Hello again Melcx!

Thanks to your tipps, I'm making nice progress with CPRT and the stacked cameras. I was hoping you could you have an idea for another issue I ran into.

The units in my scene should have a nametag above them, and I realized that by adding an empty GameObject to the unit models, and use its position to align a text field in a screen overlay canvas. I found CPRT.WorldToViewportPoint() and hoped I could use that on the position of said empty object and Vector3.Scale() the result with (Screen.width, Screen.heit) to get the correct position on my screen.

But even when the GameObject is on the very left edge of the screen, CPRT.WorldToViewportPoint() returns at least 0.3 for the X coordinate. I also don't get a value near 1 when the object is on the right edge of the screen. Do you have an idea what could be going wrong here, or if I am using WorldToViewportPoint() wrong somehow?

Thanks in advance for your input, and also thanks again for all your help so far!

Best regards,
Patrick

 

Hey !

Just passing by to say that while investigating the problem, I found out that I apparently broke something in the previous version I released. It was missing a radian to degree conversion.
I'm uploading a fix right now and it should be available soon. You'll find in it a new example script for your specific use case in the scene I previously added.
(Actually I'm also pming the fix directly to you in case the unity validation would take time)

Thank you for noticing that, and I hope everything will be fine for you from now on !

Cheers !
Melcx

 

Hello Melcx!

I'm glad I was of help for detecting the problem instead of using CPRT wrong.

Thank you for the quick fix, and the awesome support in general! I will try this as soon as I can.

Best regards,
Patrick

 

Hello everyone!

I see a strange ussue in my WebGL builds when using cameras with CPRT and without CPRT in the same scene, but only when switching to fullscreen.

Here is an example. This is non-fullscreen (the canvas is 1024x768):


and here is what it looks like in fullscreen (1920x1080):



The background and the planets are rendered by cameras which use CPRT, and the spaceship is rendered by one without.

Spoiler: Detailed description

• camera 1 has the culling mask set to the background (stars and nebulas), no CPRT, depth -2
• camera 2 only films the planets and has its clear mask set to depth only, no CPRT, depth -1
• camera 3 uses CPRT and a stacked render texture manager which captures cameras 1 and 2, depth 0
• camera 4 only films the spaceship and has its clear mask set to depth only, no CPRT, depth 1

I also tried a less complex setup, and could still reproduce this behaviour.

This is non-fullscreen (1024x768):



and this is in fullscreen (1920x1080):



Here I only have one camera with CPRT (no render textures) filming the red, green and blue spheres, and a second camera without CPRT only filming the black sphere.

I tried quite a lot of settings on the cameras, but I could not find a fix for this issue. Does anybody have an idea what could be causing this?

Thanks in advance for any input you guys might have!

Best regards,
Patrick

 

Hi Patrick,

I have a few questions about your setup :
- What is the value of the Render Size Factor in your scenes ? (visible in CPRT component)
- Have you tried using a render texture with a sufficient resolution ? (as demonstrated in the test scene)

Kind regards,
Melcx

 

Hey Melcx!

The Render Size Factor on the CPRT component is 1. Concerning the render texture: I thought this was handled by the Stacked Render Texture Manager, which creates the texture on the fly. And this also seems to adapt to resolution changes correctly (at least in the editor and a standalone windows build,the problem only occurs in a WebGL build). I tried to recreate the setup from the example scene you provided, and this worked fine before I added the additional camera.

In case it helps: this is the camera setup for the postprocessing camera:


I also attached the minimal scene from which I took the screenshots with the spheres in my previous post. (Edit: I replaced the scene attachment with a complete minimal project containing the cprt-test scene.)

Thanks in advance for any more input you can give on this issue!

Best regards,
Patrick

 

Hello again and sorry for the long wait.

Ok so, the last example scene I gave you was for a general desktop app and was not built with WebGL in mind.
Since WebGL doesn't seem to resize automatically the RenderTargets (this is what's causing the stretching on the edges), you'll have to do a slightly more complex setup in which you'll need to handle a RenderTarget yourself.

I'll leave you with another example scene attached in this message, which is closer to your use case :
- the only cam with CPRT is the background one
- the foreground cam gets the rendered image from the background cam through a RenderTarget
- the script BackgroundRenderTextureManager creates and updates the background RenderTarget

This will have a slight impact on performances equal to the drawing of the RawImage displaying the background texture.

Have a nice day !

Melcx

 

Hello Melcx!

Thank you for the explanation and the example scene. I'm sure this will help me a great deal, I will try that ASAP.

Best regards,
Patrick

 

Hi @Melcx
Thanks for the plugin, the work done is pretty impressive.
I have the following issue and I am unsure if I am using the right settings:
- for a specific application, I need to render my scene spanning on 3 displays in 4k, so for a total resolution of 11520 x 2160.
- without the CPRT script, I can get the perfect resolution without any issue
- when I attach the script, with the Pannini adaptive option, the image is losing quite a lot in quality, with a resolution noticeably lower than the target. Image is more pixelated.
- I am using Unity 2021.1.

is there a way to get the targeted resolution 3x4k with the plug in activated?

thanks a lot for your help!

best,

 

To complete my message, I attach here the current settings I am using. It looks like the full resolution is overwritten somehow.

 

Hello @unity_4B061752A5059274ADC8 ,

I'm very sorry for the late reply. I've been quite busy recently and since there's close to no activity on this thread, I missed your post.

Anyway, about your issue, it seems that what you're asking from CPRT and Unity is much more than what they can acomplish at their best. Unity has a hard cap of 16384 pixels for textures and when you try to use the panini projection, CPRT has to use MSAA and create a bigger intermediate render texture, which increases your base resolution (11520 x 2160) to a point where it far exceeds that cap.

In short, I'm sorry but there's not much I can do for you.
I would suggest to keep the vertical FoV value relatively low and to reduce the intensity of the projection, but if you really want to keep these values, you'll have to find another tool I'm afraid (and I don't think this would be doable in Unity since you'd probably need to use ray-tracing, and the one available in Unity isn't suitable for such intensive use).

I'll leave you with these settings recommendations :


Best regards,
Melcx