[URP] Problems arising after implementing custom projection matrix

Using Unity 2021.1.7f1 and URP 11.0.0

In order to have pixel art sprites and textures display correctly, I've implemented a custom projection matrix for my project. I admittedly do not understand matrix math well at all, but I came across this solution here, though I had to make some changes to get it working correctly, namely setting column 1 of the matrix rather than column 2 as they did in the example. As I understand, it is effectively making it so that the camera renders the Y axis in world space as though it were aligned with the camera's Y axis vector. Here is the script as I've implemented it:

Code (CSharp):

1. using UnityEngine;
2. using System.Collections;
3. using UnityEngine.Rendering;
4.  
5. // This script is meant to be attached to your main camera.
6. // If you want to use it on more than one camera at a time, it will require
7. // modifcations due to the Camera.on* delegates in OnEnable()/OnDisable().
8.  
9. [ExecuteInEditMode]
10. public class CustomProjection : MonoBehaviour
11. {
12.     private void OnEnable()
13.     {
14.         RenderPipelineManager.beginFrameRendering += ScenePreCull;
15.     }
16.  
17.     private void OnDisable()
18.     {
19.         RenderPipelineManager.beginFrameRendering -= ScenePreCull;
20.         GetComponent<Camera>().ResetWorldToCameraMatrix();
21.     }
22.  
23.     private void ScenePreCull(ScriptableRenderContext context, Camera[] cam)
24.     {
25.         // If the camera is the scene view camera, call our OnPreCull() event method for it.
26.         OnPreCull();
27.     }
28.  
29.     private Camera cam;
30.     public GameObject mainCamera;
31.     private void Start()
32.     {
33.         cam = GetComponent<Camera>();
34.  
35.         // Set the camera this script is attached to to use orthographic sorting order.
36.         // Instead of using the euclidean distance from the camera, objects will be sorted based on their depth into the scene.
37.         cam.transparencySortMode = TransparencySortMode.Orthographic;
38.     }
39.  
40.  
41.     [SerializeField] private Vector4 up = new Vector4(0, 1, 0, 0);
42.  
43.     // This is a Unity callback and is the ideal place to set the worldToCameraMatrix.
44.     private void OnPreCull()
45.     {
46.         // First calculate the regular worldToCameraMatrix.
47.         // Start with transform.worldToLocalMatrix.
48.  
49.         var m = GetComponent<Camera>().transform.worldToLocalMatrix;
50.         // Then, since Unity uses OpenGL's view matrix conventions
51.         // we have to flip the z-value.
52.         m.SetRow(2, -m.GetRow(2));
53.  
54.         // Now for the custom projection.
55.         // Set the world's up vector to always align with the camera's up vector.
56.         // Add a small amount of the original up vector to
57.         // ensure the matrix will be invertible.
58.         // Try changing the vector to see what other projections you can get.
59.         m.SetColumn(1, 1e-3f * m.GetColumn(1) + up);
60.  
61.         GetComponent<Camera>().worldToCameraMatrix = m;
62.     }
63. }

I think this does a good job of achieving the visual style that I am looking for, but it has of course come with some issues. The first is that the higher objects get in the world space, the further back it will appear, regardless of how far from the camera it is. Which, I suppose makes sense. I was thinking perhaps there is some way to get the matrix to "scale" with the camera's actual world Y position as the character it is following moves along the Y axis, so that it doesn't cause this distortion. Here's some screenshots of what's going on:

Character object at Y = 0 without custom projection and then with:



Character object at Y = 5 without custom projection and then with:



The second issue is that the realtime shadows begin acting strangely when the projection matrix script is active. Strangely enough, this seems to happen only when the project is using URP, the default renderer has no issues. It's almost like there's these strange cutouts at certain points of the shadowmap, and the move along with the camera. Changing settings like the shadows max distance, cascade count/split, and the camera's clip planes can "move" around these artifacts, but I can not get rid of them entirely. Is this just some sort of bug with URP's lighting? Here's a screenshot of the shadow issue:


If anyone could help with either of these issues, it would be greatly appreciated!

 

I don’t know for sure, but the shadow problem could be caused by the fact you’re modifying the view matrix and not the projection matrix. There may be code in the URP that makes the assumption that the view matrix isn’t being modified. Similar modifications to the projection matrix should produce a similar, if not identical, visual result, and may not have the shadow issues as VR rendering has to contend with similarly non-standard projection matrices for certain headsets and presumably the URP handles those correctly. But who knows, it might be broken there too.

As for the “shrinking” issue, that might also be solved when modifying the projection matrix instead of the view matrix. For the effect you’re trying to go for, it’s probably going to be better to keep the camera actually oriented so it’s not rotated up or down and use an oblique projection like this:
https://docs.unity3d.com/Manual/ObliqueFrustum.html

 

This did the trick perfectly, left the camera at 0 X axis rotation, and set the vertical obliqueness to a negative number low enough to simulate a 45 degree angle. After playing around with FOV a bit, I got the exact angled top down view of the ground and direct view of vertical objects I was looking for. Thanks a ton