Question How to pass G-buffer tensors to barracuda?

I'm interested in passing the G-buffers for a deferred shader into a barracuda image enhancement model.
Apparently in the latest versions of URP, deferred rendering (with G-buffers) is available:
https://docs.unity3d.com/Packages/c...manual/rendering/deferred-rendering-path.html

I was wondering if there were any good examples of doing this. It looks like the URP deferred render path is pretty bleeding edge.

 

I answered your other cross-post, but I'll try and address the question here, too.

You might take a look at the style transfer demo for some inspiration. It doesn't use the URP deferred renderer path, but it did mention that future work would be to utilize G-buffers, so I'd assume the structure of the project (I didn't work on this personally) is set up in such a way that would allow for that:
https://blog.unity.com/technology/real-time-style-transfer-in-unity-using-deep-neural-networks
https://github.com/UnityLabs/barracuda-style-transfer

For creating Tensor data from texture input:
https://docs.unity3d.com/Packages/c...3/manual/TensorHandling.html#texture-as-input

I suppose you could also utilize G-buffers directly in your network if you could somehow handle unpacking the data as part of the network, too. But that's without me giving it too much thought.

 

It looks like the Book of the Dead example you linked is using the legacy render pipeline, but I am not particularly interested in using that render pipeline since it's going to be deprecated, except as a fallback (In fact, it's a bit odd to even bother showcasing neural style transfer with the legacy rendering pipeline in late 2020, given the messaging from the company; is SRP so difficult to use that even people internal to the company won't use it for machine learning?).

Since this is the barracuda forum, I also had a few additional questions:
- Is it possible to stream image buffer data from Unity to pytorch in during training time, rather than writing out images to disk and training without Unity running?
- Is it possible to run Unity headlessly (ie: on AWS) for training and/or data generation purposes.

 

@RogueStargun, if you are looking to run locally or with your own setup you can also take a look at these command line parameters:
https://docs.unity3d.com/Manual/EditorCommandLineArguments.html

Specifically `-batchmode` will be helpful.

Re: streaming image buffer data from Unity to pytorch you would probably need some sort of IPC or to create a custom plugin.

Or, as @alexandreribard_unity said you could try out the Unity Simulation product.