Dynamically creating textures/materials from 2D images

Hello all,

I'm researching making an exhibit installation like the ones below. The feature they all have in common is that users take some form of 2D template, color into it, and then an action takes that image (which correct me if I'm wrong unity calls a "texture") and dynamically shades a 3D object with it (perhaps turning it into a unity "material"). As you can see from the links sometimes the template is a piece of paper with fiducial markers or sometimes it's a digital template on a touchscreen or tablet. I know that the Sketch Town install uses Unity and openCV.

https://www.breezecreativeproducts.com/draw-alive
https://futurepark.teamlab.art/en/playinstallations/sketch_town
http://www.quivervision.com/

I'm fairly experienced in Processing and openFrameworks, but have only worked on a handful of Unity projects. Can someone point me in the right direction of what techniques / terms / libraries / articles that I should look into for a project like this?

I guess the easiest example would be to have a 2D square and a 3D cube. How to take a 2D image available in a folder and dynamically create a texture/material from it through scripting not the unity editor.

Let me know if I can explain myself a little better, thanks in advance for anyone's help! Also my first time posting to this forum, so let me know if this is in the wrong format or if I should be asking for help in a different way.

Cheers,
Tom

 

Thanks Tom for sharing your problem and welcome. I am sorry to say that I cant get your point about actually what problem you are facing and the links you provide not help me anymore. Please try to explain your problem in clear words so i can help you.
sincere, custom fabrication NY

 

for the DrawAlive,
i'd guess it detects which drawing it is (from those qr codes),
then possibly crops the image in those black edges, then applies it as a 3d model texture.
(and that model is UV mapped so that the drawing shape matches it)

you can set texture to model,
https://docs.unity3d.com/ScriptReference/Material-mainTexture.html

to scan drawing image you would probably use webcam or so?
can get texture from webcam with https://docs.unity3d.com/ScriptReference/WebCamTexture.html

or if you load image from folder (on desktop)
http://answers.unity.com/answers/802424/view.html

 

This sounds like something more advanced than I've done before, but I can give you some ideas here:

1) To import an image at runtime

This method is supposed to work:
https://docs.unity3d.com/ScriptReference/ImageConversion.LoadImage.html
The example code in the docs assume that you have already loaded your file into a byte array, but if you google search for the method, you will probably find some other examples.

2) create a material from the texture:

I think the easiest way to do this is to already have some default material on your 3D object and modify just the properties of that material at runtime. You can set the texture maps using this:
https://docs.unity3d.com/ScriptReference/Material.SetTexture.html
The actual names of the Textures depend on the shader that your material uses. I think this example assumes that you are using the Unity Standard shader.