Trying find a solution to draw on seams with Ink Painter(Free Real-Time Texture Painting Asset)

Hello everyone I'm working on VR Graffiti game and all good in development unless problem with painting on UV seams on objects. I know that when I paint on something - it's find position in UV coord. to draw there some brush or texture what I am use for painting. So it paints only on one position near the seam but other side of this UV island have different position and to draw nice on seams, I think, We need to put the same brush texture on this other seam to paint perfectly. I think that it some sort of triplanar shader or something like that. For example, Blender Texture Painting have this feature to paint on model seamlessly. Maybe someone can advice me or give some link to read it. Very nice if you can help me with code - anyway Thanks) I Can attach some screens of this problem if someone don't understand about what I am talking Sorry for my bad English :/

 

Painting on an object with nice, uniform, and mostly flat UVs, is pretty easy. You find the position of the UV at the position on the mesh, and you draw your shape aligned to the UVs.

The reason why this is so easy is because the UV space and the world / model space line up so well. When you get to more complex models with curves and seams this gets much harder and you have to make some hard choices.

The short version is you can no longer just take the UV from the position you hit, but you need to find the UV position for every position in your brush's range. You could certainly do a lot more raycasts against the mesh, but this will be super slow. Instead you'll need to think of this more like a light or projector. You're painting at a position on a mesh in object space, and you'll need to map object space to UV space for every pixel of your texture. The way to get this is to render your mesh using it's UVs rather than it's position. Assuming the UVs are unique and within a 0.0 to 1.0 range, then it's just a matter of rendering the mesh to a render texture with a vertex shader like this:

Code (csharp):

1. struct v2f {
2.   float4 pos : SV_Position;
3.   float3 objectPos : TEXCOORD0;
4. };
5.  
6. v2f vert(appdata_full v)
7. {
8.   v2f o;
9.   o.pos = float4(v.uv.xy * 2.0 - 1.0, 0.5, 1.0);
10.   o.objectPos = v.vertex.xyz;
11.   return o;
12. }

Then you can pass in an object space position to test against with a 3d distance in the fragment shader. If you want to apply a texture, then you'll likely want to pass in a normal, or some pre-calculated projection matrix. You may also want to get the vertex normals to prevent back face painting, and maybe even render out a depth map using the brush's projection to do occlusions ala shadow mapping.

And that's just the basics. High end tools can handle things like overlapping UVs or UVs outside of the 0 to 1 range, or do surface distance tests instead of just normal and depth tests. Usually this involves generating unique UVs, or voxelizing the mesh and painting on that, then baking back down to the original UVs.

 

Thanks - I am gonna try this) I will write here if I have succes on it)

 

Code (CSharp):

1.  
2. float2 CalcBrushUV(float2 mainUV, float2 paintUV, float brushScale, float deg) {
3. #if UNITY_UV_STARTS_AT_TOP
4.     return Rotate((mainUV - paintUV) / brushScale, -deg) * 0.5 + 0.5;
5. #else
6.     return Rotate((paintUV - mainUV) / brushScale, deg) * 0.5 + 0.5;
7. #endif
8. }
9.  

In this asset uv's calculated in this way. But I think that is only places brush texture in one position on ONE uv island and I need to place exactly in same position at 3D world on mesh this brush but in 2D UV coordinates this 2 or more uv islands can be in different places and this is main problem I want to solve.

 

Throw all this code away and start from scratch. None of it is going to be useful anymore.

Once you get beyond a single UV island you can't think about painting in UV space anymore, you have to think about painting in 3D space. UV space is just how the results of the 3D space painting is stored, but you can no longer paint directly to UV space.

If you have a position in UV space and need to find an adjoining position in another UV island, or even along a seam, you have to convert back to 3D model space and find positions within a 3D space volume and map those back to UV space. If you have to convert back to 3D space, why not just start in 3D space? Also once in 3D space, a UV space "radius" no longer has any real meaning as UV space is not necessarily consistent in 3D space. It gets really complicated trying to stay in UV space. There's been 30+ years of research on this kind of thing, and the answer so far has been "don't do that". If you want to paint on an object, paint on the object in 3D space.

 

I do not have much time to do it from scratch unfortunately... I need to complete game, just a demo at december 25-28
I trying understand what you say about 3D and UV spaces Maybe I can ask you how you think, what method uses 3D programs like Blender? I want to draw in 3D space - but it always related with UV space cause of 3D model texture is 2D space. For now in this assets: We Have 3D model then click on model that have MeshCollider. If Raycast give us true value then we take hitInfo and using hitInfo.textureCoord pass to shader in which very bad but not at all)
I want to do reasearh of this topic by my own but only then I have more time for this((

 

https://mega.nz/#!OvYVFK5A!4jue_-zrCYPoyWuxlCpMQUxn282EJxeKll2CUCcXFws - here's gameplay video, when painting on profiled sheets and doors :/

 

Blender does a screen space projection onto the mesh, then remaps that into UV space. Basically what I described in my first post. It has options for occlusion (ie: depth / shadow map), culling (ignore backfaces based on projection), and normal rejection (fade based on projection direction & surface normal).

Think of UV space as just for storage, not the starting position. There is no solution for bridging UV island gaps that does not involve moving everything to 3D space, because the UV islands are only connected in 3D space.

 

I am want to use screen space projection... and also have such feature like stencils - for example, we can place hand between wall and spray and draw only not behind hand. It this case we have huge amount stencils and good game already) But I work alone and don't have huge knowlege of ShaderLab Cg language, but I know very well C# and Unity stuff :/ What in this case don't help me so big)

 

When you talk about screen space projection - are you mean depth texture and other stuff? So we get depth texture and use it for painting? I have another video of some guy who have this method - but i cant implement this in my game(
Can you watch it?

It's very cool method - but I dont have idea how make it. Oh, also this guy provided source code - but not all

 

Use the raycast to find a world position, pass that and the ray direction to the shader. Maybe construct a projection matrix and use that for sampling the texture, like a projector does.

Yep, that's using a depth texture from the point of "view" of the airbrush to mask where the paint sprays. You can look at that project's code since the author put it up on Github:
https://github.com/sugi-cho/ProjectionSpray

The quick rundown of what that does is this:
Save out a texture with the model's object space normals and positions baked into it:
https://github.com/sugi-cho/Project...ts/ProjectionSpray/Scripts/MeshInfoTexture.cs
https://github.com/sugi-cho/Project...jectionSpray/Shaders/Mesh-Info-Texture.shader

Render out a depth texture from the view of the airbrush using a replacement shader:
https://github.com/sugi-cho/Project...ectionSpray/Scripts/Drawer/ProjectionSpray.cs
https://github.com/sugi-cho/Project.../ProjectionSpray/Shaders/DepthRenderer.shader

Spray onto a mesh by drawing to the texture using the object's position and normals texture, along with the airbrush's projection matrix to find where the two overlap. Test against the depth and normal and reject if needed:
https://github.com/sugi-cho/Project...ectionSpray/Scripts/Drawer/ProjectionSpray.cs
https://github.com/sugi-cho/Project...rojectionSpray/Shaders/ProjectionSpray.shader

At the end you have a texture with the spray saved to it. The author has another version which doesn't bother with that first part and just renders the mesh out unwrapped rather than reading from a texture.
https://github.com/sugi-cho/Unity-ProjectionSpray-v2

 

I'm just use his scripts and use standard shader instead of unlit, but if don't use option fill crack - in texture apperas black seams, i'm put his material on this option and it seem work) In future I will understand this - but for now - thanks you for your answers Good luck!