Lighting calculations in Unity: Normals in world space or tangent space?

So I have been learning about normal mapping in Unity using the tutorial here:
https://catlikecoding.com/unity/tutorials/rendering/part-6/

According to the tutorial, when working in tangent space, the basic steps for using normal data are as follows:
1. Read the tangent and normal vectors from the mesh. Convert them to world space.
2. Use the tangent and normal vectors to calculate binormals. (This can be done either in vertex shader or pixel shader, Unity prefers to do it in the vertex shader.)
3. Read the normal data from the normal maps.
4. Use the TBN vectors from step2 to transform normals in step3 into world space.
5. Proceed to use these transformed normals for lighting calculations in the fragment shader.
In code:

Code (csharp):

1. // in vertex shader convert normals and tangents from object to world space
2.     o.worldNormal = UnityObjectToWorldNormal(v.normal);
3.     o.tangent = float4(UnityObjectToWorldDir(v.tangent.xyz), v.tangent.w);
4.     .
5.     .
6.     .
7.     .
8.     //in fragment shader
9.     //read normals from the main normal map
10.     float3 mainNormal = UnpackScaleNormal(tex2D(_NormalMap, i.uv.xy), _BumpScale);
11.  
12.     //read normals from the detail normal map
13.     float3 detailNormal = UnpackScaleNormal(tex2D(_DetailNormal, i.uv.zw), _BumpScale);
14.  
15.     //combine those normals
16.     float3 tangentSpaceNormal = BlendNormals(mainNormal, detailNormal);
17.  
18.     //calulcate the binormals from the world space normals and tangents calculated earlier
19.     float3 binormal = cross(i.worldNormal, i.tangent.xyz) * (i.tangent.w * unity_WorldTransformParams.w);
20.  
21.     //use the TBN vectors to tranform the normals to world space
22.     i.worldNormal = normalize(tangentSpaceNormal.x * i.tangent +
23.                          tangentSpaceNormal.y * binormal +
24.                          tangentSpaceNormal.z * i.worldNormal);

So my question is why perform the lighting calculations with the normals in world space? Doesn't that negate the benefits of using the tangent space normal maps altogether? Shouldn't we transform our lights and camera to tangent space and perform lighting calculations there and then transform back to world space? I am a bit confused as to how to use and interpret normals in custom shaders so any help would be really appreciated. Thanks!!

 

Normals sampled from the normal map are in tangent space. This means a normal map only contains normals for a specific mesh orientation since its a 2D texture usually viewed from +Z direction. Thats why they are blueish. So if you would rotated your object the directly sampled normals would point into a wrong direction relative to the rotated object. Thats why you need some sort of transformation.

Now there are 2 major approaches. One is to sample your normal from the map, and transform it into world/view/whatever space your lighting calculations are already performed in. In your example it looks like world space. To do this you usually transform Tagent and Vertexnormal (not the sampled one) in vertex shader using the transpose(inverse(modelMatrix)), compute Bitangent and those 3 form the TBN matrix. This TBN matrix transforms from tangent to world space. Send TBN matrix to fragment/pixel shader. Then in the pixel/fragment shader sample the normal map and use TBN to bring the sampled normal in world space and do the lighting calculation. You can also compute the Bitangent and TBN in the fragment shader instead, just like in your code.

The other approach is to do the lighting calculation in tangent space. To do this you compute again Tangent and Normal using the transpose(inverse(modelMatrix)). This will bring them into world space again as in approach 1. Also compute Bitangent and TBN. Then still in the vertex shader invert( in this case = transpose) the TBN matrix . This results in a matrix that transforms from world space to tangent space. So it transform the other way round. With this matrix transform the lightpositions and viewpos into tangent space. Also in addition transform the vertexposition to tangent space, this is needed because you also need the (later) fragmentposition in tangentspace. Send the tangent space lightspos, viewpos and vertexpos to fragment shader. In the fragment shader you sample the normal from the map. It is already in tangent space so you can perform tangent space lighting computations with this sampled normal and the lightpos, viewpos and fragmentpos that you forwarded to the fragment shader.

So to sum this up. Both is possible. If you do the computations in tangent space you can already compute the needed vectors in the vertex shader and dont have to do a vector transformation in the fragment shader. So you basically have more work in your vertex shader and less in the fragment shader. But you may have to change your fragment shader a bit if you already did lighting computations in world space before adding normal map support.