A solution to Normal Maps with mirrored UV's in UDK.
polycounter lvl 11
So I don't know if this is common knowledge or very obvious, but I decided to share since I didn't find this solution through google.
Last night I made a simple practice asset and ran into a snag when I imported it into UDK.
It ended up looking like this:
Ugh, awful.
My problem is that my normal map is mirrored, which means that half of the UV shells are flipped which cause problems with how the normals are calculated in the engine.
Look at all that red. That's a bad sign all right.
So after some research I came up with this solution.
First of all you need to fix your UV's. UDK hates mirrored UV's so I flipped the red shells so they are blue and nicely correspond to their geometry again. The problem of course is that they no longer line up with the texture any more, but there's a solution to it.
In UDK you can change the way that textures tile in the Texture Properties. The default is Wrap which is the mode we're all used to, but there's also Mirror and Clamp. Mirror is the one we're interested in because it does this:
Excellent! This means that if we lay out or UVs this way they will match the mirroring:
But its not that simple, because when you mirror the normal map horizontally you end up flipping the X axis (red channel) which causes mirrored lighting. But with a little bit of material editor magic we can fix that.
We can flip the red channel by multiplying the normal map with a float3 with the values -1,1,1, but we want that to only apply to the mirrored UVs. This is also fairly simple. All we have to do is to lerp the two normal maps with a checker pattern with its UV tiling set to 0.5. This will make it match up to the alternating mirroring and mask out the x-flipped normals from where we dont need them.
And the results?
Perfectly mirrored normals! No seams, huzzah!
Hope this is helpful to yall.
Last night I made a simple practice asset and ran into a snag when I imported it into UDK.
It ended up looking like this:
Ugh, awful.
My problem is that my normal map is mirrored, which means that half of the UV shells are flipped which cause problems with how the normals are calculated in the engine.
Look at all that red. That's a bad sign all right.
So after some research I came up with this solution.
First of all you need to fix your UV's. UDK hates mirrored UV's so I flipped the red shells so they are blue and nicely correspond to their geometry again. The problem of course is that they no longer line up with the texture any more, but there's a solution to it.
In UDK you can change the way that textures tile in the Texture Properties. The default is Wrap which is the mode we're all used to, but there's also Mirror and Clamp. Mirror is the one we're interested in because it does this:
Excellent! This means that if we lay out or UVs this way they will match the mirroring:
But its not that simple, because when you mirror the normal map horizontally you end up flipping the X axis (red channel) which causes mirrored lighting. But with a little bit of material editor magic we can fix that.
We can flip the red channel by multiplying the normal map with a float3 with the values -1,1,1, but we want that to only apply to the mirrored UVs. This is also fairly simple. All we have to do is to lerp the two normal maps with a checker pattern with its UV tiling set to 0.5. This will make it match up to the alternating mirroring and mask out the x-flipped normals from where we dont need them.
And the results?
Perfectly mirrored normals! No seams, huzzah!
Hope this is helpful to yall.
Replies
Cool stuff!
And I just realized I should have posted this in the UDK subsection. Oops!
Just FYI, the issue is with the FBX file format not containing the tangents and binormals, it only export the normals. If you export to FBX ASCII you can look in the file and see they are not present. When you import the model, Unreal looks for the tangents and binormals but uses 0 when they are not found.
Another way to fix it is to uncheck "Import Tangents" and check "Explicit Normals" But if you have custom/edited normals, they will be lost.
The Epic guys gave us a code fix that always calculates tangents from imported normals, circumventing the FBX bug. Unfortunately, I can't share it. Hopefully it makes it into a release soon.
I noticed a new UDK is out today. Have you tried that?
@monster, very happy i read that last line, lol...
I haven't checked out the new version yet. We'll see if anything is different there.
I think the imported tangents should solve the mirror seam (if Unreal imported them properly!). When the tangents are created, as long as they correctly handle your UVs (rotated, mirrored, etc.) then this decides which direction each pixel-normal will be rotated. That's the point of the tangents.
Baking flipped UVs is a different issue, if I understand you correctly. You should always move mirrored UVs out of the 0-1 UV rectangle before baking. Only the non-overlapping front-facing UVs should be baked, otherwise you get errors in the map. Moving the overlaps/mirrors 1 unit over is a good choice, and they can remain there after the bake.
I think your shader trick should be discarded if Unreal imported tangents properly, since that would give you accurate results in more situations, and use less shader instructions. But it's still a cool trick nonetheless!
So what is the relation of UV's to the tangents? Because my impression was that having a flipped UV shell would be at odds with the normal vector rather than the tangent because when you look at the lighting error it's only the vertical axis that is flipped.
Besides, if you turn off the option to detect the UV Winding Order in Maya the same error appears, which suggests that it's not an issue with the import as much as an inherent problem with UVs and normal maps.
Reversing the normals on the effected parts fixes the issue in Maya. But if I were to bring that into an engine I would need a double sided material.
Also, when I import the model into Unity and make sure to import the tangents the same issue persists.
Yeah, IIRC the tangent basis is made up of three vectors... normal, tangent, and bi-tangent (aka bi-normal). The tb normal is not the same as the vertex normal.
Good trick if you need it though!
If you place your flipped UV's to the right of 0-1 space for example:
http://i.imgur.com/JwPce5R.jpg
For other UV layout situations one could consult UDN's page on material masks:
http://udn.epicgames.com/Three/MaterialMasks.html
http://www.polycount.com/forum/showthread.php?t=108243
The Beta of February 2013 doesn't fix it, I already tried.
If they are not at the top of the stack, even if you check Tangents and Binormals in the FBX Export window, they will not be exported.
Any care to verify if Unreal or Unity imports them properly with this method?
1.No Unwrap mod, Tangents checked on export = Not working.
2.Unwrap mod on top, Tangents checked on export = working.
3.Unwrap mod on top, Tangents Unchecked on export = working!??
And it doesn't matter at all if i check the tangent while importing in UDK or not, the only thing that seems to matter is the presence of the modifier on top...
Nice find monster.
Max 2013, UDK Nov-2012...
It doesn't happen for me in max 2010 though.
I also experimented a bit with different import/export settings. Either UDK or the FBX plugin insists on flipping the binormal (blue) on the parts with mirrored UV's. The image on the left is with Import Tangents turned off, and the one on the right is with the setting turned on. Both meshes have overlapping UV's packed into 0-1 and a material where the normal map goes straight into it. No trace of my workaround.
So this seems to be an easier fix. But what do I lose by not importing the tangents? Leaving Explicit Normals on seems to import my hard/soft edges correctly so the normal map shades without error.
Import tangents -> imports the tangents defining the red/green channels of the normal map as well. If false, UDK does this itself, which again may be different to how Max or Maya did it.
Not importing tangents will be fine for most meshes, and you only need worry if you're doing some heavily distorted stuff with crazy UVs and unsplit edges. I've experimented with using this to bake normals on objects with no smoothing group splits at all, but in most sensible cases that will look much better due to less normal map bending letting UDK fill in those values is fine.
I wish it remember the last used settings for that particular asset. Maybe it's supposed to, but it doesn't.
this thread is awesome, btw
Just do like you are importing something completely new, select your mesh and name it the same as the one you want replaced, and all instances will be updated. As mentioned above, make sure you have the same group name too.
Oh, youll have to rebuild lighting/paths/etc and all that jazz, of course.
I think I might need to preform Fingus' workaround to make this work for Skeletal Meshes.
-slowly backs out of the thread
I try`d to invert the mirrored UVS or normal UVS and nothing, the mirrored uv in max is outside 1-0 uvspace..
http://www.polycount.com/forum/showthread.php?t=124923
all maps were 2k but the one that was working was about 600kb, while others were 7-8mb... can someone tell me what i did i do? :-) WHAAAAAT
EDIT: it worked without any nodes, just texture applied