The Wonders of Normal Map Baking
polycounter lvl 8
So I've done a bit of testing with normal maps in Unreal to see how optimal I could make my results, after testing different things for years. I've known about smoothing groups causing issues for normal maps for years. But only recently learned about how different engines use different tangent spaces and not all normal maps are created equal when going from 3D Package to Engine.
I know this has been talked to death on the polycount forums, but I thought I'd share my own personal results from experimenting. Note, I probably could have done more tests to control for more (triangulation, smoothing groups) but I was just going for the best results at the time.
Keeping your entire low poly mesh smooth and just baking out from a 3D package or xNormal wont always work. In my mind, this is the best workflow:
In my mind, these are the best practices for generating good normal maps (no matter the engine- works well in Unity too).
If you've baked out tangent-spaaace normals from xNormal and have been using those, don't worry! You can still create corrected normal maps out of those by loading up xNormal, and using the Object/Tangent spaaace converter tool to make an object-spaaace normal map out of your tangent spaaace map, and running that through HandPlane3D with the same steps mentioned above.
I'll probably add more clarification/pictures a bit later. I'm hoping this helps somebody- I know its not a fun process to work through. Any questions, let me know.
I know this has been talked to death on the polycount forums, but I thought I'd share my own personal results from experimenting. Note, I probably could have done more tests to control for more (triangulation, smoothing groups) but I was just going for the best results at the time.
Keeping your entire low poly mesh smooth and just baking out from a 3D package or xNormal wont always work. In my mind, this is the best workflow:
- Triangulate your mesh by hand. This removes any discrepancy between how different exporters/importers triangulate models.
- Crease the concave edges of the mesh (the inwards-facing corners)
- Finally, export the meshes. Your high poly, your low poly, and your game mesh. When exporting, make sure smoothing groups, smooth mesh, and export normals are on. Do not export bitangents/normals. Export out a bake cage if you want that level of control of your bake- I usually let xNormal do this for me and fix any problems on the normal map by hand.
- Open xNormal. Import the low and high poly models. Scale by 10. Use the raytrace calculator to make an approximate cage- or import in your own cage. Bake out an object-spaaace normal map.
- Open up Handplane 3D, target your low poly mesh, the baked out object-spaaace map, and specify the engine you will ultimately be using. Bake.
- Take this resulting normal map and bring it into your engine. Also import your mesh. In the case of Unreal, if its a static mesh, turn on Explicit Normals, so that they are read from the mesh.
In my mind, these are the best practices for generating good normal maps (no matter the engine- works well in Unity too).
If you've baked out tangent-spaaace normals from xNormal and have been using those, don't worry! You can still create corrected normal maps out of those by loading up xNormal, and using the Object/Tangent spaaace converter tool to make an object-spaaace normal map out of your tangent spaaace map, and running that through HandPlane3D with the same steps mentioned above.
I'll probably add more clarification/pictures a bit later. I'm hoping this helps somebody- I know its not a fun process to work through. Any questions, let me know.
Replies
The first mesh is the same as yours, and its smoothed. Looks really bad, for sure. The second if the same mesh as your, just with a chamfer. The smoothed shading is better, but still far from perfect. The third mesh is the same as the previous, but with custom normals, and looks perfect to me even without a normalmap. And I'm not sure about adding a baked normalmap to this mesh would make it any better.
Edit: Here is the same example with the worst case mesh:
Hope this helps!
Obscura, it's not about the shape, it's about the workflow for getting the normal to read correctly in a game engine. To be fair, I do agree beveling edges can help a lot with normal map baking too.
I wish lux would add the mikktspace normals as an option as it's not rocket science ! They are looking into it however :poly121:
What I'm interested in is better workflows for baking and maintaining normal maps. But if normal maps look correct straight out of xN for UE4 use, then I can take a huge sigh of relief for sure.
read this:
http://udn.epicgames.com/Three/XNormalWorkflow.html
http://www.polycount.com/forum/showthread.php?t=102838
You shouldn't need to use handplane for any game engine now. Most are compatible with xnormal in some capacity.
How did I not see this before?! Haha- thanks!
Seems like I'm not the only one with this problem either...
https://answers.unrealengine.com/questions/14375/normal-map-workflow-with-xnormal-is-not-working.html
My workflow using Handplane doesn't work with UE4 either. I really think Epic ought to look into implementing derivative maps- because having so many things factor into how light is bent around a surface is crazy.
I've actually found Obsura's method to work the best for making modular pieces, since it saves me time and saves on resources (you need a lot less geometry to "control a bake"), with none of the weird lighting artifacts.
This all being said, I've definitely seen some games out there use normal map baking for their modular building pieces. There has to be a de facto method out there for producing the perfect normal maps, even for tileable pieces.
I have no problems with using xnormal and baking my normal maps in UE4 using the same techniques outlined in the documentation I provided. I use Maya LT, xnormal, and zbrush.
I was having a problem with importing any normals from Maya LT because the Maya LT FBX exporter was too new for UE4. However since version 4.3.1 this has been fixed.
Perhaps you could post pictures of the problems you're having? It may have nothing to do with the normalmaps.