Home Technical Talk

Overlaying / Mixing normal maps

polycounter lvl 8
Offline / Send Message
gannonroader polycounter lvl 8
Hey guys, I was just watching a tutorial for the udk material editor, and the guy mentions that when overlaying normal maps, you should hide the blue channel in photoshop, or delete it(I forget which he said). Did I hear this wrong? How do you go about doing that? Can anyone elaborate on this method? I've been having the problem with dull normal maps that just don't seem to pop out as much as others i've seen.

Replies

  • hijak
    Hey guys, I was just watching a tutorial for the udk material editor, and the guy mentions that when overlaying normal maps, you should hide the blue channel in photoshop, or delete it(I forget which he said). Did I hear this wrong? How do you go about doing that? Can anyone elaborate on this method? I've been having the problem with dull normal maps that just don't seem to pop out as much as others i've seen.



    there is many ways to mix normal maps and they can have different results. Its not always going to be optimal to remove the blue channel however, although it works fine if you are doing it to texture sampled normal maps, as apposed to those created via surface sampling. But for in photoshop layering one normal map over another works best like this.


    put first normal map on bottom second on top of course. add a levels layer adjustment to the top normal map only, hold alt and click the space in between the levels layer adjustment and the second normal map layer to make the layer adjustment effect only the second normal map. this will be confirmed by a little arrow, my wording is confusing but youll figure it out.

    okay now open the levels adjustment, and choose channels "blue"
    and at the bottom set output levels to 128. this will sort of clamp the blue range to negate its influence.
    now set the second normal map to hard mix type. And you should have perfectly layered normal map.

    you can always just do the levels adjustment the traditional way but using adjustment layers is non destructive so its a better idea.

    This works well as it does not require normalizing the normal map but you can mix it many other ways if you plan to normalize it after wards.

    This wont work well for mixing multiple maps generated from surface sampling though as ive noticed the blue channel does contain some information that texture sampled normal maps do not.


    in the udk editor tutorial he is just not connecting the blue channel to anything.
  • Eric Chadwick
    I haven't heard of people using Hard Mix, sounds interesting. You can see a few different methods compared against one another, here on the wiki...

    http://wiki.polycount.net/Normal_Map#BNMT
  • metalliandy
    Offline / Send Message
    metalliandy interpolator
    There are many way to do the same thing really.

    You can also use a curves adjustment layer by setting the output of the blue channel to 127 or double click the overlayed layer in Photoshop and under Advanced Blending, uncheck the Blue channel as it effectively does the same as the curves adjustment but its cleaner for organisation purposes because there are no adjustment layers clogging up your layers.

    The best way would be to overlay the maps in CrazyBump using the mixer, as the result is mathematically accurate and produces much better results than using Overlay or Soft/Hard Light, but the downside is that its permanent.

    Hope that helps :)
  • PredatorGSR
    Offline / Send Message
    PredatorGSR polycounter lvl 14
    As others have mentioned, I just take the blue channel of the top map, adjust the levels down to 128, and use the overlay blending mode.

    For the final map, its usually a good idea to normalize the map just to doublecheck that all the channels add up to 1. I usually use the xnormal tool to do that.

    That may not be why your normal maps don't pop though. It doesn't deepen the map, it just stops the blue channels from becoming shallow, your map needs to be "deep" to begin with. That is achieved by using a program like Crazybump or similar program that allows you a lot of control, as well as creating a good grayscale map to start with.
  • hijak
    yeah im not sure if i meant hard light or hard mix, its been a while since i had to do this, but as stated there are many methods. Some add normal maps, some just multiply them some mix them equally, it really depends what you need to do. And then there is the whole option of doing the mixing via a shader, which offers some really cool controls.
  • Andreas
    Offline / Send Message
    Andreas polycounter lvl 11
    Anyone got the image tutorial from Mathis' site? Mathis is not hosting it anymore.
  • Eric Chadwick
  • poopinmymouth
    Offline / Send Message
    poopinmymouth polycounter lvl 19
    My host contacted me and said I had 5,600 current connections at some points in the day on that image, so too many people were posting it on forums and stuff. I'm going to rename it and upload again, just haven't gotten around to it.
  • Wells
    Offline / Send Message
    Wells polycounter lvl 18
    you can also just double-click on a layer and uncheck the B channel. this is non-destructive so completely un-doable at any time. my preferred method.
  • Ace-Angel
    Offline / Send Message
    Ace-Angel polycounter lvl 12
    In UDK you say?

    It's actually pretty simple really, use the MaskComponent node and set it to RG mask, from where you take it and multiply with a Constant, to control how deep it you want.

    After that, marry it back to the blue channel of your Normal Map (with Add or Append, always forget) and viola, presto done.

    You can also use this method for detail normal maps when you have to marry it to a base normal map.
  • EarthQuake
    Just want to drop a note here and say, that for any baked content, this is a BAD IDEA. Messing with the contrast of a normal map like this will just cause smoothing errors and possibly other bad artifacts. Hard surface type work especially, where accurate normals are essential.

    For organics and characters, you might be able to get away with it, but I wouldn't recommend it. Adjust your highpoly content if you're not happy with the bake.

    If its a tiling texture or something, its probably fine.

    [edit] Oh wait, you guys are just talking about overlaying/mixing detail maps, not sure how that has anything to do with "deepening" ie: increasing the depth, of the normals. =P The above advice stands for anyone who is thinking about messing around with the contrast/curves of their baked normals to try and make it "pop".


    From an aesthetic point of view, Ben's tutorial thing does the opposite of "deepening", because he's overlaying a high frequency detail texture on top of it, the larger forms are lost, and all you've got are these high frequency bumps, effectively flattening the end result.

    I've updated the thread title to reflect the actual discussion.

    Gannon: To address your issue specifically, adding additional overlay/detail normal maps is likely a poor way to make your model pop. If your normal map isn't very interesting, its likely because of your source content, or your shader setup. So going back to the source(highpoly) and adjusting it to produce a better normal map, or working on your materials(diffuise/spec/gloss) to really make the asset shine is the best advice I can give you. Throwing an overlay on top to give something more "pop" doesn't really work, its like stacking bad on bad but hoping the end result will be good.

    You should post some images of what you've got(high, low, etc)
  • Stromberg90
    Offline / Send Message
    Stromberg90 polycounter lvl 11
    I do the same things as Sectaurs does, very fast and non-destructive.

    I do agree with EQ, this is no way to make a bad normal to look good, I think it's a matter of personal preference, I do it with low opacity just to get a bit more crispness.

    And also be careful what you overlay, not everything needs to be in the detail normal, do not go overboard.
  • poopinmymouth
    Offline / Send Message
    poopinmymouth polycounter lvl 19
    EarthQuake wrote: »

    From an aesthetic point of view, Ben's tutorial thing does the opposite of "deepening", because he's overlaying a high frequency detail texture on top of it, the larger forms are lost, and all you've got are these high frequency bumps, effectively flattening the end result.

    Visually it does exactly what it says, making the "pits" of a made-from-2d-source normal map-overlay have sharper values in the deep parts. Is it mathematically correct? who knows, but up rezing your high poly till you can stamp actual fabric fibers into it is dumb as shit and wasteful of time.

    I very rarely take my high poly sculpts to a high level. I work them fast to have the large forms, and add all fine surface detail using overlays in photoshop. Not only does this allow me to get the density perfect for the texture resolution so the texture map size can actually hold the detail, but it makes it a lot easier to make sure these details work in the specular and diffuse layer since I have the pixel perfect source in B&W to influence them as well.

    Like is your advice to take every high poly up to the 8 billion mark so you can put in skin pores, fabric fibers, metal scratches, etc? I know that's epic's method, but they can afford to spend 2.5 man months per character, and not every studio can. Very many artist combine 2d normal map source with baked from 3d source, and this is one way to combine them, though some of the new bakers like Xnormal and crazybump have more mathematically correct ways of combining them.

    You could argue that even in old school textures, adding an overlay would make large forms disappear. It's all about balance and making sure the large forms still read from afar, with smaller details to read when up close. (which my example does just fine)
  • EarthQuake
    You could argue that even in old school textures, adding an overlay would make large forms disappear. It's all about balance and making sure the large forms still read from afar, with smaller details to read when up close. (which my example does just fine)

    Right, its basic art theory. Adding noisy overlay textures is going to do the opposite of making your forms pop. I'm not sure what the rest of your rant is about here, as I never suggested he do anything that you're talking about. I certainly do not agree that your example shows balance however, its a classic example of noise overload killing forms.

    All I said was that if the modeled lacked pop, it was likely because of the source content, or the way the materials were being handled.

    For instance, if the edges on your highpoly are too sharp and they barely show up in your normal bake, this would make your result "pop" a lot less than it should, to fix it you would go back to your high and exaggerate your edges and bevels. "Pop" is all about working your low and mid frequency details.

    For materials, if your reflective areas were weak, you play with your specular, gloss, possibly environment reflection stuff etc.

    Just to be clear, I do a 2d bump converted normals pass for virtually all of my assets and virtually never go nuts sculpting fine details, but this is to increase micro-contrast when viewed at close distances, not to make my model pop or increase the depth of my end result, which I believe is what the OP is after. Hopefully he can post some images of what he has and talk about the goal he's trying to achieve, I'm sure that will clear it up.
  • Shogun3d
    Offline / Send Message
    Shogun3d polycounter lvl 12
    Speed vs Quality. I don't bother creating high poly details for anything I can simply generate from heightmap to normal if im working in a tight time budget. On the flip side, it is higher quality to bake down from a high res piece (given a good accurate bake). If im looking at something with a lot of detail, I prefer to merge in a set of premade high stuff, like a set of bolts, nuts, rivets, generic shapes, adjustable things. Overlaying is perfectly fine, but most people don't normalize and forget to. It's essential to avoid artifacts.
  • poopinmymouth
    Offline / Send Message
    poopinmymouth polycounter lvl 19
    EarthQuake wrote: »
    Right, its basic art theory. Adding noisy overlay textures is going to do the opposite of making your forms pop. I'm not sure what the rest of your rant is about here, as I never suggested he do anything that you're talking about. I certainly do not agree that your example shows balance however, its a classic example of noise overload killing forms.

    It's almost like as an example I amped up the end result so it would be noticeable. Plus this isn't a "critique poop's texture" thread, it's asking about normal maps and how to deepen them (or just not destroy existing volume). Using a 2 layer normal combination method is what my tutorial shows, and it's up to the user to pick the right settings for their artwork and style. I'm sure if you were to bother providing a tutorial on this subject we could compare and contrast our techniques.
  • poopinmymouth
    Offline / Send Message
    poopinmymouth polycounter lvl 19
    EarthQuake wrote: »
    which I believe is what the OP is after. Hopefully he can post some images of what he has and talk about the goal he's trying to achieve, I'm sure that will clear it up.

    It's really not that cryptic to understand what he's asking. If you try to combine a normal-from-2d with a normal-bake-from-3d layer without either hiding the blue channel, or changing it to 128 grey on the 2D-normal layer, it will flatten the blue from your baked layer, resulting in a less volumetric rendering of the original baked normal. This is basic 101 level stuff and was easily comprehended from the OP.
  • EarthQuake
    It's really not that cryptic to understand what he's asking. If you try to combine a normal-from-2d with a normal-bake-from-3d layer without either hiding the blue channel, or changing it to 128 grey on the 2D-normal layer, it will flatten the blue from your baked layer, resulting in a less volumetric rendering of the original baked normal. This is basic 101 level stuff and was easily comprehended from the OP.

    Sure, if thats all he's after than its very simple, and everything I've written is pretty much moot to his problems. He mentioned that his normal's do not seem to pop as much as what he expects, and there could be a variety of causes for this however.

    I just do the basic levels clamp in the blue channel, I've seen some comparisons with crazybump and other 3rd party tools or other methods that can do it more accurately, but I've never had the need. For this sort of stuff I'm generally doing fairly abstracted work, like dents and scratches, so geometric accuracy isn't particularly important.
  • malcolm
    Offline / Send Message
    malcolm polycount sponsor
    kaburan, I did a lot of tests in our last engine and found normalizing the map in Photoshop made no difference in game. Surprised by this because everyone on polycount harps on this issue. I consulted our rendering engineer and he said he was normalizing the maps in the shader. Saves time and I don't think it cost anything either.

    When I did that eat3d old piller mudbox tutorial I followed along and tried to paint the fine detail in mudbox, total mistake. It would have looked much better and been easier to do with an overlay half blue channel trick. Would definitely never try painting high frequency detail into a sculpt again. Waste of time that will not be appreciated by the end user, especially if the textures are scaled down once they make it into game.
  • EarthQuake
    malcolm wrote: »
    I consulted our rendering engineer and he said he was normalizing the maps in the shader. Saves time and I don't think it cost anything either.

    Yeah, this is often but not always the case. In real use, unless you're really doing weird stuff it likely isn't necessary.
  • Eric Chadwick
    At work, our game shaders don't normalize. Allows us to do some cool tricks, like reverse the normals. Rarely used, but still it's important to have when we do need it.
  • throttlekitty
    Reversed normals should still work even if the shader is normalizing, right? So long as the product equals one. It's not much different than making the geometry normals point backwards.
  • Eric Chadwick
    Ah yes, you're right. Sometimes I want to shorten the normals, to reduce light contribution. This is removed when normalizing.
  • felipefrango
    Offline / Send Message
    felipefrango polycounter lvl 9
    Ah yes, you're right. Sometimes I want to shorten the normals, to reduce light contribution. This is removed when normalizing.

    How would this be achieved exactly? I presume painting black or just pasting an AO bake into the blue channel for example?
  • Eric Chadwick
    Gray is flat, iirc. So I used inverted AO as a mask for 50% gray, on all three channels. This made both ambient and diffuse fade out in the cracks. Been a while since I did this. I think it was only a benefit because the diffuse was tiled, so I couldn't just darken that.
  • felipefrango
    Offline / Send Message
    felipefrango polycounter lvl 9
    Interesting, thanks a lot Eric!
  • PhilipK
    Offline / Send Message
    PhilipK polycounter lvl 10
    I'm usually using Linear Light with a fill on 50%, then add a color overlay to it with Linear Burn with a color of 255, 255, 127. Also don't forget to check the "Blend Interior Effects as Group" checkbox in the general layer styles window. This doesn't work EXACTLY accurate, but it does a good job at keeping the blue channel very close IMO.

    I do have it as an action tho, if you wanna try it out it's part of my small old action package:
    http://philipk.net/tutorials/materials/PTActions.atn

    Thanks goes out to Teddy for this one too :)
  • RahiSan
    Hey guys!

    I am reviving this thread for a question, I hope this is okay.

    Here is my situation: We have a vehicle that is fully textured and normal mapped.I have created some projectile impacts in zbrush and exported normal and height maps. We are going to map those damage decals in realtime onto the vehicle in a game engine. So far so good.

    The problem is: How to deal with overlapping damage decals in the normal map? I did some testing in PS and marmoset and so far the darken blending mode seems to work best for the heights, but how would you do it for the normals? Any suggestions?

    Thank you.
  • Farfarer
    Don't use any of the methods suggested here, they're all going to give you incorrect results.

    Use this instead - it uses Reoriented Normal Mapping (which is a superior method to anything outlined in this thread) - to combine up to 9 normal maps together;
    http://vincentcallebaut.com/CombineNormal.html
  • cman2k
    Offline / Send Message
    cman2k polycounter lvl 17
    Hey hey hey, look at that. I'd never seen this tool (or method), but I've always called BS on overlaying being great for quality. Thanks for the link man!
  • EarthQuake
    Farfarer wrote: »
    Don't use any of the methods suggested here, they're all going to give you incorrect results.

    Use this instead - it uses Reoriented Normal Mapping (which is a superior method to anything outlined in this thread) - to combine up to 9 normal maps together;
    http://vincentcallebaut.com/CombineNormal.html

    I think it really depends on what you're doing, if you're overlaying complex geometric shapes over other complex geometric shapes I would tend to agree.

    However, if you're doing basic stuff like paint scuffs on a mostly flat plane, I wouldn't bother with extra plugins or whatever. Basic methods will work fine in this case and you won't be able to tell the difference, especially in game with texture compression and everything else.

    Personally, if I'm dealing with complex geometric shapes, I generally tend to do the work the high. Though there are some situations like complex patterns or ornate detailing that would see a benefit from this.
  • Farfarer
    I guess?

    But that's kind of like saying that having synced tangent bases isn't that big a deal if the baker and renderer are pretty much the same, because you'd never notice with texture compression and everything else.

    This plugin gives mathematically correct results and it's just as complicated (if not less so) as most of the methods outlined here. Seems kind of a moot point.
  • EarthQuake
    Farfarer wrote: »
    I guess?

    But that's kind of like saying that having synced tangent bases isn't that big a deal if the baker and renderer are pretty much the same, because you'd never notice with texture compression and everything else.

    This plugin gives mathematically correct results and it's just as complicated (if not less so) as most of the methods outlined here. Seems kind of a moot point.

    Haha well, if your baker and renderer are pretty much the same, and the difference between synced and pretty much the same is too small to actually notice in the final product, why would you put in extra effort? I've always been a huge proponent of using a synced pipeline, because it generally makes a huge difference in the quality of normals.

    I'm not a fan of doing something "correctly" just because, when it offers me little/no benefit in use. This applies to most aspects of content creation, if I can fake a shape with a floater instead of modeling in I will do it. If I can generate a normal map from a photo instead of painstakingly sculpting it in 3d I will do it. It all comes down to quality vs efficiency.

    I explained some cases where it likely would, and likely wouldn't make a difference above, though it is possible that I am simply incorrect. If you feel very strongly about it, feel free to post a real-world textured asset, in game, to show what sort of difference it makes.

    I've played with this script and personally I find it annoying to use, every time I make changes I need to run it again, with a simply layer setup in photoshop I can paint masks and get updates in realtime, adjust layer opacity in real time, etc etc all without needing to re-run a script that gives me a destructive baked down result. So unless I'm missing something entirely about how to use the tool, I fail to see how anything about it is easier. It might make sense to have a simple photoshop layer setup for quick edits, and then run something like this when you're basically done with the texture.

    There is also a very subjective question here, even if the results that this tool make are undeniably better, is a basic photoshop setup good enough? Personally I've never had a change request because my detail normal map wasn't overlayed well enough. But again, I tend to do detailed geometric shapes in 3d most of the time, as I often find it faster to do in 3d than 2d. If you do a lot of heavy lifting in 2d, I can see how tools like this would be more important.

    Edit: Sorry if this comes off as snarky. I know your intentions are good here, it just rubs me the wrong way a bit when someone pops in and says X is the only way to do things. In most cases, there are a variety of ways to do any particular task, and the best one will depend on the specific task, requirements, personal preferences, etc.
  • Farfarer
    No, I get it. I probably shouldn't have been so absolute about it.

    I know of all the folk on here you're one of the biggest proponents of working smartly and properly.

    Like all the workarounds you mentioned for other aspects of the art process, this is another of those things where you have to learn when it's acceptable to take the faster route or when you need to take the proper route.

    Just wanted to make a point that while the methods here might pass muster, there are fairly simple ways to do get the job done properly. Recently I've been seeing a lot of the faster route stuff being done when it really shouldn't - I'm maybe a bit touchy and quick to react on the subject at the moment :P
  • EarthQuake
    Yeah totally, and I agree you should always know the correct way to do something before deciding to do the fast/sloppy thing. I think a lot of people lack that primary knowledge, which probably contributes to what you're seeing.
  • malcolm
    Offline / Send Message
    malcolm polycount sponsor
    I need a normal map rotator, does anyone have that in Photoshop yet. Ryan from Crazybump had a stand alone app to rotate maps, but it would always run out of ram and not work, plus it was a bit of a pain to put the map into another app just to rotate it.

    And yes I know nDo will rotate maps, but you have to create them in that app first or it doesn't work to my knowledge.
  • cptSwing
  • haiddasalami
    Offline / Send Message
    haiddasalami polycounter lvl 14
    Just in regards to Re-Oriented Normal Mapping, heres a webgl demo that potrays the various methods (and source) so you can mimic in Material Editor etc. Definitely helps for detail Normals

    http://blog.selfshadow.com/sandbox/normals.html
  • malcolm
    Offline / Send Message
    malcolm polycount sponsor
    I require an arbitrary rotation value, rather than a flipper or 90 degree rotater.
Sign In or Register to comment.