Home Technical Talk

Why does UDK suck so bad at NMs

1
Why?! In 3 pointshader it always displays mirrored normals, flipped, rotated UV clusters anyway I please:: Correctly. It seems like this is how game engines should be! Just do it like you normally would and not have to go through a million "tricks" to try and get it to show up magically correct in udk.

I know u'll say its because the tangent basis of the baker and udk not matching up. Well damn I guess NOTHING matches up with udk.. because no matter what I bake in it will always have some sort of issue with the NM, ALWays,

This is 2012 not 1920s lol, why doesnt udks normal map display be as good as a real time viewer like 3 point, or Xoliul?

Replies

  • Joshua Stubbles
    Options
    Offline / Send Message
    Joshua Stubbles polycounter lvl 19
    I don't have issues getting my normals to look good in UDK (aside from crap mipping). I bake in Max, xNormal, use NDO, mirror normals - everything. Not sure what's going on? Can you post examples so we can debug it or are you just going to complain? ;p
  • Neox
    Options
    Offline / Send Message
    Neox godlike master sticky
    NautalusX wrote: »
    Why?! In 3 pointshader it always displays mirrored normals, flipped, rotated UV clusters anyway I please:: Correctly. It seems like this is how game engines should be! Just do it like you normally would and not have to go through a million "tricks" to try and get it to show up magically correct in udk.

    I know u'll say its because the tangent basis of the baker and udk not matching up. Well damn I guess NOTHING matches up with udk.. because no matter what I bake in it will always have some sort of issue with the NM, ALWays,

    This is 2012 not 1920s lol, why doesnt udks normal map display be as good as a real time viewer like 3 point, or Xoliul?

    because at least 3point (not sure about xoliul) is synched with max, simple as that, maps baked from maya won't look anywhere near as perfect was with 3p in max, or max maps in maya.
    did you try exporting the tangents via fbx and let unreal import those? i haven't tested in a while but at least from maya this worked for me last time i tried.
  • Xendance
    Options
    Offline / Send Message
    Xendance polycounter lvl 7
    One major point about normal suckiness is that most of the lighting is baked, which makes the situation worse.
  • Joshua Stubbles
    Options
    Offline / Send Message
    Joshua Stubbles polycounter lvl 19
    Neox wrote: »
    did you try exporting the tangents via fbx and let unreal import those

    Yeah if you're exporting from Max, this is entirely the way to go. Epic is phasing out use of ASE in UDK, so using FBX and it's explicit normals/custom binormals is really kinda required to get it as close as possible to the Max mesh.
  • Neox
    Options
    Offline / Send Message
    Neox godlike master sticky
    Yeah if you're exporting from Max, this is entirely the way to go. Epic is phasing out use of ASE in UDK, so using FBX and it's explicit normals/custom binormals is really kinda required to get it as close as possible to the Max mesh.

    i would love to say that this works with unity, so far i couldn't get correct tangents to show up there from max and as the whole production right now is max centric going via maya is also no viable option
  • AlecMoody
    Options
    Offline / Send Message
    AlecMoody ngon master
    Yeah, unreal's normal mapping pipeline is not good. I have been trying to solve this for the past few months but have had a hard time getting the required documentation. I have also been told by other developers unreal throws out a bunch of precision at a critical stage in the rendering pipeline. This could be the reason why the explicit normals workflow still has issues. On the bright side I think the qualified/explicit normals pipeline now works with skinned meshes (I haven't tested this).

    I wrote a quick guide for this here:
    http://www.3dmotive.com/exportingqualifiednormals/
  • NautalusX
    Options
    Offline / Send Message
    fvfvfv.png

    The rest of the model is great, it's just these Elements that are faceted when the normal map is turned on, their UV cluster rotation seems to have nothing to do with it. The issue is NOT present in 3point or Xoliul. It seems to be splitting the UVs.. but I am SURE that I have them weilded down before export. I am exporting it already triangulated in Max, (once again looks great in 3 point) I checked the uvs after I triangulated.. and no splits, I exported with FBX, tangents and binormals, AND smoothing groups checked.. I imported into UDK with import tangents AND remove degenerates checked... It is lit dynamically, there should be NO baked lighting issues. I checked elements for flipped normals.. they are NOT flipped. I reset Xforms. I imported as TCNormal map, I flipped channels. No avail. I also tried importing every way possible with fbx. faceting is gone IF I let udk triangulate it for me, but THEN the normal map looks horrid.
  • [HP]
    Options
    Offline / Send Message
    [HP] polycounter lvl 13
    All it would take would be for Epic to release one little .DLL file. (Xnormal TB bake plugin they use internally)
    And we would all live happily ever after, but noooo...
  • Jesse Moody
    Options
    Offline / Send Message
    Jesse Moody polycounter lvl 17
    [HP] wrote: »
    All it would take would be for Epic to release one little .DLL file. (Xnormal TB bake plugin they use internally)
    And we would all live happily ever after, but noooo...


    haha... would be nice to have a synched up bake solution now wouldn't it?
  • Stromberg90
    Options
    Offline / Send Message
    Stromberg90 polycounter lvl 11
    haha... would be nice to have a synched up bake solution now wouldn't it?

    It would :/
  • Jesse Moody
    Options
    Offline / Send Message
    Jesse Moody polycounter lvl 17
    Perhaps someone could write one for the community. That would be BEYOND helpful... Programmers out there hear my cry.... Help us!
  • NautalusX
    Options
    Offline / Send Message
    Yeah, someone put it as a sticky calling out to all programmers lol. :thumbup:
  • Ace-Angel
    Options
    Offline / Send Message
    Ace-Angel polycounter lvl 12
    Which is why Epic should be on the list of Most Trollish Companies of 2012...
  • maxivz
    Options
    Offline / Send Message
    maxivz interpolator
    Ace-Angel wrote: »
    Which is why Epic should be on the list of Most Trollish Companies of 2012...
    This
  • BluPanda
    Options
    Offline / Send Message
    has epic ever specifically said they wouldn't release this? has anyone asked?
  • Ace-Angel
    Options
    Offline / Send Message
    Ace-Angel polycounter lvl 12
    ^
    I asked them several times in their forums and even sent then email saying how I will kill a Cookie Kitten for every day they delay it and why on Earth would they hamper their own stuff.

    So far I killed over 600 Cookie Kittens, your move Epic...
  • AlecMoody
    Options
    Offline / Send Message
    AlecMoody ngon master
    As far as I know an xnormal plugin doesn't exist. I think their artists have to deal with the same workflow problems with normal maps as the rest of us who build unreal content. I have been working on a tool to solve tangent space workflows for everyone and in that process have done a considerable amount of digging into getting a perfect unreal workflow setup. I can't say a whole lot other than that at the moment I don't think it is actually possible to get smoothing error free normal maps rendering out of unreal- regardless of if you had them baked into unreal tangent space. I am not 100% sure of this since I haven't been able to get access to enough documentation to see for myself. However, other developers who I trust have essentially said it isn't possible. I would love to be wrong about this.
  • NautalusX
    Options
    Offline / Send Message
    Okay, can I get some help with my faceting issue (post 8 )
  • AlecMoody
    Options
    Offline / Send Message
    AlecMoody ngon master
    is your model force triangulated in max or are you letting fbx triangulate it for you?
  • EarthQuake
    Options
    Offline / Send Message
    AlecMoody wrote: »
    As far as I know an xnormal plugin doesn't exist. I think their artists have to deal with the same workflow problems with normal maps as the rest of us who build unreal content.

    Right, I dont think its a problem of "Epic is holding out on us". I think that the issue is that they aren't really aware of the problem or maybe more specifically do not *agree* that it is a problem. I've seen Epic artists claim on here that "UDK is synced with max"(to be fair this was a couple years ago) which is incorrect, it turns out Epic's artists are just doing the same hacks as everyone else when getting NMs into UDK.

    Its odd though, with so many Polycounters at Epic you would assume someone would have noticed by now, so maybe it is more of a technical issue.
  • NautalusX
    Options
    Offline / Send Message
    AlecMoody wrote: »
    is your model force triangulated in max or are you letting fbx triangulate it for you?

    I converted it to patch then to mesh in max, I checked the uvs after and weilded them down. If I let fbx or udk triangulate it, the normal map gets harsh seems on all smoothing group edges and looks alot worse.
  • Harbinger
    Options
    Offline / Send Message
    Harbinger polycounter lvl 8
    Depending on the situation, I've had some luck getting cleaner normals by using TC_NormalmapUncompressed instead of TC_NormalMap. DXT compression is great for efficiency, but on something like a normal map, you need exact normal colors on those edges or you'll get a small seam. The tradeoff using Uncompressed is you gain color accuracy and less noise, but your texture is quarter res (using the same memory footprint.)

    Also, many of the issues people run into are compounded by the fact that Unreal uses directional lightmapping. From what I've read, UDK calculates a tangent basis for lighting based on the UV's for channel 0 so that it can give some directionality to specular highlights. This complicates things when you want to mirror and tile materials with normal maps.

    I for one hope the next generation of hardware kills the need for tangent based normal maps. They've served their purpose and have helped us make amazing looking content, but fighting the process and tools, bouncing between multiple apps and dealing with pipelines has sucked the fun out of creating assets. Ever since I've read this whitepaper I've been drooling and crossing my fingers this becomes the next "thing" for artists:
    http://jbit.net/~sparky/sfgrad_bump/mm_sfgrad_bump.pdf
  • Quack!
    Options
    Offline / Send Message
    Quack! polycounter lvl 17
    Harbinger wrote: »
    Depending on the situation, I've had some luck getting cleaner normals by using TC_NormalmapUncompressed instead of TC_NormalMap. DXT compression is great for efficiency, but on something like a normal map, you need exact normal colors on those edges or you'll get a small seam. The tradeoff using Uncompressed is you gain color accuracy and less noise, but your texture is quarter res (using the same memory footprint.)
    http://jbit.net/~sparky/sfgrad_bump/mm_sfgrad_bump.pdf

    Epic actually recommends that you use TC_NormalMapUncompressed for normals over TC_NormalMap. If none of the standard hacks fix the issue (enough geo, uv islands all correct, smoothing groups correct, etc) Then this may be your last hope. You do lose resolution from your NM but it isn't as drastic as it sounds. Try it out.
  • AlecMoody
    Options
    Offline / Send Message
    AlecMoody ngon master
    NautalusX wrote: »
    I converted it to patch then to mesh in max, I checked the uvs after and weilded them down. If I let fbx or udk triangulate it, the normal map gets harsh seems on all smoothing group edges and looks alot worse.

    Do you have a copy from before this conversion? It is safer to select all your verts and hit connect. Converting your geometry to patch may break things in unexpected ways.

    Also, make sure you don't have other UV channels with splits in them.
  • Ace-Angel
    Options
    Offline / Send Message
    Ace-Angel polycounter lvl 12
    I had to resort to recreating my NormalMap's from a TC-Default setup, with my Blue being my AO and my Alpha my Displacement to save on texture count at the same time and SRGB disabled.

    Also, if what EQ said is right, I'm sorely disappointed at the peeps from Epic.

    Here is why;

    Lets say for the sake of argument that Epic has 50 artist working on a game. These 50 artists need to do 'something' to get the baking correct, lets say this process takes about 10-15 minutes.

    For a total of 50 artists, you're wasting between 300-500 man hours for one piece from each artist in TOTAL, if each artists does more then 1 piece, that adds up, and I'm not even talking about revisions where certain pieces might look 'old' and need a revamp or something about 6 months down the line.

    This is beyond being 'silly', this is lazy, a waste of time and man-hour all in one. Normal Maps are already though to bake, why complicate your pipeline more? Doesn't Epic want to make the most out of their time and talent, instead of doing kludge work?

    And just how lazy are the artists? I would force my artists to visit polycount as often as possible and learn about polygon/texture technicalities before they do anything in those regards.
  • Computron
    Options
    Offline / Send Message
    Computron polycounter lvl 7
    [HP] wrote: »
    All it would take would be for Epic to release one little .DLL file. (Xnormal TB bake plugin they use internally)
    And we would all live happily ever after, but noooo...

    Ditto for Crytek, since they aren't even synced to their own baker according to EarthQuake.
  • [HP]
    Options
    Offline / Send Message
    [HP] polycounter lvl 13
    Computron wrote: »
    Ditto for Crytek, since they aren't even synced to their own baker according to EarthQuake.

    AFAIK this problem is currently being looked at internally, as soon as we get it, and it's tested properly, the plugin will be released along with the FreeSDK.
    So far, Crytek's art teams have been struggling with the same problems as anyone else. Hopefully this pain will stop, for everyone using CE3, and we'll have no more Tangent Space shading discrepancies between the baker and the ingame renderer.
  • NautalusX
    Options
    Offline / Send Message
    AlecMoody wrote: »
    Do you have a copy from before this conversion? It is safer to select all your verts and hit connect. Converting your geometry to patch may break things in unexpected ways.

    Also, make sure you don't have other UV channels with splits in them.

    I tried now the connect, result is the same. It only has 1 uv channel for sure, I checked after import. I looked around other areas of the mesh in udk up close, and some other curved surfaces are also having the same faceting. I tried adding that quality normal line to my 3dxmax.ini file.. no change ;/ mesh verts are all welded up as well.

    I can tell that its importing quality normals, because it looks much better when theyre imported. BUT it only works if I triangulate it before export from max myself. Faceting. The normalmapuncompressed just makes me see the faceting clearer :D
  • AlecMoody
    Options
    Offline / Send Message
    AlecMoody ngon master
    can you copy and paste a screenshot of the max channel info?
  • Computron
    Options
    Offline / Send Message
    Computron polycounter lvl 7
    [HP] wrote: »
    AFAIK this problem is currently being looked at internally, as soon as we get it, and it's tested properly, the plugin will be released along with the FreeSDK.
    So far, Crytek's art teams have been struggling with the same problems as anyone else. Hopefully this pain will stop, for everyone using CE3, and we'll have no more Tangent Space shading discrepancies between the baker and the ingame renderer.

    Sync it to Max, you guys are already using Max's UI. :):thumbup:
  • Ace-Angel
    Options
    Offline / Send Message
    Ace-Angel polycounter lvl 12
    [HP] wrote: »
    AFAIK this problem is currently being looked at internally, as soon as we get it, and it's tested properly, the plugin will be released along with the FreeSDK.
    So far, Crytek's art teams have been struggling with the same problems as anyone else. Hopefully this pain will stop, for everyone using CE3, and we'll have no more Tangent Space shading discrepancies between the baker and the ingame renderer.

    If this happens, I'm moving my whole project to CE3!
  • NautalusX
    Options
    Offline / Send Message
    So, after literally hours of testing, I solved my faceting issue :D

    The elements that were faceted I detached into seperate objects, played with their triangulation, left them as editable polys, and exported with fbx tangents and smoothing groups checked. Before export it gave me a warning about turned edges on the affected objects, which was good cause I noticed that If i didnt get that warning about the objects the faceting would appear again in udk. I suspect that if you get the warning for turned edges, it wont use imported tangents on them objects. Then I imported it into udk with combine meshes checked.
  • Quack!
    Options
    Offline / Send Message
    Quack! polycounter lvl 17
    Use the "Turn to Poly" modifier to make a triangulated object before export. Set the max number of sides to 3. This is a non destructive act(so you don't accidentally save a triangulated mesh) and works flawlessly with fbx and udk. You won't get that turned error message.
  • Computron
    Options
    Offline / Send Message
    Computron polycounter lvl 7
    I'v read that you should turn to 3-sided poly before baking. Is that inaccurate?
  • Ace-Angel
    Options
    Offline / Send Message
    Ace-Angel polycounter lvl 12
    ^
    For a game engine export? Yeah, it should be fine.

    For baking? Not so sure, since I always get a striated edge in my bake vs. Quads which give me clean bakes.

    But then again, I use xNormal, so there is that!
  • Neox
    Options
    Offline / Send Message
    Neox godlike master sticky
    hu what? the result should be pretty much the same, as xnormal also bakes with some inner triangulation and to be sure to get the best results of course xnormal should bake with the same triangulation as your engine would use in the end.
  • Ace-Angel
    Options
    Offline / Send Message
    Ace-Angel polycounter lvl 12
    ^
    Nope, I tried that 3 times (from Max) and a triangulated mesh will give me striations in my Normal bake 100% of the time.
  • Xoliul
    Options
    Offline / Send Message
    Xoliul polycounter lvl 14
    We've looked into this solving problem at work quite heavily, to try and match up with Unreal's tangent space.
    Since I'm not too sure about the exact details, the long story comes down to: there is no way to sync correctly, due to the way the engine works internally (hence why Epic themselves can't do much).
    It loses accuracy somewhere down the render pipeline, and there is no way for normalmaps to correct that. We should just hope UE4 is better and stick with proven methods like beveling edges more.
  • Quack!
    Options
    Offline / Send Message
    Quack! polycounter lvl 17
    Computron wrote: »
    I'v read that you should turn to 3-sided poly before baking. Is that inaccurate?

    I have not tested this scientifically, but in practice it doesn't seem to matter if you triangulate before you bake if you are doing it within the same program. So if you are using max I don't think that you need to triangulate before you bake in max, I turn to poly after the bake, and my maps turn out as I would expect.

    If you are baking in an external program I would always triangulate before exporting and baking.
  • Computron
    Options
    Offline / Send Message
    Computron polycounter lvl 7
    Hmmm, ok. I just went back to the 3dmotive Normals series and Alec Moody specifically said to triangulate before baking. Either way not a big deal.
  • Neox
    Options
    Offline / Send Message
    Neox godlike master sticky
    Ace-Angel wrote: »
    ^
    Nope, I tried that 3 times (from Max) and a triangulated mesh will give me striations in my Normal bake 100% of the time.

    you mean just inside the actual texture or in the final result rendered in realtime?
  • passerby
    Options
    Offline / Send Message
    passerby polycounter lvl 12
    Computron wrote: »
    Hmmm, ok. I just went back to the 3dmotive Normals series and Alec Moody specifically said to triangulate before baking. Either way not a big deal.

    if you export out as sbm for xnormal it cut's it to tris by it's self anyways.

    i've never really had problems with triangulation, baking and the game engine before mostly just get issues with turned edges, when exporting fbx from max to maya.
  • AlecMoody
    Options
    Offline / Send Message
    AlecMoody ngon master
    triangulating before baking is safest since you eliminate any chance of the edge flow changing later on in your workflow.
  • Neox
    Options
    Offline / Send Message
    Neox godlike master sticky
    AlecMoody wrote: »
    triangulating before baking is safest since you eliminate any chance of the edge flow changing later on in your workflow.

    especially with maya in mind with no possibility to change the internal triangulation, or did i miss something there? as i see it, it is using the shortest way between verts to determine how to triangulate internally, right?
  • AlecMoody
    Options
    Offline / Send Message
    AlecMoody ngon master
    Internal triangulation can change later on in the pipeline. For example, your exporter could triangulate differently than your 3d application. Also, the shortest distance between verts should be reliable with quads but with ngons it is going to change based on how the triangulator is written. I don't fully triangulate every model before I bake. However, for anyone who isn't 100% clear on what is happening to their model all the way out to the engine it can save a lot of trouble.
  • Neox
    Options
    Offline / Send Message
    Neox godlike master sticky
    it is not even reliable with quads, in so many cases it creates ugly concave tris, or creates convex faces where concave ones might do the job way better
  • EarthQuake
    Options
    Offline / Send Message
    I know this isn't a technical solution persay, but I always try to hand triangulate non-planar(within reason) faces. If you have a quad that is very non-planar and you're worried about how your app/exporter/etc will triangulate, do it yourself! There should be an easy way to select non-planer(with a certain threshold) faces in most apps.

    I realize this isn't helpful for normal maps and tangent data, as you really want your triangulation to remain the same throughout the process to avoid those "cross" type smoothing errors, but just in the cases where non-planar triangulation is going to make a visual difference on you model.
  • Ace-Angel
    Options
    Offline / Send Message
    Ace-Angel polycounter lvl 12
    Neox wrote: »
    you mean just inside the actual texture or in the final result rendered in realtime?
    Texture and render, using RT shaders in Max.

    However, using SBM doesn't give me that issue, and baking inside Max also doesn't give me that issue.

    No idea why, and don't plan on spending time on finding out why (since let's face it, not many people will care about this point).
  • cptSwing
    Options
    Offline / Send Message
    cptSwing polycounter lvl 11
    I always slap an Edit Mesh modifier below my projection cage before handing off to xNormal, and keep it for exporting meshes to engine. That way triangulation should stay consistent.

    Xoliul wrote: »
    We've looked into this solving problem at work quite heavily, to try and match up with Unreal's tangent space.
    Since I'm not too sure about the exact details, the long story comes down to: there is no way to sync correctly, due to the way the engine works internally (hence why Epic themselves can't do much).
    It loses accuracy somewhere down the render pipeline, and there is no way for normalmaps to correct that. We should just hope UE4 is better and stick with proven methods like beveling edges more.

    d'oh. is part of the problem their uv channel 0 lookups for specular lightmap calculation? so annoying.
  • Ace-Angel
    Options
    Offline / Send Message
    Ace-Angel polycounter lvl 12
    ^
    Wait, hold on, I thought if you used the old broken baker from Max (2008, 2009, etc) you could get the correct tangents for UDK to render properly?

    Unless I'm mistaken from what I read around the net, that sounds like an issue where no one wants to use that 'broken' math anymore and simply doesn't want to implement it in it's own bakers, which is kinda like saying "Oh, we have an old 'broken' solution for a 'broken' issue, but won't implement to fix the issue because we only like clean, working stuff".

    See the paradox?
1
Sign In or Register to comment.