Home Unreal Engine

UDK, xNormal and tangents : what the hell is wrong ?

greentooth
Offline / Send Message
Froyok greentooth
Is somebody able to explain me why this happen :
importtangents_problem.jpg

I'm using Maya x64 2013 + FBX 2013.3, and also UDK November 2012. My normal maps are baked inside xNormal (3.17.16).

-When I import my static meshes I of course check "Import tangents" and "Explicit normals".
-When I export my meshes from maya as FBX files I of course check "Tangents and Binormals" in the options.
-My meshes are triangulated in Maya before export. No history on them, everything is clean.
-My normal map baked in xNormal is with the option "use exported normals"


From what I see, it looks like the "import tangent" option is inverted.

Replies

  • jogshy
    Offline / Send Message
    jogshy polycounter lvl 17
    You might need to flip the green channel or use X+Y-Z+ swizzling to fit UDK's coordsys.

    >>-My meshes are triangulated in Maya before export.
    Plus Freeze transformations.

    FBX can be problematic sometimes. Try with OBJ or SBM better.
  • Froyok
    Offline / Send Message
    Froyok greentooth
    I bake in SBM most of the time. I tried OBJ too.
    Also my normal map is flipped correctly.
    My transformations are always frozen too.

    Really, here the problem is UDK, not xNormal. :(
  • Ace-Angel
    Offline / Send Message
    Ace-Angel polycounter lvl 12
    I think explicit normals might be the issue, disable that only and try again (although Tangents shouldn't be the issue).
  • Froyok
    Offline / Send Message
    Froyok greentooth
    Ace-Angel wrote: »
    I think explicit normals might be the issue, disable that only and try again (although Tangents shouldn't be the issue).

    Here is the result, disabling "explicit normals" produce the equivalent of "averaged normals" everywhere, the problem is still here and my normals are not imported correctly (obviously, because I disabled "explicit normals") :

    importtangents_problem_explicit.jpg
  • monster
    Offline / Send Message
    monster polycounter
    I can't see your screenshots, but there are two common issues I've found when trying to get normal maps to work.
    • Make sure your normal map textures have the correct import settings.
    • FBX 2013.X Plugin is broken because it doesn't calculate the tangents correctly, I don't know about 2012. Maybe you can revert to an older plugin. (Link) So basically, you are importing incorrect tangents. Epic provides a code workaround for the FBX problem. If you have access to the licensee area. (Link)

    [edit]I can see your screenshots now. It's definately the FBX problem. If you can use ActorX that would be the easiest option.[/edit]

    Make sure you are using these settings on your texture.
    UnrealNormal.png
  • Froyok
    Offline / Send Message
    Froyok greentooth
    monster wrote: »
    I can't see your screenshots,
    Hmm, weird, images are uploaded on my personal website, they should be up.
    monster wrote: »
    • Make sure your normal map textures have the correct import settings.
    • FBX 2013.X Plugin is broken because it doesn't calculate the tangents correctly, I don't know about 2012. Maybe you can revert to an older plugin. (Link) So basically, you are importing incorrect tangents. Epic provides a code workaround for the FBX problem. If you have access to the licensee area. (Link)
    I'm aware about this FBX 2013.2 problem, but the 2013.3 should have fixed this issue from what I have read. unfortunately I'm not a licensee, just an honest user of the free UDK version. :(
    monster wrote: »
    [edit]I can see your screenshots now. It's definately the FBX problem. If you can use ActorX that would be the easiest option.[/edit]
    Does ActorX support custom/explicit normals ?
    And ActorX doesn't exists for Maya 2013 (an unofficial version yes, but not made by Epic).
    monster wrote: »
    Make sure you are using these settings on your texture.
    [IMG*]https://dl.dropbox.com/u/2904948/Temp/UnrealNormal.png[/IMG]
    To be sure with everybody : my textures are correctly imported.
  • monster
    Offline / Send Message
    monster polycounter
    I have confirmed with Autodesk that this is a bug in the FBX exporter.

    That's what the guy from Epic told me about a week ago. But I was able to use the code fix, which uses the exported normals to recalculate the tangents durring import.

    I assume ActorX supports it, because that was the alternate solution I was given by Epic. I've never tried it though.

    The only other workaround I can think of (that I haven't tried) is to export OBJ, then download an older 2011 or 2012 FBX Converter.

    http://usa.autodesk.com/adsk/servlet/pc/item?siteID=123112&id=10775920
  • Froyok
    Offline / Send Message
    Froyok greentooth
    OBJ files don't store tangents/binormals. The Collada format does, but the DAE export provided with the FBX plugin is already broken since a long time and OpenCollada is not available yet for Maya 2013.

    Looks like a dead end.

    [EDIT] I just tested the ASE exporter and from what I remember nothing as change : normals and tangents are not exported with the ASE format for Maya. So UDK recreate them (and they are wrong).

    The ASE exporter of 3DS Max is different because this one is native, not handle by a plug-in. The one in Maya with the ActorX has always been limited (this is why I quickly jumped onto the FBX format).
  • Froyok
    Offline / Send Message
    Froyok greentooth
    I will give a try to handplane then, looks like my only solution here. :(
  • monster
    Offline / Send Message
    monster polycounter
    OBJ does support normals. I was hoping the converter would generate the tangents and binormals the way the plugin is supposed to do.
  • Froyok
    Offline / Send Message
    Froyok greentooth
    I asked a friend to generate me the same mesh (the cube) from Maya 2011 x64 + FBX 2011.3. I got the same problem, which is weird...
    monster wrote: »
    OBJ does support normals.
    But not tangents/binormals.
    monster wrote: »
    I was hoping the converter would generate the tangents and binormals the way the plugin is supposed to do.
    Yeah, I tried OBJ and DAE in the FBX 2012.2 converter and it still doesn't work. :\
  • AlecMoody
    Offline / Send Message
    AlecMoody ngon master
    What exactly is the problem you are looking to correct? I see two things, there are little bands of weird shading near the edges of the cube, and the spherical bump is facing the wrong way.


    The problem with this workflow where you pass a bunch of normals/tangents between applications is that it makes it very difficult to tell what is really happening or where the error originates.

    I cant remember what the coordiante swizzles are for each of these tools but it looks like unreal is inverted on one channel compared to maya(or xnormal) which is causing the flip.
  • Froyok
    Offline / Send Message
    Froyok greentooth
    AlecMoody wrote: »
    I see two things, there are little bands of weird shading near the edges of the cube
    It's just a shadow problem with the dynamic lighting in UDK.
    AlecMoody wrote: »
    and the spherical bump is facing the wrong way.
    This is the problem I'm facing and...

    AlecMoody wrote: »
    I cant remember what the coordiante swizzles are for each of these tools but it looks like unreal is inverted on one channel compared to maya(or xnormal) which is causing the flip.
    ... it's not a swizzle error, my normal maps in UDK are correctly set for X+ Y- Z+.
  • AlecMoody
    Offline / Send Message
    AlecMoody ngon master
    The swizzle you are using is incorrect for maya and you are importing maya tangents into unreal. The reason it looks flipped is because you flipped the y direction for unreal which when you import the maya tangents is no longer the correct orientation.

    Passing tangents and normals between applications can get messy and confusing very quickly. Also, keep in mind that this workflow isn't useful for anything rigged- Which includes guns, vehicles, complicated props...
  • Froyok
    Offline / Send Message
    Froyok greentooth
    AlecMoody wrote: »
    The swizzle you are using is incorrect for maya and you are importing maya tangents into unreal. The reason it looks flipped is because you flipped the y direction for unreal which when you import the maya tangents is no longer the correct orientation.
    Are you telling me to import my normals map as X+ Y+ Z+ then ? Because this doesn't work.
    Or do you suggest to bake without the tangent and let xNormal calculate them with the algorithm of Mikk (Mikk - TSpace in xNormal) ? If so, why Epic suggests to import our tangents/binormals if we don't use them to bake ?

    I'm just following the pipeline as mentioned here :
    http://udn.epicgames.com/Three/XNormalWorkflow.html
    AlecMoody wrote: »
    Passing tangents and normals between applications can get messy and confusing very quickly. Also, keep in mind that this workflow isn't useful for anything rigged- Which includes guns, vehicles, complicated props...
    Yeah, my current tests were with static meshes only. I'm aware that the Unreal Engine 3 doesn't read our tangents with the skeletal meshes.
  • Harbinger
    Offline / Send Message
    Harbinger polycounter lvl 8
    Just wanted to chime in, I'm having what looks the same issue. Super annoying, since I've worked out these issues with my workflow in the past... Looks like the bi-normals aren't being calculated correctly with mirrored geometry.

    After some digging it looks like the current FBX exporter has some bugs, but what is weird is that older versions like FBX2012 used to work fine. I'm going nuts :poly127:
  • AlecMoody
    Offline / Send Message
    AlecMoody ngon master
    Are you all successfully running qualified normals in the viewport? Max fbx outputs whatever tangents are being displayed in the viewport and this could be another source of issues with this workflow.
  • Harbinger
    Offline / Send Message
    Harbinger polycounter lvl 8
    I'm using Maya. Normal map renders perfect, and when displaying tangents everything looks correct. Once exported to UDK, using the proper explicit normals/tangents checkboxes on both import and export, the bi-normals are wrong but the tangents and normals look correct.

    FYI, I'm mirroring wheels and some minor details on a vehicle from one side to another.
  • Harbinger
    Offline / Send Message
    Harbinger polycounter lvl 8
    OK, I had some more time to play around with this tonight. I was mistaken, FBX exports are working correctly, at least as of version 2013.3. UDK is the culprit.

    Acting on a hunch because I remembered a bug with the "Flip normal map green channel" option box not remembering its settings, I re-imported my mesh with "Import Tangents" turned off but kept "Explicit Normals" on. Guess what? My binormals import correctly and everything is peachy.

    I set up a quick test case in Maya with dummy geo and reproduced the issue. I checked my UDK version and I was on the July build. Downloading November now to see if I get different results.

    :Update: Tested the newest November Beta 2 build. Using my normal workflow of enabling both Explicit Normals and Import Tangents gave me even worse results! And again, disabling Import Tangents but leaving Explicit Normals on gave me the results I'm looking for.

    Can anybody verify? I'm going Maya 2013 -> FBX vers 2013.3 -> UDK
  • Froyok
    Offline / Send Message
    Froyok greentooth
    So, you get the same results as me then (see first post).
  • cptSwing
    Offline / Send Message
    cptSwing polycounter lvl 11
    Yeah, noticed that a while ago as well - exporting a FBX from a directx/qualified normals 3ds Max viewport to UDK definitely doesn't need the "Import Tangents" setting checked. It looks like ass if I do keep it checked, in fact.
  • Harbinger
    Offline / Send Message
    Harbinger polycounter lvl 8
    Froyok, yes. Sorry, I was so caught up in my own confusion I was mainly looking at the pic on your second post. So I spun my wheels a whole lot to find the same conclusion.

    Thanks for the verification cptSwing.
  • Froyok
    Offline / Send Message
    Froyok greentooth
    Harbinger wrote: »
    Froyok, yes. Sorry,
    No need to be sorry ! :)
    I'm glad to see some other people get the same results as me... Feeling alone in these type of problems is a bit demotivating.
  • cptSwing
    Offline / Send Message
    cptSwing polycounter lvl 11
    Ah, after re-checking it seems I was talking out of my oh-so-fine ass. I uncheck "Tangents & Binormals" in the FBX exporter settings, I do however check "Import Tangents" in UDK. This might be a Max specific thing though, the viewports tangents carrying through somehow? I dunno.
  • Froyok
    Offline / Send Message
    Froyok greentooth
    Nope, if you don't export the tang/bin, UDK will recalculate them, even if you check "import tangents". So checking the "import tangents" has no effect in this case. It's the same behavior in Maya.
  • cptSwing
    Offline / Send Message
    cptSwing polycounter lvl 11
    Welp. I did test it a while back with various import/export settings, so shrug.
  • Froyok
    Offline / Send Message
    Froyok greentooth
    I tested the February 2013 beta yesterday and well.. the problem is still here :
    udk_binormals_feb2013.jpg
  • Froyok
    Offline / Send Message
    Froyok greentooth
    I did some new tests with the help of some friends today.
    (Tested with UDK November 2012, same behaviors observed with UDK February 2013)

    Settings used when importing my tets mesh :
    -"Explicit Normals" enabled
    -"Import tangents" disabled

    List of FBX files exported from :

    -Maya 2011 [FBX 2011.3 / FBX 7.1.0] -> Not working
    -Maya 2013 [FBX 2013.3 / FBX 7.3.0] -> Not working (binormals/tangents broken : Example )
    -3DsMax 2013 [FBX 2013.3 / FBX 7.3.0] -> Not working

    Blender 2.63 [FBX 6.1.0] -> Working
    Modo 701 [FBX 6.1.0] -> Working


    So, I guess I will have to wrote a batch converter from Maya to Blender FBX files.
  • Obscura
    Offline / Send Message
    Obscura grand marshal polycounter
    I'm not sure if this is related with this, but maybe Handplane would helps?
  • Froyok
    Offline / Send Message
    Froyok greentooth
    Obscura wrote: »
    I'm not sure if this is related with this, but maybe Handplane would helps?
    Handplane require to import a mesh with "Import Tangents", "explicit normals" and "remove degenerates" settings disabled.

    Unfortunately it breaks all the normals on my meshes, I get a faceted mesh.
    So while handplane works, UDK is unable to import my mesh correctly.

    [EDIT] Ha funny, if I use the FBX generated by Blender and import it in UDK will all the settings above disabled it import my mesh correctly.

    So it's definitively related to an FBX version problem.
  • Froyok
    Offline / Send Message
    Froyok greentooth
    So here is my batch with the help of Blender :
    http://www.froyok.fr/blog/2013-04-fbx-exporter-for-udk-static-meshes-maya-to-blender-fix

    It fix the vertices count problem :

    maya_vs_blender.jpg
  • Gheromo
    Offline / Send Message
    Gheromo polycounter lvl 11
    Bumped into same issue a year ago... thought it was lightmap issue... checked now with Maya 2014, still not fixed. *sigh* Autodesk...

    Edit: Maya 2014 vs Modo 701 Ncj9BmR.png
  • JHS
    Offline / Send Message
    JHS polycounter lvl 5
    thx guys you totally help me, never knew how this works. i got a similar problem with xnormal viewer and marmoset. As i´m planning to put my mech to UDK, i wil come across with the same problems.

    http://www.polycount.com/forum/showthread.php?t=124191
  • Sinking
    I am really confused by this thread. I use Maya 2014 and export models to UDK all the time and they look perfectly fine. I baked most of the normals in Mudbox though, but some are done in XNormal.

    Was this issue resolved with the new FBX export or why didn't I ever have this problem? When I get a shading error it is usually due to wrong smoothing groups, but your problems looks like the tangents are read completely wrong, ignoring the information from the normal map mostly. I don't understand how this can work for me, even though I didn't revert to older FBX or other tweaks. Any ideas why?
  • r4ptur3
    Offline / Send Message
    r4ptur3 polycounter lvl 10
    Yeah same here -- no normal problems with the Maya FBX to UDK pipeline. Here's my settings:

    FBX EXPORT:
    smoothing groups: on
    tangents and binormals: on
    referenced containers content: on
    all other options: off

    UDK FBX IMPORT:
    override name: on
    explicit normals: on
    remove degenerates: on
    one convex hull per UCX: on (not necessary unless you have collision)

    Important! I keep "import tangents" off -- it screws up normal map mirroring, even tho epic claims it does not.

    UDK TEXTURE IMPORT
    compression settings: TC_Normalmap
    all other settings: default

    I inverted the green channel of my normal map inside my material (in UDK) by multiplying against a 1, -1, 1 vector.
  • Theodoric
    Offline / Send Message
    Theodoric polycounter lvl 10
    I have a problem, but can't decide is it normal map/tangent related or some kind of lightmap issue so I post it here first.
    The normal map perfectly works in xnormal preview and in UDK static mesh editor (with the dynamic light):

    xnormal_shot.jpg

    When I generate the lightmap (the lightmap UV channel is generated inside the UDK) some of the surfaces become concave a little:
    udk_concave_shade.jpg

    My question is why, and how to avoid this?

    BTW this model is done in XSI. When I thought maybe the XSI fbx export is bad I started try exporting from maya. When I leave the normals locked, nothing is changed, (the normals ok under the dynamic light, and strange with lightmap) but in UDK the vertex number significantly higher. If I unlock the normals, and redo the soft and hard edge setup then this number about the same, but in this case I have to redo all the normal map baking because of the different tangents... How could I keep the imported normals, and keep the vertex number low when I export this XSI model from maya? (Because probably it'll be animated in maya...)
  • Froyok
    Offline / Send Message
    Froyok greentooth
    Sinking wrote: »
    I am really confused by this thread. I use Maya 2014 and export models to UDK all the time and they look perfectly fine. I baked most of the normals in Mudbox though, but some are done in XNormal.

    Was this issue resolved with the new FBX export or why didn't I ever have this problem? When I get a shading error it is usually due to wrong smoothing groups, but your problems looks like the tangents are read completely wrong, ignoring the information from the normal map mostly. I don't understand how this can work for me, even though I didn't revert to older FBX or other tweaks. Any ideas why?

    I little time ago I found the origin of the problem. I forgot to write the answer of the problem here. It my fault in the end and how I manage my vertex normal in Maya.

    So what is the problem ? First I will explain how UDK import FBX : when UDK import any FBX it tries to find any smoothing group. With Maya there is no smoothing groups, but the FBX exporter is in charge to make them. Smoothing group while defined by Faces in any app supporting this system are stored per vertex from what I know.

    In Maya there is multiple way to manipulate normals, the most commons are :
    -Set to Face/Average normals
    or
    -Harden/Soften edge normals
    The visual result in Maya is the same, but behind the scene it's not the case. While the Soften/Harden actually work correctly and merge similar vertex normals, the Average/Set to Face orient the normals BUT don't merge them if there are similar.

    So if you make a faceted mesh (every face has its own shading) and then an "Average normal" on top, you get visually a smoothed and uniform shading, but every vertex normals are still splitted, even after the export.

    Here is a little test with two cubes :

    1376519125-maya_vertexsplit_to_udk.jpg

    Theodoric wrote: »
    I have a problem, but can't decide is it normal map/tangent related or some kind of lightmap issue so I post it here first.
    The normal map perfectly works in xnormal preview and in UDK static mesh editor (with the dynamic light):

    [IMG*]http://www.artbytheodoric.hu/temp/mech/Bug/xnormal_shot.jpg[/IMG]

    When I generate the lightmap (the lightmap UV channel is generated inside the UDK) some of the surfaces become concave a little:
    [IMG*]http://www.artbytheodoric.hu/temp/mech/Bug/udk_concave_shade.jpg[/IMG]

    My question is why, and how to avoid this?

    BTW this model is done in XSI. When I thought maybe the XSI fbx export is bad I started try exporting from maya. When I leave the normals locked, nothing is changed, (the normals ok under the dynamic light, and strange with lightmap) but in UDK the vertex number significantly higher. If I unlock the normals, and redo the soft and hard edge setup then this number about the same, but in this case I have to redo all the normal map baking because of the different tangents... How could I keep the imported normals, and keep the vertex number low when I export this XSI model from maya? (Because probably it'll be animated in maya...)

    In your case I think the problem come from the Tangent, try to import your mesh with "Explicit normals" enabled and "Import tangents" disabled. Also make a proper lightmap, this should help as automatic UVs can make seams where you don't want to, they could break the shading after the bake.

    Anyway, see my answer above about the soft/hard normals.
  • Harbinger
    Offline / Send Message
    Harbinger polycounter lvl 8
    Nice catch Froyok!
  • Theodoric
    Offline / Send Message
    Theodoric polycounter lvl 10
    Thx for explain that!

    From maya it's only works as you wrote (explicit normal:on,import tangents:off).
    For a fbx which came from XSI the import tangent has to be set ON.

    I'll export it with a 2nd UV channel for the lightmap and hope this will solve this concave shading problem. Still don't understand why it's became like that even it's perfect in xnormal and UDK model preview with dynamic light... :)
  • Shrike
    Offline / Send Message
    Shrike interpolator
    make sure you do not bake in xnormal while using FBX, and use OBJ, xnormal does the strangest things when you give it FBX files, just wanted to throw that in , I had a lot of headaches because of that

    Its just insane how much trouble you can have just because plugins and formats and programs are not working as they should and everyone doing their own thing and nothing is normed
  • r4ptur3
    Offline / Send Message
    r4ptur3 polycounter lvl 10
    FBX remembers history -- gotta freeze the objects before you put it through Xnormal. It is a weird format, but it has its advantages (vertex color, for instance).
  • Defaultsound
    Offline / Send Message
    Defaultsound polycounter lvl 14
    So I am a bit confused, I have run into this issue as well and concluded that the Tangents are also causing problems with the shading of my mesh. To fix this I have turned off Tangents in the export from Maya. Results are the same if I export from Blender as well so exporting without Tangents doesn't matter.

    So is it still the case that the FBX exporter is broken as of Maya 2013, or by doing that smoothing trick should it correct the Tangents? Otherwise I have no clue what is going on...
  • Froyok
    Offline / Send Message
    Froyok greentooth
    For the moment it's advised to let UDK recompute the tangents. I don't know about the tangents from Blender, most of the time I avoid to import them in UDK.
  • Defaultsound
    Offline / Send Message
    Defaultsound polycounter lvl 14
    Ah okay I see, so what exactly are the tangents doing? I assume they define the vertex angles of each vertex, or does the smoothing group control that?
  • Ace-Angel
    Offline / Send Message
    Ace-Angel polycounter lvl 12
    They're the math that define how they will interact with the normal map.

    For example, setting your entire mesh to 1 smooth/hardness group and backing out a map (with all the crazy gradient shifts) would results (if the engine is not synched) in pretty crappy looks models with the normal map, warping lights, etc.

    A synched engine with the baker (which takes the proper tangent and vertex information from the source) will result in MUCH better results, and usually for the cheaper, since in some cases you wouldn't spend all day creating new uv's, cutting seams, smooth grouping, etc every nook and cranny to avoid issues.

    You should read EQ's sticky in Technical, there are the A to Z's about all this stuff and tangents.
  • Froyok
    Offline / Send Message
    Froyok greentooth
    The vertex "angle" is defined by the vertex normal (aka smoothing group). The tangent and binormals are used for normal maps.
    Tangents and binormals (/bitangents) are perpendicular vectors that run along the surface of the model that describe the direction that 2 channels of the normalmap point in, for any point on a models surface. They're exported as part of DAE and FBX formats, as well as a few others. You can caclulate them yourself, using the texturecoordinates and vertex positions each vertex per triangle, then average them out ala smooth normals. Of course, because you can do this, you can also generate them in shaders but the cost is usually fairly prohibitive.
    http://www.gamedev.net/topic/609108-normal-mapping-tangents-binormals/

    That's why it's currently a problem as Ace-Angel point out : the way we pre-compute the tangent/binormals change for one application ton one another. Maya doesn't compute them as UDK for example.
  • Defaultsound
    Offline / Send Message
    Defaultsound polycounter lvl 14
    Right okay I see now, I guess that letting UDK compute them isn't the best, however it is the only way to work around the FBX issue from what it seems.
Sign In or Register to comment.