Home Technical Talk

Official handplane support thread - Now freeware!!

1234568

Replies

  • dzibarik
    Offline / Send Message
    dzibarik polycounter lvl 10
    AlecMoody wrote: »
    I am not that familiar with either of those tools. It is going to depend on how they work and what process they use to blend micro texture into the normal map. I would just test it out early in your process so you can see if it is causing issues before you spend a ton of time on it.

    yeah, I tested it and Substance Designer works alright.
  • soulty
    going from 3d max2012 object space - maya 2013 tangent(using .h3d) I some time get random edges baking into the tangent map. I have looked at a tangent map in max (max is not baking the artifact) and the object space map in maya (before using handplane) and it has no edge artifact but when processed through handplane it gets edges visible in random areas. Any thoughts on what is causing this?
  • AlecMoody
    Offline / Send Message
    AlecMoody ngon master
    Can you post a picture?
  • Chubbydan
    Offline / Send Message
    Chubbydan polycounter lvl 13
    Hi,

    I checked this thread with regards my issue which I think is related,

    http://www.polycount.com/forum/showthread.php?p=2194972#post2194972

    All down to Unity shaders. I am asking my programmer colleague to document what he did to the shader but all good now.

    cheers,
  • AlecMoody
    Offline / Send Message
    AlecMoody ngon master
    Yeah, Unity's forward rendering has some minor issues. I use the deferred rendering pipeline but for some people that isn't going to be a viable option.
  • Farfarer
    There's a couple of things you can do to work around it, but they're more expensive to compute and the good version isn't easily compatible with Surface Shaders.

    To improve things, you can normalize the light and view vectors per pixel in the surface shader.

    To fix things, you need to convert the tangent space map to object space inside of the pixel shader, then do lighting in object space. This is not really wish doing in a Surface Shader as it results in a lot of code duplication and hacky workarounds.
  • Veezen
    Offline / Send Message
    Veezen polycounter lvl 9
    I have never used this software but today I wanted to try it. I usually bake my maps in xNormal so I know how to set up proper my meshes, but handplane don't work for me, it shows really strange normals. I even watched this video but nothing helps:
    [ame="https://www.youtube.com/watch?v=l59_R7RENxE"]handplane baking tutorial - YouTube[/ame]

    And here is result after render my baked textures from xNormal in handplane. It looks terrible:

    https://copy.com/DAVA9BO4ZnKi0DUF/mmmm.JPG

    I tried everything - flip green, red channel, nothing works.

    @edit SOLUTION FOR MY PROBLEM
    I made a mistake in handplane, proper normal map is showed bellow

    Ok, so I have noticed that normals rendered in Handplane doesn't work proper in 3ds Max 2012 shader. Take a look on this:
    problem_mmm_solution.jpg

    You can read about this also here:
    http://www.laurenscorijn.com/future-xoliulshader-support.html

    If someone use 3ds Max 2012 and everything works fine, then it would be great to know what I doing wrong. I know that models are for game engines but it's good to have good maps also in 3d software.
  • Farfarer
    You need to set the normals to qualified mode via the 3dsmax.ini file.

    Enabling qualified normal mapping is done by adding the following lines to the end of 3dsmax.ini (C:\Users\UserName\AppData\Local\Autodesk\3dsmax\2011 - 64bit\enu\3dsmax.ini).

    [ViewportNormalMapping]
    ViewportNormalMappingType=Qualified[/quote]
  • Lachance
    Offline / Send Message
    Lachance polycounter lvl 5
    Will you update Handplane to support UE4 ?
  • AlecMoody
    Offline / Send Message
    AlecMoody ngon master
    Someday? Sorry, I have been busy lately and I can't give anything more specific than that.
  • TheAxiom
    Offline / Send Message
    TheAxiom polycounter lvl 3
    It seems handplane keeps flipping my blue channel. I'm baking from 3DS Max.

    i4ciIlR.png

    A few questions:
    Do I need to triangulate my low poly before importing in to handplane and/or convert it to editable mesh?

    Are these settings correct? It was -Y by default, I tried +Y but it didn't change anything.

    dfCgPEQ.png
  • unknownsock
    Offline / Send Message
    unknownsock polycounter lvl 7
    Just been having a quick look at your website, I'm wanting to implement this into my pipeline for a small indie studio I work at. I know this used to be a paid app but I have no idea about now, your website doesn't seem to mention it. Can you clarify at all?

    Thanks in advance!
  • Farfarer
    It's become a free tool now.
  • unknownsock
    Offline / Send Message
    unknownsock polycounter lvl 7
    Aight, thank you.

    Wasn't too sure as in the program is says not for commercial use still.
  • Spudnik
    Offline / Send Message
    Spudnik polycounter lvl 11
    So, I'm condensing all the info on the issues I have in this thread. I did more tests and documented the artifacts/errors that were occuring, hopefully somebody can help me get to the bottom of this.

    OBJ export settings
    ACAq3zF.jpg

    FBX export settings
    N1x5mcS.jpg


    Newest tests done in Maya 2012, files can be found HERE. The only bakes working correctly are the ones done using smoothed normals and baking in Mikk T-Space natively in xNormal.
    0fZlIYb.jpg


    Older stuff:
    http://www.polycount.com/forum/showpost.php?p=2129312&postcount=17
    Spudnik wrote: »
    Running into an issue, I reformatted my PC and now xNormal (using the Handplane plugin for normal maps) is returning some strange artifacts on my normal map, any ideas?

    AO and BentNormal maps are baking correctly and the issues in the normal map along the top of the head are present in engine too.

    EDIT: Sigh, never mind, I forgot to harden the normals along the UV seams, that fixed it ^^

    SgJDr2p.jpg





    http://www.polycount.com/forum/showpost.php?p=2129588&postcount=21
    Pior: Yep, that's what's been driving me crazy. Before the reformat, for 99% of meshes I would just do a nice smoothed normal for the whole mesh and bake without issue. It's only since the reformat that I've been having these issues.

    belkun/Andummy: Just tried that, causes the same artifacts, though to a lesser degree. Tried baking the object space with both +y and -y which yields the same results.

    sbPSxVP.jpg

    http://www.polycount.com/forum/showpost.php?p=2129629&postcount=23
    Yep, I'm aware of that fact/post but the issue persists in-engine, see image below. Additionally, I've never had those issues before the reformat with the handplane plugin. If the angle of the edge could be a factor, I've also baked way harsher angles without it trying to compensate with these artifacts :/

    ZRxoXTT.jpg



    http://www.polycount.com/forum/showpost.php?p=2202968&postcount=26
    Spudnik wrote: »
    I posted this over in the Dota2 subforums some time ago, reposting here for visibility. The OS map, generated in xNormal looks fine, but when I convert it in handplane or bake it directly in xNormal using the handplane plugin, artifacts happen, especially along UV seams.

    Check out this thread for more info and examples as well:
    http://www.polycount.com/forum/showthread.php?t=133144

    Lastly, here's a ZIP with my highpoly, lowpoly and cage files obj's, I'd really appreciate any and all help on this.
    https://dl.dropboxusercontent.com/u/38223593/handplane_tangent_error.zip

    SzGXL2Z.jpg




    http://www.polycount.com/forum/showpost.php?p=2203066&postcount=223
    Spudnik wrote: »
    Marked the areas in the image below. For a more obvious representation, check posts #21 and #23 in this thread -> http://www.polycount.com/forum/showthread.php?t=133144

    lfYLfKX.jpg





    http://www.polycount.com/forum/showpost.php?p=2203127&postcount=225
    Spudnik wrote: »
    Check the other thread, it doesn't. Background is, I've been baking this way for about a year now, never had normals come out looking like this using this exact workflow. Then I reformatted my PC, something somewhere broke along the way and I cant for the life of me figure out what.

    W0Frumg.jpg



    http://www.polycount.com/forum/showpost.php?p=2204008&postcount=228
    Spudnik wrote: »
    Alright, more info. As usual, here's a link to the files I'm working with.

    Conditions of these tests:
    1) Object Space (OS) maps are baked in +X +Y +Z
    2) Tangent Space (TS) maps are baked in +X -Y +Z
    3) Software versions are xNormal v3.18.6.28468 and handplane v1.4.1

    And the results:
    OS baked in xNormal with mesh normals hardened at UV seams -> TS baked in handplane using the Source preset
    -> visible artifacts when using normal map as diffuse, but no artifacts in engine
    -> no visible seams, but seam on mirror axis

    OS baked in xNormal with mesh normals softened -> TS baked in handplane using the Source preset
    -> visible artifacts when using normal map as diffuse, and same artifacts in engine
    -> no visible seams, but seam on mirror axis

    TS baked in xNormal using the handplane plugin with mesh normals hardened along UV seams
    -> visible artifacts when using normal map as diffuse, but no artifacts in engine
    -> seams on UV edges

    TS baked in xNormal using the handplane plugin with mesh normals smoothed
    -> visible artifacts when using normal map as diffuse, and same artifacts in engine
    -> no visible seams, but seam on mirror axis


    Z4me50a.jpg



    I did another test where I split the vertecies along the edges that seemed to be the primary source of problems, in this case the edge of a blade. Both TS maps were baked in xNormal with the handplane plugin.

    KmZiPYm.jpg
  • Spudnik
    Offline / Send Message
    Spudnik polycounter lvl 11
    Somebody was asking for the UV layout, I tried to incorporate a couple of different scenarios.

    jVibB6h.jpg
  • dzibarik
    Offline / Send Message
    dzibarik polycounter lvl 10
  • EduardoMC
    Offline / Send Message
    EduardoMC polycounter lvl 5
    Does handplane automatically flip the Y channel when outputting for Source? :poly121:
  • AlecMoody
    Offline / Send Message
    AlecMoody ngon master
    Spudnik:
    There are a couple of things going on in your posts. First, the cube and sphere mirroring test:

    You need to offset mirrored UVs by one full square like this:
    http://i.imgur.com/7nf80JQ.png

    This should be done before baking object space normals since the object space normals need to correspond to the correct side of the model. I was able to grab the correct set of shells and recalculate the tangent space map and it looks like this:

    http://i.imgur.com/y3XT3GA.png

    As far as this section goes:

    "Running into an issue, I reformatted my PC and now xNormal (using the Handplane plugin for normal maps) is returning some strange artifacts on my normal map, any ideas?

    AO and BentNormal maps are baking correctly and the issues in the normal map along the top of the head are present in engine too.

    EDIT: Sigh, never mind, I forgot to harden the normals along the UV seams, that fixed it ^^"

    Adding a hard edge will fix those patches but we are trying to come up with a solution so that you don't need to worry about those edges.
  • AlecMoody
    Offline / Send Message
    AlecMoody ngon master
    dzibarik wrote: »
    Any idea what can cause these?

    That looks like an issue with overlapping UVs to me. Try to keep in mind how all the software works.

    So first off your baking software casts a ray from your lowpoly model to the highpoly and grabs the orientation of that normal which is then stored in object space. That object space normal is specific to a single angle in the world so the mirrored normal isn't the same color. If you have overlapping UVs that sit in the 0-1 square the baking software can be confused about which of the possible pixel colors to fill into your object space map. Even if your bake software happens to pick a continuous set of shells during the bake there is no guarantee that handplane will grab the exact same set when it does it's calculation.

    Handplane works by going to each pixel in the object space map, checking where it sits on the lowpoly model, and then doing the tangent space calculation. Just like in the baking software, when you have overlapping UVs that are inside the 0-1 coordinate space, handplane can't know which copy of the shell is the correct one to do the tangent space calcution with.


    EduardoMC wrote: »
    Does handplane automatically flip the Y channel when outputting for Source? :poly121:

    If you are talking about the xnormal plugin then you should leave your xnormal swizzle settings at default and the channels will be oriented correctly. If you are using object space bakes inside handplane then everything should automatically orient itself.
  • AlecMoody
    Offline / Send Message
    AlecMoody ngon master
    As a follow up-
    Here is some testing with the head model Spudnik posted that was giving him issues. I spent hours on this model a few months ago and never thought to check the UVs.

    First I took the model and smoothed the whole thing in order to stress handplane. I re-exported that lowpoly, did a bake with his highpoly and cage using the source engine xnormal plugin. As you can see it has the issue with the off color patches:
    KYeprC2.png

    Next I fixed the lowpoly UVs so they weren't overlapping in 0-1 space:
    jgjSPGd.png

    The main patches are all gone!

    There were still some issues around the edges of the shells. I overlayed the UVs to see if they were just in the padding but they are issues within the normal map.
    Big image:
    http://i.imgur.com/wjpLYus.jpg

    The first step in debugging this is to see if the problem is caused by how the normals are being calculated or if the cause is a projection error from the bake itself. The easiest test for that is to bake to object space since that requires no tangent space math and the lowpoly normals are totally ignored.

    Result:
    http://i.imgur.com/GwfDKap.jpg

    The patches are still present, meaning the issue is with the projection not the normals calculation.

    Next up I loaded up xnormal's viewer and checked out the lowpoly, highpoly, and cage together. There were a few places where the cage was too small:
    Kl7j5ef.png

    Most of the other problem spots look like they are caused by the highpoly forms that are close to being silhouette scale but aren't reprsented in the lowpoly model.
    For example, the main exoskeleton/plate surfaces that make up the outside of the model are seperate objects and there are some small gaps:
    XAAPjQH.png

    And those same gaps with the lowpoly overtop:
    gccgcy6.png
  • dzibarik
    Offline / Send Message
    dzibarik polycounter lvl 10
    AlecMoody wrote: »
    That looks like an issue with overlapping UVs to me. Try to keep in mind how all the software works.

    thanks, I offset those by 1U, now it bakes well.
  • dzibarik
    Offline / Send Message
    dzibarik polycounter lvl 10
    one more question - I've noticed these strange errors after conversion. What can be the reason? It bakes alright in Modo in tangent space.
  • AlecMoody
    Offline / Send Message
    AlecMoody ngon master
    If you email me your files I can take a look. I can probably figure it out from just your low poly model and object space map. Also, what tangent space?
    alecmoody@gmail.com
  • dzibarik
    Offline / Send Message
    dzibarik polycounter lvl 10
    @Alec I actually forgot to post the one with the error. It's in the attachment.

    Thanks for support, I'm sending you an email!
  • AlecMoody
    Offline / Send Message
    AlecMoody ngon master
    Unreal uses smoothing groups to calculate normals (unreal calculates normals and tangents when you aren't using the explicit normals/import tangents workflow). I took a look at your smoothing groups and they correspond to the areas where you are having errors. This is what smoothing group 1 looks like:
    kbDl1w4.png

    I am assuming modo uses a hard edge system like maya. Does anyone know of a reliable way to get smoothing group data out of modo?

    Can you post an image of the modo FBX export options?


    Edit:
    Doing some research I think you are going to have a very hard time getting your modo smoothing information into unreal. Your best bet is probably to smooth everything, no hard edges at all.
  • Farfarer
    MODO does export smoothing groups to FBX, but I'd recommend not using them as they're kind of broken.

    Vertex normals export properly, but there is a bug in 801 SP3 for vertex normals if you're exporting multiple items from within a group.

    UDK had some issues importing stuff but I haven't used it in years, but UE4 seems to work just fine. Although you won't get MikkTSpace out of Handplane currently.
  • dzibarik
    Offline / Send Message
    dzibarik polycounter lvl 10
    I'm using SP3 but I also have SP2 installed. Will give it a try

    @Alec thanks for looking into this, what's strange then is that UDK doesn't give me "no smoothing group" error when I import the file and I've done extensive testing with a cube before starting this project. I've tested different cubes with smoothing groups/without them - baked with a cage and with averaged projection and everything went alright - the cube with vertex normals baked with a cage, UV splits, smoothing groups, converted in Handplane was the best one obviously.

    I'm actually ok with how the final mesh with diffuse and a couple of overlaid normal maps looks like but I want it to be as perfect as possible so I'll do more testing today and post results here.
  • AlecMoody
    Offline / Send Message
    AlecMoody ngon master
    I think the file does have smoothing groups. They are just really oddly set up.

    You could fully split your verts where you want smoothing splits and then smooth everything. It's pretty hacky but it seems like that might be your most reliable way to control smoothing.
  • Goshi
    Offline / Send Message
    Goshi polycounter lvl 5
    Farfarer wrote: »
    MODO does export smoothing groups to FBX, but I'd recommend not using them as they're kind of broken.

    Vertex normals export properly, but there is a bug in 801 SP3 for vertex normals if you're exporting multiple items from within a group.

    UDK had some issues importing stuff but I haven't used it in years, but UE4 seems to work just fine. Although you won't get MikkTSpace out of Handplane currently.


    *sigh*
    So what to use then, if FBX is STILL (!) broken? I mean, we got your wonderful Vertex Normal Kit, could harden/soften edges and such... (which is STILL not "on board in MODO")

    -but what is it all worth, if you can not export it proper?

    Can you tell, which workflow/format would be working for export?
    (Maybe DAE or OBJ, I guess?)

    It´s a pitty having to switch to another app, just for setting up hard-edges and proper fbx exporting... kind of weird ;)

    Cheers
  • WarrenM
    You can export FBX from MODO and into UE4 just fine. I do it all the time. You don't get smoothing groups (as Farfarer said you should ignore them - they suck in MODO) but you can certainly set hard edges and such using his Vertex Toolkit. It works great. You don't need another app.
  • Farfarer
    Goshi wrote: »
    *sigh*
    So what to use then, if FBX is STILL (!) broken? I mean, we got your wonderful Vertex Normal Kit, could harden/soften edges and such... (which is STILL not "on board in MODO")

    -but what is it all worth, if you can not export it proper?

    Can you tell, which workflow/format would be working for export?
    (Maybe DAE or OBJ, I guess?)

    It´s a pitty having to switch to another app, just for setting up hard-edges and proper fbx exporting... kind of weird ;)

    Cheers
    As Warren said... as far as I know exporting stuff using my toolkit works with everything I've tried it with (Dota2, UE4, Unity3D, 3DS Max, xNormal, Handplane).

    Smoothing groups do actually export correctly to FBX, it's just that MODO doesn't always blend between them correctly. So what you get from your mesh in-engine isn't guaranteed to be what it looks like in MODO.

    That said, if you want to use smoothing groups, my plugin will blend between them correctly.

    There is a bug in 801 SP3 with exporting objects with custom normals if there are multiple objects in a group.

    It's actually a bug that's arisen as a result of a bug I filed to try and get meshes exporting to UE4 correctly. But that turned out to be a bug with UE4, not with MODO. So, uh, sorry about that :P I'm told it'll be fixed in SP4 (or you can stick to SP2 for exporting to FBX).

    UE4 4.7 preview will sync properly to MikkTSpace (xNormal's default) for static meshes (not skeletal ones yet) and so no longer has that bug.
  • AlecMoody
    Offline / Send Message
    AlecMoody ngon master
    Hey James,
    Can you post a link to your plugin for dzibarik/other modo users?
  • dzibarik
    Offline / Send Message
    dzibarik polycounter lvl 10
    I was using Vertex Toolkit 2.4 and Modo SP2/SP3 for this. I've tested multiple versions of the file and Modo SP2/SP3 and still get the same error. Actually if I don't triangulate the mesh there are less errors. I haven't tried EVERYTHING (like default Modo smoothing groups setup) and that's the reason why I haven't posted screens. I just can't figure out how you do smoothing groups in Modo without Farfarer's plugin.

    I'll do everything from ground up in SP2 once again just to be sure but I think it's not the reason in this case.

    and yes, everything looks peachy in UE4 4.4 :

    This is the same mesh done with Vertex Normal toolkit 2.4 and baked in Substance Designer in Mikkt space.

    XV0sn8n.png

    FiEu7bo.png
  • dzibarik
    Offline / Send Message
    dzibarik polycounter lvl 10
    OK, I've fixed this by installing Vertex Normal Toolkit 2.61 - you can download it here - http://community.thefoundry.co.uk/discussion/topic.aspx?f=83&t=77919

    Here is how smoothing groups look in Max now:

    eYGMyMm.png

    I don't know what 2.61 has fixed but it works as intended now.

    Anyway even if you have no access to Vnorm toolkit you can set smoothing groups manually by going to Geometry - Polygon - Assign smoothing group and it'll work too.

    Now everything is ok! Also it probably explains why I was having problems with smoothing in mid-poly meshes after sending them from Modo to Maya.

    HTBomSz.png
  • Farfarer
    My toolkit doesn't really do anything with smoothing groups - in fact the only thing it's capable of doing to affect them is if you tell it to remove all smoothing groups as you convert a mesh to hard edges.

    Although it does support them and will respect then if they are on a mesh.
  • AlecMoody
    Offline / Send Message
    AlecMoody ngon master
    handplane_1_5.jpg

    v1.5 changes
    Added support for blizzard engine/ Starcraft 2
    Added a button to open output directory
    Recompiled Xnormal/source plugin for version 3.18.10


    Direct link to build 1.5:
    http://www.handplane3d.com/handplane_v1_5.rar

    handplane_15.jpg
  • leslievdb
    Offline / Send Message
    leslievdb polycounter lvl 15
  • Chase
    Offline / Send Message
    Chase polycounter lvl 9
    and next up UE4...please please please :D
  • ZacD
    Offline / Send Message
    ZacD ngon master
    As of 4.7 UE4 uses Mikktspace, so that just needs to be supported.
  • AlecMoody
    Offline / Send Message
    AlecMoody ngon master
    Yeah, Unreal 4 support is coming soon. When we found out about the change we pushed it back on our priority list but It won't be too much longer.
  • repete
    Offline / Send Message
    repete polycounter lvl 6
    AlecMoody wrote: »
    Yeah, Unreal 4 support is coming soon. When we found out about the change we pushed it back on our priority list but It won't be too much longer.


    8)
  • ZacD
    Offline / Send Message
    ZacD ngon master
  • malcolm
    Offline / Send Message
    malcolm polycount sponsor
    Does someone have a link to where Epic said they will be changing how normal maps work in UE4.7 I couldn't find anything online, we got our UE4 maps working perfectly using MightyBake so I hope this update doesn't break our existing maps or invalidate our current workflow as we've had error free bakes for a long time now in UE4.
  • Farfarer
    It's in the changelist for 4.7, although it currently only affects static meshes.
  • ZacD
    Offline / Send Message
    ZacD ngon master
    New: Tangent bases for static meshes are now computed using MikkTSpace. Special thanks to JedTheKrampus for the GitHub pull request.
  • tach
    Just to clarify, besides being able to sync the normals via the upcoming UE4 Handplane support. Is there any other way to sync the normals? I.e. we still can import tangents and binormals in 4.7? Not sure what the mikk tspace support changes in regards to that.
  • ZacD
    Offline / Send Message
    ZacD ngon master
    xNormal uses MikkTSpace if you want to bake with that until handplane supports it.
  • malcolm
    Offline / Send Message
    malcolm polycount sponsor
    There is no mention of this tangent space change in the what's new features, where are you looking?

    https://www.unrealengine.com/blog/unreal-engine-47-released
  • malcolm
    Offline / Send Message
    malcolm polycount sponsor
    Nevermind I found it, this seems a bit sketch actually if it's going to auto convert everything we've already made, I'll have to test an existing asset and see if the normals are now out of sync.
1234568
Sign In or Register to comment.