Okay, so I'd love to have some way to get "proper" tangent & bitangent vectors out of 3d studio max and into marmoset toolbag, but I can't see an obvious way to do it.

I've been learning maxscript today, writing a .mesh exporter in it. About halfway through the process I realized that maxscript provides no way to access the tangent data. I'd have to generate the tangents in the script myself, which is very doable, but would defeat the purpose of this exporter really. The goal is for the exported tangents to match those max uses for baking, so that our toolbag users who also use max would see better normal map results.

Doing some reading on here and in other places it sounds like max basically has a tangent space bug. It bakes normal maps using one tangent space, and displays them using another. From what I can tell the 3point shader "quality mode" gets around this by ignoring the display tangents and somehow retrieving the "real" ones.

Perhaps I'd have more luck writing a C++ plugin? Maybe more data is exposed through that interface?

If anyone has any knowledge in this area I'd be grateful.

Im probably stating the obvious here..
If you were willing to make the 3Point package a prerequisite you could use the data created by their modifier. it generates the values and stores them in the objects vertex data channels. 3,4,5 (i dont know which channel does what)

the calculation method was described here
http://area.autodesk.com/blogs/chris...normal_mapping

Oh - so they are generating the tangents themselves? Interesting...

Thanks for the link, i'll check that out.

ask your friends at 3point studios? they just published a shader and scripts to solve max' tangent issues

I'm trying to understand this problem too. Is it only the DirectX Shader material tangent/binormals that are barfed or does the problem also show up in a render ?

Our max real-time shader plugin bypasses DirectX Shader & generates tangent/binormals on the fly real-time using the standard D3DX approach in a DirectX Manager plugin. This means no need to add a "quality normals" modifier as far as I know.

The actual data is barfed, not just in rendering. Someone from autodesk posted on these forums in the main other thread discussing the problem in great detail as people tried to find a workaround. He was basically asking what should be done in max to fix the problem properly, and it hasn't really gone anywhere (that we can see publically) since then. I really do hope it's made so that it's just toggleable as an option somewhere to calculate tangents like every other software package in existence.

Yes, but where. What data is it that's barfed ?

If you render a frame (no D3D stuff at all) using a regular setup, does that work properly ?

> Someone from autodesk posted on these forums in the main other thread

You mean this ? http://area.autodesk.com/blogs/chris...normal_mapping

What about Mental Ray / VRay renders ? Do they come out ok ?

As we calculate tangent space independently - the same way as most D3D apps out there (using the standard D3DX utility library calls), our shader should be isolated from whatever max weirdness is going on. All you need is your vertex normals set up correctly.

Please could someone familiar with the problem check whether our implementation (Infinity Shader) renders normal maps correctly.

Infinity Shader is installed as part of the Infinity Professional installation package here : http://advance-software.com/products

... then accessed via the DirectX Manager in the material rollout. It requires a (free) authorization code for now, but we're just about to transition into open beta so the copy protection will be coming off shortly.

If this works ok, I can let Autodesk know we have a solution & they can just duplicate the calculation in their stuff.

If max is calculating vertex normals incorrectly in the first place, that's a bit more tricky and probably does need a modifier to compensate.

Nothing is 'Barfed'. The renderer and viewports use a different method of tangent basis calculation. This is because Max's solution was release before there was a solid standard. the viewport shading was released later when things were more standardised. Rendering Max baked normal maps gives flawless results in scanline. I dont think the results are quite as good in Mentalray but they are good. You can achieve excellent quality viewport display of max's normal maps using the same calculation methods used in the renederer.

If you want to use Max's normal maps as is then you will need to use the same math as the renderer, this is what the 3Point modifier is supplying to the 3Point shader.

Thanks for the explanation.

So as we're currently using the D3DX method to calculate tangent space, our shader will currently be incompatible with normal maps created in max ?

Is this a problem ? I've heard that max's normal map generation has its issues and that there are better ways of generating normal maps. Is this true ? Do the other solutions generate D3DX compatible normal maps ? (xnormal, mudbox, zbrush, etc.)

Do we need to support both ? It's easy enough to drop in a secondary tangent space calculation algorithm if required.

Thats something for a programmer to answer i think. I've read allot about this subject recently but implementations is something way above my head.

The 3Point Studios guys (Perna, CrazyButcher) would be better people to ask as they have solved the problem in their own way, im just repeating what I've read

As I understand it...

The normals that Max uses to render things are different from the normals it displays in the viewport.

So a render in Max using a Max baked normal map will look fine.

In the viewport, it'll look wrong because the normals are different from the scaline renderer.

I think this problem carries over because the normals that it exports are the same as what you see in the viewport - not what it uses in the renderer.

So, the bake is fine. The render is fine. The viewport is incorrect.

You'll need to find some way of exporting a model with the normals used by Max's renderer if you want Max bakes to display perfectly in your engine. There's already a fair bit of documentation around the place for it (see the Autodesk 3DS Max blog for the full calculation).

As I understand it, the vertex normals are the same throughout.

The issue is tangent space (aka tangent basis) is "non-standard" in scanline.

Is a normal map baked in v-ray or mental ray compatible with unmodified viewport shaders ?
(I don't actually know if you can bake a normal map with mr or vray - I'm a programmer).

If you must use normal maps created by scanline, the best solution is to flag that as being the case in your exporter somehow, then have the engine recreate tangent space in that configuration when they're used rather than export binormals. That's an extra 12 bytes per vertex you could do without having to serialize.

Summary : Is this just a scanline issue ?

I think the long and short of it is you need to choose a baking app and use the same tangent basis calculation as it. If your using max then use the suitable math. You can bake in Mentalray but you can get better results faster with max. Using Scanline + the 'global super sampler' produces very crisp hi fidelity normal maps.

Mray and Vray are not really an issue they are not used widely for normal map baking.

Anywho maybe its a good idea to start a thread for this were kind of hijacking the OP's thread

This is on topic. The OP wants to export scanline compatible tangent vectors. The algorithm has been posted so he has a solution - grab vertex normals & tex coords & run them through that algorithm. A secondary solution is to use "standard" tangent space to avoid the necessity to do this in an exporter or engine. This can be achieved by avoiding scanline bakes.

> I think the long and short of it is you need to choose a baking app and use the same tangent basis calculation as it.
Yes.

What I'm trying to understand is if scanline bakes have sufficient merit to be worth supporting. It sounds like you're saying they do.

After looking over this link in detail:
http://area.autodesk.com/blogs/chris...normal_mapping

I can't see what max is doing differently. That algorithm is almost, line for line, identical to the one marmoset uses to generate tangents. Maybe there is a small difference somewhere, I'll keep looking, but I don't see anything substantially different.

This was released soon afterwards. apologies for not posting it aswell.
http://area.autodesk.com/blogs/chris..._plot_thickens

Thanks fletch.

Does anyone have an example mesh + scanline generated normal map that they could share to help test a solution to this issue ?

What I'm looking for is something which renders ok, but which looks wrong when displayed in the viewport or a standard DirectX shader.

I don't need a diffuse - just the low poly .max file (any version) + normal map.

Thanks in advance for any assistance you can offer.

If you can help, please email an example to steve at advance-software.com or post a link here.

Any assets provided for testing will only be used for that purpose and any emails received will be treated confidentially.

There's another issue I witnessed recently with my modo pipeline exporter ...
if you're using a max version prior to 2010 (i still use max 9) it also overrides the vertex normals with its own algorithm.

So basically, if you import object with standard vertex normals (based on its geometry), you won't get the same vertex normals inside max and especially if you export .. max smoothing algorithm disregards the tri distribution, and it works in a non standard way. (computes smoothing based on vertex connection nodes)
I think 2010+ still does it, but you have out of the box tools to recalculate normals.