I keep seeing this question come up time and time again, what causes waviness? How do we fix it? I feel the reason it keeps coming up, is a fundamental lack of understanding.

To get to the root of the problem, we have to look at how normals are averaged, and thus, how the actual rays that are doing the "baking" are cast, and in what direction.

I whipped up a quick test scene here, to show a couple different types of "waviness".

First off, the highpoly, notice that the right two meshes are the same, and the left 6 are also the same, the big difference being a slope instead of a hard 90 degree angle at the ridges.


Now, we'll take a look at the lowpoly meshes. From the averaged vertex normals here we can see the beginnings of our problems, the vertex normals here are far from flat, so any and all differences between the high and the low will become exaggerated. You should be able to learn to spot potential ray trace problems, "waviness" or "skewed details" simply by looking at the lowpoly mesh normals.


And the resulting bake;

Quickly we see that even a gentle slope with our ridged details produces a much different affect, and much more acceptable waviness. A1 and B1 are really worst case examples, and you should design and model to avoid these sort of shapes, unless you have the geometry budget to accurately match the shape of the high.

A3 shows how we can account for "skewing" type details, by adding control loops on either side of the shape, B4 shows what happens when we do not do this.

B1-B4 basically show that we simply do not have enough geometry, and unless this object was going to be seen very small on screen, it simply looks terrible. No amount of tweaks or repainting or cage hacks will fix these meshes, they are hopeless. At least from the side view, from 3/4s view, the waviness actually helps to make certain areas more presentable.


As we break down what the mesh normals are doing, and as a result, the direction the rays are "looking" when we bake the mesh, we see the problem. Along these hard angles the rays are being cast out at a 45 degree angle from the surface.


Now when we look at the difference between the high and the low, the problem becomes very obvious. The greater the difference in curvature, the more problems you're going to have.


Now, the other common problem, skewed detail; is really the exact same problem, just with a different type of geometry.

Here we see that adding these supporting loops flattens out the mesh normals along the cylinder, resulting in a nice, even bake. Without the loops, the normals are averaged along the length of the mesh, and the projection is variable, resulting in "skewed" details along the length.


So, the next time you go looking for a "software fix" or a "workaround" to these problems, first stop and think: Is my geometry to blame? Am I able to simply match the low to the high more accurately? Can I simplify the design to work in a more reliable manner?

Because every software hack that we do, cage tweak, every time we paint the "errors" out in photoshop, every time we do a bunch of different bakes and combine them, we must remember that:

A. The work must be redone every time we change the mesh.
B. The work must be redone every time your boss requests you to change the mesh.
C. The work must be redone by some other poor slob, when you're unable to do it and they're forced to, requiring them to entirely debug/rework your model back to an acceptable state, or attempt to preform the same magic voodoo dance you had to perform to get a good bake. - This one here is key, you owe it to your fellow man to properly understand this stuff.



Oh, and another thing. Even if your lowpoly's normals look like A, your Cage/Projection mesh normals look like B. That is unless of course your bake is set up poorly(either using "offset" in max, match using "surface normals" in maya, or using the ray distance in XN instead of a proper, welded cage). If your cage is not averaged, you will get seams on all of your hard edges, as the normals will be facing perpendictular along those edges, and cause gaps in the projection.

Brilliant. Clear and concise explanation. Sticky it, wiki it, put it on the News page. Your next installment needs to be MOAR GEO, then 90% of the newb questions will be answered.

To reiterate what fletch said in another thread, they really need to pay you for this shit.

Originally Posted by Artifice

Brilliant. Clear and concise explanation. Sticky it, wiki it, put it on the News page. Your next installment needs to be MOAR GEO, then 90% of the newb questions will be answered.

To reiterate what fletch said in another thread, they really need to pay you for this shit.

Thanks, maybe I'll condense everything I know and write a book someday. =)

3 Point video tutorials! Between yourself and Perna and Vahl you'd make a fortune Plus it would be good information.. there is a lot of mediocre half correct training material floating about these days.

Originally Posted by r_fletch_r

3 Point video tutorials! Between yourself and Perna and Vahl you'd make a fortune Plus it would be good information.. there is a lot of mediocre half correct training material floating about these days.

I agree with this!

Thanks for the brief explanation on those, they're always a bitch.

EQ be my valentine


seriously thanks for this man, been going through the wavy cylinder thing recently and that has just helped me so much, I concur that you 3 point folks should get some video tutorials out there.

Registered just to say thanks for this. I'm new to 3D modeling, this post condenses info I had sort of figured out on my own but didn't have 100%. Definitely going keep it handy for next time I have bake issues.

Good stuff.

Alright, so I took a little time to do a more practical example. You guys have probably seen me write "model for your low" before, so I'm going to show what I mean by that.

First the high:
You'll see A is similar to the worst case example again, but is a very common type of model to see. Lots of straight edges, little regard given to how well this would translate to a low. B and C are the same(to show two different low methods), you can see here lots of low-friendly slopes, on the problem areas like the front of the barrel that we would likely add extra geometry to help keep it from skewing I've modeled a nice fat bevel there, so our extra geo will actually go towards the quality of the model.



Now, onto the low. A cool thing about the "more friendly" model is that it is actually less geometry, and is more interesting. We can add a little more geo to the larger cylindrical shapes(which again will help waviness) and STILL come out under the tri count of A. For everyone who always thinks you've gotta use 2x more geometry to get good bakes, noooo, you just need to understand the issues and plan accordingly.


Bakes, besides the obvious stuff, take note of how the high and low "bullet hole" shape is model in the front of the barrel, in the high we have a nice fat bevel in the center, and in the low we've got that nice support bevel as well, this results in a much better bake, in the A example, where the high has a simple indent and no supporting geo in the low, this area barely shows up.

Also notable, while B/C are clearly better, both of these look fine at most angles aside from a completely straight side view, so, try not to be so anal about minor waviness.



Ok, so we see that modifying our design enables us to get better bakes with less geometry, but not only in regards to waviness.

The "friendly" mesh(s) here:
1. Look better
2. Have less waviness
3. Have less skewed details
4. Use a simpler uv map with less uv island, this easier to work with + better performance
5. Use less extreme normals, thus resulting in less potential smoothing errors, which again allows us to have a simpler uv map, as we do not need to detach as many uv islands to work well with hard edges - This one is pretty huge.

So, even if all things were equal, and you could get the same/better results as far as waviness on A as we do on B/C, it simply isn't worth your time making models like this, for the variety of other advantages that proper understanding and planing can bring to your mesh.

Originally Posted by EarthQuake

That's hot.

Nice thread, EQ. Thanks for your guidance.
Why not show the flats or at least the UVs, since you mention them?

Sure, normals:



So, the A method has 5 uv islands as apposed to 4 for B/C. If you had to use hard edges to account for the normals on method A, you'd probably be up to about 10 uv islands, maybe more. If you had to use any hard edges on B/C, you'd simply use them on the natural seams, no big deal.

[Edit] Doing the math, if you had to send A into an engine/app that wasn't properly synced as far as tangent basis goes, you'de have to have a uv seam at every 90 degree angle, which would = 13 uv islands. Ralph

Thread full of win, once again.

I'm always so tempted to slope cylinders like this on real world copies (on weapons for instance), but usually end up deciding to go for reference accuracy instead, and add moar geo to make up for it... Could probably slope the suckers and no one would tell the difference though.

On a related note, do you ever modify your vertex normals? I do it alot these days and I really like what I can achieve with some simple changes (cleaner bakes, more easily reusable normal maps for modular pieces etc), mostly useful for env stuff though, less so for characters/weapons.

I dont do anything but basic hard edges, no manual vertex editing, and even then I try to just use scripts that assign hard edges to uv borders, super fast. I switch between modo/max/maya a lot so I just try to keep it basic.

One question though, can´t we just paint the wavy lines straight in photoshop?

Not that i´m bitching about waviness, just curious.

Originally Posted by SpeCter

One question though, can´t we just paint the wavy lines straight in photoshop?

Not that i´m bitching about waviness, just curious.

This pretty much sums it up:

Originally Posted by EarthQuake

So, the next time you go looking for a "software fix" or a "workaround" to these problems, first stop and think: Is my geometry to blame? Am I able to simply match the low to the high more accurately? Can I simplify the design to work in a more reliable manner?

Because every software hack that we do, cage tweak, every time we paint the "errors" out in photoshop, every time we do a bunch of different bakes and combine them, we must remember that:

A. The work must be redone every time we change the mesh.
B. The work must be redone every time your boss requests you to change the mesh.
C. The work must be redone by some other poor slob, when you're unable to do it and they're forced to, requiring them to entirely debug/rework your model back to an acceptable state, or attempt to preform the same magic voodoo dance you had to perform to get a good bake. - This one here is key, you owe it to your fellow man to properly understand this stuff.

Certainly you can paint it out in photoshop, or tweak your cage(max) or render multiple bakes and composite them in photoshop, but be aware of the drawbacks. Its better to fully understand the problem, and try to fix it in the mesh first, as painting it out for instance becomes very difficult in more complex situations.

If you've got a simple cylindrical mesh, you know you'll never make any changes to it ever again, you can really do whatever you want. I never like to give advice that isn't sound technically and applicable to a wide variety of situations however.

As shown in a few other threads, painting out waviness can also result in a worse looking asset in some cases as well, depending on the viewing angle. I might do some more examples or just dig up those threads later.

The point of the thread isnt so much to say "ALWAYS DO THIS!" or "NEVER DO THAT!" but to share information and understanding, and challenge the way we work and think, so that you can come to your own conclusion as to when it is or isn't appropriate to do extra tweaks or whatever.

Excellent explanation.

Argh sry Earthquake , so you meantioned it
I just read through your text while doing something else, so i missed that part, my bad.
I know that it can happen, that it looks worse in some situations if you paint it straight.

But in this example i think it could get rid of waviness without screwing up,
but like you said there is no exact way which can be applied generally and it´s just an example

Great write up EQ. Extremely informative and your write ups are always especially great because its a never a you should do this, or you shouldnt do that. Its always just great, solid information for people to learn from and understand why somethings happens or differing ways that an issue or problem can be tackled.

I for one second the book idea ha

Awesome thread EQ.

Originally Posted by SpeCter

Argh sry Earthquake , so you meantioned it
I just read through your text while doing something else, so i missed that part, my bad.
I know that it can happen, that it looks worse in some situations if you paint it straight.

But in this example i think it could get rid of waviness without screwing up,
but like you said there is no exact way which can be applied generally and it´s just an example

Its a question that was bound to be asked, and certainly a good question to ask as well, as i'm sure many will be wondering the same.

There was something else what you could do to get rid of the waviness at the end of the barrel(it a cage tweak again though, but can be applied with not much work involved)

If you bevel the egdes at the end to get rid of 90 degree angles you should/could tweak the cage to go straight(let the beveled part have the same diameter as the rest i guess)
And the wobblyness in this part should be nearly gone in many cases.

What exactly is a "welded cage"?

Edit - Nevermind, just read the wiki

Originally Posted by SpeCter

There was something else what you could do to get rid of the waviness at the end of the barrel(it a cage tweak again though, but can be applied with not much work involved)

If you bevel the egdes at the end to get rid of 90 degree angles you should/could tweak the cage to go straight(let the beveled part have the same diameter as the rest i guess)
And the wobblyness in this part should be nearly gone in many cases.

Cage tweaks can be used in some situations, however they do not work the same in all apps, need to be redone when editing the mesh/doing revisions as mentioned about. I'm not terribly interested in discussing band-aids to fix the symptoms of the problem, moreso the actual problem itself. =)

I think most people understand you can edit your normals in photoshop, or you can tweak your cage, or whatever other hack you want to do to get to the end result. However the mentality that this is the way to fix things, and that these problems are just because of "the way it works", this mentality that people seem to have is a huge part of the problem. And is a big reason why we see over and over, people asking the same question, how do I avoid X in X situation, because someone hasn't told them the exact little hack they should do there....

When we thoroughly understand the situation, critical thinking can be used to find a working method for any situation. When we rely on doing "X" for "Z" situation because we read it on "Y", or because "Steve" told us that is the way to do it, it becomes much more difficult to problem solve.

Yeah right, giving beginners a foundation why exactly these problems happen should be the first thing to do

I just mentioned this fix/trick because in most cases it´s just like selecting the
"end"parts of the cage and scale them up which takes very little time to none.
But like you said that only works for programs where the cage also represents the direction where the ray is coming from not just the distance)

Originally Posted by EarthQuake

Oh, and another thing. Even if your lowpoly's normals look like A, your Cage/Projection mesh normals look like B. That is unless of course your bake is set up poorly(either using "offset" in max, match using "surface normals" in maya, or using the ray distance in XN instead of a proper, welded cage). If your cage is not averaged, you will get seams on all of your hard edges, as the normals will be facing perpendictular along those edges, and cause gaps in the projection.

I did not know this, but it makes complete sense, and explains the strange offsetting in a lot of my hard edges.