FBX support for CryENGINE3

[Ruan]

Hi guys, Ruan from Crytek here; been working on this for just over a week now and I figure that an artist-centric forum is probably the best place to get feedback on this!

Because plugins are so last year, I've been developing a converter that takes an FBX file and spits out a nice CryENGINE-ready .cgf file. As well as removing the requirement to use plugins, this also means you can use your favourite 3D software rather than being forced into Max or Maya!

At present, it only supports static meshes, a fair bit more effort will be required to get rigs and animations etc working. The main caveat is that you can currently only use one material ID; everything else like UV mapping, smoothing groups etc work just fine.

Excuse the programmer art, but here's a test I created earlier using Max 2013 (for which a plugin isn't currently available):

http://i.imgur.com/WCw9g.jpg

If anyone has any suggestions for how this could added to your workflow in the best way, please just shout :)

[Computron]

Awesome this should hold up for me until, I assume, the next SDK will have an updated exporter for Max 2013?

Is it going to be possible to export custom vertex normals into Cryengine soon in any way?

[Ruan]

Yes, the next release should have the 2013 plugins. Being neither an artist nor part of the tools team, I'm not entirely sure on custom vertex normals. In what instances would you use them?

[bugo]

I HAVE BEEN WAITING FOR THAT! Thanks Ruan.

Some quick questions:

Will you make it support different materials some day? It's quite needed...
Does it support color verts?

[Ruan]

Glad people find this useful :) Yeah, multi-material support is high on the todo list, I just want to get a working version released first so I can see which DCC apps play nicely with it. It doesn't support vertex colours right now but that's probably trivial enough for me to add tomorrow before (at least hopefully, fingers crossed) the first release.

[|MM|]

@Ruan
great effort indeed, I hope it's going to be usable soon.

I'm not entirely sure on custom vertex normals. In what instances would you use them?

http://wiki.polycount.com/VertexNormal?action=AttachFile&do=get&target=TreeMakerScript_plus_NormalThiefScript.gif

Custom vertex normals would be extremely useful despite vegetation shader partially alieviating the problem
of blocky shading (but only partially).

I'm not sure it's something that can be done with an exporter is it?

[Ruan]

Ah I see. I'm not sure whether the current normal system is due to the plugins or the engine itself, but I'll see whether that's viable.

Meanwhile, looks like we're good to go for a basic release (much) later today!

http://i.imgur.com/0rR82.png

Oh, and this is definitely my project mascot:

https://dl.dropbox.com/u/53949696/CE3/fbx.png

[e-freak]

awesome work dude :) one question from your converter - how do you decide where the FBX should compile to in your build? (like Objects/Library/Architecture/Interior/bla)

[Ruan]

When you hit export, you get a save dialog :) This is just a test run really, I'm investigating proper ways of making the process more efficient.

One idea I had was to stick a sort of source directory in the engine root which could be mirrored to Game/Objects. So if you placed blob.fbx into ArtSource/Library/Misc/, you'd end up with Game/Objects/Library/Misc/blob.cgf.

That could be an automatic process, if I made the tool watch the filesystem, so you could just drop your FBX files with the tool running and get your files instantly converted for the engine.

[Computron]

Custom vertex normals can be really usefull for getting really smooth shaded low poly models and clean normals bakes as well.

It the one DCC feature I want to be able to export the most.

[cptSwing]

Looks great man!

[Ruan]

Tentatively saying that vertex colours are in:

http://i.imgur.com/oD8LN.jpg


Edit: Yup, the blend layer appears to work fine:

http://i.imgur.com/mDGUM.jpg

[e-freak]

One idea I had was to stick a sort of source directory in the engine root which could be mirrored to Game/Objects. So if you placed blob.fbx into ArtSource/Library/Misc/, you'd end up with Game/Objects/Library/Misc/blob.cgf.

That could be an automatic process, if I made the tool watch the filesystem, so you could just drop your FBX files with the tool running and get your files instantly converted for the engine.

This. A million times! Even better - if you could have the FBX within the same directory and it compares a timestamp with the last time the tool did the scan :)

[biofrost]

This would be great! My next project I was planning on using CryEngine3 but had been avoiding it because I heard it was a bit messy getting models from Maya to Cry3.

Can't wait to see this released!

[Ruan]

This. A million times! Even better - if you could have the FBX within the same directory and it compares a timestamp with the last time the tool did the scan :)
That was my first thought actually, but I think separating them is possibly easier for using version control. Rather than forcing designers/coders to check out source (ish) files too, you can instead just make the ArtSource directory a separate Git/Perforce/whatever repository and have artists commit their source files there.

Another thing I want to do in that sort of vein is investigate conversion of proper source files, similar to how Unity does it, i.e., taking a .max file and invoking Max to export to FBX which is then processed.

Anyhows, copypaste from CryDev (http://www.crydev.net/viewtopic.php?f=315&t=94539) for a very WIP test release:

You can download the v0.1 release candidate here: https://github.com/downloads/returnString/ManagedFBX/CEFBX-0.1rc.zip

Important Info for Users
Run CryEngine.RC.Frontend.exe to start the application.
This application uses version 4 of the .NET Framework. Please ensure your version is up-to-date.
This application is untested in a release environment, so I can't guarantee it won't crash (but it hasn't in my debugging sessions!).
At present, your model should be at the root of the scene hierarchy (i.e., not be the child of any other object). The application doesn't currently merge nodes.

Features
UV maps: http://i.imgur.com/IPt85.jpg
Smoothing groups: http://i.imgur.com/PZZHM.jpg
Vertex colours: http://i.imgur.com/oD8LN.jpg & http://i.imgur.com/mDGUM.jpg

Limitations
Only handles a single mesh
The mesh must be at the root of the scene hierarchy
No multi-material support right now

This is not an official Crytek tool, please don't blame me if your PC spontaneously combusts, etc etc :D

[biofrost]

Awesome thank you so much. I managed to get one of my models in but with the other I kept getting an error saying. no model in FBX scene. Not sure why this is happening with some models but not others.

Either way great tool so far!

[Ruan]

The model isn't parented to anything, is it? At the minute the converter only scans the top level of the scene and just grabs the first mesh it finds, so things like meshes composed of several objects won't work.

[cptSwing]

^ Does that count for layers as well (in max?) - and you sir, are a star. Out with the plugins, all hail FBX!

[Ruan]

^ Does that count for layers as well (in max?) - and you sir, are a star. Out with the plugins, all hail FBX!
I'm not actually sure, I just traverse an entire scene in FBX, not sure how that maps to a Max scene. Provided it's in the normal scene graph, I suspect it should still work...

The end result of this should handle multiple meshes, hierarchies etc just fine though, but right now I'm taking baby steps and ironing out issues along the way :) (read: I've never worked with mesh programming before)

Edit: Been doing some more testing/actually learning how to do things in Max and I'm pretty happy that the blend layer works just fine:

http://i.imgur.com/0xXl2.jpg

[choco]

@Ruan
great effort indeed, I hope it's going to be usable soon.



http://wiki.polycount.com/VertexNormal?action=AttachFile&do=get&target=TreeMakerScript_plus_NormalThiefScript.gif

Custom vertex normals would be extremely useful despite vegetation shader partially alieviating the problem
of blocky shading (but only partially).

I'm not sure it's something that can be done with an exporter is it?

The default exporter already supports edited vertex normals :)

Amazing work btw ruan, I've been following the progress for the last few days on crydev, I'm eager to get my hands on a finalized build.
Keep it up!

[|MM|]

@The default exporter already supports edited vertex normals

Are you sure? Can you prove that?

I don't have access to 3ds max 2013>, when I was testing it on the trees with custom projected normals it didn't have any effect.

[Ruan]

I've got a local change that I think might work if the engine does in fact support custom vertex normals; does anyone have a sample .max/.fbx file that uses custom normals to hand?

In other news, got rudimentary physics working, need to add support for proxies later :)

http://i.imgur.com/nsUA9.jpg

[|MM|]

I'll try to provide something in a minute, but maybe it would be better if
you applied 'Edit Normals' modifier to an object and e.g rotate the vertex normals randomly.

[Ruan]

Well, clicking a button (I think it was unify?) with the Edit Normals modifier and this local change certainly did something...

http://i.imgur.com/9Y4LP.jpg

Edit: Hrm, this works without my change. Interesting.

[|MM|]

What it did is average the normals as if applied to one smothing group.
It doesn't show anything.

Do something like this.
Create a cube, by default its normal look like this.

http://img98.imageshack.us/img98/1790/sddssd.jpg

Select them ,rotate them as you wish.
http://img812.imageshack.us/img812/7519/sddssdsdsdd.jpg


r_ShowNormals in the editor's console

[Computron]

Unify just puts everything into one smoothing groups. The vertex normals are still calculated by the engine. You want explicit vertex normals.


The default exporter already supports edited vertex normals :)

Amazing work btw ruan, I've been following the progress for the last few days on crydev, I'm eager to get my hands on a finalized build.
Keep it up!

It does? I can't export to cryengine through the official tool right now as I am on max 2013. And google shows nothing about cryengine support this.

[Ruan]

Ah I see, still getting the hang of this 3D lark!

In that case, the normals are certainly carried over:

http://i.imgur.com/xCn3J.png

http://i.imgur.com/QWPpU.jpg

...but to what end I'm not sure.

Gotta add support for material IDs and then 0.2 should be ready :)

[|MM|]

Both are great news to me. :)
I'm not sure still though if the shading in CE3 makes a lot of use of vertex normals.
It's even visible on the 2nd screen, maybe you could try bending them some more, and rotating just as single normals not per vertex?

[Popeye9]

Been using this on a couple meshes and so far has worked great. Looking forward to the updates

[Olli.]

So it exports the normals but doesnt use them?

oh crytek.

[Computron]

shading looks the same though.... wierd.

[seith]

I heard it was a bit messy getting models from Maya to Cry3.
It's actually quite simple if you use MayaCE3 (http://seithcg.com/wordpress/?page_id=973)... :poly121:

[Ruan]

I'll vouch for Seith's work seeing as he gave me some pointers developing this ;)

Promised myself I'd work on the pipeline a bit today, and here we go, a first pass attempt (this is in no way/shape/form the final interface, just an idea of what I want it to do!):

http://i.imgur.com/hG02R.png

I also figured out the mystical matrices (I seriously hate maths) required to bake transformations into vertex positions, so now custom pivots etc work:

http://i.imgur.com/qBCDd.jpg

[|MM|]

Waiting for exporting material indices and we can get this baby started.

[Paradan]

just wanted to say thanks for making this.

as Autodesk products aren't accessible to me (non-student/don't have money to burn), I've always wished that SDKs would release stand-alone exporters instead of plug-ins.

it's great to see something like this

[Ruan]

Waiting for exporting material indices and we can get this baby started.
Haven't abandoned this feature, I swear! :P Had some issues with extracting material IDs until I realised that Max doesn't export indices until your scene actually has a material instance... much rage when I figured that out. Now I just have to work out how material indices are represented in Collada and it'll be ready.

In the meantime, been playing around with the new pipeline tool (batch builds, woo!) and modularity:

http://i.imgur.com/rcpkl.jpg

[|MM|]

Haven't abandoned this feature

Not accusing you of that:), but the thing is when you have the material ID's exported
most of the environment stuff will be possible so it's a bit of a milestone to me.

With the materials and collada maybe it's really worth talking to angjminer (CryBlend) cause he's got that worked out.

Actually in CryBlend material indices are determined in materials' names
as material properties are, which forces having a material.(but takes a bit of time to get used to)

Also I wonder how you'd make materials properties to be assigned to the materials.
I mean in the 3d app, if you plan on supporting this anyway (material creation).

[Olli.]

Im really sorry, but I still fail to see the gain you get from using this, as opposed to the official Max exporter.

Now if we ignore possible max compatibility issues, and assume that your max version is able to use the CryExporter, then can someone explain to me the pro's and con's of using this FBX exporter instead?

Im sorry im not trying to slander this project in any way, I just thought i'd probably get a more understandable answer from someone who has a firm grip on asset create pipelines and such.

No im not Canadian, I just say like to say sorry a lot.

[Ruan]

If you have a supported Max version and you're happy with the import process, then by all means stick with it. I do hope to provide some extra value later on down the line with the new art manager tool (automatic conversion of source files inc. Max and PSD files à la Unity), but for now, yes, it's limited in comparison to the official exporter. It's also a hobby project made by someone with no prior experience in mesh programming, you get what you pay for ;)

The real aim of the project though is to enable support for anyone who can't afford (or refuses to acquire via other means) a supported DCC app. FBX is the closest thing that we have to a de facto standard*, so it seemed like the logical choice to enable support for the maximum number of users.

*if you're reading this, DCC app developers, for the love of god just agree on things for once; surely, between some highly intelligent people, we could agree on which way is up?!

[seith]

... for the love of god just agree on things for once; surely, between some highly intelligent people, we could agree on which way is up?!
Of course, it's Y. Err... No wait it's Z...!! :poly124:

[cptSwing]

I like the more open support of FBX.. works just fine in UDK, no reason not use it for Cry.

[Ruan]

Of course, it's Y.
Burn the Maya user!

I like the more open support of FBX.. works just fine in UDK, no reason not use it for Cry.
No reason except Autodesk's excuse for an SDK >_> Although, in parallel, I'm developing a .NET wrapper for the format which should make life easier for anyone else looking to create an FBX pipeline in the future.

[|MM|]

Ok, I've tested the converter with 3ds Max and Blender on a simple mesh.
Everything seems to work rather fine.

Talking of Blender as far as I know it doesn't support mat id's at all.

I don't know how fbx stores materials, maybe the exporter assigns
the id's in some way but there's no control over it.

There should be an option to assign the id's and physicalization type in the UI
of the converter IMO.

PS: I'm writing all this since I'd just really like to use it right now, cause I'm making a new environment
and all I need is the id's and a possibilty to export a mesh with no
default phys or a custom proxy for the sake of speed.

[Ruan]

No material IDs? Do you create multi-materials by assigning materials directly to polygons then? If you have any sample Blender-created FBX files I could take a look at, that'd be great :) (between work and projects like this, I can't invest too much time in learning extra software)

Agreed that physicalisation options etc need exposing but I'm not sure how to best do this. I could use material names, but that feels clunky. I also can't depend on app-specific material params like our in-house plugins do.

Perhaps a checkbox for physicalisation, and then the app would first try to use the mesh's proxy (a child named "proxy"), then otherwise just physicalise the render geometry.

And the obligatory progress shot:

http://i.imgur.com/0zU6b.jpg

[|MM|]

No material IDs? Do you create multi-materials by assigning materials directly to polygons then?

Yes.


Here are two cubes one with one material the second with two.

https://www.dropbox.com/s/47rr8069il37eqd/blender_fbx.zip?m


Perhaps a checkbox for physicalisation, and then the app would first try to use the mesh's proxy (a child named "proxy"), then otherwise just physicalise the render geometry.

Not sure what you mean here. There's 5 types of physicalisation right?
So each material should perhaps have it's dropdown menu to choose the type it needs. (phys_None by default)


Also as I said the CryBlend material naming convention takes some time to get used to,then comes pretty natural, but if it can be done with the UI then whatever.

It goes like this
| libraryname__ID__materialname__phystype |
So for example
tree__1__canopy__physNone
tree__2__trunk__physNone
tree__3__proxy__physProxyNoDraw

[Ruan]

Thanks! Fortunately, material IDs for Blender objects with more than one material appear to export just fine:

http://i.imgur.com/BcoGW.png

And this works just fine with my FBX library:

http://i.imgur.com/kThcC.png

So now this just needs mapping to Collada...

I'm familiar with the 'CryBlend' naming conventions because those are actually what we use internally for our Collada variant. :) However, in my quest for usability (futile, I know), manual material naming feels clunky and not overwhelmingly artist-friendly. I'll have to look into the best way to achieve this a bit more...

[Ruan]

Apologies for the bump, but I made a breakthrough and got material IDs working!

http://i.imgur.com/JY1sW.jpg

[Olli.]

those are some nice walls you got there!

I think we might just make an artist out of you yet.

[biofrost]

Awesome progress! Any idea of when the next release is?

[Ruan]

lololli. I hope not, I spend enough time in Visual Studio, I don't need more reasons to not leave my cave.

@Biofrost: The code's all in place and committed, so I've just gotta add a checkbox for physicalisation (no proxy support yet, that's coming next); I'll hopefully be releasing this tomorrow unless anything explodes in the meantime :)

[cptSwing]

awesome :)

[Ruan]

And we're live with the second release! Updated the thread on CryDev: http://www.crydev.net/viewtopic.php?f=315&t=94539

You can download the latest build here: https://github.com/returnString/CryEngine.RC/downloads

Summarised (and apologies once again for the 'art', but this is fun!):

https://dl.dropbox.com/u/53949696/CE3/FBX/range.png

Usage Guide

Extract the contents of the .zip file to your CryENGINE/Tools folder.
Run CryEngine.RC.Pipeline.exe to start the application.
Save your FBX files to CryENGINE/Assets, using the same hierarchy that you'd like them to use in Game/Objects (e.g., C:\Dev\CryENGINE3\Assets\Props\test.fbx will be saved as C:\Dev\CryENGINE3\Game\Objects\Props\test.cgf)
To export individual files, right click the file in the list, and then select "Export". You can do the same for specific folders to avoid full rebuilds.


Features

UV maps
Smoothing groups
Vertex colours
Multi-materials
Basic physicalisation


Limitations

Only handles a single mesh
The mesh must be at the root of the scene hierarchy
Some 3D applications use different unit scales; use the scale factor to work around this
Some applications may require a material to exist in order to export material IDs for submaterials


Licence
Both the source code and builds for CryEngine.RC and ManagedFBX are available for use under the following terms:


The software is only available for non-commercial entities, and may not be relicensed.
You must make changes available to the public.
In the rare event that we meet, you owe me a pint.

[cptSwing]

Works well on some meshes, and not at all on others, though I blame the finicky dae conversion and not your tool.
Half of my meshes don't convert because of vertices allegedly having the same coordinates - and outside of actually deleting parts of the geometry, or slapping on a ridiculously distorting noise modifier, nothing works.
In Max, no STL errors show up, Xview is fine, no overlapping UVs, nothing to weld within sensible ranges, and there's usually nothing (wrong) at the coordinates specified within the export log.. sigh :)

[Ruan]

Yeah, it's hard for me to tell what's caused by the RC/engine and what's caused by the converter :/

Unless I find some obvious flaws in my application, I'll discuss this with the tools team and see if anything's up.

Also, 0.2.1 is out which has a fix for UV coords being in layers other than 0. Probably not relevant for most users though :)

[cptSwing]

Looks to me like the classic mesh errors the CryTools would spit out as well. Since I'm using max 2013 though, I can't test that ;-)

[cptSwing]

Hm, a little heads-up: Using Max 2012 and the Cry exporter, getting the disputed meshes into engine works flawlessly.. so maybe the collada part of your tool is doing strange things? :)

[Ruan]

Hey, thanks for confirming that! :) I've been meaning to get around to installing 2012 but I've been caught up in other things. This is going to be some fun debugging, I'm sure...

One query; is this with pre-triangulated meshes, or meshes with quads/ngons? I'll be the first to admit that my code is possibly not working entirely as expected but to the best of my knowledge I just list mesh data exactly as provided. There is, however, an exception to this: I do perform a rough triangulation pass using the FBX SDK if a mesh isn't already triangulated. I would assume this uses a similar algorithm to whatever CryMaxExport does to handle quads, but it's worth me investigating.

[cptSwing]

Hmm, I can't give you a definitive answer on the triangulation question, but I usually have an Edit Mesh modifier sitting on top of my objects, and I also usually have "Triangulate" checked on when exporting to an FBX.

The files were the exact same FBX models that I tried using with your transmogrifier, so hmm. Could send you a mesh tonight, if it'd help.

[Ruan]

Yeah, if you have a reliable test case for the issue, feel free to send it over and I'll take a look at it :)

[Ruan]

I've probably fixed the issue; 0.2.2 is available now and in theory handles all vertices just fine: https://github.com/returnString/CryEngine.RC/downloads

[choco]

For those who are still wondering if edited vertex normals alters the shading in CE3. I did some tests on some bushes and it does work like a charm. I'm using maya and MayaCE3 plugin tho.

[|MM|]

@choco

We got that figured out, they're exported through FBX exporter, I installed max 2012
and they're exported too.

If they make any difference to shading is a different thing though.

[Computron]

For some reason my meshes normals are getting flipped when I use the exporter, I have to flip them before exporting. It is a rather large mesh, maybe that has something to do with it:

http://i.imgur.com/PmIUj.jpg

[bugo]

Is it supporting different materials in Maya as well?

[bugo]

I'm getting a msg "Failed to compile file: C:\cryengine_3.4\Assets\Props\arch01.fbx"
Is there a way to see what's going on with this mesh? Is it because I have 5 sided faces?

edit: yes it was, I made it work not having 5 sided faces. Would be awesome if you get that info on a log or something, or if automatically triangulate it for you.

[Ruan]

Hi guys, sorry about the lack of responses, my email subscription to this thread seemingly died :/

The normals bug is very strange. I think there have been cases before where symmetry modifiers have screwed up normals even using the official plugins, but I'll look into this.

With regards to materials: I grab the material IDs from the FBX format which is, at least in theory, universal. If material IDs aren't working on a mesh, please attach it so I can take a look at it.

I already perform triangulation (https://github.com/returnString/CryEngine.RC/blob/master/CryEngine.RC/Fbx.cs#L38-42) on meshes, seems odd that it failed. If you run the Frontend tool rather than Pipeline, you'll get the full RC log. Pipeline is very much a WIP :)

[AdanR]

wow works nice! thanks Ruan!, moving to modo workflow lol!

[pior]

Sorry if this is a bit off topic, but this seems to be the best place to ask.

Is there any official word regarding custom vertex normals support with the regular Max exporter ? The current Crydev tools and editor (3.4.0) still don't support them (shading wise) even tho they do seem to export fine as pointed out earlier.

I think I remember reading that this functionality is available to the Crytech team internally - it's a bummer that it is not featured in the public tool! Its a big deal breaker when it comes to efficient modular modeling and texturing ...

[Torch]

Not slating the engine as it seems to yield some really superior results, but I would've thought it would have some kinda FBX support by now, considering how easy it is to get an FBX out of a 3D app into UDK. Would be cool to see it in the next update :D

[cptSwing]

slightly related and possibly useful for '13 users: http://72.27.230.245/3ds-max/scripts/cryengine-3-export-preparation