3D Crono Cross model extraction?

[Luminaire85]

Here's the good news: When I looked at the *.mtl files in a text editor, I recognized the syntax. There's a very good chance that the *.obj and *.mtl files are a 3D file format called Wavefront OBJ. These files are designed to hold all of the data needed to render the 3D models. And Blender (free, but steep learning curve) has an importer for the *.obj file format.

Here's the bad news: The *.obj files in your attachment are all 0 bytes in size, so there's no geometry data.

I'm hoping that you can fix this with your 3D Ripper DX/psxfin technique, because I'm really excited by all of this. Kudos to you guys.

[FaustWolf]

More bad news, Luminair. I've never been able to get the .OBJ files to be anything other than 0 bytes; I probably shouldn't have even included them, but wanted to do so to show the community what 3D Ripper/psxfin produces. Thanks for checking these out though - are the .mtl files model skeletons, perhaps?

Say, do you know of any way to identify 3D model data, like given a model of unidentified format, would you be able to tell  that model data from, say, bitmap data? I've got these Chrono Cross .BIK archives in another thread that contain .bmp data for sure (or at least that's what PSicture rips from the archives), and I'm wondering what else might be in them.

[FaustWolf]

Another question for Luminair, jono, and any other experienced modelers who happen through here:

How, exactly, does model data work in a console game? I'm used to thinking in terms of Final Fantasy VII, where actual shaded models are stored on the game disc and the game tells the system to apply textures to those models. Is my understanding vaguely correct, and would this rudimentary description be applicable to Chrono Cross? For example, could we find, say, a "physical" model file within the Chrono Cross iso in the form of Kid's shoe, but grayed out because the texture is wrapped around that model in-game?

I'll rephrase that if necessary; I'm still trying to wrap my mind around just how character models work. I've found what *may* be archives/containers in the form of .BIK files on the Chrono Cross Disc, and I just wanted to make sure it's worth the effort to decompress or otherwise break these containers down if the goal is to obtain model files. I realize that some plugin or program would eventually have to be created to view Square's proprietary model format for Chrono Cross, of course. Not that I'd be capable of doing that .

[Luminaire85]

I think you've got the gist of it. I'll try to expound a little more for those interested:

3D models are made up of lots of little triangles, quadrilaterals, and other polygons. Each polygon has a bunch of information associated with it: what points make up its vertices, what color it should be, how it should reflect light, what texture is associated with it, etc. All of this "model data" is stored in a file until needed by the game. The game is able to recognize the model data, adjust it as necessary (e.g. for animations), and call the appropriate DirectX/OpenGL/etc. code to do the rendering.

Among other things, this means that any 3D model file should contain a certain set of information: vertices, normals, material and texture data, etc. Plus we don't even know if Square Enix uses a proprietary format yet. I think that if we can figure out what parts of the archives contain model data, we can reverse engineer the format.

Hope that helps.

[FaustWolf]

Pure and utter sweetnes, Luminair! I know Final Fantasy VII's model data was stored as .p files (stands for "polygon", I guess). At least in the PC version; I haven't done any rips from the PSX version. Is that considered a proprietary format to your knowledge? Is it possible that Square would have gone with a .TMD format even in 1999 or whenever this game was developed?

Thanks again for lending us your expertise!

[Luminaire85]

Well I haven't heard of the *.p file format (for 3D models anyway), but since someone has figured out enough of that format to explain how to rip the models, I would expect to be able to do the same with whatever format the models end up coming in. The description of the *.p file I linked to is very much what I would expect to be in that sort of file, so that's encouraging. 

[FaustWolf]

*Jaw drops*

If you can actually understand all that stuff Mirex is saying about the .p format, Luminair, I must commend you. I've used Mirex's programs, but always without having any inkling about the nuts and bolts of what was going on.

I know already that there's uncompressed .TIM files for the textures within the Chrono Cross CDs, or at least within the .BIK archives that Square apparently stuffed their files & data into. I and hopefully lots of others will be working on locating where all the .TIM texture data is, so at least we can tell where the model data isn't stored.

Say -- is there any rule in game design that model data should be stored right next to the applicable textures? In Final Fantasy VII things seem that way, what with everyone's eyeballs as textures stored right along with their model data. But Chrono Cross is from a more advanced era, of course - maybe textures are clumped together in one place and model data is clumped together at another location on disc. I dunno.

Thanks again for your attention to these model matters!

[FaustWolf]

Double-post. Sorry, sorry.

The fact that .TIM files aren't compressed within the Chrono Cross .iso and .BIKs makes me wonder...Could it be that Square used the .TMD format for Chrono Cross' models? I believe that's a general format (just as .TIM is a general format for 2D images), and there might be .TMD viewers out there that can scan the Cross iso and make our work easy.

So, community, is there a .TMD viewer somewhere that has file scanning capabilities? I've seen .TMD converters, but they need .TMD files as input and are thus useless in helping us sift through Chrono Cross' data.

[FaustWolf]

SOMEONE CHECK THIS OUT, but don't get too excited. Luminaire, can you make sense of the info contained in the image files I link to below? The red lines symbolize truncation so you can see the beginning and end of the TMD ripper's DOS messages.

I threw my Cross iso at a program called TMD Rip available at www.megagames.com/psx/psx_utils_prg.shtml

There are three types of results I'm getting. The most promising is this:

But that's the only one of its kind I've found, after dumping about 25 supposed TMD files from the iso.

The second type is a 0 byte dump -- the program's even telling me the file size is negative(!) WTF? But it says there's objects and vertices and stuff:



And the third type is *really* bizarre - TMD files that are impossibly large - I'm talking about up to 500MB here. Double WTF!! This one is a more manageable 70MB once out of the rar, but still...

And here's the physical product if anyone wants to take a look: http://rapidshare.com/files/62539486/TMD24.rar.html

Any ideas on what the heck's going on here? I only get 0 byte files when I toss a .BIK at the program. And the summation of all the extremely large files I've gotten so far is larger than the file size of the entire Disc 1 iso!! Is TMD ripper just taking data out of the iso, inflating it, and then calling it a TMD file?

Examples of the smaller TMD files I got are attached.

EDIT: And it sure would be interesting to get our hands on another TMD ripper, just to see if the results are consistent between rippers.


[attachment deleted by admin]

[Luminaire85]

Those TMD files that appear to be impossibly large are probably not actually TMD files. The numbers you're seeing that are on the order of 10^9 are garbage, probably due to the program outputting integers that weren't initialized to the screen.

After a brief glance at a .tmd specification, it looks like the only way to identify TMD file sequences is by the 0x00000041 at the beginning of the file. As you might expect, not every 0x00000041 sequence in an iso or bik corresponds to a .tmd file. Compressed data makes this even harder, so that may be why you're getting garbage.

Also, to respond to something you asked earlier, there is probably no correlation between the location of a character's 3D model data and the corresponding textures. It's somewhat more likely that the model data is in one area of the archive and the textures are together in another area, but we can't say anything definitive about that until we know more about the archive.

Nevertheless, later tonight I will try to import the more promising .tmd file you attached into a 3D modeling program.

[FaustWolf]

Thanks, as always!

EDIT: Haven't been able to view that 13KB "TMD1" with a viewer yet, but judging by the first byte in hex it does seem to be valid.

EDIT: Eww, but I see the same beginnig byte on the 70MB "TMD24" as well. I guess we'll just have to see if TMD1 can be viewed.

[Luminaire85]

Well, TMD1 is something, all right: the P and S of the Playstation logo. Screenshot from Milkshape 3D attached.



[attachment deleted by admin]

[FaustWolf]

Well, how about that! We've got one model already! Though that's no guarantee the rest are in .TMD format, right? All Playstation game CDs probably have that one.

Judging from the few sample PSX .TMDs I've gotten my paws on, it appears that the TMD file header in hexadecimal could be either

00000000 4100 0000 0000 0000                     A.......       
 -OR-
00000000 4100 0000 0000 0000 0100 0000     A...........   

There are 4 instances of the bottom hex string in the iso, but if I search based on the truncated top version, I get 1370 instances in the first .BIK alone. So either we've got a ton of .TMDs or almost none at all.

What I'd like to do next is rip some of the data that follows the "full" TMD "headers" in the iso and see if other TMDs are present at all - the "full headered" hex strings would be the most likely candidates. But once I've got the hex strings copied, Luminaire, how would I go about transforming them into files Milkshape could view? Is that even possible? And then there's the problem of guessing how long of a hex string would be sufficient to form a .TMD file, as they don't have a standard length and I don't believe they have any kind of "footer."

For my own recording purposes, the PlayStation logo is contained within the following offsets of the iso:
00002800 ~ 00005A78. Once I start a datamap that'll be put in there along with .TIM locations.

Thanks again, Luminaire!

[ZeaLitY]

Would those numerous TMDs be intramap solidity stuff? Like, a map shows a prerendered house, and there's a solid, transparent polygon around the house to block you from walking on it? Or is solidity handled some other way?

[Luminaire85]

My guess is solidity is handled some other way since the backgrounds aren't really 3D anyway.

The header of a valid .tmd file appears to be 12 bytes in length:

• The first four bytes of a .tmd file represent an identifier tag, and are always 41 00 00 00.
• The next four bytes are some info about what the file contain. If this is not 00 00 00 00, hopefully it is at least consistent across the various .tmd files in the archive(s).
• The next four bytes are the number of objects in the file, so if each .tmd file corresponds to only one object, then the hex would be 01 00 00 00. A .tmd file with 70 objects, like one of the examples you posted earlier, would be 46 00 00 00.

You're right that there is no footer to .tmd files. It looks like the only way to figure out the length of a .tmd file is to fully parse it via TMD Rip or a similar utility. If we can isolate a .tmd file from the iso (that is, find a valid header, parse the hex data to determine the length of the file, and copy the appropriate bytes to a new file) then importing into Milkshape is trivial.

The screenshots from before suggest TMD Rip wasn't really working; if this is the case and no alternatives can be found I can maybe try to code something up, but that would take at least a few days.