Chrono Compendium
Kajar Laboratories - Fan Works and Submissions => Chrono Cross Modification => Topic started by: Chickenlump on December 10, 2006, 11:31:35 am
-
I found this topic over at Emutalk, and thought it would be neat if we could apply it to Crono Cross, to extract and have a closer look at it's 3D models of characters and enemies and who knows what else.
http://www.emutalk.net/general-gc-emulation-discussion/37091-universal-model-map-ripper.html (http://www.emutalk.net/general-gc-emulation-discussion/37091-universal-model-map-ripper.html)
There are other programs like this as well, for OpenGL.
It would make a nice addition to the downloads section if we were able to extract and save the models and texture mappings for people to import into 3D programs for modeling. It would take a bit of time to do this though, if it works, then it could be made a bit easier to get the models through codes that load up characters or monsters instead of playing through the game to capture them all.
-
Oh yeah, definitely. This is one of the holy grails of the encyclopedia along with translations and Chrono Cross locations. My computer sucks, so I'll ask on that thread if someone can try Chrono Cross.
-
I'll try it out.
>___> Keep getting "Unable to Create Process"
Edit Edit: I realize, I don't have directx 9 applications anywhere >_>
-
There are other programs that do pretty much the same thing. I followed a couple of the links at the 3D Ripper DX site. I wonder which Epsxe plugin is DX9 based? I know there are Open GL plugins for it, so one of the OpenGL alternative ones might work. We just need to find the proper combination of emulator and plugins.
http://glintercept.nutty.org/index.html (http://glintercept.nutty.org/index.html) OGL alternative
http://denull.ucoz.ru/load/1-1-0-40 (http://denull.ucoz.ru/load/1-1-0-40) Another 3D ripper of DX9 applications.
http://developer.nvidia.com/object/nv_texture_tools.html (http://developer.nvidia.com/object/nv_texture_tools.html) Some tools and plugins to view/edit DDS texture files (some of these rippers save the textures as this file-type).
-
Wow, I will rip some models, and with those models, I will make out sprites for RMXP.
-
I might give it a go when I can get a hold of 3DS max.
-
Buuuuuuuuuuump.
-
Quite a while ago I tried to capture geometry from chrono cross which was run on epsxe. Unfortunatly I could never quite get the opengl capturing program to successfully capture anything from any psx game ran with epsxe, my own simple tests and other programs seemed to work fine though. I'd be very interested to see if anyone has managed to successfully capture a model from a psx game.
-
I've tried 3D Ripper DX on other PlayStation games without success. I believe it only works with the Gamecube emulator "Dolphin" from what people are saying on the message boards on the 3D Ripper DX site, even though epsxe is a DX9 application. So that option is probably out.
I wonder if ripping the models directly from the game CD could be done? Does anyone know any details *at all* about the structure of the archives Chrono Cross files are stored in? I'll do some basic investigation when I get a chance and see if I can't get any help from the xentax community, which specializes in extracting files from game archives. Though we'd still have the difficulty of finding a suitable viewer/editor for the models either way.
-
I have just managed to rip flat untextured models from epsxe using ogle and glintercept.
I thought the flatness was strange but I found some info that explains it. Epsxe only uses opengl to render the already projected polygons, this means that opengl only ever recieves sets of triangles that are already projected onto the view plane. It therefore isn't possible to capture models from epsxe that arn't flat on a plane. A bit unfortunate. Perhaps another emulator could be used, if there is any others that support chrono cross.
-
Now that we know how to rip a clean iso from Chrono Cross and get textures, does anyone know if there's a program like PSicture that can scan a file and detect model data? Would that even be possible?
I strongly urge anyone interested in Chrono Cross models to check out the xentax forums (http://forum.xentax.com/) and take a gander at what those people have accomplished as far as ripping game resources. Unfortunately though, most of their accomplishments have been for PC games, which historically store graphics in a manner that makes them far more accessible than the graphics in console CDs. Their wiki states that someone developed a script for getting at Xenosaga: Episode I files, but I haven't had any success with that and haven't had the interest to pursue the matter any further.
Their Guide to Exploring File Formats (at the bottom right of http://multiex.xentax.com/) might prove useful. Honestly I haven't looked at it in-depth yet and I don't even know if it's applicable to the .iso file that can be ripped from the Chrono Cross CD. Mr. Mouse's MultiEx Commander program isn't all that useful for console games just yet, for the reason stated above, and the free version has very limited functionality.
If none of this stuff is applicable, the Chrono Community will have to slog through the emulator-rip method that Jono and Chickenlump have been exploring. That'll be a bit more work in the long run, but hopefully something will work out.
Mayhap the era of Chrono Cross hacks lurks nigh?
-
I don't think we'll ever be able to rip them from epsxe, it's impossible to get any depth data for the model :( Disappointing. I'm not going to continue trying.
Also directly ripping from the disk poses a couple of problems, first up we don't know what format the models are actually stored in. If it's the format same as final fantasy vii then I guess we could work something out.......if we could get to them. This is the second problem, after the first generation or so, developers started compressing and bunching together files on the disc for two reasons. To protect their resources from those who would rip them and also to save space on the disc, as the psx became older the developers continued to improve their methods.
I would have a go at writing a program to attempt this but I simply don't know enough about it. I've just started reading and I can see why no one else has yet attempted to extract this sort of thing from square's games (with the exception of ffvii).
I think the best way would be to take reference shots and attempt to model over the top of the images (very time consuming and probably not perfectly accurate). If you could get enough interested modelers I guess it wouldn't be so bad, but this in itself would be a problem.
Also animation would have to be done by hand, but I guess this would be the case if we used the glcapture/epsxe way as well.
It doesn't look too bright at the moment, it's just so frustrating because they're right there on that disc and we just can't get 'em.
-
Does anyone have 3DS Max or Lightwave, or a modeling program that can view any of the following filetypes?
3DR
DDS
MTL
OBJ
I dumped some stuff using 3D Ripper DX and psxfin, first time I've used it on Chrono Cross. It pulled 3MB of something, but what, I wonder? The stuff I pulled is attached.
No rush, it's most likely nothing. It's just that I can't view it myself, and I'm curious to see if anything even remotely useful is in here. It'll probably be sporadic flat textures like Jono's been talking about above.
[attachment deleted by admin]
-
I can view the .dds files only.
It's not junk, they are pictures from the opening sequence; you know, the squaresoft title swirling in the ripple of water and the rest are from the Serge character naming screen and the font base image that is used in battle, I think.
I can view them using Nero PhotoSnap viewer, by the way.
-
Okay, this is going to be all disjointed but bear with me.
First, thanks a million for that, Dankun. The DDS are 2D graphics files then. I'll see if I can view them myself when I get a chance.
It's really weird how the program ripped title screen and naming screen stuff, because I did the rip right when I got into the intro dungeon. I suppose this tells us that psxfin stores stuff in memory that occurred minutes beforehand, and that 3D Ripper DX can access that history - but it's supposed to rip what's onscreen at the moment. *Is confuzzled*
Any takers on the other files in the rar attached above? Most of the files with identical extensions will probably be duplicates because I hit the rip key multiple times to see if I could get frame-by-frame model movements if I was lucky.
And as a side-note, does anybody know which PSX emulators are DirectX and which are OpenGL programs? Maybe we could get a list of what doesn't work going, so we can quickly see which options have been explored and not explored.
Jono, do you happen to remember which OpenGL plugin you were using with ePSXe when you attempted to model capture? I wonder if it might make a difference.
I believe some of the pages Chickenlump referred to at the top may be in Russian. Hey -- isn't Den Schwarz Russian? The user who submitted snippets of his Chrono Trigger novel in the Dream Splash! forum? Hmm...
And this is going to sound totally stupid, but is it possible to convert an iso such that it would play in a different system's emulator? For example, I've read that people can convert PSX iso files into eboot files to play Playstation games on the PSP. The reason why I'm asking is that I know 3D Ripper DX works with the Gamecube emulator "Dolphin," and it would be sweet if Chrono Cross could be somehow played by that emulator. Haha. Please be gentle in your responses to this one. :lol:
Must get back to work. Nothing keeps you from studying the Keynesian cross like Chrono Cross.
EDIT: Added another attachment to my post above. This is to check for consistency in 3D Ripper DX's ripping process.
-
Here's the good news: When I looked at the *.mtl files in a text editor, I recognized the syntax. There's a very good chance that the *.obj and *.mtl files are a 3D file format called Wavefront OBJ (http://en.wikipedia.org/wiki/Obj). These files are designed to hold all of the data needed to render the 3D models. And Blender (http://www.blender.org/) (free, but steep learning curve) has an importer for the *.obj file format.
Here's the bad news: The *.obj files in your attachment are all 0 bytes in size, so there's no geometry data.
I'm hoping that you can fix this with your 3D Ripper DX/psxfin technique, because I'm really excited by all of this. Kudos to you guys.
-
More bad news, Luminair. I've never been able to get the .OBJ files to be anything other than 0 bytes; I probably shouldn't have even included them, but wanted to do so to show the community what 3D Ripper/psxfin produces. Thanks for checking these out though - are the .mtl files model skeletons, perhaps?
Say, do you know of any way to identify 3D model data, like given a model of unidentified format, would you be able to tell that model data from, say, bitmap data? I've got these Chrono Cross .BIK archives in another thread that contain .bmp data for sure (or at least that's what PSicture rips from the archives), and I'm wondering what else might be in them.
-
Another question for Luminair, jono, and any other experienced modelers who happen through here:
How, exactly, does model data work in a console game? I'm used to thinking in terms of Final Fantasy VII, where actual shaded models are stored on the game disc and the game tells the system to apply textures to those models. Is my understanding vaguely correct, and would this rudimentary description be applicable to Chrono Cross? For example, could we find, say, a "physical" model file within the Chrono Cross iso in the form of Kid's shoe, but grayed out because the texture is wrapped around that model in-game?
I'll rephrase that if necessary; I'm still trying to wrap my mind around just how character models work. I've found what *may* be archives/containers in the form of .BIK files on the Chrono Cross Disc, and I just wanted to make sure it's worth the effort to decompress or otherwise break these containers down if the goal is to obtain model files. I realize that some plugin or program would eventually have to be created to view Square's proprietary model format for Chrono Cross, of course. Not that I'd be capable of doing that :(.
-
I think you've got the gist of it. I'll try to expound a little more for those interested:
3D models are made up of lots of little triangles, quadrilaterals, and other polygons. Each polygon has a bunch of information associated with it: what points make up its vertices, what color it should be, how it should reflect light, what texture is associated with it, etc. All of this "model data" is stored in a file until needed by the game. The game is able to recognize the model data, adjust it as necessary (e.g. for animations), and call the appropriate DirectX/OpenGL/etc. code to do the rendering.
Among other things, this means that any 3D model file should contain a certain set of information: vertices, normals, material and texture data, etc. Plus we don't even know if Square Enix uses a proprietary format yet. I think that if we can figure out what parts of the archives contain model data, we can reverse engineer the format.
Hope that helps.
-
Pure and utter sweetnes, Luminair! I know Final Fantasy VII's model data was stored as .p files (stands for "polygon", I guess). At least in the PC version; I haven't done any rips from the PSX version. Is that considered a proprietary format to your knowledge? Is it possible that Square would have gone with a .TMD format even in 1999 or whenever this game was developed?
Thanks again for lending us your expertise!
-
Well I haven't heard of the *.p file format (for 3D models anyway), but since someone (http://mirex.mypage.sk/FILES/pformat2.txt) has figured out enough of that format to explain how to rip the models, I would expect to be able to do the same with whatever format the models end up coming in. The description of the *.p file I linked to is very much what I would expect to be in that sort of file, so that's encouraging. :)
-
*Jaw drops*
If you can actually understand all that stuff Mirex is saying about the .p format, Luminair, I must commend you. I've used Mirex's programs, but always without having any inkling about the nuts and bolts of what was going on.
I know already that there's uncompressed .TIM files for the textures within the Chrono Cross CDs, or at least within the .BIK archives that Square apparently stuffed their files & data into. I and hopefully lots of others will be working on locating where all the .TIM texture data is, so at least we can tell where the model data isn't stored.
Say -- is there any rule in game design that model data should be stored right next to the applicable textures? In Final Fantasy VII things seem that way, what with everyone's eyeballs as textures stored right along with their model data. But Chrono Cross is from a more advanced era, of course - maybe textures are clumped together in one place and model data is clumped together at another location on disc. I dunno.
Thanks again for your attention to these model matters!
-
Double-post. Sorry, sorry.
The fact that .TIM files aren't compressed within the Chrono Cross .iso and .BIKs makes me wonder...Could it be that Square used the .TMD format for Chrono Cross' models? I believe that's a general format (just as .TIM is a general format for 2D images), and there might be .TMD viewers out there that can scan the Cross iso and make our work easy.
So, community, is there a .TMD viewer somewhere that has file scanning capabilities? I've seen .TMD converters, but they need .TMD files as input and are thus useless in helping us sift through Chrono Cross' data.
-
SOMEONE CHECK THIS OUT, but don't get too excited. Luminaire, can you make sense of the info contained in the image files I link to below? The red lines symbolize truncation so you can see the beginning and end of the TMD ripper's DOS messages.
I threw my Cross iso at a program called TMD Rip available at www.megagames.com/psx/psx_utils_prg.shtml
There are three types of results I'm getting. The most promising is this:
(http://img132.imageshack.us/img132/1339/tmd1ch7.gif) (http://imageshack.us)
But that's the only one of its kind I've found, after dumping about 25 supposed TMD files from the iso.
The second type is a 0 byte dump -- the program's even telling me the file size is negative(!) WTF? But it says there's objects and vertices and stuff:
(http://img132.imageshack.us/img132/5755/tmd6ks5.gif) (http://imageshack.us)
And the third type is *really* bizarre - TMD files that are impossibly large - I'm talking about up to 500MB here. Double WTF!! This one is a more manageable 70MB once out of the rar, but still...
(http://img67.imageshack.us/img67/6154/tmd24th8.gif) (http://imageshack.us)
And here's the physical product if anyone wants to take a look: http://rapidshare.com/files/62539486/TMD24.rar.html
Any ideas on what the heck's going on here? I only get 0 byte files when I toss a .BIK at the program. And the summation of all the extremely large files I've gotten so far is larger than the file size of the entire Disc 1 iso!! Is TMD ripper just taking data out of the iso, inflating it, and then calling it a TMD file?
Examples of the smaller TMD files I got are attached.
EDIT: And it sure would be interesting to get our hands on another TMD ripper, just to see if the results are consistent between rippers.
[attachment deleted by admin]
-
Those TMD files that appear to be impossibly large are probably not actually TMD files. The numbers you're seeing that are on the order of 10^9 are garbage, probably due to the program outputting integers that weren't initialized to the screen.
After a brief glance at a .tmd specification (http://wiki.xentax.com/index.php/Playstation_TMD), it looks like the only way to identify TMD file sequences is by the 0x00000041 at the beginning of the file. As you might expect, not every 0x00000041 sequence in an iso or bik corresponds to a .tmd file. Compressed data makes this even harder, so that may be why you're getting garbage.
Also, to respond to something you asked earlier, there is probably no correlation between the location of a character's 3D model data and the corresponding textures. It's somewhat more likely that the model data is in one area of the archive and the textures are together in another area, but we can't say anything definitive about that until we know more about the archive.
Nevertheless, later tonight I will try to import the more promising .tmd file you attached into a 3D modeling program.
-
Thanks, as always!
EDIT: Haven't been able to view that 13KB "TMD1" with a viewer yet, but judging by the first byte in hex it does seem to be valid.
EDIT: Eww, but I see the same beginnig byte on the 70MB "TMD24" as well. I guess we'll just have to see if TMD1 can be viewed.
-
Well, TMD1 is something, all right: the P and S of the Playstation logo. Screenshot from Milkshape 3D attached.
[attachment deleted by admin]
-
Well, how about that! We've got one model already! :D Though that's no guarantee the rest are in .TMD format, right? All Playstation game CDs probably have that one.
Judging from the few sample PSX .TMDs I've gotten my paws on, it appears that the TMD file header in hexadecimal could be either
00000000 4100 0000 0000 0000 A.......
-OR-
00000000 4100 0000 0000 0000 0100 0000 A...........
There are 4 instances of the bottom hex string in the iso, but if I search based on the truncated top version, I get 1370 instances in the first .BIK alone. So either we've got a ton of .TMDs or almost none at all.
What I'd like to do next is rip some of the data that follows the "full" TMD "headers" in the iso and see if other TMDs are present at all - the "full headered" hex strings would be the most likely candidates. But once I've got the hex strings copied, Luminaire, how would I go about transforming them into files Milkshape could view? Is that even possible? And then there's the problem of guessing how long of a hex string would be sufficient to form a .TMD file, as they don't have a standard length and I don't believe they have any kind of "footer."
For my own recording purposes, the PlayStation logo is contained within the following offsets of the iso:
00002800 ~ 00005A78. Once I start a datamap that'll be put in there along with .TIM locations.
Thanks again, Luminaire!
-
Would those numerous TMDs be intramap solidity stuff? Like, a map shows a prerendered house, and there's a solid, transparent polygon around the house to block you from walking on it? Or is solidity handled some other way?
-
My guess is solidity is handled some other way since the backgrounds aren't really 3D anyway.
The header of a valid .tmd file appears to be 12 bytes in length:
- The first four bytes of a .tmd file represent an identifier tag, and are always 41 00 00 00.
- The next four bytes are some info about what the file contain. If this is not 00 00 00 00, hopefully it is at least consistent across the various .tmd files in the archive(s).
- The next four bytes are the number of objects in the file, so if each .tmd file corresponds to only one object, then the hex would be 01 00 00 00. A .tmd file with 70 objects, like one of the examples you posted earlier, would be 46 00 00 00.
You're right that there is no footer to .tmd files. It looks like the only way to figure out the length of a .tmd file is to fully parse it via TMD Rip or a similar utility. If we can isolate a .tmd file from the iso (that is, find a valid header, parse the hex data to determine the length of the file, and copy the appropriate bytes to a new file) then importing into Milkshape is trivial.
The screenshots from before suggest TMD Rip wasn't really working; if this is the case and no alternatives can be found I can maybe try to code something up, but that would take at least a few days.
-
Wow Luminaire, that info helps me immensely. I didn't know about the bytes representing the # of objects in the file. Because of that, my search may have excluded a considerable number of .TMDs, so I'll look through it again.
Community, can we find a better .TMD viewer/scanner so Luminaire doesn't have to code one? That's a lotta work I imagine, though you'd be the hero of any other game community looking for .TMDs in their isos. We can wait; take your time if such a program becomes necessary to further progress, since there's a ton of other stuff to keep us occupied in the meantime.
Meanwhile, I'll search for .TMD headers in the .iso and in the .BIKs this week.
-
Wow, you guys have done alot while I've been away. I'm not confident .TMD would actually be used by Square for the actual games model format. TMD was a format that Sony used in tech demos and I think also early games (like Ridge Racer). So it's quite likely that most playstation games contain the Sony logo in .TMD format somewhere on the disc not compressed.
Also if Square was likely to ever use TMD then it would make sense to use it in early games like FFVII which it didn't (FFVII used a format called .lzs with a really funky animation system i've heard.)
Like luminaire said, it's likely that the program you used looked for a 'magic number' in the file headers to identify a TMD file If that number occours at the right place then the program tries to read the file as a TMD, producing gibberish if it is in fact not the correct format.
I am interested that you managed to use dx-ripper with a playstation emulator, are you making sure that the character whose geometry you want to capture is on screen at the time of capture? Sorry if this point has already been asked, I've only skimmed through the thread to catch up :)
On the note about changing the graphics plugin for epsxe, unfortunatly it wont make a difference as the plugin only takes care of drawing these pre-projected polygons to the view plane. All the projection is handled by the emulation of the graphics chip which is actually handled internally by epsxe. It makes sense to do it this way for compatibility I guess, I wonder if any psx emulators actually do all the projection work in directX/opengl.
On Zeality's point, it is possible because the character models would have to be interacting with some type of invisible 3d geometry on top of the background. I cant't think of another way to do it. They probably have a "bounding structure" that the character can't walk off, so any 'walkable' area would be a part of this geometry.
In the end I believe that square probably used a proprietry format (maybe an evolution of .lzs), without a bit more information I think I'd be at a loss to figure out exactly how a model format works without any documentation. It is possible but would rely on finding a file that we believe is a model then starting to experiment with it.
Another possiblity, and this is just a reproducing something I read a couple of years ago, is that the models are stored simply in psx graphics assembly, ready to be sent straight to the chip. I think quite a few software house started doing this with the psx, maybe square aswell. I'm not sure exactly how this would be formatted and everything but I just thought that it was interesting.
-
Good to see you back, jono, and you bring up lots and lots of good points. Take a look at the .BIK thread as well, since there might be stuff applicable to 3D model stuff going on there as well. We've ripped the PlayStation logo in .TMD format, at least, but as you point out that could be ripped from any PlayStation CD.
As far as 3D Ripper DX is concerned, I did the rip with Serge, Kid, and an extraenous third character standing around in Fort Dragonia, a few seconds after the game begins. I had hoped to capture some of their models, but I haven't viewed the 3DS Max-applicable files yet. I did a similar rip with Front Mission 3 over the summer, but could only get random stuff like spheres that are part of buildings in the background. I'm assuming that if the Chrono Cross 3DS, etc., files can be viewed, some totally pointless stuff will be in there.
Did you get those textures you wanted to look at, jono? If you haven't ripped 'em yet, I'll send them to you via email or my MediaMax account eventually, maybe with a split .rar.
-
I haven't got around to it actually, I'll try tonight, If I fail I may come crawling back for an email :)
I'll track down this other emulator and have a look.
-
The psxfin emulator can be obtained at: http://psxemulator.gazaxian.com/
You've probably heard of the emulator under the name pSX. It says "psxfin" once you've got it unzipped though, if I'm right. Strangeness.
-
Running with an idea jono gave me in another thread, I decided to run TMD Rip on a VRAM dump. It produced one file, 4KB IIRC. Luminaire or jono, could you try opening this in lightwave or another program capable of viewing .TMDs and see what's inside? The file's attached.
Also, I just rediscovered a 5MB .TMD file on my PC probably produced during my earlier TMD Rip attempts. Is it possible to open up such a large file without crashing Lightwave or another 3D viewer? The hexadecimal header checks out as a .TMD with lots of objects. Take a gander if possible and when time is aplenty.
http://rapidshare.com/files/62630448/Unknown_TMD_Data.zip.html
[attachment deleted by admin]
-
Ah, unfortunately it'll be probably a fair bit more difficult than that to find vertex arrays in memory. The models will only be stored as a .TMD (provided that is indeed the format square used) on the disc, once loaded they will exist in memory as linear arrays of values (16 bit values if memory serves me correctly). I really have to get some time together to have a look with you.
Also, it is a bit unlikely that the geometry will exist in vram, generally it's used to store texture data. The vertex data is more likely to exist in the main memory of the psx. This just brought an interesting thought into my head, i'll start a new paragraph i think :)
Save states - a save state is actually a complete dump of the state of the emulated system. The state of the system would have to include the exact state of any memory that the system uses, with a discbased system like the sony all the data being used at a given time is loaded off of the disc and into memory. Basically this means that model data exists in save states taken when models are on screen, in one way or another. It would be in the section of the save state that stores the current state of the system's main memory.
If you can find any information about the epsxe save states post it, I had a quick look and found that ram starts at offset 1BA in an uncompressed epsxe save state. Now would be the difficult part, how do we know what data in ram is representing what?
I guess you could get a hex editor and have a look for a series of 16 bit numbers with very few zeros and little repetition. This 'could' be a sign of meaningful vertex data. Then you would either have to copy the sequence and feed it to through a script in blender or write a simple program to load a series of arrays from a simple file and attempt to draw them as a vertex array.
The trouble is I have no idea how the psx gu takes it's texture co-ordinates, normal, spacial co-ordinates ect. I'll make some time on the weekend and have a look. This could make figuring out the order of the vertex array very hard (provided we can identify one).
Also one good thing about this is, we would be likely to find the model's key-frames, meaning we could re-create the animation.
Maybe someone knows a bit more about the way the psx works and can give a hand with this idea. Still unsure if it would be correct though. I'm just hoping that the psx has very little ram otherwise this would be the worst job ever.
Wow, ended up being alot of writing.
-
Once the TIMs are lined up, then perhaps we'll find that main location textures are stored next to their dynamic counterparts. Like, Termina's prerendered background might be right next to Van's paintings / piggy bank (which we can view) in the ISO.
-
@Zeality: I'll get a .TIM location-finding tutorial up soon so lots of Compendiumites can join in the fun. It's really quite easy, and I think everyone should be able to do it, and it doesn't even involve a hex editor anymore; it can accurately be done with a hex calculator and TIMViewer.
@jono: I usually take the VRAM dumps from savestates, so getting into those should be no problem. I can probably find .TIM data in the savestate since that's in VRAM, provided the character portraits and textures are still in .TIM format after it's put into VRAM. That might narrow things down a bit.
-
Obviously last resort options here, but:
1. Yasuyuki Honne *MIGHT* know what file format was used for textures, but he probably doesn't since he wasn't an engineer. Honne did most of the game's environmental art (he also worked on Trigger, and can be considered part of the "true" Chrono Dream Team). We have his blog, and I wanted to see if I could get in touch with the almighty Kato through him, but we don't have someone to translate Japanese.
2. Cross ripping may have been done in the JP Chrono community. Again, no way of finding out whatsoever without someone to translate Japanese. That we can't find someone to even help with Missing Piece / Ultimania is becoming a matter of embarrassment (let alone getting a chance to speak with Kato).
3. I plan on mailing Richard Honeywood to see if he'd like to talk about Chrono Cross sometime. He probably wouldn't know much on the technical side, but he did invent the accent system.
At any rate, you might want to open a thread at ROMHacking.net's General ROM hacking forum with the most up to date information on this effort. I offended the administration's delicate sensibilities with my dig at fan translation groups recently, so I'm staying away from there for the time being.
-
!!!
This is from a 2002 post on GameDev.net:
The trick is to have an emulator dump you the model. With the playstation, you can find the 3d-to-2d projection that the model underwent when being projected to the screen by hacking the GTE portion, and all the faces and textures (in screen coordinates) from the GPU portion (in the graphics plugin for most emulators). Those 2 can be used to reconstuct an original textured mesh (which can be easily animated with the skeleton system I'm using for my current project).
My particular tools were AdriPSX + the PEOPS GPU plugin (both open source).
The only way a game can prevent this is by not using software rendering (which is not particularly fast on a playstation), so I think it's safe to assume that most games will fall to this trick. I've tested it with Chrono Cross and managed to get a well animated Serge model:
screenshot of Serge: (http://bingweb.binghamton.edu/~bj92077/serge.jpg)
modeller + source + 2 dumped models: (http://bingweb.binghamton.edu/~bj92077/modeller.zip)
Original thread here:
http://www.gamedev.net/community/forums/topic.asp?topic_id=93782
Anyone able to decipher exactly what this person's saying regarding GTE and GPU? I'll see if I can track down this AdriPSX and PEOPS GPU plugin. All links in the quote are long since broken. I haven't even tried contacting the person. Should we?
EDIT: Lower in the GameDev thread this user describes things slightly more clearly. Reproducing whatever process he used sounds like a lot of work. AdriPSX might be an emulator; wonder if there'd be any way to do this with epsxe?
-
GTE = geometry transfer engine (does fast matrix math needed for graphics rendering)
GPU = graphics processing unit (handles actual rendering)
Source: http://darkwatcher.psxfanatics.com/console/details/psx.html
I read it as saying we can get the model data by modifying the source code of a PSX emulator and/or graphics plugin at the appropriate locations to, for example, spit out the data to a file. Normally I'd be wary of the practicality of such an approach, but since this person appears to have been successful (and with Chrono Cross, no less) we might be able to duplicate his efforts. If I can find some time I will also give it a try.
I'd be surprised if you could get a hold of the person, but I suppose you could try.
-
Thanks again, Luminaire. This will probably be a last, last, last resort unless you can do such things easily. But it's good to know it can be done.
-
Apparently I can't private message on Slashdot. I'll just ask R_D to do it tonight, since he might be subscribed.
-
The user's slashdot posts seem recent, too.
-
It seems that he modified the source code of the emulator to dump all the models drawn from a given save state. It would be impossible to do with epsxe because it is/was a closed source project. I'd have a llok if anyone can find the source code to Adripsx and the plugin specified, the only thing I'm not sure about is the re-construction process. I'm pretty sure I'll be abl to find documentation somewhere.
Really want uni to be over so I can have a really good go at this. If you guys know of any other open source emulators that run chrono cross then maybe we could look for them.
-
Good deal, jono. Maybe during winter break you and Luminaire can take a look at this process. In the meantime I'll be on the lookout for open source emus, as well as AdriPSX.
-
The only open-source PSX emulators I could find are AdriPSX and PCSX, neither of which appear to have been updated since 2003. After several tries with Google I finally found the source to AdriPSX, but the author must have despised commenting, because I have yet to find a single one. Plus the variable names are extremely generic. The PCSX (http://www.pcsx.net/) source is much nicer in terms of comments, plus there's a Visual Studio project file in addition to the makefiles. (Everything is written in C.)
The P.E.Op.S. (http://sourceforge.net/projects/peops/) GPU plugin is on Sourceforge. There's a Visual Studio project file here too. The post from GameDev suggests to me that maybe only the plugin would need to be modified in order to get the model data. Perhaps this would allow us to continue to use a more current emulator?
Just thinking out loud.
-
Okay, this is a wild shot in the dark, but I have enough circumstantial evidence to suspect that .KMD files might be in Chrono Cross:
(http://img143.imageshack.us/img143/9891/image1vw0.gif) (http://imageshack.us)
That's offsets 1F0320 ~ 1F036F in the iso. As for what .KMD files are, or might be, a thread at the xentax forums indicates more or less that it's model data: http://forum.xentax.com/viewtopic.php?p=19004&sid=faf16ae844b2da1f5ef414c4082c1f7f
This all won't be enough to get me excited until I can get my hands on a .KMD model example. In the meantime, does anyone know anything about the .KMD model format? I'll post an example if I happen to come across one.
EDIT: Eww, but the thread also says that it might stand for "Konami Model." Well, I'll follow this lead as best I can.
-
Re-creating a post I made elsewhere here, as it is also relevant to model extraction:
In a moment of beautiful, beautiful mental clarity, I re-discovered a .zip file called "cctools" that's been sitting in a folder on my desktop since two days ago. It has a CCP file called "Main Disk Dump"!! :shock: :shock:
It says: "Dump the complete content of Chrono Chross CD1/CD2 US and Japanese version"
Then it's got a bunch of Computer Science speak, and I'm an economist, so --
I'm attaching the .CCP in a zip to this post b/c I'd like some confirmation as to whether the code contained within is C/C++ and if anyone knows of a good compiler if that's the case. This might just save us a lot of work!
EDIT: Oh, it is C/C++, I think. Anyone know of an appropriate compiler for the code contained within? If it works, we owe it to Yazoo, wherever he is.
[attachment deleted by admin]
-
I downloaded and compiled the dump program you found, I've modified it a bit for usability but I have a small problem, it relies on the disc image being exactly 736,651,104 bytes long, I can only get an image of my chrono cross to be 641,437,696 long.
If anyone has a cd image exactly that long, I'll tidy up the program so it's a bit simpler to use an put it up. It's currently not creating directories properly in windows and is hard coded to look for an iso image that is called Chrono1.iso in the sam directory as the program so i'll just fix these couple of things up and it should work fantasticly.
We still have the problem of understanding the formats even though the files will all be output and be readily available.
EDIT: oh I see it's already been done, never mind :)
-
Hm, uploading would take a while...
perhaps something can be arranged. But for now, I'll go ahead and 7zip the two character model folders and pass it off to you. Maybe you can throw it at a program and see if it's recognized.
http://chronofan.com/Black/Cross/DUMP.7z
-
Oh, sorry, I wasn't asking for you to upload the iso for me :) I'm a little bit anti-piracy so I was just hoping to send the program to someone to rip the files. I'd rather not be sharing around iso's :)
Doesn't matter anyway, as you guys are a few steps ahead of me :)
Thanks very much for those model files, I'll have a peek at them now, this is the first bit of free time I've had for ages.
-
Jono, to get a CD image that's the proper length for the Ramsus/Yazoo dumper, you can rip with IsoBuster and use the settings "Extract CD <Image> -> RAW"
That should give you a 700+MB .bin file that will work.
-
So if I'm interpreting tonight's posts correctly, then in the file I've attached are the 3D models of all playable characters as seen at the status screen. (These are all the "3" files as produced by Yazoo's cd-decompress.exe.)
I believe this brings us to the point of trying to figure out how the model data is stored in these files.
[attachment deleted by admin]
-
Pardon my foolish question Luminaire, but did you produce these files from a single file in the CD directory or did you decompress all of them? *Haven't tried CD Decompress yet; must go do that now*
But to answer your question about the character models, I, er...don't know yet. Gemini tells me in another thread that the model data is only a "copy" of the true model, and not the model itself - I think. It's toward the end of the "FILE EXPLORATION..." thread. BUT - battle/status screen textures are certainly in there, as Gemini has proven. I'm hopeful that the CD directory might contain everything.
-
I just set up the file/folder structure that Ramsus explained and ran cd-decompress.exe (after running the dump program first, of course). A few seconds later all 51 .out files in the CD directory had a corresponding character-name folder with the six files Gemini mentioned in each. Out of those I only posted the "3" files in order to make the attachment <1 MB.
-
Aha, thank you. I'll try as well.
EDIT: Luminaire and jono, I'm not sure if having a file header in front of you would help out any in determining whether there could potentially be model data in these, but here goes:
http://img525.imageshack.us/img525/9369/possiblemodelske8.gif
The first 8 bytes seem to constitute a header. Any rule of thumb for determining whether there's vertex data or other 3D data stuck within hex code?
EDIT: I'm linking to some documentation Halkun gave us at the Qhimm forums regarding FFVII battle models, which may be at least somewhat similar to Cross'. It might be useful to compare what's in this wiki entry with the (potential) models' headers I've linked to in my previous edit.
http://wiki.qhimm.com/FF7/Battle_model_format_%28PSX%29
-
I've had a little look at these files, again I'm snowed under at uni but I'll try and post later if I find something of interest.
I would have guessed that they are compressed but they may not be. I hope we do find uncompressed model data of some form in there, then a simple model converter could easily by written to output to whatever format is required.
Exciting stuff indeed.
-
A tip from Gemini in the texture thread:
the drp files *should* (not verified, but it must be like that because of the method used for loading all the data) contain the field 3D model depending on the room.
It appears the drp files in Rooms\Disk1\####\unknown may be some sort of environmental model data.
I believe everything we're producing now is uncompressed, jono. The file sizes for the folders appear to be way bigger than the files from which Yazoo's decompressors extracted them. We're most likely looking at real model data. If you haven't had a chance to rip any of this stuff yourself, I'll gladly post examples of both environmental 3D files and what I think are character battle models (the "3" files Luminaire has already posted).
EDIT: Just for my own notes, it looks like we know of three sources for potential model files:
CD\Disk1\(CharacterName)\"3"
OMCM\Disk1\ (Though I think there's .TIMs in here too; maybe there's no models at all?)
Rooms\Disk1\(Room#)\unknown\
Also, I take back my earlier statement about our files being decompressed. Even the room decompressor and CD decompressor utilities seem to produce a collection of folders that are about the same size as the OUT files they came from; in fact, the CD folders are about 36 bytes smaller in size than the OUT files they came from if I'm interpreting things correctly (with Serge, Kid, and Guile coming from 0075, 0076, and 0077 OUTs respectively).
And I have no clue as to where the enemy overworld and battle models are; maybe in the MISC folder?
-
Just a thought. Anyone know of a PS2 emulator written in DirectX or OpenGL? I assume a PS2 emulator would play PS1 games, and possibly provide us with another avenue of ripping the models in-game using one of the ripping utilities we've tried previously on PS1 emus without success.
-
I was able to successfully rip Chrono Cross models (along with models from almost any PS1 game) some years ago. The key is to record the texture information, the projected polygons, AND the projection.
You get the projection by keeping close tabs on the project opcode in the PS1's vector processing unit (naturally requiring hacking an emulator).
It appears that you already know how to get the projected polygons. It's from the PS1's graphics processing unit.
The texture information can be dumped from the PS1's vram. The palette information is in there too.
I implemented it by adding dumping code (fprintf(...)) to adriPSX in those three spots to dump the raw information, then I wrote a program to use the projection information to to a reverse projection (take the projected 2-D polygon and find out the original 3-D polygon) and outputs the model in my own 3D format.
The model will be in whatever shape it was when you took the snapshot. This can be fixed by hand in a modelling program or, with some skeletal animation systems, such as the one I wrote for the game that used the ripped 3d models, it just doesn't matter.
The code is buggy and it outputs if a my own own rather limited format (triangles only, fixed point notation, not expandable at all), so I doubt it will be of use, but I don't mind giving guidance and I might very well write some better code should I find the motivation to do so.
-
:lee: YARTREBO!! THANK YOU FOR SHOWING UP!! :lee:
Yes, the Compendium has been aware of your work and one of our ultimate goals was to replicate your success with model ripping, so your appearance is a major boon to the Chrono Community.
So far we've only had luck with textures, and that was accomplished through the heavy-handed means of ripping straight from the CD image. I was hoping programs like 3D Ripper DX would capture the projected polygons, but alas, we have had no such luck. At best, the programs we've tried out just give us 2D data from VRAM as far as I know.
So really, we are still in the very beginning stages of the model capturing process, and you're unquestionably lightyears ahead of us in this area.
Now some questions for you, if you don't mind:
1.) Is the dumping code (fprint(...)) written in C/C++? Could it be inserted into any emulator, you think, or is there something about AdriPSX that makes such hacking easier? I assume we'd need the emulator source code, then we'd have to compile the code after making the necessary additions? I'm totally clueless when it comes to coding, but many members of the Community would be able to work with any documentation on the process you could provide.
2.) How much knowledge have you acquired regarding the native model formats stored on the Chrono Cross CDs? We have some file dumps from the game iso that we suspect contain models, but we're not sure yet. At least I'm not. I could provide samples if you're interested in taking a look at them.
Any knowledge or utilities you can share with us would be ABSOLUTELY appreciated, yartrebo.
EDIT: For everyone interested, I'm linking to the most complete list of PSX emulators I've found to date. PCSX has open source code available as Luminaire and jono have pointed out already I think, and AdriPSX source should be available elsewhere.
http://www.aldostools.com/psxemus.html
-
- Is the dumping code (fprint(...)) written in C/C++?
It is in C. The language you use is dictated by the language of the emulator, which is C or C++ in most cases.
- Could it be inserted into any emulator, you think, or is there something about AdriPSX that makes such hacking easier?
I was able to compile AdriPSX for my computer. I wasn't able to get other emulators to compile, or they were closed source. Nothing intrinsically easier about that emulator, and the source code is actually very messy. I have also managed to get dumps from PCSX, so that emulator will work also.
- I assume we'd need the emulator source code, then we'd have to compile the code after making the necessary additions?
That is correct. If the emulator uses plugins, then you need the code for the main emulator as well as the GPU plugin.
- How much knowledge have you acquired regarding the native model formats stored on the Chrono Cross CDs?
I have no knowledge of the format.
As far as any other information, I would recommend reading up on the PS1 architecture, particularly the vector processing unit and the GPU. There are special operations for projection and drawing polygons. That both of these operations are hardware accelerated is what makes my process feasible. Find those parts in the emulator software, and start poking around (dumping information, running it through a debugger, etc).
If you're particularly ambitious, you can try tracing the game to find where it gets the models from on the disc and how it interprets them. This would be far harder than what I did, but I have little experience in assembly programming and an experienced person should be able to succeed.
Good luck, and if you want any more information, feel free to ask.
I plan to post some source code once I can find it (I did this many years ago) and clean it up.
-
Thanks Yartrebo! If we could just get the fprint function you used and the lines it needs to be on in the PCSX and/or AdriPSX source code, that would be awesome.
I imagine we could simply cut & paste whatever code you could provide into the emulator source code and we'd be good to go as far as dumping is concerned.
-
You guys are kind of barking up the wrong tree.
I'm guessing you are trying to capture the 3d data using that program that captures OpenGL and makes then into 3D scenes.
The problem is that when the PSX GPU gets the triangle data, (Both in emulators and in the real thing), the GTE has already taken care of the polygon rotation and z-sorting, so all the GPU does is draw the triangle on the flat framebuffer. This is why your models are all coming out flat. All the OpenGL plugins do is accelerate the flat 2D drawing on the framebuffer as the 3D math was taken care of internally.
You best bet is actually getting the raw poly data from the model in the CD. It's a lot less painless.
-
Yeah, the OpenGL and DirectX 9.0 rippers have been useless precisely for that reason. Though yartrebo has had a fair amount of success by getting the AdriPSX emulator GPU to dump the data it's working with via inserting an fprint function into the sourcecode. Then he converted the GPU's output back into the original 3D model through reverse projection, it seems. Only problem is that it only allows capturing one pose at a time, so it doesn't seem to capture animation data.
-
Your best bet is to get the pieces from the CD and place them in a proper hierarchy. Bone/joint angle information is going to be lost as it's precalculated before the GTE projection. You are only going to have vertex movements (morphs). The data you are getting will be good for maybe a 3D print, but wildly inefficient for anything else.
-
Hmmm...I suppose yartrebo's method would be great for capturing battlefield models, though. No animation associated with those, right? Just a static 3D environment.
-
Those are probably the easiest to find too ^_^
-
The source code to PCSX is still around. Go and abuse it? :D
Actually it would be interesting to attack the GTE and pick off the information passed to that and that passed on to the GPU. I've always wanted to do 'deep' playing with PCSX (by that I mean emulate the GTE and GPU pairing together).
Anyhow I digress. PCSX is available although it would have a fairly sizable learning curve to modify.
Cyb
-
The source code to PCSX is still around. Go and abuse it? :D
Actually it would be interesting to attack the GTE and pick off the information passed to that and that passed on to the GPU. I've always wanted to do 'deep' playing with PCSX (by that I mean emulate the GTE and GPU pairing together).
Anyhow I digress. PCSX is available although it would have a fairly sizable learning curve to modify.
Cyb
There is good debugger wrote by Agemo. I use it for all my reversing. It is modified PCSX.
http://www.enixfans.com/html/agemo/
-
Would be great if the site wasn't "service" unavailable for all the links :)
I'll try later I guess. I can't find the debugger version Pixel made of PCSX I made a nice UI for that.
Cyb