Chrono Cross Pointer Investigation

[FaustWolf]

Alrighty, another CC thread to adorn Kajar Labs. This time I'm snooping for pointers, specifically those in the Table of Contents at the beginning of the first game CD (some will call this the Index File). I've decided that the Table of Contents reports everything in sectors (dec 2048; x800 bytes apiece). I'm still trying to figure out the "stride" of the data, i.e., how many bytes are needed to specify all the info for any one file in the TOC.

The TOC for the first CD at address 0xC000 and starts out like so:

0000C000  26 00 00 00 4E 01 00 F7-4F 01 80 00 4F 01 80 00  &...N...O...O...
0000C010  4F 01 00 33 D6 01 00 FC-D8 01 00 62 4E 02 00 4F  O..3.......bN..O
0000C020  51 02 00 25 5F 02 00 2B-77 02 00 28 7A 02 00 F1  Q..%_..+w..(z...
0000C030  9D 02 80 00 9D 02 80 00-9D 02 80 00 9D 02 80 00  ................
0000C040  9D 02 80 00 9D 02 80 00-9D 02 80 00 9D 02 80 00  ................
0000C050  9D 02 80 00 9D 02 00 74-AF 02 00 1C B5 02 00 BC  .......t........
0000C060  0C 03 00 13 98 03 80 00-98 03 80 00 98 03 80 00  ................
0000C070  98 03 80 00 98 03 80 00-98 03 80 00 98 03 80 00  ................

I want to focus on that first stride of four bytes. I'm not sure that it's a complete stride, but read in Little Endian, it would be 0x00000026. It's essentially a pointer to a file; multiply it by the sector size (x800) and you get the address x13000 -- that's the beginning of the game executable.

So we have some idea of what's going on here. But I'd be interested in input from hackers and programmers regarding a few questions:

1.) If I, say, shift the game executable one sector backward - from 0x13000 to 0x13800 - and enter in x27 as the first byte of the Table of Contents, have I just done a pointer change? Is that all there is to it? Like, will the game be hard-coded to look at address x13800 when it wants the game executable? This is assuming there's enough "free space" following the executable's original position to perform such a move without running into other data.

2.) Some files do NOT fall on sector boundaries, which really turns my world topsy-turvy. For example, Flea's overworld texture starts at address 0x0E196C98. When the game looks for a pointer to this texture, how the heck can it find it? Certainly fractional sectors can't be reported in the TOC, can they?

[MDenham]

Yes, everything seems to be four-byte entries.  Interestingly, there seems to be a lot of "xx xx 80 00" entries which repeat, followed by "xx xx 00 yy".  Presumably, therefore, it works as follows:

* 2 bytes: sector #.
* 1 byte: flags - 80 = deleted? extended information? Who knows?
* 1 byte: Not sure.

The "flags" byte may contain part of the sector number as well (if it does, it's going to be the high-order nibble of the sector number in the lower 4 bits; 20-bit sector numbers are enough to map out 2GB overall).

And as far as that fourth byte on every entry, when I say "not sure" I'm REALLY in the dark there. We'd need at least 9 bits to get a workable offset into a sector (and then just assume the bottom two bits are 0, which generally works - essentially, shift the number left two places).  Since we only have 8 viable bits there...  well, maybe it's an offset shifted right by three, but I wouldn't put money on it.

[FaustWolf]

Ah, thanks M. So it's possible to have a sector number and an offset within the sector communicated by four bytes.

I've confirmed a four-byte stride -- the next four bytes over, the ones that will show up in green shortly, point to something that looks like this:

BOOT   = cdrom:\SLUS_010.41;1
TCB      = 4
EVENT   = 16
STACK   = 801fe000

That begins at address 0xA7000, which is sector x014E. So, looking at the code posted above, the first two bytes, at least, definitely communicate whole sector starting addresses. Now to figure out what that xF7 does...

EDIT: The purple bytes point to an instruction set of some sort judging by the ASCII, at sector x014F. Then there's the x80 the next byte over.

All that I can see is that the sector "green" points to has data that is x45 bytes long, then 00 buffers fill the rest of the sector until the instruction set "purple" points to begins.

EDIT AGAIN: Hmm, this Table of Contents seems to be pointing into various instruction sets within the executable so far. I'll need to make sure that this TOC actually DOES point to the data eventually...

[FaustWolf]

Here we go, I very strongly suspect this section of the TOC is pointing at data we've studied:

0000C390  42 16 00 00 44 16 00 00-48 16 00 89 4F 16 00 00  B...D...H...O...
0000C3A0  50 16 00 3B 60 16 00 A8-75 16 00 B9 A1 16 80 00  P..;`...u.......
0000C3B0  A1 16 80 00 A1 16 80 00-A1 16 80 00 A1 16 00 D2  ................
0000C3C0  AE 16 00 BC B7 16 00 BB-C3 16 00 BC CC 16 00 54  ...............T
0000C3D0  DB 16 00 BC E4 16 00 8B-F1 16 00 BC FA 16 00 22  ..............."
0000C3E0  06 17 00 BC 0F 17 00 82-1B 17 00 BC 24 17 00 B7  ............$...
0000C3F0  34 17 00 BC 3D 17 00 E7-4B 17 00 BC 54 17 00 DA  4...=...K...T...
0000C400  61 17 00 BC 6A 17 00 93-79 17 00 BC 82 17 00 52  a...j...y......R
0000C410  8D 17 00 BC 96 17 00 5D-A2 17 00 BC AB 17 00 4F  .......].......O
0000C420  B8 17 00 BC C1 17 00 B6-D0 17 00 BC D9 17 00 BC  ................
0000C430  E7 17 00 BC F0 17 00 B5-00 18 00 BC 09 18 00 4C  ...............L
0000C440  17 18 00 BC 20 18 00 31-2F 18 00 BC 38 18 00 B7  .... ..1/...8...
0000C450  46 18 00 BC 4F 18 00 2B-5D 18 00 BC 66 18 00 89  F...O..+]...f...
0000C460  74 18 00 BC 7D 18 00 ED-8C 18 00 BC 95 18 00 FF  t...}...........
0000C470  A3 18 00 BC AC 18 00 8D-BD 18 00 BC C6 18 00 E4  ................

Dudes and dudettes, get this.
Sector x1734 "Red" is Riddel's overworld texture. (fourth byte BC)
Sector x173D "Purple" is Riddel's overworld model. (fourth byte E7)
Sector x174B "Blue" is somebody's overworld texture. (fourth byte BC)
Secor x1754 "Green" is somebody's overworld model. (fourth byte DA)
Sector x1761 "Orange" is somebody's overworld texture. (fourth byte BC)

I'm becoming fairly confident that the last byte is a file type identifier. A similar scheme is used in FF9 according to the intrepid explorers over at Qhimm, so I think that's what we have here. What I can't reconcile yet are the different "labels" applied to the models...

SUPER EDIT: Alrighty, I've found the pointers to Serge's weapon and battle model. I'm coming back with either the Dark Serge Hack or utter, painful failure.

DISAPPOINTMENT EDIT: Meh. This is merely a list of files on the CD. They are not, I repeat, they are NOT active pointers as I was hoping for. For example, I replaced Serge's texture, battle model, and weapon model TOC entries with those for Dark Serge's data. Nothing. Zilch. Null. Nada. I'm left staring at our normal, run-of-the-mill Serge. I would have expected the game to freeze at least.

Thus, this list of files is extremely useful for dumping, but not for hacking. The active pointers are somewhere else further on in the data, I'm afraid to say. And I imagine they'll be relative pointers (and not absolute pointers), making them all the more difficult to find.

Can anyone confirm that a TOC is not a list of active pointers? I just want to make sure I'm not missing something huge.

PS: As an added bit of food for thought, it would appear that there is no other list of absolute pointers to particular models, textures, etc., anywhere on the CD. That is, the *real* pointers would have to be stored in some format other than in sectors counting from the beginning of the CD.

[Gemini]

Consider those pointers as an array of unsigned int. So:
int lba=index[id]&0x7FFFFF;
int size=((index[id+1]&0x7FFFFF)-lba)*2048+(index[id]>>24)*8;

When byte 3 in index[id] contains 0x80 you can skip that value and go to the next one. Byte 4 is NOT a type classifier.

[FaustWolf]

Yay! Gemini, I was about to look for you at Romhacking.net, so I might as well ask here. 

These pointers don't seem to be the ones the game uses to find model and texture data in real-time. How would we go about finding pointers for graphical data in a PlayStation game?

I've read Duo's PlayStation pointer doc (http://www.romhacking.net/docs/372/), which says that one needs to dump memory from pSX using the debugger. I dumped the r3000 processor memory, and got what appears to be 940kb of data from the game executable. Duo's doc was for text though, and I'm not sure how much it's going to help in finding graphical data.

In the game executable, I can see instructions like ()kzgetmodel in the ASCII sidebar of my hex editor. Pointers wouldn't directly follow these instruction sets by any chance, would they?

[Gemini]

These pointers don't seem to be the ones the game uses to find model and texture data in real-time. How would we go about finding pointers for graphical data in a PlayStation game?

Pointers are pointers, no matter what they point to. No specific rule to find them, and they don't follow any specific format either. I guess you just have to cross your fingers and hope to find something.

[FaustWolf]

Heh, heh, heh, that's what I was afraid of. From your experience, do pointers typically occur in a specific section of the game CD (like the executable) or do developers just throw pointers in wherever they want to? I think in SNES games pointers can be just about anywhere.

[Gemini]

File pointers can be stored in the main exe, Parasite Eve is an example of them, but they are usually stored in other files if not in the archive itself (most frequent case). Games like Innocent Sin have files for dialogues and events and then the pointers to each subsegment stored inside the section module/exe.

[FaustWolf]

Alright, that gives me a starting point. Thanks Gemini!

EDIT: For the heck of it, and so I can dump this info from my memory for awhile, here's a pointer table for character names. It is stored in the executable, and comes just before the character names themselves.

0005BD60                                      84 00 8A 00              ....
0005BD70  8E 00 94 00 9B 00 A1 00-A7 00 AE 00 B4 00 B9 00  ................
0005BD80  BF 00 C6 00 CD 00 D4 00-DB 00 E1 00 E7 00 EE 00  ................
0005BD90  F4 00 FA 00 00 01 06 01-0B 01 11 01 18 01 1F 01  ................
0005BDA0  26 01 2D 01 32 01 39 01-40 01 46 01 4D 01 54 01  &.-.2.9.@.F.M.T.
0005BDB0  5B 01 5F 01 64 01 68 01-6E 01 75 01 79 01 80 01  [._.d.h.n.u.y...
0005BDC0  87 01 8D 01 91 01 95 01-99 01 9D 01 A1 01 A5 01  ................
0005BDD0  AA 01 AC 01 AE 01 B0 01-B2 01 B4 01 B6 01 B8 01  ................
0005BDE0  BA 01 BC 01 BE 01 C0 01-C2 01 C4 01 C6 01 C8 01  ................

[Vehek]

While I know you can't be around much right now, I want to ask something.
What were you doing to test pointer changes? I've tried messing with the pointers and I've gotten almost nothing but freezes.

edit-
My problem must be with what it does to determine the file size.

edit 2:
If you were testing by loading savestates, then that was why nothing seemed to happen.

[FaustWolf]

I had been altering the ISO itself, saving the file in a hex editor, and then loading the ISO. However, I should probably check that I was doing a fresh load from the actual load screen and not using save-states to test.

From my discussions with halkun and Cyberman, I gather that pointers to battle models are possibly stored in the LZSS-compressed battlescripts. However Vehek, could you provide an example of a pointer change you made that caused the game to freeze?

[Vehek]

Well, to be more accurate, I mostly got the battle screen stuck unable to load the PC battle models.
For example, I changed a pointer that was originally A5 29 02 84 to one of the later pointers. I can't remember what pointer changes actually crashed the game.

[FaustWolf]

BLAARGH! Turns out Vehek discovered that the table of contents is, indeed, the pointers we're looking for! I was testing with savestates all this time, but Vehek found out that these pointers work as long as you load up the game from scratch. If we get these working flawlessly, it'll be a major step in Chrono Cross hacking!!

Vehek's had some success in repointing battle and field models. I've successfully got Kid's battle model moved and repointed to make space for a Serge > Dark Serge hack. However, when I implement the changes necessary for Serge > Dark Serge I get a nasty red screen of death just as the battle model begins to load, so something's up.

Simply repointing Serge's battle model pointers to Dark Serge's causes the red screen of death as well.

To facilitate exploration of the pointers, I've attached a spreadsheet detailing where all the models and pointers are for the battle models on the first CD. The CD needs to be ripped in 2048 byte/sector format. Note that by looking at the spreadsheet, you can tell that the fourth byte we were wondering about before serves some kind of vital purpose -- it's got to be some kind of file identifier or a pointer to an instruction set that the game engine has to use to interpret the file, or something. Neither of those descriptions is qutie correct, however, because every unique model has its own identifier. Radius (both player and enemy) have 0xD2 for example, and all sword weapon packs have 0x18. All player character textures are assigned the fourth byte 0x73, and enemy textures are assigned 0xBA.

The most important rule I've discovered so far with the pointers is that no two pointers should ever overlap, even if one of the pointers applies to a character not being loaded. This is ridiculous because there's tons of duplicate pointers in the pointer table, but there are no duplicates within the range of pointers used for character battle models.

The above rule was kicking my ass while I tried in vain to move Kid's battle model for awhile. I erased Guile's battle model data, shoved Kid's backward in the ISO to make room for Dark Serge, altered the pointers accordingly, and the red screen of death occurred. The pointers for Kid's texture and model just happened to duplicate the pointers assigned to Guile at the time; when I repointered Guile's positions in the table of contents to blank sectors following Kid's, it finally worked!

However, I haven't been so lucky with Dark Serge. It *shouldn't* be a problem with model length because NeoFio happens to have the same amount of sectors for its model as Dark Serge does, believe it or not. I'll keep trying out various things I guess.

[attachment deleted by admin]

[MDenham]

However, when I implement the changes necessary for Serge > Dark Serge I get a nasty red screen of death just as the battle model begins to load, so something's up.

Okay, stupid question (and GRAAAAAAAA RETURN OF ME time also, been working on numerous other side projects entirely unrelated to CC, such as getting my server up and running): is this "red screen of death" just a flat red, same shade every time no matter what, or are there differences (flickers between several shades, different shade every time, etc.)?