Jak 2 Header Differences

[xTVaser]

Thanks for making this tool public. I've been trying to make it work for Jak 2 and potentially Jak 3 files but it seems like the headers are different than they are in Jak 1 and I haven't been able to make sense of it yet. Figured I would make an issue in case anyone else was also trying, or had any advice.

Instead of it being a simple 4-byte number for either the number of files in the .CGO or the length of the individual files there are many numbers before the .CGO's name none of which seem to correspond to either (the first 4-bytes seems to be common across all files though). Likewise the file names do not have reserved size but seem to just be whitespace delimited.

I've tried several of these numbers as 4-bytes and 2-bytes but nothing seems to pass through the file properly as it does in the Jak 1 files. Some output comparing the Jak 1 and 2 files.

[rmitton]

Yeah no idea, sorry, I haven't look at Jak 2/3 yet.

I had a quick look at the disc and it looks to me like the file may be compressed with some basic LZSS-style compression? The first four bytes ("BlZo") make me think it possibly could be using LZO compression.

[xTVaser]

Compression would make a lot of sense but kind of above my knowledge. I assume if it turns out to not be a standard type of compression than it will be a lot more difficult to figure out, probably more work than its worth.

One thing I guess I'll add is I noticed that none of the files are <256kb, while Jak 1 has many files that are much smaller. Many of the files are exactly the same size at this 256kb, so maybe the files are padded? Maybe somewhat related to the compression? not sure.

Thanks for taking a look!

[VodBox]

One of the files on the Jak 2 disc (DRIVERS/OVERLORD.IRX) has debug symbols mentioning miniLZO. I believe it would be possible to decompress the files, if you can find the length for each section, should be simple enough to put that length and the section into miniLZO's decompress function. Unfortunately, I don't know how to pick out the different sections, yet alone determine their lengths.

[xTVaser]

Ya I've heard about the usage of miniLZO and I gave it a really quick shot one day and didn't have much success...however I was totally rushing it so its probably worth another attempt.

An even easier way is to use the files off the PS3 collection as they are individually separated, uncompressed. On my fork (https://github.com/xTVaser/goaldis plenty of bugs still remain) I've been able to figure out the differences and disassemble all .go files, but no luck with the .o files yet.

[pe1uca]

I started some work on this.
The first 12 bytes of the files are the header of the compressed file.
4 bytes for the magic number for the file: 0x6f 0x5A 0x6C 0z42 == "oZlB"
4 bytes I'm guessing for the decompressed file size.
4 bytes the buffer size for the decompression.
After this 12 bytes you can decompress the rest of the file with miniLZO.

I'm not completely sure about the buffer size, since when the second buffer is passed to decompress it crashes, but when I pass the file as a whole the decompression finds and eof at the buffer size, still working on that.

[pe1uca]

Great news!
I successfully decompressed some files, I'll include the code soon.
In the mean time I'm including the COMMON.CGO for both Jak 2 and 3. I tried with both goaldis, this and @xTVaser fork, but with both files crashed in both (Well maybe I don't know how to use xTVaser's one, only give it one try).

CGO.zip

[xTVaser]

@edgaralvarado Nice work! The decompressed .CGO files look very similar to the Jak 1 ones now. I also encounter crashes when attempting to disassemble them though :(.

I wouldn't bother too much with my fork at this point, it only made edits to the disassembler to handle the PS3 files. On the PS3 releases the CGO/DGO files were not archived into a single zip-like file like on the ps2 releases. I was mostly trying to make a simple gui wrapper for the disassembler so it was more accessible.

I'll see if I can help in identifying the underlying issue causing the crash. At this point I'm not sure if the crashes are caused by the same reason why the PS3 files could not be disassembled. If that's the case, then my fork resolved the issue for .go files, but not .o files. I've included the PS3 files for jak 2's COMMON for comparison purposes

Jak2 - PS3 - COMMON.zip

[xTVaser]

Initial investigation - Looks like the first file in COMMON.CGO for Jak 2 has an opcode which results in trying to resolve a symbol name/type that does not exist with id 26.

So either something is not being properly initialized, or judging from this comment

goaldis/goaldis/machine.cpp

Lines 184 to 193 in a3a8987

	set_fixed_symbol(s_f, "#f", &s7[s_f]);
	set_fixed_symbol(s_t, "#t", &s7[s_t]);
	set_fixed_symbol(s_nothing, "nothing", make_nothing_func());
	set_fixed_symbol(s_zero_func, "zero-func", make_zero_func());
	// TODO- the rest of these
	set_fixed_type(s_object, "object", &s7[s_object], (uint64_t(9)<<32) | 4, NULL, NULL);
	set_fixed_type(s_structure, "structure", &s7[s_object], (uint64_t(9)<<32) | 4, NULL, NULL);
	set_fixed_type(s_basic, "basic", &s7[s_structure], (uint64_t(9)<<32) | 4, NULL, NULL);
	set_fixed_type(s_string, "string", &s7[s_basic], (uint64_t(9)<<32) | 8, NULL, NULL);
	set_fixed_type(s_function, "function", &s7[s_basic], (uint64_t(9)<<32) | 4, NULL, NULL);

maybe there are just some missing symbols that are used in Jak 2/3 that weren't an issue for Jak 1