[BUG] Control flow issues when attempting to analyze or merge F-Zero GX DOL with REL files

[sndrec]

Repository URL

No response

Game Name

F-Zero GX

Game Version

USA, Japan

Description

When attempting to analyze the F-Zero GX dol with the dol info command, control flow errors are returned. Furthermore, when attempting to merge the DOL and REL files together to create an ELF which can be statically analyzed, even more control flow errors are returned, along with a "Tail call analysis" failure.

[Altazimuth]

Looks like GX has two functions that are effectively the same here and they overlap, possibly to save on space. The function at 8006d1c0 does a creqv and moves on as normal, the function at 8006d1b8 does a crxor and then unconditionally bs to the instruction at 8006d1c4, so right after the first instruction of 8006d1c0.

What sort of change might be necessary to fix dtk's analysis of this kind of scenario?

[encounter]

After investigating, FZGX appears to have many functions with odd control flow. It's possible it was a big ASM file with branches that confuse the analyzer. I was able to get further by manually adding function symbols + sizes to symbols.txt for affected functions (though I don't think they're 100% accurate). Here is the configuration I'm currently using: GFZE01.zip

However, now the analysis is stuck because there are RELs that are missing in the final game:

Failed: Creating relocations for module car_colchg

Caused by:
    Failed to locate module 1

Looking at the game's fze.str, it appears that fze.test.rel is module ID 1, and doesn't appear to be present in the game files. Every REL I've checked has various relocations pointing to module ID 1, with one exception: fze.sample.rel. Notably, this is the only REL I can find a reference to in main.dol. Are the other RELs totally unused?

[Altazimuth]

The other RELs I think are used but I think are mostly mentioned in line__.rel. In the JP line__.rel the list of the strings for them starts at 19FAB0, and the actual filled-out data in RAM for the string table ends up being what would be 19FB48 in line__.rel.

The list is: advertise, sel, (blank string), option, test, customize, car_colchg, acsetup, movie, story, title, testmode, replay, pilotpoint, winning, profile, interview, and (another blank string). I'd note that acsetup has no REL file for GX, but maybe AX does have that file.

These all correlate to the modes in the game, of which there are 18. Thankfully there's a table of modes, and the modes have the corresponding enum label stored. This table is in line__.rel at 16A5E0, (each entry being char[32] then four fptrs that're filled at runtime). Either way the two modes with blank string in the rel listfiles are MD_GAME, and MD_ERRORDISP.

At 18FB90 in JP line__.rel there's a string, fz.%s.rel, which is used in conjunction with the values acquired by indexing into the REL name list with the appropriate mode to load the appropriate module.

From what I could glean, fz.sample.rel is the first-loaded REL, and line__.rel is loaded after that? I forget the exact specifics of that stage of execution.

[encounter]

Ah, nice, JP does have fz.test and fz.testmode unlike US, so I should be able to work from there. Where is line__.rel?

[Altazimuth]

line__.rel isn't hanging out in the open. It's encrypted and compressed at files/enemy_line/line_.bin. I would suggest using gfz-cli to get the file in REL form. https://github.com/RaphaelTetreault/gfz-cli/blob/main/docs/usage-guide.md#linerel-file

After decryption and decompression the JP version should have a CRC32 hash of 86B9B4F6, I believe.

As for fz.test and fz.testmode I believe they pertain to the debug functionality, which was removed from non-JP releases.

[RaphaelTetreault]

It's worth noting that GFZJ01 is the "most complete" ROM in terms of REL files. fz.test.rel is the debug mode in the game. It was stripped from future releases such as GFZE01.

Speaking with Lawn Meower that figure out the decryption, main.dol loads fz.sample.rel which stores the code which decrypts and loads line__.bin (decrypted and decompressed as line__.rel). line__.rel is basically most of the game code and sort of completes main.dol.

With line__.rel that should be everything.

As for acsetup mentioned; doesn't look like there is an fz.acsetup.rel in the arcade version GFZJ8P, though looking at it it does contain an enemy_line/line__.bin too, so perhaps it's hidden in there. AFAIK that is uninvestigated. (EDIT: specifically, I recall Lawn Meower saying that AX's main.dol contains what GX puts in line__.rel, so unsure what AX uses its line__.rel for.)

[Altazimuth]

OK so I ended up accidentally tackling understanding the exact function that causes the primary issue while trying to name some F-Zero GX maths functions. It's three functions that all share a main body but start differently.

Here's what a rough minimum ASM example would look like of code that causes the issue seen in the image up top (I've never really written proper PowerPC ASM before):

.global math_sincos_in_sincos
.global math_sincos_in_sin_in_cos
.global math_sincos_registers_only

; All functions have an angle param in r3, and return in r4
math_sincos_in_sincos:
; r4 is pointer to two floats
addi    r5, r4, 0x4 # Set second param to the 2nd (index 1) index of what r4 points to

math_sincos_in_sin_in_cos:
; r4 is pointer to float, r5 is pointer to float
crclr   4*cr1+eq # Clear eq bit in cr1
b       math_sincos_base

math_sincos_registers_only:
; r4 and r5 don't matter
; This function/instruction is the equivalent of 1:8006D1C4 in the top image
crset 4*cr1+eq # Set eq bit in cr1

math_sincos_base:
; Do maths stuff here using r3 so that f1 and f2 have the sine and cosine in them
; This instruction is the equivalent of 1:8006D1C8 in the top image
; There's three additional breaks in here (a ble, bge, and bne) but I am assuming they won't affect the control flow analysis?
beqlr   cr1 # math_sincos_registers_only bails out here
stfs    f1, 0x0(r4) # float version of f1 into address pointed to by r4
stfs    f2, 0x0(r5) # float version of f1 into address pointed to by r5
blr

; The first instruction in the function after this is the equivalent of 1:0x8006D21C in the top image

Long-winded text explanation:

At `0x8006d1b4` we have what I call `math_sincos_in_sincos`, which look like this in : * `8006d1b4: f1_f2 math_sincos_in_sincos(ushort angle, float *sincos)` * `8006d1b8: f1_f2 math_sincos_in_sin_in_cos(ushort angle, float *out_sin, float *out_cos)` * `8006d1c0: f1_f2 math_sincos_registers_only(ushort angle)`

All return the sine and cosine of the angle in f1 and f2.
math_sincos_in_sincos takes a ushort angle (r3) array of two floats (r4), sets the r5to point to the second (addi r5, out_sincos, 0x4) of r4 and then falls through to the second. The results of the function end up in the floats pointed to (in addition to f1 and f2).
math_sincos_in_sin_in_cos is like the first but you explicitly set the address of the second float (r5) before calling it. It clears the eq bit in cr1 (crxor 4*cr1+eq, 4*cr1+eq, 4*cr1+eq) then bs to the second instruction in math_sincos_registers_only (which we'll call math_sincos_base from now).
math_sincos_registers_only only cares about the angle coming in and does not place anything into whatever r4 and r5 point to. Its first instruction is to set the eq bit in cr1 (creqv 4*cr1+eq, 4*cr1+eq, 4*cr1+eq).

From here on in they all share the same code in math_sincos_base. The key thing here is the final four instructions of the function.
The fourth-from-last instruction is beqlr cr1, which means that math_sincos_registers_only returns at that point, and the other two functions do not. The other two functions then store the floats in the pointers and do an ordinary blr.

[Altazimuth]

OK I peered into dtk's code a bit. Looks like check_tail_call is marking math_sincos_base as a tail call because, when analysing math_sincos_in_sincos, math_sincos_registers_only is between it and the jump target (known_functions.range(function_start + 4..=addr).next().is_some() is true).
I'm not sure if this technically counts as a real tail call since it's not setting LR or what have you, meaning I'm not clear on the "proper" solution. For my mind the two options I can presently see are:

• Support overlapping functions in general
• Figure out that it's not a tail call to a real "function" and so don't add it to the functions list (maybe check the specific break opcode used)
• Intuit that for math_sincos_registers_only it is technically only one instruction long, and then tail calls math_sincos_base

None of these solutions seem all that trivial to me, though maybe somebody with more understanding of the codebase would find it easier.

FWIW swapping for

                        if first_end > second {
                            match skip_alignment(section, second, first_end) {
                                Some(addr) => addr,
                                None => continue,
                            }
                        } else {
                            match skip_alignment(section, first_end, second) {
                                Some(addr) => addr,
                                None => continue,
                            }
                        };

seems to work specifically for doing dol info on F-Zero GX's main.dol (and elf merge on main.dol and all the FZGX .rel files), but I'm uncertain if this causes any issues elsewhere. dol split will not work