[Windows] FDS games don't start if system path contains a special character

[bslenul]

Hey!

If the disksys.rom file is in a path that contains a special character, FDS games will fail to boot on Windows.
As a test I moved the BIOS in a "é" folder on my desktop and changed Settings > Directory > System/BIOS path in RetroArch:

[INFO] [Environ]: SYSTEM_DIRECTORY: "C:\Users\B-S\Desktop\é".
[INFO] [Environ]: GET_SAVE_DIRECTORY.
[libretro INFO] Loading D:\programmes\RetroArch\cache\Zelda no Densetsu - The Hyrule Fantasy (Japan).fds...

[libretro WARN] Not an iNES file!
[libretro INFO]  FCEU_MakeFName: C:\Users\B-S\Desktop\é\disksys.rom
[libretro WARN] FDS BIOS ROM image missing!
[libretro WARN] An error occurred while loading the file.

Full log: retroarch_fceumm.log
It works fine on my Linux Mint VM however.

[negativeExponent]

this was suppose to be fixed in RA long time ago. would like to look into it, nut not a windows user here.

just check in src/general.c, func FCEU_MakeFName() is there's any in there that affects how windows reads path with special chars.

or maybe fopen itself is the issue:

libretro-fceumm/src/file.c

Line 78 in ff7d528

FCEUFILE * FCEU_fopen(const char *path, const char *ipsfn,

[bslenul]

This is mostly Chinese to me :p But I can compile if you have any idea, I'd be happy to test the changes on Windows!

And to be clear, with this core it seems to only affect the system dir path, I can load a game with a special character in its path with no issue.

[bslenul]

Yay, the game loaded with #449 👍

[negativeExponent]

good to hear

lemme know which other cores needs this update

[bslenul]

The ones I've noticed:

• DeSmuME: If the ROM name or the path where the ROM is located has Chinese characters, desmume will make an error desmume#70
• PicoDrive: The core doesn't like special characters in path on Windows. picodrive#123
• Flycast: Japanese characters in a folder directory do not boot a game flycast#1068
• Mesen: [Windows] FDS games don't start if system path contains a special character Mesen#5
• Nestopia: [Windows] FDS games don't start if system path contains a special character nestopia#33

Also noticed, but haven't opened an issue yet:

• PCSX ReARMed: https://github.com/libretro/pcsx_rearmed
• fixNES: https://github.com/libretro/fixNES

And probably more I'm not aware of... 😓

Really appreciated anyway! ❤️ Some people use special characters in their Windows username, and since default install dir for RetroArch with the installer is in %APPDATA% (which contains the username in the path...) it can be REALLY frustrating for them because they did everything correctly but it's still not working :p And it's not always easy to troubleshoot either...

[negativeExponent]

Updating pcsx_rearmed. If you are able to test, kindly test with cd-images in special character-named folders as well, if possible if you have ccd, pbp, cue, bin files as well.

thanks.

[bslenul]

Hm I can't build it, I get this at the end:

G:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: libretro-common/compat/fopen_utf8.o:fopen_utf8.c:(.text+0xd): undefined reference to `utf8_to_utf16_string_alloc'
G:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: libretro-common/compat/fopen_utf8.o:fopen_utf8.c:(.text+0x18): undefined reference to `utf8_to_utf16_string_alloc'
collect2.exe: error: ld returned 1 exit status
make: *** [Makefile:369: pcsx_rearmed_libretro.dll] Error 1

[negativeExponent]

ok wait

@bslenul - pcsx updated. try to clone again and recomple

[bslenul]

OK, so no issue with system dir, no issue with bin+cue/pbp/ccd+img+sub either (tested in both dir name and filename), however chd didn't load:

[libretro INFO] Loaded CD Image: F:\Roms\Sony - PlayStation\日本語\日本語.chd.
[libretro INFO] unsupported/invalid CD image: F:\Roms\Sony - PlayStation\日本語\日本語.chd
[ERROR] [CONTENT LOAD]: Failed to load content

[negativeExponent]

is that suppose to be valid? cant use " \ " as part of filename

[bslenul]

No no \ isn't part of the filename, it's 日本語.chd inside a 日本語 folder, sorry I should've pick different names for the folder and the game to avoid confusion :p

edit: Here, renamed to avoid confusion (folder 日本語, game ïïï.chd):

[libretro INFO] Loaded CD Image: F:\Roms\Sony - PlayStation\日本語\ïïï.chd.
[libretro INFO] unsupported/invalid CD image: F:\Roms\Sony - PlayStation\日本語\ïïï.chd
[ERROR] [CONTENT LOAD]: Failed to load content

[negativeExponent]

this is rather confusing since chd path have its own file handling function, so its not part of these changes. can you verify if this chd file actually works on other psx cores?

[bslenul]

Oh I see, didn't know. Beetle PSX HW crashed on load content:

[libretro INFO] Loading F:\Roms\Sony - PlayStation\日本語\ïïï.chd...
[libretro INFO] chd_read_toc: finished
[libretro ERROR] TOC first(99)/last(0) track numbers bad.

The game starts with SwanStation however.

[negativeExponent]

can you try changing this line:

https://github.com/libretro/pcsx_rearmed/blob/master/deps/libchdr/include/libchdr/coretypes.h#L24

to this?

#include <compat/fopen_utf8.h>
#define core_fopen(file) fopen_utf8(file, "rb")

recompile and test

@twinaphex libchdr need to be changed as well. you have any inputs with this?

[bslenul]

That did it 👍

[negativeExponent]

ugh.. so libchdr had to be change as well...

[inactive123]

@negativeExponent Thanks for letting me know about libchdr.

[negativeExponent]

@bslenul I am starting a different approach of solving this issue using vfs approach. If you would try testing again if there are still special character issues (like CHD).

get the source from my repo/branch
https://github.com/negativeExponent/pcsx_rearmed_libretro/tree/vfs

thanks.

[bslenul]

My test games loaded fine (bin+cue/pbp/ccd+img+sub/chd/m3u), tested special characters in system path as well and still no issue 👍

[negativeExponent]

My test games loaded fine (bin+cue/pbp/ccd+img+sub/chd/m3u), tested special characters in system path as well and still no issue

just want to confirm, this is off frm the vfs branch of my repo, right? can you show the log when you have the time.

thanks.

[bslenul]

Yup, here you can see the commit version in bottom-left + path and filename for the CHD:

And here's the log: https://pastebin.com/09Txs6JX

Commit version:

[libretro INFO] Starting PCSX-ReARMed r15-1202-gca4e3f8

and I renamed the system folder in Russian to mix even more:

[INFO] [Environ]: SYSTEM_DIRECTORY: "C:\Россия".
[libretro INFO] found BIOS file: SCPH5501.bin

BIOS loaded properly, and the game too as you can see (Japanese folder + special character ï in filename):

[libretro INFO] Loaded CD Image: F:\Roms\Sony - PlayStation\日本語\ïïï.chd[+chd].
[libretro INFO] Track 01 (DATA) - Start 00:02:00, Length 03:11:10
[libretro INFO] CD-ROM Label: SLUS_01488
[libretro INFO] CD-ROM ID: SLUS01488
[libretro INFO] CD-ROM EXE Name: SLUS_014.88;1

[negativeExponent]

@bslenul thanks for testing and confirmation. Ill try compiling in on other platforms i am able to compile on and see if there are issues. ill forward final PR soon.

thanks

UPDATE:
@bslenul. I am making the final PR for pcsx. if you are able to test and see if there are no regressions, that would be cool
libretro/pcsx_rearmed#516

[negativeExponent]

@bslenul Hi we can probably close this one now, and leave the other affected cores open. As for fceumm case, this is probably fixed now

[bslenul]

Sure! Thank you again :D