Having trouble with a cpk file from a 3DS game

[RocketRobz]

NOTE: This issue has been fixed.
Only one issue remains for Style Savvy: Styling Star. See my next post for more info.

As this tool seems to be more up to date than others, I thought I might have a better luck here.

I'm having trouble with a cpk file (character_motion.cpk) from a Nintendo 3DS game called Style Savvy: Styling Star.
Whenever I use a newly created one by this tool or others (CriPakTools and YACpkTool), the game is unable to boot, and stays on black screens.

I've tried all of the CPK modes from 1-3 (0 refused to work, due to an error related to the compress flag), with and without compression, and even forcing Tver to CPKMC2.49.32, DLL3.24.00, all with no luck.
In addition, the extraction feature only extracts the first 445 files out of the 3,570 files from the cpk file.

To look into the file, download it here: https://www.dropbox.com/s/gqpagh7fa311fo7/character_motion.cpk?dl=1

Inside the file are only ID-numbered files with no filenames.
The header is structured like this in the file, and doesn't seem to match what's currently in the source code for the CPK modes.

UpdateDateTime
FileSize
ContentOffset
ContentSize
TocOffset
TocSize
TocCrc
HtocOffset
HtocSize
EtocOffset
EtocSize
ItocOffset
ItocSize
ItocCrc
GtocOffset
GtocSize
GtocCrc
HgtocOffset
HgtocSize
EnabledPackedSize
EnabledDataSize
TotalDataSize
Tocs
Files
Groups
Attrs
TotalFiles
Directories
Updates
Version
Revision
Align
Sorted
EnableFileName
EID
CpkMode
Tvers
Comment
Codec
DpkItoc
EnableTocCrc
EnableFileCrc
CrcMode
CrcTable

[Youjose]

Hello, thanks for reporting the issue, I am actually having another issue with another user as well with the CPK packing, so this is a great time to tackle both.

Also to note, I was writing this tool as I was learning python, so it's being a lot newer doesn't mean it's the best, it's filled with bad code that I want to rewrite at some point. Some parts are rewritten but some are still old.

Anyhow, about the issue with extraction, I've found the issue, the code used to assume that the maximum ID number is the maximum number of files present in the CPK. However, that's not the case, so it will be an easy fix with just merging the ID's, and I have already pushed it.

As for packing, I will have to look deeper into this, I did verify that not using CRILAYLA compression will work on some games, however, this is the 2nd game I encountered using only ITOC ID's, with CpkMode 0. So this is a great chance to revise the packer. I will likely test without CRILAYALA compression first since it's unstable as it is.

My theory on this matter is, because this CPK does not contain incrementing ID's, and they are seemingly increasing but are not incrementing by 1, the best course of action is to take the extracted filenames, check if they are all numbers, and if so, use the numbers as ID's, if not, make them increment by 1 as I currently pack them as. Will be working on this and will be back to you when I get somewhere.

[Youjose]

Hello, I pushed a quick fix that might fix this issue, I did see what you mean by the compress error, and that was a silly mistake added by me.

Moreover about the CPK table structure, not everything must be present there to make the CPK valid, so I still used the existing default table of CpkMode 0, since it's only missing the HGTOC information, which is zeroed out anyway so it might not affect the CPK. I did notice some other changes, for example this CPK uses a different align value than the default one. I don't think these differences will affect the CPK as much, however, if it still didn't boot, I will push a change that allow custom dictionaries, since they seem rather arbitrary.

I used this code here after the recent push and it works, packs the CPK, it's not compressed so it's a whopping 129MB's, here character_motion is a folder extracted by PyCriCodecs, where the directory is just full of files numbered by the ID's.

Extracting:

from PyCriCodecs import CPK

u = CPK("character_motion.cpk")
u.extract()

Packing:

from PyCriCodecs import CPKBuilder

u = CPKBuilder("character_motion", "test.cpk", 0, "CPKMC2.49.32, DLL3.24.00")

You can test those and see if it still works, tell me how it goes.

[RocketRobz]

The fix has worked, thank you!
The game now boots again with a new uncompressed cpk file made.
Replacing a file inside the extracted folder is working as well.

Sorry if this seems sudden, but there's another cpk file of which the previously mentioned tools have trouble with.
https://www.dropbox.com/s/42zu1igny9517qg/character_model.cpk?dl=1

Apparently, every file inside the cpk file (totaling up to 9,993) is named a, and as a result when extracting, keeps overwriting the extracted file named a, being the only file in the extracted folder.

According to YACpkTool, the ID numbers are also higher than 65535, which I think is the maximum signed integer number.

[Youjose]

Hmm, this game and its CPK's are weird, perhaps they are artificially edited? The only reason why this could work if only they had a lot of folders, each folder had a file named a, and they toggled an option to not preserver folder names in the CPK, and so we end up with this hell of a CPK.

Not to mention, the ID's in these CPK's are all over the place and are not incrementing properly. Also this CPK you posted is using CpkMode of 2, and CpkMode of 2 can accept ID's up to maximum 4 byte unsigned integer which is 4294967295.

I think the official CPK tool has an option for editing/patching CPK's, which explains why we see an ETOC here, and that could also explain why the ID's are just insane. I am not sure how to tackle this, I do think this is a rare case and not a general case to adapt my extractor/packer for it. It could be an easy fix for extraction which is just to tell my extractor to go over the ID's, append the filename with the ID, and extract it.

And for packing the same as well, just to tell it to split the ID/Name and pack as normal. However, we cannot possibly preserve the folder structure now that it's lost (if there's one to exist in the first place), so packing/modding this way is nigh impossible, since we cannot possibly discern if it's a filename, or a split ID_filename that we are trying to pack.

I might be late to patch the packer for this, perhaps I can do it on Monday when I am a bit more free. The extractor would be an easy fix, but it might go through some changes as I write the packer, so I will patch both at the same time. Sorry for the long rant, I just like to give my thoughts in text sometimes.