Skips links with a subdirectory

[cowbutt]

Most downloads have links that begin https://dl.humble.com/$DOWNLOAD_NAME, but some begin with a subdirectory, e.g. https://dl.humble.com/teamcherry/Hollow_Knight_1.5.80.11835_WindowsDRMFree.zip from NOVEMBER 2018 HUMBLE MONTHLY
or https://dl.humble.com/popcannibal/kind-words-windows-64-76.zip from JULY 2019 HUMBLE MONTHLY
and the downloader seems to skip these links, not even checking to see if the MD5 sum matches (e.g. if they've been updated).

[mmarcincin]

Thx for feedback.
I've found what's causing this issue, I'll fix it as soon as possible.
The workaround for now would be using all section switch.
@all

[cowbutt]

That agrees with my further testing and debugging; I think the subdirectory was a red herring, but those with a subdirectory seem to also have a platform name of 'windows mac linux', which seems to through the section filtering unless one uses @ALL, as you recommend.

[mmarcincin]

Yeah, it affects all downloads with multiple OS options, it either works like @all when it literally finds 'windows mac linux' in platform/section switch string or not at all.

[mmarcincin]

I just fixed it in version 0.4.2.
btw, I edited my comments above (I've put @all into code so that I do not refer to user :))
Let me know if it works for you now.

[cowbutt]

That seems to have fixed it for me. Thank you!

[cowbutt]

Hmm, nearly fixed it for me: it seems to get out of sync on HUMBLE BUNDLE WITH ANDROID 5 and HUMBLE BUNDLE: PC AND ANDROID 8, resulting in files being downloaded to the wrong $humbleTitle directory names.

[mmarcincin]

thanks, is that $humbleTitle the directory of previous download ?
I don't have that specific bundle so can't test it but I have at least 1 with multiple OS options (HUMBLE GAMEMAKER REBUNDLE) and it works fine there.
Check if the download which is going to wrong folder has its title represented in $titleList,
If it doesn't have that, it could be because of missing $humbleTitle source (that should not happen and would be most likely incomplete source info).

[cowbutt]

Hi - Yes, that appears to be what's happening. From the LOG-ALL for HUMBLE BUNDLE: PC AND ANDROID 8:

9 / 19 - Jack Lumber
Download - JackLumber_android_1389644733.apk
File downloaded already, skipping...
OK - File integrity (MD5) verified.
MD5: 1aea7e0f5d81198b40a4956434ae7642
Download - JackLumber_android_1389644733.apk
File downloaded already, skipping...
OK - File integrity (MD5) verified.
MD5: 1aea7e0f5d81198b40a4956434ae7642
32-bit .zip - JackLumber_Linux_x86_1389644733.zip
OK - File integrity (MD5) verified.
MD5: 2ec174d1fc08a877ea48dc457d552571
Download - JackLumber_Windows_1389644733.zip
File downloaded already, skipping...
OK - File integrity (MD5) verified.
MD5: 7593dc0b00adcf81a27cd89580e949b2
Download - JackLumber_Windows_1389644733.zip
File downloaded already, skipping...
OK - File integrity (MD5) verified.
MD5: 7593dc0b00adcf81a27cd89580e949b2

10 / 19 - Jack Lumber
MP3 - geminirue_ost_mp3_1387230441.zip
OK - File integrity (MD5) verified.
MD5: 19297c8831289b358d7b6d10f957c64d

11 / 19 - Gemini Rue
MP3 - solar2_ost_mp3_1409159048.zip
OK - File integrity (MD5) verified.
MD5: 744a4e81a3d84b307f06842a69be8df3
.tar.gz - Solar2_v1.24_amd64.tar.gz
OK - File integrity (MD5) verified.
MD5: 2e42a2574a9e15b9e1de898621d8a309
.deb - Solar2_1.24.0_amd64.deb
OK - File integrity (MD5) verified.
MD5: c370f709a87ea4f95c3146930affa599
Download - Solar2_Android_1.13_1388267491.apk
OK - File integrity (MD5) verified.
MD5: 62bc4d088bdfeb1dd79f84c96ee02d08
Download - Solar2_Android_1.13_1388267491.apk
File downloaded already, skipping...
OK - File integrity (MD5) verified.
MD5: 62bc4d088bdfeb1dd79f84c96ee02d08
Download - Solar2_v1.24_x86.exe
OK - File integrity (MD5) verified.
MD5: 413e686dbfd8adaec1d16fdcf646ca78
Download - Solar2_v1.24_x86.exe
File downloaded already, skipping...
OK - File integrity (MD5) verified.
MD5: 413e686dbfd8adaec1d16fdcf646ca78

12 / 19 - Solar 2
MP3 - jacklumber_ost_mp3_1386031151.zip
OK - File integrity (MD5) verified.
MD5: 5900491a81c1c7a7e6fc847c279f6544

13 / 19 - Jack Lumber
MP3 - bardstale_ost_mp3_1381717883.zip
OK - File integrity (MD5) verified.
MD5: 24e0be9862f4036ff2f852d4e54414dc

14 / 19 - AaaaaAAaaaAAAaaAAAAaAAAAA___ for the Awesome
Download - Aaaaa_Awesome_Windows_1389663697.zip
File downloaded already, skipping...
OK - File integrity (MD5) verified.
MD5: d0ddb81ea5d5c575af9ae306d9567723
Download - Aaaaa_Awesome_Windows_1389663697.zip
File downloaded already, skipping...
OK - File integrity (MD5) verified.
MD5: d0ddb81ea5d5c575af9ae306d9567723
32-bit .zip - Aaaaa_Awesome_x86_Linux_1389647240.zip
OK - File integrity (MD5) verified.
MD5: c513f1e8c5f1e2b7651e6c1e315326cc

15 / 19 - The Bard's Tale Soundtrack
Download - BardTaleHumble.apk
OK - File integrity (MD5) verified.
MD5: 5755e4335740089a49cbf568748ec7d7
Download - BardTaleHumble.apk
File downloaded already, skipping...
OK - File integrity (MD5) verified.
MD5: 5755e4335740089a49cbf568748ec7d7
.tar.gz - TheBardsTale_linux_1385084953.tar.gz
OK - File integrity (MD5) verified.
MD5: 5c8f4e5f0e2b0172cc42c6e019a1f5e7
Download - bardtale_linux.zip
OK - File integrity (MD5) verified.
MD5: 78c2a60f8c31a47f6af2e075ed48834b
Download - bardtale_linux.zip
File downloaded already, skipping...
OK - File integrity (MD5) verified.
MD5: 78c2a60f8c31a47f6af2e075ed48834b

I guess there's only so many heuristics a script can incorporate to mitigate against GIGO... :-)

[mmarcincin]

Open your browser and go to https://www.humblebundle.com/api/v1/order/<key text string after ?key=> and download the json file for this bundle.
If you have notepad++, open the json file and ctrl+f for find, then go to replace:
First select search mode as regular expression (matches new line is unchecked for me)
find what: \?gamekey.+(")
replace: \1
This should remove your personal gamekey from the bundle links.
You can also remove gamekey and uid at the top of the json file (replace with xxxxxxxxx).
You can post the json content here afterwards and I'll look at what's wrong with it.

Observation of log file:
I can see that for some reason gemini rue soundtrack is still taking Jack Lumber $humbleTitle which is strange cause each $humbleTitle is saved only once per folder download, It could theoretically happen in the soundtrack didn't have the $humbleTitle source but based on the next entry it looks like It's assigned to the next bundle which is strange.
The 13 / 19 - Jack Lumber and 15 / 19 - The Bard's Tale Soundtrack suggest the whole json info is not put in order as it should be.
I presume that The Bard's Tale Soundtrack folder should have been for 13/19 and The Bard's Tale folder for 15/19 (which doesn't exist at all based on this log)

[cowbutt]

humble-download-debug.json.txt

[mmarcincin]

It was malfunctioning because some of the entries didn't have download link.
I had check for 'download link' length but not for the whole section of the script which starts by saving the $humbleTitle in $titleList. The condition was there only for different part of code which separates downloads from different $humbleTitle.

If you could count all titles in the 'Humble Bundle_ PC and Android 8', how many are there. I guess 17 right ? not 19
subproducts[8], subproducts[13] (from json file) are only placeholders for websites.

machine_name                  : jacklumber_asm
url                           : http://jacklumbergame.com
downloads                     : {@{desktop_app_only=False; machine_name=jacklumber_asm;
                                download_struct=System.Object[]; options_dict=; download_identifier=; platform=asmjs;
                                download_version_number=}}
library_family_name           :
payee                         : @{human_name=Owlchemy Labs; machine_name=owlchemylabs}
human_name                    : Jack Lumber
custom_download_page_box_css  :
custom_download_page_box_html :
icon                          :

machine_name                  : aaaaaa_fortheawesome_asm
url                           : http://www.dejobaan.com/awesome/
downloads                     : {@{desktop_app_only=False; machine_name=aaaaaa_fortheawesome_asm;
                                download_struct=System.Object[]; options_dict=; download_identifier=; platform=asmjs;
                                download_version_number=}}
library_family_name           :
payee                         : @{human_name=Dejobaan Games; machine_name=dejobaangames}
human_name                    : AaaaaAAaaaAAAaaAAAAaAAAAA!!! for the Awesome
custom_download_page_box_css  :
custom_download_page_box_html :
icon                          : https://hb.imgix.net/...

I'll make the fix today :)

[mmarcincin]

I just uploaded version 0.4.3, it should be fixed there.
I added some of the pull requests' ideas/improvements as part of my version 0.4.3 (I hope you don't mind).

[cowbutt]

Excellent. It'll be a while before I have cause to use it as I've just finished syncing my DRM-free downloads - but if I get chance, I'll give it a spin on a real bundle link.

I'm pleased you found my contributions useful enough to include in 0.4.3!