Can't restore converted JD 800 sysex to JD-08

[kolutshan]

Hi and thanks so much for this tool!!!! (I just donated).

The conversion of any JD 800 sysex to the JD-08 format works well and easy but I'm unable to restore the "JD08Backup.svd" to my JD-08 following this process: https://static.roland.com/manuals/jd-08_reference/eng/17812038.html
I can copy it back to the JD-08 but restoring ends with a "Err" message on the JD-08 display. What looks interesting to me is, that the original JD-08 backup file (2 mb) and the converted sysex (131 kb) differ in file size:

Maybe the converted file is missing additional information needed by the JD-08?

[sagamusix]

Hi, many thanks for your donation. :) As mentioned in the changelog, JD-08 support is still experimental because I don't actually have a JD-08. I was hoping that I can create a backup file without all the other data that JDTools cannot generate on its own (such as sequencer backup data), but apparently that might not be possible. Which is surprising me a bit because the SVD file format is clearly meant to be modular and it should be possible to create a file with just the patch data. But if that doesn't work out, I will have to change this functionality so that it takes an existing JD800Backup.svd and just replaces the patch data inside that file. I will try to get something for you to test later today. As you provided a macOS screenshot, can I assume that you built the software yourself and I can send you a patch for the code that you then compile yourself?

[sagamusix]

Maybe you could also attach the existing JD08Backup.svd from your unit to this issue so that I have another data point to look at. Right now I only have a single file of that type to look at, and the more files I can compare against, the easier it will get to analyze what's going wrong.

[kolutshan]

Thanks for your quick reply!
Unfortunately I'm not really a developer (only limited
stuff in certain products) and used my other Windows gaming machine for the actual converting actions. I'm happy to assist as much as I can though and please find attached my original JD-08 backup file.
JD08Backup.svd.zip

[sagamusix]

Okay, no worries, I will send you a compiled version of the updated tool then, once I have a potential fix for the problem. Hopefully later today. :)

[kolutshan]

Thanks so much!

[sagamusix]

Can you check if you can load this file? JD08Backup.zip

If all goes well, it should overwrite patches A11...A88 and nothing else.

[sagamusix]

And if that file still doesn't load, try this one next:
JD08Backup-256.zip

This will overwrite all four banks A-D instead of just the first one.

[kolutshan]

I tried both files and none of them worked; same error

[sagamusix]

Okay, then the only solution I see for now is to take an existing JD08Backup.svd file and overwrite the patch data in that inside JDTools. Not really happy about that, but I guess it's the only way to proceed at this point in time.

While I work on that, could you do me another favor? I found that the JD-08 appears to have a patch parameter that the JD-800 VST doesn't have: Pan position of effect group A (item 6 in https://static.roland.com/manuals/jd-08_reference/eng/17812184.html) - could you modify a few patches to use different values for this feature and then provide the corresponding JD08Backup.svd file? I think I know which byte in the resulting patch data corresponds to this feature, but I want to be sure about it, so that I can warn in conversion that this feature is not supported.

[kolutshan]

Yup that solution seems reasonable but surely also more complicated. Of course I'm right on it :)

[kolutshan]

I changed A11 to A14 in the attached backup file
JD08Backup.svd.zip
.

[sagamusix]

Thanks, that confirms my suspicion.

[sagamusix]

Actually... I have one more idea to avoid relying on an existing backup. Do any these files load?
JD08Backup-3-4-5.zip

[sagamusix]

If neither of those three files work, here is a first attempt at creating a full backup file: jd08backup.zip

If the "64patches" variant works, that would be the best. Otherwise I'd hope that the "256patches" file works instead.

Oh, and I have one more favor to ask... Does the JD-08 store arpeggiator settings with patches? If it does, could you modify a few patches to use the arpeggiator, similar to how you did with the effect group A panning? Thanks :)

[kolutshan]

None of "3-4-5" worked but both the "64" and "256" patches variants worked just fine!! Will check for arp settings now :)

[kolutshan]

It does not look like I can save arp on or off on an individual patch level. I can either turn it on and its just on for each patch I change or I can turn it off. Saving with arp on does not make any difference if I turn it off and or change patches

[sagamusix]

Good, one feature less to care about during conversion. :) And great to hear that the 64-patch variant works as well, that simplifies the conversion a bit. Knowing that this version works does show that there's some flexibility with regards to writing valid SVD files (like in this case, just writing less patch data), and it gives me a bit of hope that we can still find a solution that doesn't require an existing backup file.

[sagamusix]

Here's another attempt at writing an SVD file without an existing backup file. Maybe this one works? JD08Backup-6.zip

[kolutshan]

Meh, Backup 6 unfortunately was not very cooperative and said nein ;)
Super cool that it seems to work with the 64 and 256 versions though, even if we have to include / merge preexisting backup file data...

[sagamusix]

Fair enough. Overwriting an existing JD08Backup.svd file will be the way to go then for now. I have just published a new release, v0.11: https://github.com/sagamusix/JDTools/releases - maybe you can test it a bit and see if it works for you. The usage for the conversion command remains the same as before, with the only difference that the output file must already exist. Let me know how it works for you!

[sagamusix]

And because I really want to get this to work now, here is one last attempt for tonight to make it work without an existing backup file. :) I think I already know the answer, but... does this file perhaps work? JD08Backup-7.zip

[kolutshan]

It was worth another try but the JD did not like it :/
I'll test release 0.11 now :)

[kolutshan]

I can already report, that version 0.11 works like a charm :)

[sagamusix]

Awesome! 🎉

And stubborn as I am, here's really the last try for tonight to make this work without an existing backup: JD08Backup-8.zip - I hope you don't mind testing another file.

[kolutshan]

I'm super happy to try as much as needed and can't say enough how happy I'm about your tool!

[kolutshan]

Holy cow, backup number 8 works! However all patches seem to have no sound

[kolutshan]

Oh wait, they have but not when I use the docked Roland keyboard but the sequencer. Maybe it has something to do with the settings

[sagamusix]

That's interesting! Basically the difference with this version is that it now tells the JD-08 that yes, there are system settings and pattern data, but they are empty. My hope would have been that it simply keeps using the existing system settings but from your observation it seems like something different happens. This is really helpful to know. But for now I guess you should restore one of the other backup files so that input via the keyboard works again. :)

[kolutshan]

Ok it gets weirder: the JD-08 now seems detected as a Jupiter Xm (midi device):

And works when I trigger it via midi but not with the docked Roland K-25 keyboard. I supposte the reason for that might be, that the Jupiter Xm does not support it?

[kolutshan]

I guess this must have something to do with the ZEN core eco system which technically should run on all ZEN based Roland hardware

[sagamusix]

I guess this must have something to do with the ZEN core eco system which technically should run on all ZEN based Roland hardware

That's what I think as well. Internally, I think all those synths share exactly the same ZenCore engine with small variations. I have no idea how restoring the backup file could have changed anything about how the device identifies itself or is identified by other devices, though. Does it go back to normal if you restore one of the other backup files?

[kolutshan]

I just rebooted the device and now the keyboard is working again but all patches are now identical. I'll restore now to the converted backup

[kolutshan]

Ok, things are back to normal now

[sagamusix]

Good to know, that means I can sleep well tonight. :) I'll try to think a bit more about these observations in the next few days and see if I can come up with another way of generating the file. I'll let you know here when there's something new to test. Until then, I hope you have lots of fun with your converted patch banks!

[kolutshan]

Thanks so much for your work and I'm looking forward for more tests and yes, there are literally thousands of patches to try now. Thanks for opening this true legacy of patches for the JD-08. I'm sure a lot of people will be super happy about that!

[kolutshan]

Oh and good night :)

[spicemix]

BTW without reading the rest of this thread I think it's useful to mention that the JD-08 will Err on anything that isn't named JD08Backup.svd so you need to remember to rename your file that when you drag it into the RESTORE folder if it has a different name.

I should also mention there's no good instructions on building this for Mac. Google installing homebrew and once homebrew is installed, download the latest JDTools.zip source code release from the releases list here, unzip in whatever folder you want this to live in, and open Terminal app. Once in terminal,

cd <drag the JDTools folder from the Finder here to paste in the full path to it>
mkdir build
cd build
brew install cmake
cmake ..
make

If all goes well (I may have some developer privileges enabled and I forget if they are necessary?) you now have JDTools as a Unix executable file inside the build directory you made there. Now you can drag that out to copy it to somewhere more convenient to use, like your JD-08 banks folder. Finally you are now able to follow the instructions provided. Only thing is you need to call JDTools in the folder it is installed, using the prefix ./ because it won't be installed in your global path. So e.g. with a pre-existing JD08Backup.svd file also in that directory,

% ./JDTools convert svd MyJD800Card.syx JD08Backup.svd

You have my permission to copy this or a derivative into your general README if it's that helpful. Thanks so much for your work on this!