Support Kawai K5000S/W/R #72

christofmuc · 2021-01-09T13:20:08Z

There is good documentation available, and generally there is nothing that should not work already in KnobKraft Orm.

https://www.cyborgstudio.com/wimpy/synthmp3s/kawai/k5000/manual/k5000midiimplementation.pdf

Just that the device is really complex, and you probably want to support different data types.

I'd say we wait for the multi data types implementation before trying.

markusschloesser · 2025-02-08T14:43:21Z

https://github.com/coniferprod/k5ktools
Explanation at https://digitalsynth.net/posts/2022/07/30/making-sense-of-kawai-k5000-patch-data-files/
Will probably be helpful

markusschloesser · 2025-02-08T14:44:32Z

And I'd suggest to focus on S and R, the extra W stuff is just rompling which nobody needs anyway 😇
Edit: S and R only have the additive synth part, W additionally a ROM part.
So suggesting to focus only on the additive parts, which all 3 have in common.

W, S and R have Bank A
S and R have Bank D
If expansion installed also Banks E and F
W additionally for the Rom parts Bank B

markusschloesser · 2025-02-09T18:27:55Z

Valid file sizes

File Size Number Of Sources
254 bytes 2 PCM
340 bytes 3 PCM
426 bytes 4 PCM
512 bytes 5 PCM
598 bytes 6 PCM
1060 bytes 1 PCM + 1 ADD
1146 bytes 2 PCM + 1 ADD
1232 bytes 3 PCM + 1 ADD
1318 bytes 4 PCM + 1 ADD
1404 bytes 5 PCM + 1 ADD
1866 bytes 2 ADD
1952 bytes 1 PCM + 2 ADD
2038 bytes 2 PCM + 2 ADD
2124 bytes 3 PCM + 2 ADD
2210 bytes 4 PCM + 2 ADD
2758 bytes 3 ADD
2844 bytes 1 PCM + 3 ADD
2930 bytes 2 PCM + 3 ADD
3016 bytes 3 PCM + 3 ADD
3650 bytes 4 ADD
3736 bytes 1 PCM + 4 ADD
3822 bytes 2 PCM + 4 ADD
4542 bytes 5 ADD
4628 bytes 1 PCM + 5 ADD
5434 bytes 6 ADD
From https://www.optimolch.de/jens.groh/K5000/Kenji/digests/ka1formt.html

So if we only do the additive part, valid file sizes are:
1866 bytes 2 ADD
2758 bytes 3 ADD
3650 bytes 4 ADD
4542 bytes 5 ADD
5434 bytes 6 ADD

markusschloesser · 2025-02-09T18:36:41Z

Patch management utility:
https://github.com/coniferprod/KSynthLib
And
https://github.com/coniferprod/k5ktool

markusschloesser · 2025-02-11T00:30:13Z

ADD Singles (additive Presets)

ONE SINGLE DUMP REQUEST (ADD A/D 1 -128) (Only for K5000S/R)
Format: F0 40 <ch> 00 00 0A 00 00 <data> F7 (Bank A)
Format: F0 40 <ch> 00 00 0A 00 02 <data> F7 (Bank D)
Format: F0 40 <ch> 00 00 0A 00 03 <data> F7 (Bank E, only when Expansion ME-1 installed)
Format: F0 40 <ch> 00 00 0A 00 04 <data> F7 (Bank F, only when Expansion ME-1 installed)

<ch>: MIDI channel (00-0F)
<DATA>: patch no.: 0 - 7f

coming from the synth according to manual:

ONE SINGLE DUMP (ADD, A1 - 128)
Format: F0 40 <ch> 20 00 0A 00 00 <sub1> <DATA> F7 (Bank A)
Format: F0 40 <ch> 20 00 0A 00 02 <sub1> <DATA> F7 (Bank D)
Format: F0 40 <ch> 20 00 0A 00 03 <sub1> <DATA> F7 (Bank E, only when Expansion ME-1 installed)
Format: F0 40 <ch> 20 00 0A 00 04 <sub1> <DATA> F7 (Bank F, only when Expansion ME-1 installed)

<ch>: MIDI channel (00-0F)
<sub1>: Tone No.(00-7F)
<DATA>: One ADD tone data

The Structure of ADD tone data:
(Single Tone DATA) + (ADD Wave KIT DATA) * No. of src (included muted src)

The structure of the Single Tone DATA:
(check sum) + (effect DATA) + (common DATA) + (source DATA) * No. of used sources

The structure of ADD Wave KIT DATA
(check sum) + (HC KIT DATA) + (HC code1 DATA) + (HC code2 DATA) + (Formant Filter DATA) + (Harmonic Envelope data) + Loudness sense select)

Combis/Multis
request 01 F0 40 00 00 00 0A 20 00 00 F7
request 02 F0 40 00 00 00 0A 20 00 01 F7

Combis/Multis only reference the ADD

You don't wanna bulk request things, looks nasty

markusschloesser · 2025-02-16T16:42:08Z

Acknowledge Format
K5000W/S/R transmit these data after receiving the dump data:
F0 40 cc aa 00 0A F7

cc: Unit midi channel number: 0-f (ch. 1 - ch. 16)

aa: Function Number
40: Write Complete
41: Write Error
42: Write Error by protect
44: Write Error by memory full
45: Write Error by no expansion board

markusschloesser · 2025-02-16T16:46:39Z

ID Request(KAWAI)
K5000W/S/R do not transmit these data but receive only.
F0 40 cc 60 F7
cc: Unit channel number: 0-F (ch.1 - ch.16)

ID request reply
K5000 transmit these data after receiving the KAWAI ID request.
F0 40 cc 61 00 0a ii F7
cc: Unit channel number: 0 - F (ch 1-16)
ii: Sub ID number
01: K5000W
02: K5000S
03: K5000R

markusschloesser · 2025-02-18T11:18:36Z

I have device detection, bank descriptors working, program dumps never finish because of #423
K5000 do not have an edit buffer

markusschloesser · 2025-02-18T11:44:17Z

Acknowledge Format
K5000W/S/R transmit these data after receiving the dump data:
F0 40 cc aa 00 0A F7

cc: Unit midi channel number: 0-f (ch. 1 - ch. 16)

aa: Function Number
40: Write Complete
41: Write Error
42: Write Error by protect
44: Write Error by memory full
45: Write Error by no expansion board

@christofmuc how could I implement something that uses this?
I've searched issues and code, but all mentions of Acknowledge are about sending ack from kk to synth.
This would be especially useful as I have yet to find out if a k5000 has the expansion installed or not

christofmuc · 2025-02-19T00:14:13Z

We have a handshake possibility - isPartOfSingleProgram() can return a Tuple instead of just True or False.

When you want to send a message in reply to the message you just got you can return (True, [f0 .... ]), with the second item the next message to send to the synth. Then the Write complete would become part of the bank dump (they'd need to be removed before sending).

But I agree, we need a new way to terminate the download. Currently. it always assumes the number of patches to be downloaded is known upfront. We can either add a way to flag that bank requests should stop, or we can use the isComplete type of call that inspects all messages and decides it has enough.

Maybe the simplest thing that could work is to extend the return value of the isPartOfSingleProgramDump, allowing something like this:

requestBankDump(bankNo) -> returns first program dump request message. 
All subsequent received messages are fed into this loop:
for receivedMessage:
    keep, handshakereply, stop = isPartOfBankDump(receivedMessage)
    if keep:
       bank.append(receviedMessage)
    if stop:
      break
    if handshakereply is not None:
      send(handshakereply)
    if isBankDumpFinished(bank):
      break

# After the loop, present all kept messages to the bankloader, that can assemble an arbitrary amount of patches from all kept messages:
patches = extractPatchesFromAllBankMessages(bank)

I think the only thing missingis to allow the triple return value. And isBankDumpFinished() could be implemented empty (return False) because there is a different way to stop the loop.

markusschloesser · 2025-02-19T00:23:03Z

We have a handshake possibility - isPartOfSingleProgram() can return a Tuple instead of just True or False.

When you want to send a message in reply to the message you just got you can return (True, [f0 .... ]), with the second item the next message to send to the synth. Then the Write complete would become part of the bank dump (they'd need to be removed before sending).

the handshake mentioned in #72 (comment) is about the synth sending a message to kk (not the other way around)

But I agree, we need a new way to terminate the download. Currently. it always assumes the number of patches to be downloaded is known upfront. We can either add a way to flag that bank requests should stop, or we can use the isComplete type of call that inspects all messages and decides it has enough.

Maybe the simplest thing that could work is to extend the return value of the isPartOfSingleProgramDump, allowing something like this:
requestBankDump(bankNo) -> returns first program dump request message. 
All subsequent received messages are fed into this loop:
for receivedMessage:
    keep, handshakereply, stop = isPartOfBankDump(receivedMessage)
    if keep:
       bank.append(receviedMessage)
    if stop:
      break
    if handshakereply is not None:
      send(handshakereply)
    if isBankDumpFinished(bank):
      break

# After the loop, present all kept messages to the bankloader, that can assemble an arbitrary amount of patches from all kept messages:
patches = extractPatchesFromAllBankMessages(bank)
I think the only thing missingis to allow the triple return value. And isBankDumpFinished() could be implemented empty (return False) because there is a different way to stop the loop.

I haven't implemented the bank dump feature, mainly because the unpacking looks especially nasty.
Can this also be done with program dumps? Also, this does not rely on handshakes (see first reply), the synth just doesn't send ANYTHING when the patch entry is empty (and of course kk never advances)

christofmuc · 2025-02-19T00:26:45Z

the handshake mentioned in #72 (comment) is about the synth sending a message to kk (not the other way around)

well, every message the synth sends will be presented to the isPartOf.ProgramDump(), so you will be able to inspect it.
Are you talking about upload or download now?

I haven't implemented the bank dump feature, mainly because the unpacking looks especially nasty. Can this also be done with program dumps? Also, this does not rely on handshakes (see first reply), the synth just doesn't send ANYTHING when the patch entry is empty (and of course kk never advances)

That's nasty. We currently guarantee that hangs when the synth stops sending. I think we had a ticket with timeouts somewhere, but that was hard to implement. What is the logic? Can you just send all program requests in advance?

markusschloesser · 2025-02-19T00:33:52Z

the handshake mentioned in #72 (comment) is about the synth sending a message to kk (not the other way around)

well, every message the synth sends will be presented to the isPartOf.ProgramDump(), so you will be able to inspect it. Are you talking about upload or download now?

this is when sending a patch to the synth ("K5000W/S/R transmit these data after receiving the dump data:"). it will reply with one of those messages

I haven't implemented the bank dump feature, mainly because the unpacking looks especially nasty. Can this also be done with program dumps? Also, this does not rely on handshakes (see first reply), the synth just doesn't send ANYTHING when the patch entry is empty (and of course kk never advances)

That's nasty. We currently guarantee that hangs when the synth stops sending. I think we had a ticket with timeouts somewhere, but that was hard to implement. What is the logic? Can you just send all program requests in advance?

I'd suggest a timeout, wait 5 seconds and move on to the next number (until end of bank defined in bankdescriptors).
Currently kk sends and receives:

06:56:51.997: Out UFX+ MIDI Port 2 Sysex [f0 40 00 00 00 0a 00 02 00 f7]
06:56:53.196: In  UFX+ MIDI Port 2 Sysex [f0 40 00 20 00 0a 00 02 00 46 00 00 1e 26 17 25 33 15 00 06 4a 00 00 1a 47 06 4a 00 31 0f 00 00 3e 31 00 0c 34 63 00 63 02 42 40 40 40 40 40 42 00 44 61 4c 65 61 64 20 20 7f 00 06 05 1f 01 02 00 40 02 00 40 00 73 13 0b 12 0d 07 08 0c 00 58 4a 58 31 27 27 5f 40 01 0a 01 0a 00 7f 10 00 7f 02 00 03 50 00 40 0d 5b 01 4a 02 5f 00 40 00 00 40 00 00 40 00 00 40 04 00 58 40 00 03 40 03 40 03 40 40 00 00 04 00 00 7f 77 40 40 00 4e 40 0d 40 3e 40 40 7f 40 40 02 00 34 7f 4e 00 01 40 40 40 40 3f 40 40 40 00 5d 01 0e 0a 03 40 00 40 00 40 00 7f 10 00 7a 02 00 03 50 00 40 03 54 00 40 02 5f 00 40 00 00 40 00 00 40 00 00 40 04 00 34 3a 00 00 40 04 40 40 40 40 00 00 06 00 00 4b 77 40 7f 00 56 40 36 40 3e 40 40 7f 40 40 00 00 00 7f 00 7f 11 48 40 40 40 36 40 40 40 00 5d 01 0e 0a 08 40 00 40 00 40 00 7f 10 00 77 02 00 02 44 03 50 03 54 00 40 02 5f 00 40 00 00 40 00 00 40 00 00 01 04 00 34 44 00 00 40 04 40 40 40 40 01 00 04 00 00 0d 77 40 7f 00 76 7f 76 7f 76 40 40 7f 40 40 07 00 00 7f 00 7f 11 48 40 40 40 3f 40 40 40 00 5d 01 0e 0a 08 40 00 40 00 40 00 7f 10 00 55 00 00 01 5f 03 50 00 42 03 54 02 5f 00 40 00 00 40 00 00 40 00 00 40 03 27 34 43 00 00 40 04 40 40 40 40 01 00 04 00 00 0d 77 40 7f 00 76 7f 76 7f 76 40 40 7f 40 40 07 00 0a 7a 0a 7a 11 48 40 40 40 3f 40 40 40 00 5d 01 0e 0a 08 40 00 40 00 40 00 7f 10 00 77 02 00 02 44 03 48 03 54 00 40 02 50 00 40 00 00 40 00 00 40 00 00 7f 04 00 34 3c 00 00 40 04 40 40 40 40 00 00 04 00 00 0d 77 40 7f 00 6b 7f 6b 7f 3e 40 40 7f 40 40 07 00 00 7f 00 7f 11 48 40 40 40 3f 40 40 40 00 5d 01 0e 0a 08 40 00 40 00 40 71 00 37 00 40 04 00 00 00 00 00 00 00 00 00 40 40 40 40 00 7f 00 7f 2c 01 00 5e 00 0d 00 40 02 61 7f 00 00 00 75 7d 7b 7b 72 72 6e 71 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7e 76 7f 7f 7f 7a 7e 7b 74 75 74 73 7c 7a 7c 7b 7d 6c 72 6c 74 6e 77 6c 70 66 6e 72 72 73 6b 73 79 6b 70 6d 73 6d 6d 6c 70 64 71 6a 70 75 7d 7b 7b 72 72 6e 71 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7e 76 7f 7f 7f 7a 7e 7b 74 75 74 73 7c 7a 7c 7b 7d 6c 72 6c 74 6e 77 6c 70 66 6e 72 72 73 6b 73 79 6b 70 6d 73 6d 6d 6c 70 64 71 6a 70 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7c 79 70 66 5d 54 4a 40 37 2d 23 1a 10 06 00 00 00 00 00 04 0c 15 1e 27 30 39 41 4a 53 5b 64 6d 75 77 7a 7c 7e 7f 79 77 74 71 6e 6b 68 65 62 5f 5d 5a 57 54 51 4e 4b 48 45 43 40 3d 3a 37 34 31 2e 7f 3f 4a 3f 55 3f 1d 00 7f 3f 5d 33 55 34 1d 00 7f 3f 4a 34 55 35 1d 00 7f 3f 35 33 43 34 1d 00 7f 3f 4a 33 55 34 1d 00 7f 3f 36 33 45 34 1d 00 7f 3f 4a 33 55 34 1d 00 7f 3f 38 33 46 34 1d 00 7f 3f 4a 33 55 34 1d 00 7f 3f 3a 33 47 34 1d 00 7f 3f 4a 33 55 34 1d 00 7f 3f 3b 33 48 34 1d 00 7f 3f 4a 33 55 34 1d 00 7f 3f 3d 33 49 34 1d 00 7f 3f 4a 33 55 34 1d 00 7f 3f 3f 33 4a 34 1d 00 7f 3f 4a 33 55 34 1d 00 7f 3f 40 33 4b 34 1d 00 7f 3f 4a 33 55 34 1d 00 7f 3f 42 33 4d 34 1d 00 7f 3f 4a 33 55 34 1d 00 7f 3f 44 33 4e 34 1d 00 7f 3f 4a 33 55 34 1d 00 7f 3f... (3746 more bytes)]
06:56:53.196: Out UFX+ MIDI Port 2 Sysex [f0 40 00 00 00 0a 00 02 01 f7]
06:56:53.824: In  UFX+ MIDI Port 2 Sysex [f0 40 00 20 00 0a 00 02 01 3d 00 00 00 2d 12 64 3c 1e 09 44 0a 00 01 0d 0b 26 48 2b 00 2b 00 14 13 2b 00 0b 46 00 05 09 00 41 41 40 40 3f 3f 42 00 44 69 6e 6f 73 61 75 72 7f 01 00 03 07 00 02 00 40 02 00 40 00 73 13 0b 12 0d 07 08 0c 00 58 4a 58 31 27 27 5f 40 01 0a 01 0a 00 7f 10 00 7f 02 00 01 58 00 40 0d 5e 00 40 02 5f 00 40 00 00 40 00 00 40 00 00 20 04 00 40 42 00 00 30 01 40 01 40 40 00 00 04 01 01 6a 60 40 7f 00 0c 62 64 62 3e 40 40 7f 40 40 04 00 00 7f 00 7f 07 40 40 40 40 06 40 40 40 00 5e 01 00 00 00 40 00 40 00 40 00 7f 10 00 7f 02 00 01 58 00 40 0d 21 00 40 02 5f 00 40 00 00 40 00 00 40 00 00 60 04 00 40 3e 00 00 50 01 40 01 40 40 00 00 04 01 01 61 60 40 7f 00 0c 62 64 62 3e 40 40 7f 40 40 04 00 00 7f 00 7f 07 40 40 40 40 06 40 40 40 00 5e 01 00 00 00 40 00 40 00 40 00 7f 10 00 7f 02 00 00 40 00 40 01 59 00 40 02 5f 00 40 00 00 40 00 00 40 00 01 40 03 19 40 40 00 00 40 04 40 40 40 40 00 00 05 01 00 7f 40 40 71 00 3e 60 65 40 65 40 40 78 40 40 00 00 5b 72 73 7b 34 40 40 40 40 37 40 40 40 00 5b 01 0e 0a 00 40 00 40 00 40 01 00 37 00 30 04 7f 00 00 00 00 00 00 00 00 40 40 40 40 00 62 00 74 3f 7f 3f 6c 09 1c 3f 71 00 7f 40 0f 00 3f 69 36 69 36 69 36 5c 1a 68 2c 50 17 4a 17 4a 17 4a 17 4a 17 4a 17 4a 17 4a 17 4a 17 4a 17 4a 17 4a 17 4a 17 4a 17 4a 17 4a 17 4a 17 4a 17 4a 17 4a 17 4a 17 4a 17 4a 17 4a 17 4a 17 4a 17 4a 17 69 36 69 36 69 36 69 36 79 55 79 55 79 55 79 55 79 55 79 55 79 55 79 55 79 55 79 55 79 55 79 55 79 55 79 55 79 55 79 55 79 4f 79 49 73 36 64 2c 5a 21 4e 17 4a 17 4a 17 4a 17 4a 17 4a 17 4a 17 7f 7f 75 7b 79 76 74 72 7f 77 6a 6f 6c 68 71 62 00 00 00 00 00 00 00 00 00 00 79 72 6b 7f 78 68 7f 79 6f 00 00 00 00 00 00 00 00 00 00 7b 75 77 79 7b 7d 7f 76 6c 7b 00 00 00 00 00 00 00 00 00 00 76 6d 7b 78 75 75 75 78 72 6b 6f 00 00 00 00 00 00 00 00 00 00 7f 7f 79 74 79 7b 7c 7d 7f 7f 7f 00 00 00 00 00 00 00 00 00 00 7f 7f 7f 7f 7f 7f 7d 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 79 70 68 11 7f 3f 7f 3f 26 3f 28 00 00 3f 7f 3f 26 00 28 3f 7f 3f 7f 3f 26 3f 28 00 00 3f 7f 3f 26 00 28 3f 7f 3f 7f 3f 26 29 28 00 00 3f 7f 3f 26 00 28 3f 7f 3f 7f 3f 26 3c 28 00 00 3f 7f 3f 26 00 28 3f 7f 3f 7f 3f 26 27 28 00 00 3f 7f 3f 26 00 28 3f 7f 3f 7f 3f 26 39 28 00 00 3f 7f 3f 26 00 28 3f 7f 3f 7f 3f 26 26 28 00 00 3f 7f 3f 26 00 28 3f 7f 3f 7f 3f 26 34 28 00 00 3f 7f 3f 45 00 28 3f 7f 3f 7f 3f 45 28 29 00 00 3f 7f 3f 45 00 28 3f 7f 3f 7f 3f 45 29 28 00 00 3f 7f 3f 45 00 28 3f 7f 3f 7f 3f 45 2b 28 00 00 3f 7f 3f 45 00 28 3f 7f 3f 7f 3f 45 2d 28 00 00 3f 7f 3f 45 00 28 3f 7f 3f 7f 3f 45 2f 28 00 00 3f 7f 3f 45 00 28 3f 7f 3f 7f 3f 45 31 28 00 00 3f 7f 3f 45 00 28 3f 7f 3f 7f 3f 45 32 28 00 00 3f 7f 3f 5c 00 28 3f 7f 3f 7f 3f 5c 33 28 00 00 3f 7f 3f 5c 00 28 3f 7f 3f 7f 3f 5c 33 28 00 00 3f 7f 3f 5c 00 28 3f 7f 3f 7f 3f 5c 33 28 00 00 3f 7f 3f 5c 00 28 3f 7f 3f 7f 3f 5c 32 28 00 00 3f 7f 3f 5c 00 28 3f 7f 3f 7f 3f 5c 32 28 00 00 3f 7f 3f 5c 00 28 3f 7f 3f 7f 3f 5c 31 28 00 00 3f 7f 3f 5c 00 28 3f 7f 3f 7f 3f 5c 30 28 00 00 3f 7f 3f 5c 00 28 3f 7f 3f 7f 3f 5c 2f... (1962 more bytes)]
06:56:53.825: Out UFX+ MIDI Port 2 Sysex [f0 40 00 00 00 0a 00 02 02 f7]
06:56:54.480: In  UFX+ MIDI Port 2 Sysex [f0 40 00 20 00 0a 00 02 02 5c 00 00 25 07 1b 2d 33 2a 19 0c 0e 63 00 19 0c 09 35 00 2d 15 20 06 4a 00 00 0c 04 63 00 63 02 40 40 40 40 40 40 40 00 4f 74 61 53 74 61 63 6b 78 00 00 04 0f 00 02 00 40 02 00 40 00 73 13 0b 12 0d 07 08 0c 00 58 4a 58 31 27 27 5f 40 01 0a 01 0a 00 7f 10 00 7b 02 00 01 54 00 40 03 59 00 40 02 5f 00 40 00 00 40 00 00 40 00 00 40 04 00 40 40 00 00 40 04 40 40 40 40 00 00 05 02 01 0a 77 40 71 02 3e 60 46 45 3e 40 40 7f 40 40 01 00 0c 7d 0c 7d 18 48 40 40 40 3f 40 40 40 00 5b 00 0e 0a 03 40 00 40 00 40 00 7f 10 00 78 02 00 01 54 00 40 03 59 00 40 02 5f 00 40 00 00 40 00 00 40 00 00 40 02 63 40 7c 00 00 40 04 40 40 40 40 00 00 05 03 01 17 54 40 71 00 3f 60 3f 47 3e 40 40 7f 40 40 01 00 0a 7d 0a 7d 18 3a 40 40 40 3f 40 40 40 00 5b 01 0e 0a 00 40 00 40 00 40 00 7f 10 00 75 02 00 01 54 00 40 03 59 00 40 02 5f 00 40 00 00 40 00 00 40 00 00 40 04 00 40 46 00 00 40 04 40 40 40 40 00 00 06 03 02 1a 77 40 0e 05 3a 40 3d 01 3e 40 40 6c 40 40 01 00 3a 50 3d 7f 18 40 40 40 40 3f 40 40 40 00 5b 01 0e 0a 00 40 00 40 00 40 00 7f 10 00 60 02 00 01 54 00 40 03 59 00 40 02 5f 00 40 00 00 40 00 00 40 00 00 40 03 26 40 40 00 00 40 0a 01 00 40 40 00 00 06 03 02 5b 40 40 04 05 3c 4f 54 01 3e 40 40 54 40 40 01 00 3a 50 3d 7f 18 40 40 40 40 3f 40 40 40 00 5d 02 0e 0a 00 40 00 40 00 40 76 00 3f 00 36 06 7f 00 00 00 00 00 00 00 00 40 40 40 40 00 34 00 40 3f 54 1e 2c 16 54 3f 40 01 40 40 40 00 00 7f 72 65 65 58 50 6a 66 59 64 5d 62 59 64 61 54 5a 62 60 54 58 5f 57 59 54 59 52 59 54 5b 53 56 57 59 53 55 52 59 51 56 4d 53 4d 53 4c 52 51 49 4e 4f 55 4a 4d 4a 51 48 50 48 4e 43 4c 49 42 49 7f 74 68 68 75 53 5a 6e 5c 67 60 65 5c 67 64 57 5d 65 63 57 5b 62 5a 5c 57 5c 55 5c 57 5e 56 59 5a 5c 56 58 55 5c 54 59 50 56 50 56 4f 55 54 4c 51 52 58 4d 50 4d 54 4b 53 4b 51 46 4f 4c 45 4c 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 7f 3d 51 7b 4e 70 52 32 72 36 56 72 56 7c 4c 37 75 3f 57 7d 53 75 47 3e 6f 39 4c 7c 4d 72 53 3c 7f 38 57 7a 4b 73 4c 32 73 39 47 74 50 7a 5c 36 7f 3c 5c 7c 44 77 55 34 7f 39 5a 74 5a 7a 4b 3a 72 37 55 78 4a 6f 52 3a 7f 36 53 74 4d 7b 50 38 78 3c 54 79 50 72 44 34 7f 3d 5f 73 4e 79 4e 34 66 39 54 70 4b 76 49 32 7d 3c 3f 7a 4f 72 52 33 70 3d 55 72 5e 79 50 37 78 3b 44 74 46 7b 5b 37 70 3b 56 71 4e 78 51 37 7e 39 48 77 53 6e 5d 3b 7f 35 5d 7a 4c 73 55 31 7d 37 4e 7d 54 76 57 33 6c 3a 56 79 52 71 50 3b 75 38 52 7a 52 6d 5d 37 7d 3a 4b 79 52 73 5e 37 75 3a 56 7c 4d 71 4e 3a 7f 3d 56 79 54 74 56 38 7f 3a 64 74 64 7c 62 36 7e 37 41 72 47 79 4e 36 77 3a 52 77 4c 70 56 37 7b 3a 52 77 4b 72 53 36 74 3a 51 6f 66 76 58 33 7f 37 4f 77 53 71 49 38 66 36 53 72 44 7a 48 34 71 38 4f 7a 45 73 60 38 73 3b 48 6f 59 78 4e 31... (2048 more bytes)]
06:56:54.481: Out UFX+ MIDI Port 2 Sysex [f0 40 00 00 00 0a 00 02 03 f7]

etc.
So it just goes through the patch numbers, which is fine.

markusschloesser · 2025-02-20T18:46:07Z

Unfortunately I have to implement bank import as well, as quite a few sounds out there come in form of a bank.
Is there any reason in the c code that this cannot work with patches which have a variable size > I'll need other delimitors (which I'm still trying to find)

christofmuc · 2025-02-20T21:15:05Z

Why would this not work with variable size?

Bank import comes in two forms:

The easy way was one message contains multiple patches -> one message in, e.g. 16 patches out.
The more complex way - messages get presented to isPartOfBankDump(). Then, all messages which were accepted get presented to extractPachesFromAllMessages which can produce as many patches as it wants. It gets called once isBankDumpComplete given all messages returns True.

christofmuc · 2025-02-20T21:17:11Z

I made a new issue for the missing handshake on upload/send to synth. This is still a gap, the C++ code can do it and the Python code not.

markusschloesser · 2025-02-20T21:28:14Z

Why would this not work with variable size?

Bank import comes in two forms:

The easy way was one message contains multiple patches -> one message in, e.g. 16 patches out.

The more complex way - messages get presented to isPartOfBankDump(). Then, all messages which were accepted get presented to extractPachesFromAllMessages which can produce as many patches as it wants. It gets called once isBankDumpComplete given all messages returns True.

Good to know that kk doesn't rely on a fixed patch size for patch separation in bank dumps!

Tbh I have trouble understanding the functional difference between extractPatchesFromBank extractPachesFromAllMessages.
The K5000 sends one message as reply to a bank dump request, so I would assume this is a use case for extractPatchesFromBank?
But I can of course request 2 banks.

Is extractPachesFromAllMessages only for synths that reply to a bank dump request with multiple messages?

christofmuc · 2025-02-20T22:50:43Z

The algorithm is:

requestBankDump() 

all_messages
for all incoming messages:
  if isPartOfBankDump(message):
    all_messages.append(message)
  if isBankDumpFinished(all_messages):
    return extractPachesFromAllMessages(all_messages)

which is the more powerful way.

The old way wasI think:


requestBankDump()

patches = []
for all incomingMessages:
    patches.extend(extractPatchesFromBank(message)
return patches

This needs to go into the adaptation programming guide.

markusschloesser · 2025-02-20T22:59:47Z

the synth just doesn't send ANYTHING when the patch entry is empty (and of course kk never advances)\n\nThat's nasty. We currently guarantee that hangs when the synth stops sending. I think we had a ticket with timeouts somewhere, but that was hard to implement. What is the logic? Can you just send all program requests in advance?\n\nI'd suggest a timeout, wait 5 seconds and move on to the next number (until end of bank defined in bankdescriptors).
So it just goes through the patch numbers, which is fine.

Any thoughts about the time out thing?

christofmuc · 2025-02-20T23:25:48Z

That's a bigger thing. how about sending all requests at once?

markusschloesser · 2025-02-20T23:28:52Z

That's a bigger thing. how about sending all requests at once?

can't, don't know upfront which will be available. Or what do you mean?
Hopefully I get bank dump working (the dividing the fucking mess part 🙄😂)

christofmuc · 2025-02-20T23:36:11Z

I meant sending all possibel requests upfront :-) How many are these?

markusschloesser · 2025-02-20T23:39:36Z

I meant sending all possibel requests upfront :-) How many are these?

You mean send the synth 128 program dump request at the same time and see what happens? 🤨

markusschloesser · 2025-02-20T23:41:08Z

I have trouble finding the delimitors, and thought I could just feed complete bank dumps to ChatGPT, but with 105kb for Bank A probably not 🙄

christofmuc · 2025-02-20T23:43:32Z

There is the function knobkraft.findSysexDelimiters

you can do

for start,end in knobkraft.findSysexDelimiters(messages):
  print(len(messages[start:end],"Message")

christofmuc · 2025-02-20T23:44:33Z

I meant sending all possibel requests upfront :-) How many are these?

You mean send the synth 128 program dump request at the same time and see what happens? 🤨

Yes

markusschloesser · 2025-02-20T23:54:43Z

Notepad for later:
when I send a bank dump request to an empty bank (F0 40 00 01 00 0A 00 04 00 F7 (Bank F on Expansion Board), the synth replies with F0 40 00 01 00 0A 00 04 00 F7 F0 40 00 21 00 0A 00 04 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 F7

markusschloesser · 2025-02-21T01:46:07Z

how do I tell kk to wait after bank dump request? It currently sends the bank dump request 4 times with ": error Giving up retrying initiating bank dump"

(while a bank dump is being sent from the synth, but there usually around 100kb and that takes a while over midi din)

christofmuc · 2025-02-21T23:19:56Z

You can set the general message delay, but that is more inbetween the message it sends itself. How long does it take for the reply to be generated by the K5000? It seems we need the handshake protocol implemented...

markusschloesser · 2025-02-21T23:23:40Z

You can set the general message delay, but that is more inbetween the message it sends itself. How long does it take for the reply to be generated by the K5000? It seems we need the handshake protocol implemented...

the reply comes in pretty quick, but it takes around 1-2min until it's done.
I thought by using

def isBankDumpFinished(messages):
    return any(isPartOfBankDump(message) for message in messages)

kk would wait until retrying?

markusschloesser · 2025-02-21T23:25:00Z

It seems we need the handshake protocol implemented..

nope, got nothing to do with it. the handshake thing by the k5000 is only for when kk sends to synth, NOT the other way round

christofmuc · 2025-02-21T23:30:14Z

Currently the retry is fired already after 500 ms of not a single message having been registered positively by isPartOfBankDump().
We could make the timeout configurable?

markusschloesser · 2025-02-21T23:33:49Z

Currently the retry is fired already after 500 ms of not a single message having been registered positively by isPartOfBankDump().
We could make the timeout configurable?

"registered positively" the start of the bank dump OR the completing? (EDIT typo)
If start then my isPartOfBankDump() is not working correctly

christofmuc · 2025-02-22T00:03:06Z

No, the first message having been accepted by isPartOfBankDump() - if after 500 ms the list of accepted messages is still empty, it will fire the retry. At least, this is how I read the C++ code.

markusschloesser · 2025-02-22T00:24:16Z

what comes from the synth F0 40 00 21 00 0A 00 02 .......

and the function

def isPartOfBankDump(message):
    return (
            len(message) > 4
            and message[0] == 0xF0
            and message[1] == KawaiSysexID  # Kawai manufacturer ID (the 40)
            and 0x00 <= message[2] <= 0x0F  # Unit channel (0-F for ch 1-16)
            and message[3] == AllBlockDump  # Bank Dump identifier (21)
            and message[5] == 0x0A 
    )
``


the synth definitely starts sending before 500ms

christofmuc · 2025-02-22T15:09:40Z

Show me the MIDI log with timestamps, then we can see what happens before the retry kicks in?

markusschloesser · 2025-02-22T16:27:25Z

Show me the MIDI log with timestamps, then we can see what happens before the retry kicks in?

That's the thing, midi ox immediately shows the bank sysex coming in. Kk midi log shows 4 outgoing retries and then the first incoming bank after about 1-2 min. After that it also receives the answer to the other 3 requests. Will do video later.
EDIT: video here https://photos.app.goo.gl/psM9oxAnny3SigBh9

also you see from the screen shot, that the midi log takes around 33 seconds to register the incoming dump.

Anyway, that's not my main problem, even with "receive manual dump" and then initiating ONE bank dump request from midi ox, it still imports nothing. Will write later where I am struggling with

markusschloesser · 2025-02-22T23:54:26Z

There is the function knobkraft.findSysexDelimiters

you can do
for start,end in knobkraft.findSysexDelimiters(messages):
  print(len(messages[start:end],"Message")

unfortunately cannot use it, as patches are not marked with 0xF0...0xF7.
It's 1 message, F0, header stuff , a tone_map and then the actual patches, F7

explanation of Tone Map:
// In a tone map, only the bottom seven bits of each byte are used.
// So to indicate that tones A001...A007 are included, the first byte
// must be 0b0111_1111.
// The whole tone map that includes A001...A101 (or 0...100) would be:
// sub1 = 0b0111_1111 -- tones A001...A007
// sub2 = 0b0111_1111 -- tones A008...A014
// sub3 = 0b0111_1111 -- tones A015...A021
// sub4 = 0b0111_1111 -- tones A022...A028
// sub5 = 0b0111_1111 -- tones A029...A035
// sub6 = 0b0111_1111 -- tones A036...A042
// sub7 = 0b0111_1111 -- tones A043...A049
// sub8 = 0b0111_1111 -- tones A050...A056
// sub9 = 0b0111_1111 -- tones A057...A063
// sub10 = 0b0111_1111 -- tones A064...A070
// sub11 = 0b0111_1111 -- tones A071...A077
// sub12 = 0b0111_1111 -- tones A078...A084
// sub13 = 0b0111_1111 -- tones A085...A091
// sub14 = 0b0111_1111 -- tones A092...A098
// sub15 = 0b0000_0111 -- tones A099...A101
// sub16...sub19 = 0
from https://github.com/coniferprod/KSynthLib/blob/master/KSynthLib.Tests/K5000/ToneMap.cs#L15

markusschloesser · 2025-02-25T00:12:11Z

current error messages:

01:09:48: info Adaptation: base_ptr calculated: 3425792
Pointer table has 123 pointers
Clean up.......... 
Processing patch index 116: tone_ptr=2135636351
Index out of range for patch 116, tone_ptr=2135636351, skipping
01:09:48: info Got bank result with 1 patches
01:09:48: error Adaptation[KawaiK5000]: Error calling extractPatchesFromAllBankMessages: Unable to cast Python instance of type <class 'list'> to C++ type '?' (#define PYBIND11_DETAILED_ERROR_MESSAGES or compile in debug mode for details)
01:09:48: info Loaded bank dump with 0 patches
01:09:49: warning No patches contained in data, nothing to upload.

markusschloesser · 2025-02-25T00:21:16Z

@christofmuc just pushed to main fa51671

christofmuc · 2025-02-25T00:23:18Z

I will have a look!

The last error seems to indicate that the number format of the return value is wrong.
extractPatchesFromAllBankMessages() takes a list of ints as parameter (one large list of values), but returns a list of lists (a list of patches).

Do you know how to switch a branch in git? We can then stage your changes in a separate view. I will create one!

markusschloesser · 2025-02-25T00:28:28Z

full bank dumps from Midi OX from my K5000r for testing

K5000r Banks A,D,E.zip

christofmuc · 2025-02-25T00:30:16Z

Here is our pull request https://github.com/christofmuc/KnobKraft-orm/pull/433/files

I moved your changes to a branch called "synths/k5000" so we can stage them and can see and comment on them in the pull request view. So best you checkout the branch to continue working on it.

markusschloesser · 2025-02-25T00:33:07Z

Here is our pull request https://github.com/christofmuc/KnobKraft-orm/pull/433/files

I moved your changes to a branch called "synths/k5000" so we can stage them and can see and comment on them in the pull request view. So best you checkout the branch to continue working on it.

switched to it in pycharm! thanks for helping! :-)

optimolch · 2025-02-27T07:17:08Z

[...] From https://www.optimolch.de/jens.groh/K5000/Kenji/digests/ka1formt.html
[...]

that website's owner/author here, reading this with interest, though probably not being able to help.
Greetings (from Munich, Germany, too! ;-) Jens

christofmuc · 2025-02-27T09:03:31Z

[...] From https://www.optimolch.de/jens.groh/K5000/Kenji/digests/ka1formt.html
[...]

that website's owner/author here, reading this with interest, though probably not being able to help. Greetings (from Munich, Germany, too! ;-) Jens

Nice! Thanks for hosting this, as you can see, useful for some! I'm sure we'll figure the K5000 out - it might be the most ambitious so far, but that's why we have waited for 4 years before finally tackling it ;-)

markusschloesser · 2025-03-02T18:27:10Z

Show me the MIDI log with timestamps, then we can see what happens before the retry kicks in?

That's the thing, midi ox immediately shows the bank sysex coming in. Kk midi log shows 4 outgoing retries and then the first incoming bank after about 1-2 min. After that it also receives the answer to the other 3 requests. Will do video later. EDIT: video here https://photos.app.goo.gl/psM9oxAnny3SigBh9
also you see from the screen shot, that the midi log takes around 33 seconds to register the incoming dump.
Anyway, that's not my main problem, even with "receive manual dump" and then initiating ONE bank dump request from midi ox, it still imports nothing. Will write later where I am struggling with

@christofmuc btw kk still does this (sends 4 requests), even with the new code. But manages to load the bank successfully. But also still shows giving up, which is confusing

christofmuc · 2025-03-02T22:29:55Z

I think I understand the problem with the retries - what you see in MidiOX is the incoming Sysex continue messages - those can be displayed, but don't make KK register it doesn't need to retry. KK only gets to process the message once it is fully in, which takes longer than 4 seconds. I think this is fixable, if I recall correctly there is a sysex continue handler possibiltiy in JUCE, I need to check again. I hope the giving up/loading confusion comes from the fact and will go away when we fix the retries.

markusschloesser · 2025-03-02T23:32:12Z

does the opening the tree on the left side > "in synth" > "Bank A,...." do something differently than Menu/Midi/Import patches from synth/get Bank? because the latter works fine, while the former shows
Extracted 40 valid patches.
00:25:12: info Got bank result with 40 patches
00:25:12: info Loaded bank dump with 40 patches
00:25:12: info Retrieved 40 patches from synth
00:25:12: info All patches already known to database
00:25:12: error DATABASE ERROR in putPatchList: SQL Exception FOREIGN KEY constraint failed
00:25:12: info Bank of 0 patches retrieved from database
00:25:12: error Failed to create list!
00:25:12: error Program Error: Invalid synth bank, not stored in database. Can't load into panel

it does show the error for Bank A and E, but not for Bank D

christofmuc added the New device label Jan 9, 2021

markusschloesser changed the title ~~Support Kawai K5000/S/W~~ Support Kawai K5000/S/W/R Feb 8, 2025

markusschloesser changed the title ~~Support Kawai K5000/S/W/R~~ Support Kawai K5000S/W/R Feb 9, 2025

markusschloesser mentioned this issue Feb 16, 2025

How to deal with device family #422

Closed

christofmuc mentioned this issue Feb 19, 2025

using Bank Descriptors for banks with variable bank size / variable amount of patches #423

Closed

christofmuc mentioned this issue Feb 20, 2025

Send patch mechanism currently has no handshake protocol implemented (K5000) #426

Open

markusschloesser mentioned this issue Feb 25, 2025

Kawai K5000 #433

Open

Support Kawai K5000S/W/R #72

Support Kawai K5000S/W/R #72

Comments

christofmuc commented Jan 9, 2021

markusschloesser commented Feb 8, 2025 • edited Loading

markusschloesser commented Feb 8, 2025 • edited Loading

markusschloesser commented Feb 9, 2025 • edited Loading

markusschloesser commented Feb 9, 2025 • edited Loading

markusschloesser commented Feb 11, 2025 • edited Loading

markusschloesser commented Feb 16, 2025

markusschloesser commented Feb 16, 2025

markusschloesser commented Feb 18, 2025 • edited Loading

markusschloesser commented Feb 18, 2025

christofmuc commented Feb 19, 2025

markusschloesser commented Feb 19, 2025

christofmuc commented Feb 19, 2025

markusschloesser commented Feb 19, 2025

markusschloesser commented Feb 20, 2025

christofmuc commented Feb 20, 2025

christofmuc commented Feb 20, 2025

markusschloesser commented Feb 20, 2025

christofmuc commented Feb 20, 2025

markusschloesser commented Feb 20, 2025 • edited Loading

christofmuc commented Feb 20, 2025

markusschloesser commented Feb 20, 2025

christofmuc commented Feb 20, 2025

markusschloesser commented Feb 20, 2025

markusschloesser commented Feb 20, 2025

christofmuc commented Feb 20, 2025 • edited Loading

christofmuc commented Feb 20, 2025

markusschloesser commented Feb 20, 2025

markusschloesser commented Feb 21, 2025 • edited Loading

christofmuc commented Feb 21, 2025

markusschloesser commented Feb 21, 2025

markusschloesser commented Feb 21, 2025

christofmuc commented Feb 21, 2025

markusschloesser commented Feb 21, 2025 • edited Loading

christofmuc commented Feb 22, 2025

markusschloesser commented Feb 22, 2025 • edited Loading

christofmuc commented Feb 22, 2025

markusschloesser commented Feb 22, 2025 • edited Loading

markusschloesser commented Feb 22, 2025

markusschloesser commented Feb 25, 2025 • edited Loading

markusschloesser commented Feb 25, 2025

christofmuc commented Feb 25, 2025

markusschloesser commented Feb 25, 2025

christofmuc commented Feb 25, 2025

markusschloesser commented Feb 25, 2025

optimolch commented Feb 27, 2025 • edited Loading

christofmuc commented Feb 27, 2025

markusschloesser commented Mar 2, 2025

christofmuc commented Mar 2, 2025

markusschloesser commented Mar 2, 2025

markusschloesser commented Feb 8, 2025 •

edited

Loading

markusschloesser commented Feb 8, 2025 •

edited

Loading

markusschloesser commented Feb 9, 2025 •

edited

Loading

markusschloesser commented Feb 9, 2025 •

edited

Loading

markusschloesser commented Feb 11, 2025 •

edited

Loading

markusschloesser commented Feb 18, 2025 •

edited

Loading

markusschloesser commented Feb 20, 2025 •

edited

Loading

christofmuc commented Feb 20, 2025 •

edited

Loading

markusschloesser commented Feb 21, 2025 •

edited

Loading

markusschloesser commented Feb 21, 2025 •

edited

Loading

markusschloesser commented Feb 22, 2025 •

edited

Loading

markusschloesser commented Feb 22, 2025 •

edited

Loading

markusschloesser commented Feb 25, 2025 •

edited

Loading

optimolch commented Feb 27, 2025 •

edited

Loading