Created SF3 files are invalid RIFF

[mirabilos]

I’m stumbling over an issue… I’ve got a nifty Python3 script that outputs the structure and metadata of a soundfont, and I’m trying it on an SF3 for the first time today and it crashes. The cause can be traced to the soundfont file itself:

 25F42D0  D0 FF BE B2 37 77 E2 D1  61 65 DB B9 93 6E 0E 4C  ....7w..ae...n.L
 25F42E0  49 53 54 5A C3 02 00 70  64 74 61 70 68 64 72 50  ISTZ...pdtaphdrP
 25F42F0  2E 00 00 43 6F 6E 74 72  61 62 61 73 73 00 00 00  ...Contrabass...

As you can easily see, the LIST<pdta> chunk is not properly aligned because the preceding chunk misses its mandatory padding.

My parser comes up to:

START
BEGIN level=0
 CHUNK b'RIFF<sfbk>' of size 39978553
  BEGIN level=1
   CHUNK b'LIST<INFO>' of size 2818
    BEGIN level=2
     CHUNK b'ifil'(4): b'\x03\x00\x01\x00'
     CHUNK b'INAM'(54): b'MuseScore_General_Lite.sf3 (MuseScore_General v0.2.1)\x00'
     CHUNK b'isng'(10): b'E-mu 10K2\x00'
     CHUNK b'IPRD'(8): b'SBAWE32\x00'
     CHUNK b'IENG'(134): b'MuseScore_General by S. Christian Collins; based on Fluid (R3) Mono GM by Church Organist; based on Fluid (R3) SoundFont by Frank Wen\x00'
     CHUNK b'ISFT'(10): b'Polyphone\x00'
     CHUNK b'ICRD'(32): b'Friday 23 March 2018, 13:27:43\x00\x00'
     CHUNK b'ICMT'(2292): b'MuseScore_General SoundFont (MIT, parts PD or CC0):\n\nMuseScore Drumline (MDL) samples (CC0) by S. Christian Collins & Amir Oosman\nSplendid Grand piano from AKAI S5000 (verified Public Domain)\n\nMarching Cymbals: open crash samples from Versilian Community Sample Library\nhttps://github.com/sgossner/VCSL (CC0) by Sam Gossner\n\nAdaptation for MuseScore_General Copyright \xc2\xa9 2018-2021 S. Christian Collins\nDebian packaging, Metronome/documentation \xc2\xa9 2018-2021 mirabilos <[email protected]>\n\nFluid (R3) Mono GM soundfont (MIT):\n\nMono version Copyright \xc2\xa9 2014-2017 Michael Cowgill\nTemple Blocks instrument Copyright \xc2\xa9 2002 Ethan Winer\nDrumline Cymbals Copyright \xc2\xa9 2016 Michael Schorsch\n\n(Original Stereo version) Fluid (R3) GM SoundFont (MIT):\n\nCopyright \xc2\xa9 2000-2002, 2008 Frank Wen <[email protected]>\nInclusion into Debian derivatives \xc2\xa9 2008 Toby Smithe\n\nFluid was constructed in part from samples found in the public\ndomain that I [Frank Wen] edited/cleaned/remixed/programmed and\nlargely from recordings of my own and in conjunction with the\npeople below who helped along the way: Suren M. Seron,\nScott Hanan, Steve Aupperle, Chris Gillman, Alex Taubr,\nChris Prola, Andrew Klenk, Winfried Hubbe, Dylan, Tim, Gort,\nUros Katic, Ethan Winer (http://www.ethanwiner.com)\n\nPermission is hereby granted, free of charge, to any person\nobtaining a copy of this software and associated documentation\nfiles (the "Software"), to deal in the Software without\nrestriction, including without limitation the rights to use,\ncopy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the\nSoftware is furnished to do so, subject to the following\nconditions:\n\nThe above copyright notice and this permission notice shall be\nincluded in all copies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,\nEXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES\nOF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND\nNONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT\nHOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,\nWHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING\nFROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR\nOTHER DEALINGS IN THE SOFTWARE.\n\x00\x00'
     CHUNK b'ICOP'(198): b'Copyright (c) Frank Wen, Michael Cowgill, S. Christian Collins and others; see ICMT chunk or Debian source package musescore-general-soundfont-small (= 0.2.1-1) for entire terms and the MIT licence\x00'
    END level=2
   CHUNK b'LIST<sdta>' of size 39794625
    BEGIN level=2
     CHUNK b'smpl' of size 39794613
    END level=2
   CHUNK b'ISTZ' of size 1879048899

The unaligned chunk is therefore LIST<sdta>.smpl and LIST<sdta> itself as well.

The file was generated by sf3convert.

I cannot find any reference to SF3 disabling RIFF padding… plus, the LIST<INFO> chunks clearly still do use padding. So I cannot even understand why this file even loads/plays in MuseScore in the first place…

@davy7125 do you perhaps know? Polyphone’s sf2/sf3 loading code is very… nōn-validating, I think. Did you run into this?

I searched through all (old and current) issues and PRs here but could not find any hint regarding padding…

[mirabilos]

I implemented a workaround in the script (if the input filename ends with .sf3 it will switch to unpadded mode and treat the file as read-only). This yields:

START
BEGIN level=0
 UNALIGNED CHUNK b'RIFF<sfbk>' of size 39978553
  BEGIN level=1
   CHUNK b'LIST<INFO>' of size 2818
    BEGIN level=2
     CHUNK b'ifil'(4): b'\x03\x00\x01\x00'
     CHUNK b'INAM'(54): b'MuseScore_General_Lite.sf3 (MuseScore_General v0.2.1)\x00'
     CHUNK b'isng'(10): b'E-mu 10K2\x00'
     CHUNK b'IPRD'(8): b'SBAWE32\x00'
     CHUNK b'IENG'(134): b'MuseScore_General by S. Christian Collins; based on Fluid (R3) Mono GM by Church Organist; based on Fluid (R3) SoundFont by Frank Wen\x00'
     CHUNK b'ISFT'(10): b'Polyphone\x00'
     CHUNK b'ICRD'(32): b'Friday 23 March 2018, 13:27:43\x00\x00'
     CHUNK b'ICMT'(2292): b'MuseScore_General SoundFont (MIT, parts PD or CC0):\n\nMuseScore Drumline (MDL) samples (CC0) by S. Christian Collins & Amir Oosman\nSplendid Grand piano from AKAI S5000 (verified Public Domain)\n\nMarching Cymbals: open crash samples from Versilian Community Sample Library\nhttps://github.com/sgossner/VCSL (CC0) by Sam Gossner\n\nAdaptation for MuseScore_General Copyright \xc2\xa9 2018-2021 S. Christian Collins\nDebian packaging, Metronome/documentation \xc2\xa9 2018-2021 mirabilos <[email protected]>\n\nFluid (R3) Mono GM soundfont (MIT):\n\nMono version Copyright \xc2\xa9 2014-2017 Michael Cowgill\nTemple Blocks instrument Copyright \xc2\xa9 2002 Ethan Winer\nDrumline Cymbals Copyright \xc2\xa9 2016 Michael Schorsch\n\n(Original Stereo version) Fluid (R3) GM SoundFont (MIT):\n\nCopyright \xc2\xa9 2000-2002, 2008 Frank Wen <[email protected]>\nInclusion into Debian derivatives \xc2\xa9 2008 Toby Smithe\n\nFluid was constructed in part from samples found in the public\ndomain that I [Frank Wen] edited/cleaned/remixed/programmed and\nlargely from recordings of my own and in conjunction with the\npeople below who helped along the way: Suren M. Seron,\nScott Hanan, Steve Aupperle, Chris Gillman, Alex Taubr,\nChris Prola, Andrew Klenk, Winfried Hubbe, Dylan, Tim, Gort,\nUros Katic, Ethan Winer (http://www.ethanwiner.com)\n\nPermission is hereby granted, free of charge, to any person\nobtaining a copy of this software and associated documentation\nfiles (the "Software"), to deal in the Software without\nrestriction, including without limitation the rights to use,\ncopy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the\nSoftware is furnished to do so, subject to the following\nconditions:\n\nThe above copyright notice and this permission notice shall be\nincluded in all copies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,\nEXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES\nOF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND\nNONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT\nHOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,\nWHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING\nFROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR\nOTHER DEALINGS IN THE SOFTWARE.\n\x00\x00'
     CHUNK b'ICOP'(198): b'Copyright (c) Frank Wen, Michael Cowgill, S. Christian Collins and others; see ICMT chunk or Debian source package musescore-general-soundfont-small (= 0.2.1-1) for entire terms and the MIT licence\x00'
    END level=2
   UNALIGNED CHUNK b'LIST<sdta>' of size 39794625
    BEGIN level=2
     UNALIGNED CHUNK b'smpl' of size 39794613
    END level=2
   CHUNK b'LIST<pdta>' of size 181082
    BEGIN level=2
     CHUNK b'phdr' of size 11856
     CHUNK b'pbag' of size 4920
     CHUNK b'pmod' of size 7520
     CHUNK b'pgen' of size 20872
     CHUNK b'inst' of size 4532
     CHUNK b'ibag' of size 10576
     CHUNK b'imod' of size 10040
     CHUNK b'igen' of size 52960
     CHUNK b'shdr' of size 57730
    END level=2
  END level=1
END level=0
OUT

I still think this violates the RIFF standard and all kinds of spec-y things. Maybe it’s also responsible for some amount of playback bugs with SF3 soundfonts?

[mirabilos]

Link to the script and the workaround applied in case someone is interested.