Improve Performance

[spessasus]

somehow improve performance of the synthesizer

Either:
Port the worklet system to wasm/emscripten in c++ to improve performance
AudioWorkletProcessor would be a wrapper for a cpp renderOut(left, right) function

or improve the current JavaScript implementation.

[TS53]

When the website is loaded and before a MIDI file is selected or something is changed, the RAM usage, on my PC, is already at 470 MB (but likely Firefox itself uses two third here independent of the website). Replacing algorithms here can lower the amount of RAM quite a bit. I assume, the soundfont loader is not optimized, maybe loads the complete soundfont into RAM even though only small parts are required for a given MIDI file later.

Replacing js core code by wasm will most likely reduce the amount of RAM by half.
But it is not clear whether the runtime will be faster. Benchmarks indicate 2 times faster to 2 times slower, depending on the task.

My guess is, that changes to the pure js code can already achieve the wanted performance improvements, but first, it has to be clear what are the morst resource-hungry task w.r.t. RAM usage and CPU runtime. My guess without looking into the code: soundfont loader and soundfont renderer, gui renderer (especially the notes and effects at high FPS), calculating and rendering the wave effects.

Nonetheless, so far I have not seen the performance to be problematic on my laptop. It could play all MIDIs I have tested and RAM usage was also still OK.

[spessasus]

Unfortunately RAM is a big problem, because of a few reasons:

1. The entire soundfont HAS to be stored in the RAM. Unlike native synths (like fluidsynth) spessasynth, being in the browser, cannot operate on the file on the disk and load specific elements, but has to fetch the file as a whole in order to read it.
2. Forced 32-bit float rendering: WebAudio uses Float32 as the bit depth of choice. It's a great and incredibly high quality choice, but given that soundfonts use 16-bit audio, spessasynth has to resample it to 32-bit and keep it in its cache. Note that during the MIDI loading only samples that the said file uses are actually cached, so at least there's that
3. Transferring the data between contexts: Since AudioWorklets operate in a different thread than the main thread, the entire soundfont binary has to be copied there. This is fine for smaller soundfonts, but if we're talking about 4GB, it can freeze my system and take a good two minutes to copy it. Note that this only works on Firefox because Chrome has a hard-coded 4GB memory limit for a tab, so maybe its Firefox's fault for taking so long to copy it. The parser itself seems to be quite fast, as after the said 4GB soundfont has been succesfully copied to the audio thread, the parser has it ready in about a second or so.

[TS53]

1. is unfortunate, there are ways to still solve this by splitting the soundfont into different chunks and each has an own webadress. Load only those which are needed. But for small soundfonts this will be overkill and for others it will add much complexity.
2. This sounds like difficult to improve.
3. This sounds like it is possible to pass through the threads by allocating it to some static adress and only pass the adress. But maybe there is a good reason that this does not work.

Maybe it is possible to supply a "bad quality" default soundfont which uses high compression but has a lower filesize.

If it's mainly about RAM, then WASM can indeed give a factor 2 improvement according to the linked benchmarks. Have you tried WASM and were there any problems with it? I could port the code to Rust if the c++ code has issues.