Re: [fluid-dev] Proposal for a new feature: lazy-loading of SoundFonts

Hi all,

I would like to propose (and then implement) a new feature to Fluidsynth; lazy-loading of Soundfont data.

Let me first explain the problem I want to solve:

I'm building a musical instrument (including the hardware) that uses Fluidsynth for sound output. The player has up to 10 channels available onto which he can load different sounds/presets from any of the Soundfonts that are stored on the instrument. Now suppose the player has a few large Soundfonts and wants to use them all at the same time, but only one or two presets from each one. Currently this means that every Soundfont has to be completely parsed and loaded into RAM, even though the player only uses a fraction of that data for playback. And although the instrument has 1 GB of RAM, it's still very easy to exceed that limit with a few larger Soundfonts. To make matters worse, all of this data has to be loaded from the built-in SD-Card... which takes quite a while.

So my idea was to add an optional feature that enables selective or lazy loading of preset and sample data from the chosen Soundfonts. When this feature is enabled and a Soundfont is loaded, the file is not parsed completely, but only so much that we can extract the high-level preset information: preset name, bank and program number. Only when a preset is actually selected for a channel is the file opened and parsed again. This time, only the information that is relevant for this particular preset is read and parsed, including all the samples that the preset uses. And when that preset is unselected again and not used anymore on any channel, the related structures and RAM will be freed again.

I had a good look through the Soundfont loading code and think this feature could be added in such a way that it won't impact the "normal" use. So if the feature is disabled (which will be the default), Fluidsynths behaviour and performance will be unchanged. I also think it won't add a lot of new complexity to the codebase, but that will have to be seen once the implementation is done.

The feature would bring two major drawbacks (when enabled):
>

1. The Soundfont structure will not be completely checked for structural errors at loading time. Doing so would defeat the purpose of this feature, as we would have to parse the entire file. Most checks will be done eventually, as presets are selected for playback. But errors in those parts will surface at a later stage, not during initial load of the Soundfont. And some checks, like testing if all zone indices of all instruments are monotonic, won't be possible at all.

For my use case, this isn't really a problem. And after all... there is always the possibility to disable the feature if you want to validate a Soundfont completely.

2. Selecting a preset will definitely incur memory allocation and disk access. That means that a program change event will most likely take much more time than it does now. The work will be done in a separate thread so that parsing the file will not lead to audio buffer underruns. But it would mean that program change events will be delayed and take effect only after loading has finished.

Again, for my use case, this isn't a problem at all. I'm not dealing with automatic program changes, a player has to explicitly change the channel setup to choose a different one. And currently players of my instrument have to wait many seconds when he switched to a new setup that uses a few previously unloaded Soundfonts. With the lazy loading, this time will actually reduce drastically.

And just make this clear: this feature is *not* intended to be used with MIDI file playback. It's mainly useful for situations where a player chooses a fairly static channel/preset setup and uses Fluidsynth for a real-time musical performance.

I would really like to get your thoughts on this. Do you think it's a feature that other Fluidsynth users could benefit from? Do you see any major obstacles or drawbacks that I've missed?

In preparation for this feature, I've started a larger refactor of the Soundfont loading code: Breaking it up into more manageable chunks, cleaning it up and fixing smaller bugs I've found. I hope those changes are useful even if this new feature doesn't make it into Fluidsynth. https://github.com/FluidSynth/fluidsynth/pull/360

All the best, and sorry for the long post...

Marcus

From:	Ceresa Jean-Jacques
Subject:	Re: [fluid-dev] Proposal for a new feature: lazy-loading of SoundFonts
Date:	Thu, 5 Apr 2018 19:17:38 +0200 (CEST)