Re: [fluid-dev] Getting severe clicking/static with FluidSynth (on Android)

[Element Green]

Hello Ien,

On Mon, Feb 6, 2017 at 8:32 AM, Ien Cheng <address@hidden> wrote:

As promised, here are findings with clicks/static noted in this thread. It turns out there are multiple issues:

A. Some clicks/scratch sounds are simply due to signal clipped to [-1,+1] range, as Jean-Jacques noted. This is not Android specific and we can solve this by different gain settings.

B. Other clicks are caused by our application-specific logic for panning, again not an Android problem. Specifically, we are panning based on pitch (low pitches all the way left, high pitches all the way right), not orchestral positioning. This means we are setting the pan before each note on message. We thought this only affects the new note on messages. But it affects all existing notes on the channel, which end up "jumping" between different pan settings, causing clicks.

--> NEW QUESTION 1. Is it possible to set FluidSynth to set panning by note rather than by channel? If not, would this be a hard change to do? I'm guessing the synth maintains a list of signal generators for the current notes, and these currently use a global pan setting -- they would need to be changed to use their own note-specific pan setting.

Yes, this is possible.

If the panning can remain fixed always for a specific instrument, you could potentially just hard code the panning per note in the SoundFont itself.  This would definitely be the easiest route.

Another possibility is utilizing the SF2 loader API in FluidSynth - fluid_synth_add_sfloader().  This would complicate your implementation a lot.  Swami is the only application I'm aware of that utilizes the SF2 loader, besides FluidSynth itself.  This would require that you instantiate your own voices in FluidSynth when a note-on occurs.  Swami does it's own SoundFont processing using the libInstPatch library.  Doing this from scratch without the help of something like libInstPatch would require a lot of coding to load up a SoundFont, manipulate it, and do the voice synthesis calculations (with the panning generator modified as desired for the note).  Though FluidSynth has a bunch of this kind of code in its default SoundFont loader which it utilizes internally.  The license of FluidSynth may not be compatible with your application if you were to just copy FluidSynth code into it and unfortunately it doesn't look like the default SF2 loader API is public, which could allow for modifications of its functionality without direct inclusion.

There may be some other trick I'm not thinking of..

 

3. The wall of noise problem (in the recording from my original message) is solved by reducing number of voices from 256 to 16. This feels like a workaround, not a real diagnosis of the underlying problem, which may still be related to Android performance limitations. I also suspect it has to do with the specific SF2 characteristics. For instance it was happening using a harp preset, which has global attenuation set to a relatively high value of 20. 

As I mentioned in a previous email.  My guess is that this is a performance limitation of the platform and would be greatly affected by whether the system has an FPU or not and whether FluidSynth has been built properly to utilize the FPU.  The harp instrument probably has really long release times, which means that even after a note-off occurs the voice will continue to be synthesized until its volume drops below the perceptible silence threshold.  This means that voices can stack up really quick, even if there are only a few notes still in the on state.

--> NEW QUESTION 2. I think I need to understand the meaning of the various SF2 settings like attenuation, vol env. delay/attack/hold/decay/sustain/release. Any tips on learnings resources for these things for a laymen (i.e. non-music person)?

Well, I think one of the best resources is the SoundFont 2.04 specification itself (http://freepats.zenvoid.org/sf2/sfspec24.pdf).  But that is certainly a technical one.

 

Thanks all for the responses so far, really helpful.

--Ien

Best regards,

Element