Re: [fluid-dev] Getting severe clicking/static with FluidSynth (on Andro

As promised, here are findings with clicks/static noted in this thread. It turns out there are multiple issues:

A. Some clicks/scratch sounds are simply due to signal clipped to [-1,+1] range, as Jean-Jacques noted. This is not Android specific and we can solve this by different gain settings.

B. Other clicks are caused by our application-specific logic for panning, again not an Android problem. Specifically, we are panning based on pitch (low pitches all the way left, high pitches all the way right), not orchestral positioning. This means we are setting the pan before each note on message. We thought this only affects the new note on messages. But it affects all existing notes on the channel, which end up "jumping" between different pan settings, causing clicks.

--> NEW QUESTION 1. Is it possible to set FluidSynth to set panning by note rather than by channel? If not, would this be a hard change to do? I'm guessing the synth maintains a list of signal generators for the current notes, and these currently use a global pan setting -- they would need to be changed to use their own note-specific pan setting.

3. The wall of noise problem (in the recording from my original message) is solved by reducing number of voices from 256 to 16. This feels like a workaround, not a real diagnosis of the underlying problem, which may still be related to Android performance limitations. I also suspect it has to do with the specific SF2 characteristics. For instance it was happening using a harp preset, which has global attenuation set to a relatively high value of 20.

--> NEW QUESTION 2. I think I need to understand the meaning of the various SF2 settings like attenuation, vol env. delay/attack/hold/decay/sustain/release. Any tips on learnings resources for these things for a laymen (i.e. non-music person)?

Thanks all for the responses so far, really helpful.

--Ien

On Wed, Feb 1, 2017 at 1:23 PM, Ien Cheng <address@hidden> wrote:

Thanks Jean-Jacques, very helpful, I will look into your advice (and share any findings for Android)...

On Wed, Feb 1, 2017 at 1:17 PM, Ceresa Jean-Jacques <address@hidden> wrote:
Hi Ien,

There are 2 situations that can produce pop/clics and noise.

First, pop/clics due to high level signal when clipped in the range [-1..+1] by the audio thread.

In FluidSynth 1.1.6 there is no "vue meter API" allowing amplitude level monitoring.

But there are an API to set the gain at run time.

Second, as Element stated, when the cpu load is greater than 100% this audio underrun situation produce

clics and permanant noise. This situation arises on CPU with relative low performance (Fan less machine, smartpone, tablet,..).

In FluidSynth 1.1.6 there is an API that returns the CPU load (in percent) at run time.

An other solution allowing to master any hardware capability in term of CPU load is to use the patch proposed

"Adding profiling commands Interface" (24 February 2016)". "http://lists.nongnu.org/archive/html/fluid-dev/2016-02/msg00009.html"

Among others things, this patch allows the measurement of the maximun number of simultaneous voices that a given plateform can produce.

When this value is known it is easy to set the "polyphony setting" to this value to ensure that at run time this platform will remain always below 100% cpu load.

As an example, i have used this patch intensively to compare a set of differents plateform (Tower machine, Fan less machine, Rasberry PI).

I have no experience on building Fluidsynth on Android or Ios. If you plan to apply this patch , it will be very useful to share theses platforms measurement results.

regards

jean-jacques

> Message du 01/02/17 14:34
> De : "Ien Cheng" <address@hidden>
> A : "FluidSynth mailing list" <address@hidden>, "Ceresa Jean-Jacques" <address@hidden>
> Copie à : "Element Green" <address@hidden>

> Objet : Re: [fluid-dev] Getting severe clicking/static with FluidSynth (on Android)
>
>
Thanks all for this great info.

>

One more general question -- is there any way to programmatically detect when there is static, clicks, pops, or other kinds of unintended audio artifacts due to CPU load or other problems?

>

I'm going to instrument the thread that is running FluidSynth to get a cycle time or frames per second rate. Other ideas welcome.

>

The issue is we are getting different behavior on different Android devices so it would be very helpful to be able to detect problems programmatically.

>

Thanks, Ien

>
On Jan 31, 2017 12:53 PM, "Ceresa Jean-Jacques" <address@hidden> wrote:

> Hello Ien,

>

> In conjonction with the informations given by Element (i.e "high cpu load"), it is important to note that

> "reverb" and "chorus" processing effects consume cpu time. For example the "reverb load" is about 4 times than

> a simple "voice load".

>

> The following commands turn off effects processing.

> >reverb off

> >chorus off

>

> regards

> jean-jacques

>

>

>

> Message du 31/01/17 17:46
> > De : "Element Green" <address@hidden>
> > A : "FluidSynth mailing list" <address@hidden>
> > Copie à : "Ceresa Jean-Jacques" <address@hidden>
> > Objet : Re: [fluid-dev] Getting severe clicking/static with FluidSynth (on Android)
> >
> >
Hello Ien,

> >

This sounds like the typical symptoms of running out of CPU. Checking the CPU load would easily confirm this. One question I have is if your hardware platform has a floating point unit. Without that, FluidSynth will run pretty inefficiently. This also can relate to how FluidSynth was built, as far as wait floating point support it is using. For example, you could have a hardware floating point unit, but if FluidSynth is not built to take advantage of it because it wasn't built with it enabled, then its the same as not having one.

> >

If all that checks out. Then perhaps there are more voices being synthesized simultaneously than the system can handle. This can happen if there are too many notes being played or if the instruments have long release times. This can be adjusted by setting the polyphony amount to something which your system can handle. When the number of voices exceeds this number, the least perceptible notes will be terminated.

> >

I hope that helps.

> >

Best regards,

> >

Element Green

> >

> >
On Mon, Jan 30, 2017 at 5:15 PM, Ien Cheng <address@hidden> wrote:
> >

Thanks Ceresa, I will try that. Any reason why a high gain would be fine for the first 15 seconds and then start to deteriorate? (Asking to help myself understand how it all works...) --Ien

> >
On Mon, Jan 30, 2017 at 7:12 PM, Ceresa Jean-Jacques <address@hidden> wrote:
> >

> > Hi, Ien

> >

> > After a quick glance on the recording please try with a lower gain value.

> > You can set the gain value using the command "gain".

> > Example:

> > > gain 0.5

> > or

> > > gain 0.2

> >

> > regards.

> >

> > jjc

> >

> Message du 30/01/17 16:03
> > > De : "Ien Cheng" <address@hidden>
> > > A : "FluidSynth mailing list" <address@hidden>
> > > Copie à :
> > > Objet : [fluid-dev] Getting severe clicking/static with FluidSynth (on Android)

> > >
> > >
Hi all,

> > >

I'm using the synth to play a piece of music with a series of note on/off commands, using a harp soundfont. This is on an Android phone using this Android wrapper for FluidSynth.

> > >

It starts off great, with clean sound, but after around 15, there starts to be clipping/clicking/static sounds which quickly worsen to the point of being unusable. Here is a recording.

> > >

I'm hoping this is a common problem due to some kind of mistake with the settings. Any hints or help on how to solve this would be much appreciated!

> > >

--Ien

> >
> >
> >

_______________________________________________
> > fluid-dev mailing list
> > address@hidden
> > https://lists.nongnu.org/mailman/listinfo/fluid-dev
> >
> >

> > _______________________________________________
> > fluid-dev mailing list
> > address@hidden
> > https://lists.nongnu.org/mailman/listinfo/fluid-dev
> >
> >

> >

> > _______________________________________________
> > fluid-dev mailing list
> > address@hidden
> > https://lists.nongnu.org/mailman/listinfo/fluid-dev
> >
> >

> >

> _______________________________________________
> fluid-dev mailing list
> address@hidden
> https://lists.nongnu.org/mailman/listinfo/fluid-dev
>
>

From:	Ien Cheng
Subject:	Re: [fluid-dev] Getting severe clicking/static with FluidSynth (on Android)
Date:	Mon, 6 Feb 2017 10:32:27 -0500