Re: [fluid-dev] callbacks

[David Henningsson]

On 2010-11-14 02:34, Element Green wrote:
> On Sat, Nov 13, 2010 at 5:28 AM, David Henningsson<address@hidden>  wrote:
> > On 2010-11-08 19:32, Element Green wrote:
> > > My current FluidSynth API wishlist:
> > > - 24 bit and/or floating point sample data support
> >
> > ...and the easiest way forward, would that be to depend libInstPatch? How
> > much does libInstPatch currently do and what would need to be implemented in
> > FluidSynth?
>
> That could very well be the easiest way forward.  libInstPatch
> currently does loading/saving of 24 bit SoundFont files.  The
> FluidSynth plugin in Swami is GObject oriented and has a libinstpatch
> based SoundFont loader.  Currently it is an on demand loader, which
> caches samples in RAM when a preset is selected (rather than loading
> the entire sample data).  But the same functionality as current
> FluidSynth could be achieved by caching all samples at file load time
> (though this could also be made an option).
>
> libInstPatch also has a SoundFont voice cache system
> (IpatchSF2VoiceCache), which converts SoundFont presets, instruments
> or samples into an array of potential SoundFont voices (selected by
> MIDI note and velocity values).  This means faster note-on
> calculations (than current FluidSynth), since a simple array of
> potential voices needs only to be searched, rather than doing
> calculations on the SoundFont tree at note-on time (its done at preset
> selection time instead).

Hmm, interesting, but wouldn't this make preset selections very slow?

> The way forward for adding this functionality into FluidSynth:
> - Create a new SoundFont loader using the Swami plugin as a basis
> (I'll contribute it as LGPL to FluidSynth, since its GPL in Swami).
> - Add 24 bit sample support to the FluidSynth SoundFont loader API
> - Probably some other things that aren't occurring to me at the moment

We must also change/rewrite the interpolation routines as well to handle 
sample data of different sizes, right? That's where the actual data 
comes in.

> Perhaps at some point FluidSynth might want to be GObject oriented, or
> it could add that as a wrapper around existing FluidSynth.  The Swami
> plugin could be used as a basis for that purpose as well.

I'm a little hesitant to adding GObject stuff to FluidSynth as I don't 
see what we would gain from it.

> So actually really not that much needs to be done.  Since most of the
> code is already there, just needs to be properly integrated into
> FluidSynth as another loader option.  Could even be made build time
> optional.  I'd be happy to do this work!

Contributions are welcome :-) But as you say, libInstPatch should 
probably be a build time option. And we might want to split 
fluid_sample_t into an fluid_rsample_t for the hard real-time side. It 
needs some design work.

> > > - Voice playback pointer querying
> >
> > The 1.1.2 engine currently makes it hard to read anything that is changeable
> > by rendering. I'm not saying it's impossible though, we could add atomic
> > stuff that updates the playback pointer (and perhaps other stuff as well). I
> > would like to do that configurable though in order to get maximum
> > performance from use cases that does not need this functionality.
> >
> > The other option would be that you do this yourself from the rendering
> > thread, but then you would have to hook into the audio driver and you'll
> > have to be careful in order to avoid underruns.
>
> I don't think it is possible though to get access to a single voice's
> current sample playback position, even if handling the rendering, from
> the public API though.  Or am I overlooking something?

You're correct, this is more of a discussion on how such a thing 
could/should be implemented.

> > Callbacks leaves us the question as of from what thread(s) the callback
> > should be called. We have two options, either directly from the hard
> > real-time side, or transferred back to the soft real-time side. Now remember
> > that the engine does not own any threads on either side, so you'll get the
> > callback inside a call into the engine (which in turn could be done by
> > yourself or by a midi/shell/audio driver).
>
> I didn't understand what callbacks you are referring to.  Was this
> related to the voice sample playback position item?

I'm referring to the original question, which was "is there any way to 
receive callbacks when voices finish/loop etc".

> > > - Ensure sample garbage collection can occur in a thread safe fashion
> >
> > Hmm, I haven't looked into this. Looking at the code now, there seem to be
> > calls for fluid_sample_decr_ref from both threads. It would be fairly easy
> > to move all calls to the soft real-time side, we should probably do that. As
> > for thread safety, you would then be guaranteed to only get one notify
> > callback at a time.
>
> So fluid_sample_decr_ref() ultimately calls the "notify" method of the
> fluid_sample_t structure.  So as long as that is thread safe, as you
> mention, that method could be used to do garbage collection.  Assuming
> the sample wont somehow get used again.

Could you elaborate on what "thread safe" means for you here?

// David