Re: [fluid-dev] Help about EWI behavior - Soundfonts generator

[Louis B.]

Hi jjc,

Thanks for the reply, I'll see I can summarise the situation from my point off view.

1. We need to produce a custom soundfont for breath data. I'll have a go at producing one using my live flute playing and share the result. Please share any soundfonts you create or modify. However it would be good if we could have an easy way to reuse existing woodwind soundfonts.
2. I cannot get a usable MIDI note on velocity out of the EWI, this is because it is impossible tell how loudly I am going to playing a note just from the very start of my blowing.  I suggest we hardcode the velocity to 50% or 64 and ignore the EWI midi note on velocity output. (not max velocity as that often is a hard over blown sample)
3. We need to get fluidsynth accurately adjust the amplitude to track the breath data. Somehow we want to magically extract a breath profile from the soundfont sample and compare that to the players midi breath stream and then adjust the output according to the difference between the two profiles. 
   
4. Overlapping or Linked notes or Legato notes. There is a distinction between tonged notes and legato notes in that the later does not have any "attack" part of the note. I don't know if it is possible to switch to another note skipping the attack and decay section (which may contain the tonging sound) and go straight into the sustain part of the note. So for example if the musician plays note 'A' for one second and then plays note 'B' legato (without tonging), could fluidsynth play note B not from the beginning but as if note B had already been playing for one second. The EWI output always has the next note on BEFORE the old note off for legato notes so making the notes overlap.

I'll let you know how I get on with my custom soundfont. 

At the moment when I try connect the EWI to fluidsynth using a fixed velocity and map the breath sensor to MIDI _expression_ (using the EWI) the outputs sound dead. I am not sure what is wrong.

Louis.

On Wed, May 6, 2015 at 1:22 PM, jean-jacques.ceresa <address@hidden> wrote:

Hi,

Thanks again for theses useful informations about EWI MIDI data.

jjc> Here the list of default modulators to change the amplitude of a sounding note in real time (called "Initial Attenuation" in SoudFont terms) .

Louis> What is needed is "Continual Attenuation"  rather than "Initial Attenuation".

jjc>  Yes, and it is the case, please read:
- Initial Attenuation is the name of a SoundFont Parameter. This value set the  "maximum" amplitude on ADSR envelope output.
- On a note start  this "maximum" is read from the soundfont and set. For the volume ADSR generator, this set the ("max") level of the voice(s) note during its life.
   This parameter is useful when a note have more than 1 voice (this is instructed by the soundfont) as each voice may have different ("max") level.
- This "maximum" value  can be  "modulated" by a list of default modulator (this is instructed by the soundfont). This means that during note life from the start, this
  "maximum" value can be changed by the modulators in real time. This is why this parameter is called "Initial..".
- For the "Initial Attenuation", the list of defaults modulators are :
  "Note-On Velocity to Initial Attenuation", the source of the modulator is the noteOn velocity.
  "_expression_ (CC 11) to Initial Attenuation", the source of the modulator is CC 11.
  "Volume  (CC 07) to Initial Attenuation", the source of the modulator is CC 07.
  The result is:  Voice Amplitude= volADSRout x Initial x Vel x Exp x Vol
  Note than Initial,Vel,Exp,Vol are normalized  to [0..1]  (0 for minimum value, 1 for maximum value).
  So Amp = Initial at Peak envelope when,  noteOn velocity= 127,  CC 11=127, and CC07=127.

  We see that if noteOn velocity is low (i.e 20) , the maximum amplitude will be lowered to this low value 20
  This is why for any MIDI breath controler (EVI,EWI,..) we need to:
     -1)set a fixed noteOn velocity value to 127, to keep the "max amplitude" possible value to the value set by the soundfont designer.
         This setting can be done,
         1.a) On the MIDI controler (EWI) (if possible).
         1.b) On the synthesizer (if possible)
         1.c) By the soundfont designer (generator GEN VELOCITY)
    
     -2)Have a way to modulate "Initial Atenuation" by CC Breath"
         This setting can be done,
         2.a) On the MIDI controler (EWI) (if possible), by routing Breath sensor to CC 11 (or CC 7)
         2.b) On th synthesizer (if possible). In FluidSynth we can use the MIDI Router function.
         2.c) By the soundfont designer, by adding a modulator "Breath CC  02 to Initial Attenuation" in the soundfont.

    Method 1.a,b, 2.a,b are interresting for quick result. But this is not the best because at sound design time, if
    there are modulators with noteOn velocity to modulate other sound parameters, at playing time theses parameters
    will be modulated by a fixed velocity (i.e not modulated at all). So the Sound design is damaged.

    Method (1.c, 2.c) are by far the best method. Unfortunately this is not a quicker method

Louis>It may be better to work with sound fonts that use fluidsynth for the modulation as it is possible to override this with the breath modulation.
jjc>Now, we know that modulation in a SF2.01 synthesizer is done by the musician via MIDI CC ->Modulators-> Synthesis parameters.
We also must know that modulators (described in soundfont) are additive to the value of the parameter set by the sound designer.
So if the musician put the CCx value to "0", the result is "no modulation x". So  it is always possible to externally disable/enable
any CC modulator source, without the need to re-edit the soundfount or add specific code at synthesis engine.

Further, for EWI it will be interresting to try to "overide" the effect of Volume ADRS generator. Again this is possible at Soundfont Design time.
For now this post is definitively too long !

Jean-jacques
 

Le 03/05/2015 11:23, Louis B. a écrit :

Hi, jcc,

Here are my replies. Sound  + midi recording to follow.

jjc> Technically 'vibrato' is a pitch modulation and 'tremolo' is an amplitude modulation. What are you talking about ?

I am talking about the way I play the flute and the sound produced when I don't play a steady note.

jjc> 2.a) Do you mean that this "modulation"  is blended in the sound signal sample ?

Yes, I meant that. The problem with this "modulation"  in the sound signal sample is that that the breath data steam includes it own natural modulation/vibrato following how the player plays.  It may be better to work with sound fonts that use fluidsynth for the modulation as it is possible to override this with the breath modulation.

jjc> Here the list of default modulators to change the amplitude of a sounding note in real time (called "Initial Attenuation" in SoudFont terms) .

What is needed is "Continual Attenuation"  rather than "Initial Attenuation".

I don't think that the 'note on' velocity is used so much in a breath capable synth instead the amplitude is set from the breath stream.

jjc> When MIDI noteOn event occurs ? When you start blowing ?

Yes, and also when you play a different note.

jjc> When MIDI noteOff event occurs ? When you stop blowing ?

Yes, and also when you play a different note.

There is a subtlety here, if you change notes while blowing steadily the new 'note on' comes just before the old 'note off'. When this happens maybe you should skip the attack part of the new note and go straight into the sustain part of the next note. Lets called these 'linked notes', more about this in following emails.

jjc> If possible, MIDI file format 2 will be appeciated as well the kind of synthesizer used when recording audio.

I use the built in sound generator of the EWI5000. There are not many breath capable synths out there.

It would be great if we could tweak fluidsynth to make it a breath capable synth but there are lots of challenges.

Louis

On 1 May 2015 at 18:21 "CERESA Jean-Jacques ENAC/ENAC"  wrote:

>  Hi, Louis

>  

> Thanks for you response about this subject.
> I need some precision from your last post and to ovoid overload on  fluidsynth maillist i prefer using direct mailing.
>  
> >However playing a note on the EWI it always sends a midi note on with a velocity just above zero.
> >This is because when you start blowing the sound level starts from zero then builds up.
> When MIDI noteOn event occurs ? When you start blowing ?
> When MIDI noteOff event occurs ? When you stop blowing ?
>  
> >but this is a large sound fount and has the vibrato built into the sound loops
> 1) Often musician use 'vibrato' word talking  about amplitude modulation.
> Technically 'vibrato' is a pitch modulation and 'tremolo' is an amplitude modulation. What are you talking about ?
>  
> 2.a) Do you mean that this "modulation"  is blended in the sound signal sample ?, or
> 2.b) Do you mean that it is described as a modulator with a MIDI CC on input. ?.
> In case 2.a, while playing it is impossible "to say" to the synthesizer engine to change anything during synthesis.
> In case 2.b, while playing it alway possible "to say" to the synthesizer engine to change any parameters via MIDI CC and modulators.
> Soundfont (2.01) have a lot of default modulator. For example Mod wheel (MIDI CC 1) to Vibrato LFO pitch depth.
> It is just an example describing a MIDI CC that change the pitch depth of Vibrato LFO.
> Notice that 'Vibrato LFO' in soudfont 2.01 synthesizer model is an LFO that change the pitch of sound.
>  
> > What is needed is a custom sound font that uses a breath controller to control the volume. There is natural vibrato in the MIDI breath data.
> > So what would be great if we could somehow add a magic flag to fluidsynth that overrides the global vibrato modulation used on many sound fonts
> > and instead control that with the breath controller input. This would save having to manually edit all the sound fonts to respond to the breath controller.
>  
> To control the amplitude of a note in real time you need a soundfont with a modulators to do that.
> Fortunately, when any soundfont is loaded, a list of default modulators is build inside the engine. You can use theses defaut modulators
> as far they are not changed by custom modulators having the same name in the soundfont.
>  
> Here the list of default modulators to change the amplitude of a sounding note in real time (called "Initial Attenuation" in SoudFont terms) .
>
> - MIDI Note-On Velocity to Initial Attenuation
>
> - Volume (MIDI CC 7) to initial attenuation
>
> - _expression_ (MIDI CC 11) to initial attenuation
>
>  
>
> As you see there is no default modulator with breath controler (CC 2 ) on its input. But when using fluidsynth application you can "say" it to transform
>
> any MIDI message CC 2 in a MIDI CC 11 using the router command line interface.
>
> Using fluidsynth application, you need to enter the following command in the shell to instruct the router.
>
> # Remove current rules (to remove any default rules):
> router_clear
>
> # Set the rule to transform CC breath (CC 02) to CC _expression_ (CC 11)
>
> router_begin
>
> cc
>
> router_par1 02 02 0 11
>
> router_end
>
> # Set the rules to pass through other messages types (note, prog, pbend, cpress, kpress)
>
> router_begin note
>
> router_end
>
> router_begin prog
>
> router_end
>
> router_begin pbend
>
> router_end
>
> router_begin cpress
>
> router_end
>
> router_begin kpress
>
> router_end
>
>  
>
> >I will post over the weekend a MIDI recording (with matching audio).
>
> If possible, MIDI file format 2 will be appeciated as well the kind of synthesizer used when recording audio.
>
>  
>
> Thanks a lot for your contribution.
>
>  
>
> jjc
>
>  
>
>  
>
>  

 

_______________________________________________
fluid-dev mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/fluid-dev

_______________________________________________
fluid-dev mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/fluid-dev