fluid-dev
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [fluid-dev] Loss of quality with --enable-float option?


From: Carlo Bramini
Subject: Re: [fluid-dev] Loss of quality with --enable-float option?
Date: Tue, 11 Jun 2019 19:26:41 +0200 (CEST)

Hello,

This is the relative sample quantization error that we make when using

single precision. Let's just ceil this value, i.e. for every sample
computed in single precision we may have an integer quantization error
of +-1 .

BTW, it should be noted that since dithering is applied when writing 16bit samples, the LSB of each sample is not significant because it moves in a pseudo-random way. So, probably this quantization error is a "do-not-care" error when emitting with 16bit samples, indipendently by the math resolution.


Sincerely.


reply via email to

[Prev in Thread] Current Thread [Next in Thread]