Re: [avr-gcc-list] update on the last issue

[E. Weddington]

intiha Ho gai wrote:

> Thank you. 
> Okay.. i can live with that.. for now.. but my issue is.. will the
> long long addition be correct?
> Also if it is, then why isnt the solution for the equation to
> calculate a (slope) and b(offset) coming out incorrect. For the above
> code where I hand coded the values of all long longs
>   table.sumX = 12338578LL;
>       table.sumY = -3336078LL;
>       table.sumXSquared = 7260107358192LL; 
>       table.sumXY = -1646741910392LL; 
>        denom = ( table.n * table.sumXSquared - table.sumX * table.sumX);
>        if (denom != 0)
>               a = (double)( table.n * table.sumXY - table.sumY *
> table.sumX) / denom;
>        b = (double)(table.sumY - a * table.sumX) / table.n;
>
>     sprintf(OutputMsg,"**************** A=%f     B=%f ***********\r\n",a,b);
>     UARTOutput(DBG_ERROR,"%s", &OutputMsg);
>
> I get on the UART
>
> **************** A=-0.045255     B=-106838.632813 ***********
>
> While the actual values of a and b that i have precomputed should be a
> = -0.000208 , and b = -133340.419011
>  
What is the value of table.n that you are using in the code above to get 
the results that you gave?

Is table.sumXSquared supposed to be the square of table.sumX?

Please give us more information about what you are doing here.

Also, could you post/attach the assembly output of the above code snippet?

Eric