avr-gcc-list
[Top][All Lists]

## Re: [avr-gcc-list] update on the last issue

 From: E. Weddington Subject: Re: [avr-gcc-list] update on the last issue Date: Tue, 05 Apr 2005 10:54:44 -0600 User-agent: Mozilla Thunderbird 0.7.3 (Windows/20040803)

```intiha Ho gai wrote:

```
Thank you. Okay.. i can live with that.. for now.. but my issue is.. will the
```long long addition be correct?
Also if it is, then why isnt the solution for the equation to
calculate a (slope) and b(offset) coming out incorrect. For the above
code where I hand coded the values of all long longs
table.sumX = 12338578LL;
table.sumY = -3336078LL;
```
table.sumXSquared = 7260107358192LL; table.sumXY = -1646741910392LL; denom = ( table.n * table.sumXSquared - table.sumX * table.sumX);
```       if (denom != 0)
a = (double)( table.n * table.sumXY - table.sumY *
table.sumX) / denom;
b = (double)(table.sumY - a * table.sumX) / table.n;

sprintf(OutputMsg,"**************** A=%f     B=%f ***********\r\n",a,b);
UARTOutput(DBG_ERROR,"%s", &OutputMsg);

I get on the UART

**************** A=-0.045255     B=-106838.632813 ***********

While the actual values of a and b that i have precomputed should be a
= -0.000208 , and b = -133340.419011
```
What is the value of table.n that you are using in the code above to get the results that you gave?
```
Is table.sumXSquared supposed to be the square of table.sumX?