RE: reading data using fgets in while loop

[financial engineer]

> Date: Mon, 12 Mar 2012 17:11:27 +0000
> From: address@hidden
> To: address@hidden
> CC: address@hidden
> Subject: Re: reading data using fgets in while loop
>
> On Mon, Mar 12, 2012 at 12:13:50PM -0400, financial engineer wrote:
> >
> >
> >
> > > Date: Mon, 12 Mar 2012 15:48:40 +0000
> > > From: address@hidden
> > > To: address@hidden
> > > CC: address@hidden
> > > Subject: Re: reading data using fgets in while loop
> > >
> > > > > Date: Mon, 12 Mar 2012 11:37:37 +0000
> > > > > From: address@hidden
> > > > > To: address@hidden
> > > > > CC: address@hidden
> > > > > Subject: Re: reading data using fgets in while loop
> > > > >
> > > > > On Sun, Mar 11, 2012 at 04:00:36PM -0700, newbie_octave wrote:
> > > > > > I have a .csv file with the following data
> > > > > >
> > > > > > 01/03/2012,H (Mar 12),26.750000,2584
> > > > > > 01/04/2012,H (Mar 12),26.280000,2330
> > > > > > 01/05/2012,H (Mar 12),26.000000,3198
> > > > > > 01/06/2012,H (Mar 12),25.500000,3045
> > > > > > 01/09/2012,H (Mar 12),25.350000,2314
> > > > > > 01/10/2012,H (Mar 12),25.050000,2685
> > > > > >
> > > > > > and I am a newbie to octave. I ran the following command to read the above
> > > > > > data into octave
> > > > > >
> > > > > > X=csvread("/tmp/hist.csv")
> > > > > >
> > > > > > but it returns the following matrix.
> > > > > >
> > > > > > 1.0000e+00 0.0000e+00 2.6750e+01 2.5840e+03
> > > > > > 1.0000e+00 0.0000e+00 2.6280e+01 2.3300e+03
> > > > > > 1.0000e+00 0.0000e+00 2.6000e+01 3.1980e+03
> > > > > > 1.0000e+00 0.0000e+00 2.5500e+01 3.0450e+03
> > > > > > 1.0000e+00 0.0000e+00 2.5350e+01 2.3140e+03
> > > > > > 1.0000e+00 0.0000e+00 2.5050e+01 2.6850e+03
> > > > > >
> > > > > >
> > > > > > Obviously, it is not reading the text correctly
> > > > > > so I am now using fgets() as follows:
> > > > > >
> > > > > > fid=fopen(fname);
> > > > > > fout="out.mat"
> > > > > > global tline;
> > > > > > global tempstr;
> > > > > > while 1
> > > > > > tline = fgets(fid);
> > > > > > sep=",";
> > > > > > tempstr=strsplit(tline, sep);
> > > > > > dt=tempstr(1);
> > > > > > cname=tempstr(2);
> > > > > > price=str2double(tempstr(3));
> > > > > > volume=str2double(tempstr(4));
> > > > > > if ~ischar(tline), break,end;
> > > > > > end
> > > > > > disp(tline);
> > > > > > fclose(fid);
> > > > > >
> > > > > > but when I run the script, it returns -1 due to disp(tline) being after the
> > > > > > end. I want to be able to compute the volume-weighted average price, and not
> > > > > > have to do it all within the script that reads the data from the file. But,
> > > > > > given that I cannot access any of the variables outside the while loop, I am
> > > > > > stuck. Ideally, I could have used csvread() or dlmread() but that is messing
> > > > > > up the string fields in my file. Can anyone please suggest how I fix this so
> > > > > > I can manipulate the variables outside the while loop that gets the data
> > > > > > from the .csv file. thanks!
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > View this message in context: http://octave.1599824.n4.nabble.com/reading-data-using-fgets-in-while-loop-tp4464844p4464844.html
> > > > > > Sent from the Octave - General mailing list archive at Nabble.com.
> > > > >
> > > > > Hi,
> > > > > I would try while(!eof(fid)) instead of while(1).
> > > > > This would stop the while loop when the whole file is read.
> > > > >
> > > > > as an example, I've copied a part of a script that I use myself. This also does some date conversion to 'seconds since epoch'.
> > > > > To read the manual, do 'doc eof' or 'doc mktime', etc.
> > > > >
> > > > > ------------------------------------------------
> > > > > params27C.date:
> > > > > 05/12/11 18:56:37 0.001728386 16.43405 1.323835
> > > > > 05/12/11 19:26:39 0.001676934 16.41423 1.334429
> > > > > 05/12/11 19:56:40 0.001599179 16.40604 1.351362
> > > > > 05/12/11 20:54:10 0.001547091 16.35170 1.361025
> > > > > ------------------------------------------------
> > > > >
> > > > > script:
> > > > >
> > > > > pos27f=fopen('./fits2012/27C/params27C.date');
> > > > > n=0;
> > > > > while (!feof(pos27f))
> > > > > n++
> > > > > line=strsplit(fgetl(pos27f),' ')
> > > > > datestring=sprintf('%s %s',line{1},line{2});
> > > > > pos27sec(n)=mktime(strptime(datestring,'%d/%m/%y %T'));
> > > > > pos27(n)=str2num(line{4});
> > > > > endwhile
> > > > > fclose(pos27f)
> > > > >
> > > > >
> > > >
> > >
> > > On Mon, Mar 12, 2012 at 11:23:56AM -0400, financial engineer wrote:
> > > >
> > > >
> > > > Thanks for the script...I rewrote mine as follows:
> > > > fname="/tmp/VIXts.csv"
> > > > fid = fopen(fname);
> > > > n=0;
> > > > while (!feof(fid))
> > > > n++
> > > > tline=strsplit(fgetl(fid),',')
> > > > datestring=sprintf('%s',tline{1});
> > > > fid(n)=str2num(tline{3});
> > > > endwhile
> > > > fclose(fid)
> > > >
> > > > but when it encounters endwhile, it gives the following error:
> > > > > endwhile
> > > > error: feof: invalid stream number = -1
> > > >
> > > >
> > >
> > > Dear financial engineer,
> > >
> > > please put your 'reply-text' at the bottom of the thread (like I do here), so that the thread is readable from top-to-bottom.
> > >
> > > feof() complains because the fid is not a positive number. The fid is not positive, which most certainly means that the file
> > > was never opened. For instance, the file doesn't exist, the name or directory is spelled wrong, the file permission on the filesystem is
> > > wrong, etc, etc.
> > >
> > > Can you try to do it manually, step by step, by starting typing the commands on the octave command-line?
> > > Try other files, skip the while() loop in first instance, check the file name, try fid=fopen('/tmp/VIXts.csv') directly, etc, etc.
> >
> > hi Indium,
> > In fact the file was missing, so now it is capturing the price and volume, but I only get one date. To fix that I tried, using datestring(n) as below:
> >
> > here is my updated code:
> > fname="/tmp/VIXts.csv"
> > fid = fopen(fname);
> > n=0;
> > while (!feof(fid))
> > n++
> > tline=strsplit(fgetl(fid),',')
> > datestring(n)=sprintf('%s',tline{1});
> > %cname(n)=sprintf('%s',tline{2});
> > price(n)=str2double(tline{3});
> > volume(n)=str2num(tline{4});
> > endwhile
> > fclose(fid)
> >
> > Now when I run it, I get the following error due to datestring(n)=sprintf('%s',tline{1});
> > fname = /tmp/VIXts.csv
> > ans = 0
> > tline =
> >
> > {
> > [1,1] = 01/03/2012
> > [1,2] = H (Mar 12)
> > [1,3] = 26.750000
> > [1,4] = 2584
> > }
> >
> > error: A(I) = X: X must have the same size as I
> >
> > how do I capture all the datestrings.
> >
> > thanks,
> > Bobby
> >
> >
>
> Hi Bobby,
>
> sprintf will give you a string, but you index datestring as a matrix.
>
> octave:1> e(1)="werwerwe"
> error: A(I) = X: X must have the same size as I
> octave:1> e{1}="werwerwe"
> e =
> {
> [1,1] = werwerwe
> }
> octave:2> e{2}="rrrrrrr"
> e =
> {
> [1,1] = werwerwe
> [1,2] = rrrrrrr
> }
> octave:3>
>
> So, you can store your strings with {} instead of ().
>
> Another possibility is converting the date to a integer, as I have shown in my script. Then you can store those in a matrix/vector using the () index.
>
> Be aware with these cells: if you generate them with {}, you can retrieve elements back with {}, but also with (). I don't always grasp the idea behind that myself...
>
>
>
>
>
>
>

hi Indium,
understood, the {} works vs. ().
but then I also tried the date conversion as in your code (since my date format is mm/dd/yyyy), I wrote the following:
datestring=sprintf('%s',tline{1});
pos27sec(n)=mktime(strptime(datestring,'%m/%d/%Y'));

pos27sec =

 Columns 1 through 6:

   1.3256e+09   1.3257e+09   1.3257e+09   1.3258e+09   1.3261e+09   1.3262e+09

 Columns 7 through 12:

   1.3263e+09   1.3263e+09   1.3264e+09   1.3268e+09   1.3269e+09   1.3269e+09

 Columns 13 through 18:

   1.3270e+09   1.3273e+09   1.3274e+09   1.3275e+09   1.3276e+09   1.3276e+09

 Columns 19 through 24:

   1.3279e+09   1.3280e+09   1.3281e+09   1.3282e+09   1.3282e+09   1.3285e+09

 Columns 25 through 30:

   1.3286e+09   1.3287e+09   1.3288e+09   1.3288e+09   1.3291e+09   1.3292e+09