Re: How to read integers from file faster?

[Pascal J. Bourguignon]

Darren Hoo <address@hidden> writes:

> It is way too slow to read numbers from a file simply by using `read' 
>
> for example a txt file contains 10,000,000 line of numbers:

To get speed, I would:

1- open a binary file,

2- perform double-buffering I/O,  (ie. read a buffer while you process
   another),

3- assume the input format is as specified, ie. no check.
   …
   (let ((byte (vector-ref buffer i)))
     (if (= byte 10)
        (begin (set! values (cons value values))
               (set! value 0))
        (set! value (+ (* value 10) (mod byte 16)))))
   …


Actually, only using buffered I/O matters.  For big files, what slows
down is I/O, not the handling of characters or checking of the syntax or
anything that's done in memory.  If you want to read your data faster,
keep it in RAM, or else in SSD.

-- 
__Pascal Bourguignon__
http://www.informatimago.com/