[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
dlmread / matrix mem consumption
From: |
Levente Torok |
Subject: |
dlmread / matrix mem consumption |
Date: |
Tue, 23 Apr 2013 19:30:04 +0100 |
Hi All,
I am dealing with large data set for this reason I had to see how much
it can be loaded for digesting.
I used dlmread since data comes from a csv.
The memory consumption of a 1M x 22 table is stored rougly in 200MB
which means roughly 9..10 Bytes / double.
During the load it went up temporarily to 410MByte which means which
is a 2x overhead.
But loading a 10M x 22 table required 8GB during the load and dropped
back to 3.4 GBytes.
This drop is similar to the one above but for storing a double it eats
16..17 Bytes.
Can anybody tell me why this is happening?
Is there a way of reducing excessive mem. consumption during dlmread()?
Thanks,
Lev
- dlmread / matrix mem consumption,
Levente Torok <=