Parallel and Large Results

[Alberto Simões]

Hello

I am trying to use parallel package, namely the parcellfun function.

I was able to make it work, but given that each iteration result is a huge cell with matrices (about 20 MB each), my machine gets out of memory before all processes finish their work.

What I need is a way to "reduce" results during the parallelism (and not just at the end). These matrices should be all summed up later, so the reduce algorithm is associative, and therefore, I can do it whenever I like.

The problem is that I do not see a way to do that. I tried to do something stupid, creating a global cell, and making each process change it. But as you might guess, the global cell is not copied to the forked octaves.

An option would be to create a file with each data, and then open them, read, and sum values. But this is worse in processing time.

Is there any way to make this work? A method to make the global variable really global? Or a method to reduce the results for each process (chunk) before returning?

If possible, some way I do not need to change the parallel code O:-)

Thank you very much,
Alberto

--

Alberto Simões