Re: Parallel octave ??

[Michael Creel]

On Mon, Sep 8, 2008 at 11:01 AM, Javier Fernández <address@hidden> wrote:
> address@hidden wrote:
>> Today's Topics:
>> [...]
>>    4. Re: Parallel octave ?? (Bill Denney)
>> [...]
>>
>> ------------------------------------------------------------------------
>>
>> Subject:
>> Re: Parallel octave ??
>> From:
>> Bill Denney <address@hidden>
>> =?ISO-8859-1?Q?Javier_Fern=E1ndez_ wrote:
>>
>>> On the other hand, I found mpirun easier to use with Open-MPI, since no
>>> daemon has to be previously started [...]
>>>
>>
>> Hi Javier,
>>
>> I'm curious how easy it would be to modify this to write a parfor
>> command?  I'm probably completely off on this, but it sounds like this
>> could translate into parfor without huge issues.
>>
>> Thanks,
>>
>> Bill
>>
> Well, Octave is GPL'ed so anybody has the freedom to do that, if they
> want. FWIW, my advice would be not to. I have Googled for parfor, and
> after reading the first two pages, I see some issues :-)
>
> 0.- parfor is a grammatical structure of the language. It's not a
> command. You cannot implement it as a function. It's the kind of
> reserved word that matches a closing "end". You better talk to JWE and
> collaborators to get your parser patch applied. For me, that qualifies
> as "huge" issue, ie: I won't do the job :-)
>
> 1.- The very moment you provide a parfor, you'll have users asking for
> the corresponding matlabpool. I'm not sure they would be satisfied with
> an octavepool command, probably they'll require matlabpool :-)
>
> 2.- Particularly, the idea of having an XXX-pool command would be more
> appropriate if/when using LAM/MPI which could easily lamboot and mpirun
> on the fly from a pure sequential Octave session. When you replied, I
> was just explaining that I found easier not to do that under OMPI.
> Probably now OMPI can do that too, after the 1.3.x series and the list
> discussion I showed last week, but I have not tried it yet.
>
> 3.- OMPI is the way to go. In MPITB "readme" file I commented on the
> difficulties I had trying to convince users to setup their clusters so
> that they could send back terms to debug their parallel applications...
> it soon becomes a nightmare, trying to MPI_Comm_spawn("xterm",...), it's
> easier to mpirun. I even deleted the "tutorial" file, where no mpirun
> was used, everything was done with MPI_Comm_spawn("xterm",...). Now
> MPITB has demos, which are to be executed using mpirun. OMPI is the way
> to go. mpirun is the way to go. Let "spawning on the fly" for advanced
> users. In fact, MPI_Comm_spawn was only included in the standard after
> many many discussions (and there are those who believe that it was due
> to pvm_spawn() and bad user habits, not to a sound rationale :-)
>
> 4.- I hope I'm not doing anything illegal if I quote the following from
> the second link to TMW in the Google search, which better illustrates
> the grammatical issues of parfor (and its semantics :-)
>> parfor i = 1:n
>>    if p(i)   % assume p is a function
>>       s = s + 1;
>>    end
>> end
>>
>> The key point of this example is that the conditional adding of 1 to s
>> can be done in any order. After the parfor statement has finished
>> executing, the value of s depends only on the number of iterations for
>> which p(i) is true. As long as p(i) depends only upon i, the value of
>> s is deterministic. This technique generalizes to functions other than
>> plus (+).
>>
> I suppose that "s=s+int(p(i))" does not generalize, ie, the technique
> refuses to trying to understand arbitrary code, and has a pre-defined
> casuistic of what can and cannot be done.
>
> The example above this one (in the original TMW page) is also
> interesting. They both prove that some (non-obvious) gramatical analysis
> is required to implement the desired functionality. Some part inside
> Octave (the one analyzing the parfor command) will need to know and
> decide a lot of things:
> a) decide the "control index variable" values to be assigned to each host
> b) do some index translation (better understood in the other example) so
> you won't need to repeat the same storage allocation on each host. Ie,
> if you are computing a 1GB array in parallel on N "workers", it would be
> nice you needed 1GB/N storage on each, not 1GB on each
>
> parfor i = 1:n;   t = f(A(i));   u = g(B(i));   C(i) = h(t, u); end
>
> c) do some Frank-ssembly to get the results back, and according to this
> other example, 3 cases are in order
> - scalars assigned (before used: i,t,u) inside the loop: no need to send
> or receive them. That includes the index.
> - arrays with indexing based in the control var (C(i), or C(3*i^2)):
> They must be assembled back.
> - scalars (like s in the 1st example) "known" to have a predictable
> final value: They must be somehow reduced back.
>
> The idea behind MPITB is that the final users knows better.
> The idea behind parfor seems to be that final users don't want to know
> there is a cluster (or multiple workers) behind parfor... well, they
> only need to know that indexes must be ascending row vector of
> consecutive indexes, if they are to be iterated with parfor. And that
> they shouldn't expect scalars to keep their "last assigned value" in the
> loop (which is rather sensible :-). And other things I'm not sure I
> foresee them all ;-)
>
> Again, that's my 2p. Octave is GPL, so anyone can play with the code. A
> different issue is getting the patch commited, which renders JWE and/or
> collaborators responsible for its maintenance. parfor has more to do
> with Octave parser than with MPITB, really.
>
> -javier
> _______________________________________________
> Help-octave mailing list
> address@hidden
> https://www-old.cae.wisc.edu/mailman/listinfo/help-octave
>

Thanks, Javier, for your characteristically clear and detailed
discussion. I just want to agree that parallelization done by design,
chosen to take advantage of granularity, is a model that works for me.
For the stuff I do, an automatic parallel loop would certainly not
perform nearly as well as breaking the problem down intelligently
(semi, at least).  I'm so sure about that that I've never bothered to
try to verify that it's true!

About mpirun being the way to go, I think that I agree in most cases.
But the on-the-fly spawning works well, too, and for some of my work I
can't really figure out how to easily adapt it to the mpirun model
without loosing a lot of flexibility.
Cheers, Michael