[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Octave Interpreter

From: Stefan Seefeld
Subject: Re: Octave Interpreter
Date: Thu, 02 Oct 2014 08:34:38 -0400
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.1.1

On 2014-10-01 21:16, Bipin Mathew wrote:
> Hello Manu,
>     I was actually thinking of working on something along the same
> lines. Googling around, I see a few efforts along these lines:
> http://octave.sourceforge.net/parallel/
> http://octave.sourceforge.net/mpi/index.html
> While I have not used these personally, looking at the examples, it
> appears one needs to be quite aware of MPI / parallel programming to
> make effective use of these package. It would be nice if all these
> things could be made more transparent to the end user.
> Many years ago, I actually worked with a team at MIT Lincoln
> Laboratory, designing this sort of  thing for Matlab:
> http://www.ll.mit.edu/mission/cybersec/softwaretools/pmatlab/pmatlab.html
> The idea was to hide as much of the parallelism as possible. This was
> done by using a special "dmatrix" class to represent your matrix,
> which was, in reality, spread across several servers. I don't know if
> this is still an active project there ( Any of my old Lincoln friends
> can comment? :-) ).

Interesting that you bring this project up, as I was considering
proposing something similar. While I haven't worked at the MIT/LL, I
have collaborated with them.

> Of course technology has moved on since then and there is probably
> alot of new technology we can leverage to make something similar that
> cleaner/faster/smarter ( i.e. more awsome ). For example, metadata for
> where the data was was stored in a static file back in days. I reckon
> we can just query a distributed file system to get this same
> information these days, along with all the benefits of replication /
> performance / resilience.

I believe Jeremy Kepner's original implementation was deliberately using
very simple means to implement the transport protocol (all file-based),
to make it easy for users to get off the ground.

In collaboration with the MIT/LL et al. I have been working on a C++ API
specification (http://portals.omg.org/hpec/content/specifications) as
well as library implementation (http://openvsip.org/) incorporating much
of the concepts of pMatlab's distributed arrays (i.e., using "maps" to
define the distribution of blocks of data across many computing nodes).

I was considering working on scripting frontends for that implementation
in Python and Octave, and I have been watching with interest the recent
work on classdef support in Octave, as I was considering using that to
define the high-level API for Maps, Blocks, and Views.
(In fact, I presented the idea to add Python and MATLAB "frontends" to
VSIPL++ backends for a seamless integration into a high-performance
computing platform at an IEEE conference two years ago:
http://ieee-hpec.org/2012/index_htm_files/Seefeld.pdf, and the ideas
were received quite enthusiastically.)

>  All this being said, perhaps someone more familiar with the source
> code could comment on how easy / impossible such a thing would be
> given the current state of Octave? In terms of your direct question of
> how you can build a compiler for Octave, I believe a good place to
> start is to look at the lex/yacc grammar files:
> octave/libinterp/parse-tree/lex.ll
> octave/libinterp/parse-tree/oct-parse.yy

I'm honestly not entirely sure I understand the question: What
additional support in terms of language support is needed to provide a
"distributed Octave" ? What I was expecting (though I haven't
investigated much of it yet) is infrastructure work akin to
to allow an Octave "frontend" to dispatch computation to multiple "engines".

Would any of this require changes or additions to the language /
interpreter, rather than just additional functions ?

I think a good path towards support for pMatlab in Octave would be

* infrastructure work that allows multiple "octave engines" to be
controlled from one "octave controller" via some scheduler, akin to
ipython's implementation)
* an Octave package that adds an MPI-like API on top of the above
* A new API defining distributed arrays in terms of "maps"

I'd be happy to collaborate on this, in particular as there are
discussions in the OMG HPEC Working Group (http://portals.omg.org/hpec/)
to add new language bindings for Python and MATLAB/Octave to the VSIP



      ...ich hab' noch einen Koffer in Berlin...

reply via email to

[Prev in Thread] Current Thread [Next in Thread]