Re: help! Octave, connecting laptops to "supercomputers"

[Pedro]

On Sun, Feb 3, 2013 at 12:19 PM, Thomas Weber <address@hidden> wrote:
>
> Hi,
>
> reading through your mails gives me the impression that you have no idea
> what you actually want, sorry.

It doesn't matter ;)
I think that every response is giving interesting information about
that field. And helps me, and I hope also other people that is having
the same problems.

> If you process data on a host and want to look at a graphics from this
> data processing, then this image must be sent to your local machine. It
> doesn't matter at all what kind of connection you use; the image must be
> sent to your machine. If you need to sync lots of data (like > 10 MB)
> over a slow connection, then rsync is the best general purpose solution.
>
> Yes, there are options that are better then forwarding an X connection
> to your laptop via ssh. But then it depends on what you want. Because,
> if you just want to run a script via the Octave command line, then you
> wouldn't even forward your X session, but just run Octave inside GNU
> screen or tmux.

Yes, I see screen or tmux really useful for keep running processing
when you disconnect session with ssh [1] [2]

> On Thu, Jan 31, 2013 at 02:00:33AM +0100, Pedro wrote:
> > First of all, Writing this mail I wanted to know what use octavian
> > people like me (or not) when need to work with hard processes. 4/4 for
> > going in "ssh -X", well, I have to re-think on that!
> >
> > I said no to "ssh -X" because I'm looking for alternatives on
> > connecting with computer/computers that process the data.
> >
> > - I've never used. (read last line of this mail)
> >
> > - from ssh -X I read that is innerently insecure, because run a root
> > process: the X server, read this (old) review about this [1].
> >
> > - Also, because is slow (or inefficient), that's because is encrypting
> > all the communications to the server.
> > But I only want to "pass the code strictly necessary".
> > Because if I use the ssh connection as a workstation it's excessive
> > for what I want.
> > Imagine looking images, sound or video with a ssh connection in a
> > "streaming way".
> > I could do rsync? ... Yes, but are things that take you off your
> > concentration, no? It means, that make you thing on two machines
> > constantly.
>
> I suggest you actually start with ssh and rsync and then solve problems
> as they come along.
>
>
> > I would like to imagine "one machine", could be with virtualization?
> > cloud computing? "p2p processing" ? Assymetric processing?
>
> Your problem definition is so broad that even "pocket calculator" seems
> to be in the solution set. What do you actually want to do?

I wanted to know what are doing other people with the problem of "big
calculations", and discuss different methods.
And I put that keywords for inspire people to post a solution in that way.

> > I don't understand a lot how it works distcc, but see this:
> > compilation of kernel distributed: the guy is in his computer, and
> > divide tasks to others, after it, they have his output file. [1]
> > Compiling means that a personalized task (hardware, processor
> > architecture...) but delivered to process by others
>
> distcc is a specialized tool for a very specific problem. In other
> words, it is the complete opposite of your approach.
>
> > For "ssh -X" users, could they give some tricks to make life easier?
> Depends; what problems do you have?
>
> For the record, I used to have Octave running on a 40 node cluster
> several years back. One Octave session was started on each host insided
> a screen session with 40 windows. Data was put onto the central master
> node,
> home directories were NFS-mounted into each node. Numeric results were
> saved on each node into the shared home directory and finally images
> from this data was generated on the master node. The only data that I
> then downloaded from this system was the images. This worked reasonably
> well for me. If your problem is different, then your solution will be
> different.
>
> All of this only makes sense if your calculations takes some time. If
> you have to wait like 1 minute, then the time to set up your specific
> solution will be much larger than the time for the calculations on your
> small machine.

I appreciate your sharing.
And I have a question:
How you manage this 40 screens (nodes)? In every screen you put
different commands/scripts (by hand), to have the processing
distributed?
Or what tool / interface you use to do it?

Thanks!
Pedro

[1] 
http://askubuntu.com/questions/8653/how-to-keep-processes-running-after-ending-ssh-session
[2] 
http://serverfault.com/questions/19634/how-to-reconnect-to-a-disconnected-ssh-session