Re: [Fab-user] Output of local()

[Jeff Forcier]

Hi all,

This has turned into a much bigger issue than I originally thought!
Here's a quick run-down:

- Started out trying to working with subprocess + on-disk files, as
per that thraxil.org link. This turned out to be a headache (partly
due to my lack of expertise in such matters, admittedly) although it
looked like it might be possible with a lot of work. And with code
likely to be inefficient and bug-prone.
- I then did a sanity check to try and reproduce that 64KB limit issue
and...couldn't. At least, not on OS X running from-MacPorts Python
2.5.2, where the command run via subprocess was "cat /some/file.txt"
and "/some/file.txt" was 444KB.
- Given that, I am assuming the issue has been fixed, or only occurs
on some specific platform (the link didn't say). So I went ahead using
PIPEs.
- Unfortunately, I then ran into crazy intermittent "no subprocesses
found" errors, which appears to be the long-standing and unfixed
Python bug http://bugs.python.org/issue1731717 . And if I read that
correctly, it would have cropped up with the file-based approach
anyway -- basic issue being that subprocess.Popen is not thread-safe.


I've been thinking about trying to update Fabric to use Twisted
instead of Paramiko, as it could theoretically make things much
simpler -- using asynchronous callbacks* instead of the current
thread-based, more "raw" recv()/read() loop approach. And after the
above, it seems even more tempting.

This will obviously take some time to investigate so I won't bother
with a play-by-play, but will let you all know if anything interesting
comes up. Comments are welcome! (Especially if you know of any decent
and not-widely-linked Twisted tutorials...)

Regards,
Jeff

* Async callbacks is exactly how Capistrano's underlying module,
Ruby's net-ssh, handles this problem, and I liked what I saw of it.