bug#14820: 24.3; elisp manual: How to write good idle timer worker functions?

[Daimrod]

"Phil Sainty" <psainty@orcon.net.nz> writes:

> Stefan Monnier wrote:
>> Currently, Emacs doesn't support such "background tasks" well.
>> Documentation could help, but I think it's not really worth the
>> trouble.  Better come up with a package that provides support
>> for it.
>
> Well surely you'd agree that writing such a package is going to be
> more much difficult without documentation? That is essentially the
> problem I ran into here, after all.
>
> At the absolute minimum, that page definitely needs to mention
> `sit-for' / "(elisp) Waiting" and `accept-process-output' /
> "(elisp) Accepting Output". After all, when specific pit-falls
> are mentioned, I think that the available facilities for working
> around them warrant at least a passing mention.
>
> (Honestly, the merest mention of these two functions after the
> description of the problems of blocking other timers and processes
> would have saved me many hours, so it's worth it just for the
> potential to save someone else from being similarly stumped. I'd
> simply never had cause to use them before, so it wasn't *at all*
> obvious where to look.)
>
> Having found my way to the latter, and run some experiments with
> generating output from a shell while running my idle timer code,
> I've managed to answer my own question about `sit-for' (which is
> that it does not solve the problem of accepting process output), but
> determined that (unsurprisingly) calling `(accept-process-output)'
> does provide a solution, so I believe I'm on my way to resolving my
> original issue.
>
> I do still think that if there were some nice easy-to-use wrappers
> around the functionality that Emacs *does* provide, we could say
> something a little more positive about the situation -- that
> *despite* not supporting "background tasks" well, Emacs makes
> it easy to fake them!
>
>
>> IIRC someone posted some packages that try to provide support
>> for related issues, but I can't find the corresponding email.
>
> Ah, that's a shame; but thank you for looking. Maybe someone
> else remembers what those were?
>
> Failing that, if I end up with something that seems useful and
> generic, I'll submit it for consideration.

A bit of background:
I'm GSoC student working on the XWidget branch and I wanted to add test
to it. But I've encountered two problems while trying to write "classic"
ERT tests. One is that since I'm working on the C side on an
experimental branch, sometimes Emacs crashes. The second problem is that
some tests needs to be run in a graphical Emacs. So I can't run the
tests in batch mode to circumvent the first problem.


I've wrote a package, emacs-parallel[1], (the API isn't very clean but
it's good enough for my current needs) to eval some stuff in another
Emacs process. I've been heavily inspired by emacs-async[2], but I've
made differents choices:
- it can start a graphical Emacs (not Batch Mode);
- it communicates through a Unix socket instead of standard I/O stream
(because, AFAIK, it is not possible to use them without Batch Mode);
- you can send data from the remote process while the process isn't
finished instead of just sending the result of the call;
- the main entry point is a function, not a macro;
- it tells you whether the process terminated normally, and if not, what
happens (exit code or signal number).
- it handles timeout (you can stop the remote instance after X seconds);

I haven't wrote much doc yet (I've started it monday) so maybe
emacs-async is a better choice (ATM) if you don't need an graphical
Emacs or if you don't need to send data without interrupting the remote
process.

(parallel-start (lambda () (parallel-send 42) (sleep-for 3) 12)
                :post-exec (lambda (results _status)
                             (message "%s" results)))

(emacs-async equivalent)
(async-start (lambda () (sleep-for 3) (list 12 42))
             (lambda (result) (message "%s" result)))

=> after 3 sec it displays (12 42)

However, since those packages simulate parallel/async jobs with another
Emacs instance, you cannot execute code with side effects. Well, you
can, but it won't have any effect in your current Emacs instance, only
on the remote. You have to isolate the side effects and to put them in a
function to be executed once the computation (free of side effect) is
done (`finish-func' in `async-start' or `:post-exec' in
`parallel-start').

> -Phil
>
> p.s. Although I *think* I now know what I'm doing here, I don't
> want to close this issue just yet. I'm very interested in any and
> all comments on this subject, if any other people wish to chip in
> with their thoughts?

[1] https://github.com/daimrod/emacs-parallel.git
[2] https://github.com/jwiegley/emacs-async.git

-- 
Daimrod/Greg