bug#46388: 27.1; emacs -batch does not output messages immediately when invoked outside of the command prompt

[Ioannis Kappas]

Hi Eli,

    the reported issue and the draft patch is only concerned with the
    behavior of the MESSAGE function in emacs -batch, not with
    the communication of any emacs subprocess in general.

    I think I now understand where you are coming from.

    Let me summarize once more where we are, introducing buffering in
    the description (assuming MESSAGE length is < 2048):

    | # | System     | emacs -batch invoked from | MESSAGE behavior

        |
    
|---+------------+---------------------------+----------------------------------------------------------------------------------------------------|
    | 1 | Linux      | bash                      | any MESSAGE is
immediately printed, i.e. stderr is unbuffered
              |
    | 2 | Linux      | emacs eshell/shell etc    | >>

        |
    | 3 | Windows 10 | command prompt            | >>

        |
    | 4 | Windows 10 | mintty                    | MESSAGEs are only
displayed when emacs -batch is about to exit, i.e. stderr appears to
be buffered |
    | 5 | Windows 10 | emacs eshell              | >>

        |

    The issue here is that if someone invokes emacs -batch with #4 and
    #5, they won't be getting any MESSAGEs/feedback on their terminal until the
    program exits. This, I consider to be unacceptable if the
caller/user is expecting
    to get feedback on their terminal from a long run emacs -batch
script. And thus the bug report.

    I think I've just realized what you were saying from the
    beginning. That the difference in behavior is expected, since
    it is the parent process which decides the buffering regime to be
    used for the subprocess. Thus in #5, it is emacs on windows that
    decided to invoke emacs -batch as a subprocess using pipes, which
    has resulted in emacs -batch's stderr to be buffered.

    From my side, seeing that #4 and #5 behaving similarly, I had made the wild
    assumption that it is was windows that is enforcing the emacs
    -batch's stderr to be buffered, only based on the fact that the subprocess
    was not attached to the console alone. And thus the suggested
    patch to correct this as such.

    I will need to look into how exactly mintty and emacs invoke a
    subprocess and confirm indeed that stderr is buffered because they
are both using
    pipes (or similar methods) as you suggested,  and is not as I have
arbitrarily assumed it to be.

    If the current behavior is indeed the correct expected behavior, how do I
    flush text message to stderr (or even stdout) from an emacs
    -batch script/eval?

    Hope this clarifies things a bit.

On Tue, Feb 9, 2021 at 9:14 PM Eli Zaretskii <eliz@gnu.org> wrote:
>
> > Date: Tue, 09 Feb 2021 22:52:13 +0200
> > From: Eli Zaretskii <eliz@gnu.org>
> > Cc: 46388@debbugs.gnu.org
> >
> > > For #2, emacs batch stderr behavior when invoked from outside the
> > > command line is not compatible with the posix standard, since it uses
> > > "fully buffered" mode, which actually makes it behave very differently
> > > from Linux (which is posix compliant by always having stderr as being
> > > unbuffered). Thus, the suggested patch actually addresses this
> > > concern, rather than invalidating it.
> >
> > Again, are you sure it isn't the pipe that imposes buffering?
>
> Moreover, even if we did make the change you propose, it will only
> affect Emacs as a subprocess, but not when Emacs is the parent process
> and some other program (e.g., Python) is the subprocess.  IOW, the
> main problem, which is that interactive subprocesses behave on
> MS-Windows differently from their behavior when invoked from the
> console -- this problem will remain unsolved as long as Emacs can only
> use pipes for communications with subprocesses.