bug#46881: 28.0.50; pdumper dumping causes way too many syscalls

[Pip Cet]

On Fri, Mar 5, 2021 at 7:55 AM Eli Zaretskii <eliz@gnu.org> wrote:
> > From: Pip Cet <pipcet@gmail.com>
> > Date: Fri, 5 Mar 2021 07:38:27 +0000
> > Cc: Daniel Colascione <dancol@dancol.org>, eggert@cs.ucla.edu, 
> > 46881@debbugs.gnu.org
> >
> > > I'm not sure I understand: what's wrong with fseek?
> >
> > Nothing, assuming you're fine with the current performance. Many libcs
> > aren't going to be smart enough to avoid I/O when you fseek through a
> > "large" file and write a word here and there, and my suspicion is that
> > would include glibc.
>
> Could we benchmark the two implementations instead of acting on
> suspicions?

Sure.

My patch:

real    0m1.988s
user    0m1.916s
sys    0m0.073s

fwrite-based patch:

real    0m3.576s
user    0m2.571s
sys    0m1.006s

This is as I expected: glibc just isn't doing a very good job for this
buffered stream.

> In general, I'd prefer not to reinvent the wheel, and trust modern
> libc's that they are efficient enough in handling buffered streams,
> unless we have hard evidence to the contrary.

We do, now.

> If nothing else, it
> would prevent people asking, like Daniel did, why didn't we use stdio
> in the first place.

I think it's a very good question (in fact, the brach I'm working on
is called pdumper-fwrite because I decided only after creating it that
all the seeking would hurt performance too much). I'll try including a
comment explaining why.

> > Also, we're not currently using fseek-and-write anywhere in Emacs.
>
> I don't see why this would be important.

Because the stream returned by emacs_fopen might not be generally seekable?

> Since we open the file in
> binary mode, fseek should work correctly even on non-Posix systems.

I guess I should have used emacs_fopen :-)

> > We're talking about a file which Emacs is going to have to keep in
> > memory anyway, when reading the dump. The only case in which there
> > might be a problem is if the build machine has significantly less
> > available memory than the machine we intend to run on, and I just
> > don't think that's going to happen.
>
> You are thinking about memory consumption, while I am thinking how to
> avoid implementing our own private buffered streams.

By preparing the data in memory and writing it in one go, which
doesn't require any of the major complications of implementing
buffered streams.

Pip