[Top][All Lists]

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

bug#32228: sed -i does not buffer, causing excessive writes

From: Assaf Gordon
Subject: bug#32228: sed -i does not buffer, causing excessive writes
Date: Fri, 20 Jul 2018 19:48:28 -0600
User-agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1


On 20/07/18 02:23 PM, Vidar Holen wrote:
I'm using noticing that `sed -i` does not do any kind of output
buffering. While behavior is correct, this causes an excessive number of
small writes:

I'm seeing this on Debian with the latest sed commit (c52a676) and libc
2.27-2. The root cause appears to be `ck_mkstemp` using `fdopen`, which
unlike `fopen` does not buffer by default.

Thank you for reporting this issue, and providing clear way to reproduce
it. I can confirm seeing the same behavior on Debian 9.4 with glibc 2.24 (and latest sed).

I think the technical culprit is slightly different:
It is not due to 'ck_mkstemp/fdopen', but because sed explicitly flushes
the output after every line, except if the output is STDOUT.

This happens in execute.c:flush_output():

Changing this function enables default buffering with "sed -i"
(using whatever the buffering default is for glibc).

This change does have a side-effect:
It also changes the buffering of other write commands such as 'w', 'W'
and 's///w'.
It might have other side effects I'm haven't spotted.
Based on the ChangeLog-2014 file, I see this function was added in 2003.

I wonder if there are good reasons to explicitly flush all output -
ideas anyone?

If you can give this a quick test and let me know if you also notice
improvements on your system - would be appreciated.

Comments and suggestions welcomed,
 - assaf

Attachment: 0001-sed-do-not-flush-output-stream-unless-in-unbuffered-.patch
Description: Text Data

reply via email to

[Prev in Thread] Current Thread [Next in Thread]