Re: Excessive redisplay from lots of process output

[Po Lu]

XFlush is not too expensive if not many requests have been generated.  What is 
expensive is _XReply, which waits for a reply from the X server on the other 
side of the connection.

Anyway, please give Emacs 29 a try.  I tried to reduce the number of locations 
where Emacs waits for replies, which makes it much more usable over a network 
connection.

Also, try using a no toolkit build.  GTK has a bug where changing the scroll 
bar waits for replies from the X server.

If the problem is still there in Emacs 29, please run Emacs under xscope, and 
tell which request(s) are being made.  Emacs should not make X requests when 
nothing on the display has actually changed.


On February 25, 2023 3:14:55 AM GMT+08:00, Spencer Baugh 
<sbaugh@janestreet.com> wrote:
>On Fri, Feb 17, 2023 at 11:06 AM Eli Zaretskii <eliz@gnu.org> wrote:
>>
>> > From: Spencer Baugh <sbaugh@janestreet.com>
>> > Cc: azeng@janestreet.com
>> > Date: Fri, 17 Feb 2023 10:16:11 -0500
>> >
>> > Emacs calls redisplay every time it reads process output.  This causes
>> > Emacs to become painfully slow when these two conditions are met:
>> >
>> > - When redisplay is expensive (e.g. when using ssh X forwarding)
>> >
>> > - And when there is frequent process output, say hundreds or thousands
>> >   of times a second (e.g. ERC receiving network traffic, shells
>> >   running commands which log heavily, other things which run in the
>> >   background)
>>
>> This cannot be the whole story, because usually redisplay exits almost
>> immediately after being called, having determined that nothing needs
>> to be changed on display.
>>
>> So the important part of your situation that you didn't disclose is:
>> why does Emacs in your case perform the expensive redisplay each time
>> redisplay is entered?  Can you figure that out by stepping through
>> redisplay_internal?  (Let me know if you need guidance.)
>
>Actually, I'm looking a bit more into it, and I think there is still
>some issue causing excessive X traffic when there's lots of process
>output to be read.  On emacs -Q with Emacs 28, flush_frame/XFlush is
>called on every redisplay, and does seem to be actually writing X
>requests on the X socket every time.
>
>Here's a backtrace of where flush_frame/XFlush is being called from:
>
>#0  0x00007ffff4abc5c0 in XFlush () at /lib64/libX11.so.6
>#1  0x00000000004d23d6 in x_flush (f=<optimized out>) at xterm.c:250
>#2  0x00000000004d2416 in x_flip_and_flush (f=0xf956d0) at xterm.c:246
>#3  0x00000000004d2416 in x_flip_and_flush (f=0xf956d0) at xterm.c:1230
>#4  0x0000000000461374 in redisplay_preserve_echo_area (f=<optimized
>out>) at frame.h:1700
>#5  0x0000000000461374 in redisplay_preserve_echo_area
>(from_where=from_where@entry=12) at xdisp.c:16527
>#6  0x00000000005bee33 in wait_reading_process_output
>(time_limit=time_limit@entry=0, nsecs=nsecs@entry=0,
>read_kbd=read_kbd@entry=-1, do_display=true,
>wait_for_cell=wait_for_cell@entry=0x0, wait_proc=wait_proc@entry=0x0,
>just_wait_proc=just_wait_proc@entry=0) at process.c:5808
>#7  0x000000000050aea4 in read_char (end_time=0x0,
>used_mouse_menu=0x7fffffffda5b, kbp=<synthetic pointer>) at
>keyboard.c:3926
>#8  0x000000000050aea4 in read_char (used_mouse_menu=<optimized out>,
>local_getcjmp=<optimized out>, end_time=<optimized out>) at
>keyboard.c:2198
>#9  0x000000000050aea4 in read_char (used_mouse_menu=<optimized out>,
>prev_event=<optimized out>, local_getcjmp=<optimized out>,
>end_time=<optimized out>) at keyboard.c:2261
>#10 0x000000000050aea4 in read_char (commandflag=commandflag@entry=1,
>map=map@entry=0x127fed3, prev_event=<optimized out>,
>used_mouse_menu=used_mouse_menu@entry=0x7fffffffda5b,
>end_time=end_time@entry=0x0) at keyboard.c:2892
>#11 0x000000000050cc5e in read_key_sequence
>(keybuf=keybuf@entry=0x7fffffffdb60, prompt=prompt@entry=0x0,
>dont_downcase_last=dont_downcase_last@entry=false,
>can_return_switch_frame=can_return_switch_frame@entry=true,
>fix_current_buffer=fix_current_buffer@entry=true,
>prevent_redisplay=prevent_redisplay@entry=false) at keyboard.c:9635
>#12 0x000000000050e390 in command_loop_1 () at keyboard.c:1392
>#13 0x0000000000578754 in internal_condition_case
>(bfun=bfun@entry=0x50e180 <command_loop_1>,
>handlers=handlers@entry=0x90, hfun=hfun@entry=0x504990 <cmd_error>) at
>eval.c:1450
>#14 0x00000000004ff4aa in command_loop_2
>(handlers=handlers@entry=0x90) at keyboard.c:1133
>#15 0x0000000000578704 in internal_catch (tag=tag@entry=0xe850,
>func=func@entry=0x4ff490 <command_loop_2>, arg=arg@entry=0x90) at
>eval.c:1181
>#16 0x00000000004ff46a in command_loop () at keyboard.c:1111
>#17 0x00000000005045cb in recursive_edit_1 () at keyboard.c:720
>#18 0x00000000005048e5 in Frecursive_edit () at keyboard.c:803
>#19 0x000000000041e909 in main (argc=4, argv=<optimized out>) at emacs.c:2361
>
>What could be causing this?
>