--- Begin Message ---
|
Subject: |
26.1; Can `count-lines' be rewritten to use the newline cache? |
|
Date: |
Fri, 17 Aug 2018 13:01:38 +1200 |
|
User-agent: |
Orcon Webmail |
I saw this story the other day:
https://fuco1.github.io/2018-08-12-WAR-STORY:-When-turning-to-the-profiler-turns-out-to-be-a-good-call.html
The summary is that some very slow code turned out to be spending the
vast bulk of its time inside `line-number-at-pos' (which was used
frequently), and once the author discovered what that function
actually entailed they were able to reduce their processing time from
42 seconds down to 5 seconds (processing a file of ~10,000 lines) by
finding an alternative approach which did not involve calling
`count-lines'.
`count-lines' uses a regexp search to find all the newlines (and/or
carriage returns -- I don't know if that's a problem) and I recall
that internally Emacs uses a newline cache to make certain
line-oriented functionality performant. I know nothing about the
cache other than that it exists, but I wondered whether `count-lines'
might be able to use it to avoid most of the work that it currently
does?
-Phil
In GNU Emacs 26.1 (build 1, x86_64-pc-linux-gnu, X toolkit, Xaw scroll
bars)
of 2018-04-10 built on shodan
Windowing system distributor 'The X.Org Foundation', version
11.0.11501000
System Description: Ubuntu 14.04.5 LTS
Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.
Configured using:
'configure --prefix=/home/phil/emacs/26/26.1rc1/usr/local
--with-x-toolkit=lucid --without-sound'
Configured features:
XPM JPEG TIFF GIF PNG RSVG IMAGEMAGICK GPM DBUS GSETTINGS NOTIFY
LIBSELINUX GNUTLS LIBXML2 FREETYPE XFT ZLIB TOOLKIT_SCROLL_BARS LUCID
X11 THREADS LCMS2
Important settings:
value of $LANG: en_NZ.UTF-8
value of $XMODIFIERS: @im=ibus
locale-coding-system: utf-8-unix
Major mode: Dired by name
Minor modes in effect:
tooltip-mode: t
global-eldoc-mode: t
electric-indent-mode: t
mouse-wheel-mode: t
tool-bar-mode: t
menu-bar-mode: t
file-name-shadow-mode: t
global-font-lock-mode: t
font-lock-mode: t
blink-cursor-mode: t
auto-composition-mode: t
auto-encryption-mode: t
auto-compression-mode: t
buffer-read-only: t
line-number-mode: t
transient-mark-mode: t
Load-path shadows:
None found.
Features:
(shadow sort mail-extr emacsbug message rmc puny seq byte-opt gv
bytecomp byte-compile cconv cl-loaddefs cl-lib format-spec rfc822 mml
easymenu mml-sec password-cache epa derived epg epg-config gnus-util
rmail rmail-loaddefs mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums
mm-util mail-prsvr mail-utils dired dired-loaddefs advice elec-pair
time-date mule-util tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel term/x-win x-win term/common-win x-dnd tool-bar
dnd fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode elisp-mode lisp-mode prog-mode register page menu-bar
rfn-eshadow isearch timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core term/tty-colors frame cl-generic cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese eucjp-ms cp51932 hebrew greek romanian slovak czech european
ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray minibuffer
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote dbusbind inotify lcms2
dynamic-setting system-font-setting font-render-setting x-toolkit x
multi-tty make-network-process emacs)
Memory information:
((conses 16 99206 10028)
(symbols 48 20474 0)
(miscs 40 101 169)
(strings 32 29605 992)
(string-bytes 1 777027)
(vectors 16 14293)
(vector-slots 8 495220 11440)
(floats 8 55 100)
(intervals 56 315 0)
(buffers 992 14)
(heap 1024 30317 1420))
--- End Message ---
--- Begin Message ---
|
Subject: |
Re: bug#32462: 26.1; Can `count-lines' be rewritten to use the newline cache? |
|
Date: |
Sun, 19 Aug 2018 00:10:27 +1200 |
|
User-agent: |
Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 |
On 18/08/18 02:40, Eli Zaretskii wrote:
>> From: Phil Sainty <address@hidden>
>> `count-lines' uses a regexp search to find all the newlines (and/or
>> carriage returns
>
> It uses regexp search only when selective-display is in effect, which
> means almost never. Otherwise, it uses forward-line, which uses
> scan_newline_from_point, which already uses the newline cache (unless
> the cache is disabled).
Ah, thanks Eli; I see that now. A look at the C code suggests that the
newline cache is also a more complicated arrangement than I'd imagined,
so I don't think my original thoughts about this were actually viable.
I'm closing this bug.
--- End Message ---