[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
bug#31427: 27.0.50; libxml-parse-html-region stripped a big node
From: |
Noam Postavsky |
Subject: |
bug#31427: 27.0.50; libxml-parse-html-region stripped a big node |
Date: |
Sun, 13 May 2018 10:06:54 -0400 |
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) |
tags 31427 + notabug
quit
stardiviner <numbchild@gmail.com> writes:
> As the background here: https://github.com/AdamNiederer/elquery/issues/5
>
> ,----
> | (search "\"content" ; changing to "\"menu" reveals the <div id="menu">
> | (prin1-to-string (with-temp-buffer
> | (insert-string (with-current-buffer
> (url-retrieve-synchronously "https://nginx.org/en/docs/dirindex.html")
> | (buffer-string)))
> | (let ((tree (libxml-parse-html-region
> (point-min) (point-max))))
> | tree))))
> `----
>
> libxml-parse-html-region stipped the <div id="content"> node (which is big).
The problem is that the buffer given by `url-retrieve-synchronously' is
unibyte, then you insert the unibyte buffer-string into a multibyte
buffer (the temp buffer), which gives bad results.
The functions `search' and `insert-string' don't exist, so I
reformulated the code above:
(let ((case-fold-search nil)
(str (prin1-to-string
(with-temp-buffer
(insert (with-current-buffer (url-retrieve-synchronously
"https://nginx.org/en/docs/dirindex.html")
(buffer-string)))
(let ((tree (libxml-parse-html-region (point-min) (point-max))))
tree)))))
(and (string-match "\"content" str)
(match-string 0 str)))
;=> nil
(let ((case-fold-search nil)
(str (prin1-to-string
(with-temp-buffer
(set-buffer-multibyte nil) ; <--------- set the temp buffer to
unibyte
(insert (with-current-buffer (url-retrieve-synchronously
"https://nginx.org/en/docs/dirindex.html")
(buffer-string)))
(let ((tree (libxml-parse-html-region (point-min) (point-max))))
tree)))))
(and (string-match "\"content" str)
(match-string 0 str)))
;=> "\"content"
Instead of setting the temp buffer to multibyte, it would probably be
more correct to check the "Content-Type: text/html; charset=utf-8" HTTP
header, and decode it accordingly. It looks like the libxml-parse-*
functions assume utf-8, so it works out either way in this particular
case.