bug-gnu-emacs
[Top][All Lists]
Advanced

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

bug#31427: 27.0.50; libxml-parse-html-region stripped a big node


From: Noam Postavsky
Subject: bug#31427: 27.0.50; libxml-parse-html-region stripped a big node
Date: Sun, 13 May 2018 10:06:54 -0400
User-agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux)

tags 31427 + notabug
quit

stardiviner <numbchild@gmail.com> writes:

> As the background here: https://github.com/AdamNiederer/elquery/issues/5
>
> ,----
> | (search "\"content" ; changing to "\"menu" reveals the <div id="menu">
> |         (prin1-to-string (with-temp-buffer
> |                            (insert-string (with-current-buffer 
> (url-retrieve-synchronously "https://nginx.org/en/docs/dirindex.html";)
> |                                             (buffer-string)))
> |                            (let ((tree (libxml-parse-html-region 
> (point-min) (point-max))))
> |                              tree))))
> `----
>
> libxml-parse-html-region stipped the <div id="content"> node (which is big).

The problem is that the buffer given by `url-retrieve-synchronously' is
unibyte, then you insert the unibyte buffer-string into a multibyte
buffer (the temp buffer), which gives bad results.

The functions `search' and `insert-string' don't exist, so I
reformulated the code above:

    (let ((case-fold-search nil)
          (str (prin1-to-string
            (with-temp-buffer
              (insert (with-current-buffer (url-retrieve-synchronously 
"https://nginx.org/en/docs/dirindex.html";)
                (buffer-string)))
              (let ((tree (libxml-parse-html-region (point-min) (point-max))))
            tree)))))
      (and (string-match "\"content" str)
           (match-string 0 str)))
    ;=> nil

    (let ((case-fold-search nil)
          (str (prin1-to-string
            (with-temp-buffer
              (set-buffer-multibyte nil) ; <--------- set the temp buffer to 
unibyte
              (insert (with-current-buffer (url-retrieve-synchronously 
"https://nginx.org/en/docs/dirindex.html";)
                (buffer-string)))
              (let ((tree (libxml-parse-html-region (point-min) (point-max))))
            tree)))))
      (and (string-match "\"content" str)
           (match-string 0 str)))
    ;=> "\"content"

Instead of setting the temp buffer to multibyte, it would probably be
more correct to check the "Content-Type: text/html; charset=utf-8" HTTP
header, and decode it accordingly.  It looks like the libxml-parse-*
functions assume utf-8, so it works out either way in this particular
case.






reply via email to

[Prev in Thread] Current Thread [Next in Thread]