--- Begin Message ---
|
Subject: |
Unhandled case in module/web/response.scm |
|
Date: |
Sat, 02 Mar 2013 00:21:55 -0700 |
|
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/24.3.50 (gnu/linux) |
response.scm does not seem to handle the case where the server does not
specify a content length. Here's a minimal example that should work,
but doesn't:
--8<---------------cut here---------------start------------->8---
#!/usr/local/bin/guile -s
!#
(use-modules (srfi srfi-8)
((web uri) #:select (string->uri))
((web client) #:select (http-get)))
(receive (res-headers res-body)
(http-get (string->uri
"http://www.blogger.com/feeds/4777343509834060826/posts/default";))
(display res-body)
(newline))
--8<---------------cut here---------------end--------------->8---
Now the reason that I started experimenting with guile in the first
place was that I wanted to learn more about scheme, and fixing this
seemed like a good opportunity at a practical application of my basic
scheme skills.
So I did a little debugging and created this little patch that fixes
this issue.
>From 74af03051fec22e01376f3f4597eb8de2dab87b5 Mon Sep 17 00:00:00 2001
From: Jason Douglas Earl <address@hidden>
Date: Fri, 1 Mar 2013 23:58:41 -0700
Subject: [PATCH] Handle case where server does not specify a Content-Length.
* module/web/response.scm (response-body-port,
make-undelimited-input-port): The server does not have to return a
content length. Handle that case.
---
module/web/response.scm | 28 +++++++++++++++++++++++++---
1 file changed, 25 insertions(+), 3 deletions(-)
diff --git a/module/web/response.scm b/module/web/response.scm
index 7e14f4d..33dcf1e 100644
--- a/module/web/response.scm
+++ b/module/web/response.scm
@@ -264,6 +264,26 @@ closes PORT, unless KEEP-ALIVE? is true."
(make-custom-binary-input-port "delimited input port" read! #f #f close))
+(define (make-undelimited-input-port port keep-alive?)
+ "Return an input port that reads from PORT, until EOF. Closing the
+returned port closes PORT, unless KEEP-ALIVE? is true."
+ (define bytes-read 0)
+
+ (define (read! bv start count)
+ (let ((ret (get-bytevector-n! port bv start count)))
+ (if (eof-object? ret)
+ 0
+ (begin
+ (set! bytes-read (+ bytes-read ret))
+ ret))))
+
+ (define close
+ (and (not keep-alive?)
+ (lambda ()
+ (close port))))
+
+ (make-custom-binary-input-port "undelimited input port" read! #f #f close))
+
(define* (response-body-port r #:key (decode? #t) (keep-alive? #t))
"Return an input port from which the body of R can be read. The
encoding of the returned port is set according to R's ‘content-type’
@@ -277,9 +297,11 @@ response port."
(make-chunked-input-port (response-port r)
#:keep-alive? keep-alive?)
(let ((len (response-content-length r)))
- (and len
- (make-delimited-input-port (response-port r)
- len keep-alive?)))))
+ (if len
+ (make-delimited-input-port (response-port r)
+ len keep-alive?)
+ (make-undelimited-input-port (response-port r)
+ keep-alive?)))))
(when (and decode? port)
(match (response-content-type r)
--
1.7.10.4
With that patch my little test program works.
Now, please forgive my ignorance. I probably misunderstand what
"delimited" means in this context, and I am probably using it
incorrectly. I would be shocked if this even slightly resembles how
this should be fixed. I have been lurking on the guile mailing lists
for a few months and I don't understand half of what you guys are
talking about (which is actually why I am so keen to play with guile).
Sharing this patch seemed to be the easiest way to explain what is
happening.
I am not even going to pretend that I have spent a great deal of time
reading the HTTP 1.1 protocol specs, but it does appear that the server
does not have to return a Content-Length. I certainly have run across
servers that don't.
Poking at this issue has been quite a bit of fun for me. So, thanks for
all of your hard work on the system. Now I must admit that I am
interested in seeing how (and if) this gets fixed.
Jason
--- End Message ---
--- Begin Message ---
|
Subject: |
Re: bug#13857: Unhandled case in module/web/response.scm |
|
Date: |
Sun, 10 Mar 2013 19:24:13 +0100 |
|
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/24.2 (gnu/linux) |
On Sat 09 Mar 2013 23:59, Jason Earl <address@hidden> writes:
> That did not work for me. [I]s it possible that you mean something
> like this?
Indeed -- good catch! Thank you for the report, patch, and testing :-)
commit 84dfde82ae8f6ec247c1c147c1e2ae50b207bad9
Author: Jason Earl <address@hidden>
Date: Sun Mar 10 19:23:31 2013 +0100
fix response-body-port for responses without content-length
* module/web/response.scm (response-body-port): Correctly handle cases
in which EOF terminates the body.
Cheers,
Andy
--
http://wingolog.org/
--- End Message ---