Re: [Wget-dev] wget2 | Unimplemented option '--ignore-length' (#439)

[Archit Pandey]

Archit Pandey commented:


> First of all, when `Content-Length:` is missing there is no 
> `Transfer-Encoding: chunked`, we only can say
> that the body has been transferred when the connection remotely closes. At 
> least that's the best guess
> (the connection could have been closed prematurely).
I could find code in `libwget/http.c` that does this although was not able to 
test it since when there is no user specified `Content-Length` in a test, 
_libmicrohttpd_ seems to adds its own. I couldn't find an option to disable 
this, but will confirm with their mailing list.

Currently, _libwget_ does not read the body when it sees a `Content-Length: 0` 
in the header. That was leading to the failed test case. However, I believe 
this is being done intentionally according to [RFC 2616 
4.4](https://tools.ietf.org/html/rfc2616#section-4.4). 

I made a test case with 2 files and it passed. From the test logs I can see 
that these files were downloaded using two GET requests. I think I am missing 
something. 
```
        wget_test(
                // WGET_TEST_KEEP_TMPFILES, 1,
                WGET_TEST_OPTIONS, "",
                WGET_TEST_REQUEST_URLS, "index.html", "second.html", NULL,
                WGET_TEST_EXPECTED_ERROR_CODE, 0,
                WGET_TEST_EXPECTED_FILES, &(wget_test_file_t []) {
                        { urls[0].name + 1, urls[0].body },
                        { urls[1].name + 1, urls[1].body },
                        //{ urls[2].name + 1, urls[2].body },
                        {       NULL } },
                0);
```
I tried setting different values of `Content-Length` but still no error. Could 
it be that _libwget_ already accounts for incorrect `Content-Length` values. 

Found this code in `libwget/http.c`:
```c
         if (nbytes < 0)
             error_printf(_("Failed to read %zd bytes (%d)\n"), nbytes, errno);
         if (body_len < resp->content_length)
             error_printf(_("Just got %zu of %zu bytes\n"), body_len, 
resp->content_length);
         else if (body_len > resp->content_length)
             error_printf(_("Body too large: %zu instead of %zu bytes\n"), 
body_len, resp->content_length);
         resp->content_length = body_len;
```

-- 
Reply to this email directly or view it on GitLab: 
https://gitlab.com/gnuwget/wget2/issues/439#note_235082539
You're receiving this email because of your account on gitlab.com.