[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Bug-wget] New option "--no-list-a"
From: |
Giuseppe Scrivano |
Subject: |
Re: [Bug-wget] New option "--no-list-a" |
Date: |
Sun, 25 Aug 2013 22:49:19 +0200 |
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/24.3 (gnu/linux) |
Hi Andrea,
would you mind to rework the patch against the current git repository?
Some of the fixes (the documentation bits) are already there.
Also, before I could accept a non-trivial change, you should complete
the copyright assignment to the FSF process. I will send you a separate
email with more details.
Thanks,
Giuseppe
"Andrea Urbani" <address@hidden> writes:
> Good afternoon,
> some systems (at least info-zip.org) have problems with the ftp command "LIST
> -a".
> I.e. trying
>
> wget --timestamping --no-directories --recursive --level=1 --accept
> "zip*.t*,unzip*.t*" --reject "*.zip,unzip55?.*,zip23?.*" --tries=5
> --waitretry=10 --timeout=15 --debug
> ftp://ftp.info-zip.org/pub/infozip/src/
>
> you get
>
> Created socket 4.
> done. ==> LIST ...
> --> LIST -a
>
> 150 List started.
> done.
>
> [ <=> ] 0 --.-K/s in 0s
>
> Closed fd 4
> 226 Transfer completed.
> 2013-08-25 06:40:42 (0.00 B/s) - â.listingâ saved [0]
>
> that means no errors but also an empty list of files.
> Within the attached patch I have added a new parameter, --no-list-a, that
> tells WGET just to use the "LIST" command.
> If you try the new parameter
>
> wget --timestamping --no-directories --recursive --level=1 --accept
> "zip*.t*,unzip*.t*" --reject "*.zip,unzip55?.*,zip23?.*" --tries=5
> --waitretry=10 --timeout=15 --debug --no-list-a
> ftp://ftp.info-zip.org/pub/infozip/src/
>
> you will get
>
> Created socket 4.
> done. ==> LIST ... (Skipping "LIST -a")
> --> LIST
>
> 150 List started.
> done.
>
> [ <=> ] 1,855 --.-K/s in 0.06s
>
> Closed fd 4
> 226 Transfer completed.
> 2013-08-25 06:45:08 (30.5 KB/s) - â.listingâ saved [1855]
>
> PLAINFILE; perms 645; size: 1063171; month: Oct; day: 23; year: 1992 (no tm);
> calgarycorpus.zip
> PLAINFILE; perms 645; size: 24979; month: Feb; day: 9; year: 2004 (no tm);
> check-4_3-src.zip
> ...
> ...
>
>
> Please, make attention that inside the wget.texi I have changed some "itemx"
> into "item" as reported me by texinfo ver. 5.1
>
> Bye
> Andrea
> matfanjol
> http://matfanjol.users.sourceforge.net/
>
> Index: src/ftp-basic.c
> ===================================================================
> --- src/ftp-basic.c.orig 2012-05-12 17:18:27.000000000 +0200
> +++ src/ftp-basic.c 2013-08-24 16:40:29.000000000 +0200
> @@ -975,8 +975,14 @@
> fail, but will never do what is desired here, skip directly to the
> simple "LIST" command (assumed to be the last one in the list).
> */
> - if (rs == ST_VMS)
> - i = countof (list_commands)- 1;
> + /* 2013-08-24 matfanjol: Other systems seems to have problem with
> + "LIST -a", so I don't use it if requested
> + */
> + if ( (rs == ST_VMS) || (opt.nolista) )
> + {
> + DEBUGP ((" (Skipping \"LIST -a\") "));
> + i = countof (list_commands)- 1;
> + }
>
> do {
> /* Send request. */
> Index: tests/FTPServer.pm
> ===================================================================
> --- tests/FTPServer.pm.orig 2013-08-25 12:10:59.024413164 +0200
> +++ tests/FTPServer.pm 2013-08-25 12:13:31.000000000 +0200
> @@ -78,8 +78,11 @@
>
> sub _LIST_command
> {
> - my ($conn, $cmd, $path) = @_;
> + my ($conn, $cmd, $path, $empty_list_if_list_a) = @_;
> my $paths = $conn->{'paths'};
> + my $ReturnEmptyList = (defined ($empty_list_if_list_a)
> + && $path eq '-a');
> +
>
> # This is something of a hack. Some clients expect a Unix server
> # to respond to flags on the 'ls command line'. Remove these flags
> @@ -94,11 +97,15 @@
> # working directory.
> local $_;
>
> - $dir = FTPPaths::path_merge($dir, $path);
> - my $listing = $paths->get_list($dir);
> - unless ($listing) {
> - print {$conn->{socket}} "550 File or directory not found.\r\n";
> - return;
> + my $listing;
> + if ( ! $ReturnEmptyList )
> + {
> + $dir = FTPPaths::path_merge($dir, $path);
> + $listing = $paths->get_list($dir);
> + unless ($listing) {
> + print {$conn->{socket}} "550 File or directory not found.\r\n";
> + return;
> + }
> }
>
> print STDERR "_LIST_command - dir is: $dir\n" if $log;
> @@ -112,8 +119,11 @@
> return;
> }
>
> - for my $item (@$listing) {
> - print $sock "$item\r\n";
> + if ( ! $ReturnEmptyList )
> + {
> + for my $item (@$listing) {
> + print $sock "$item\r\n";
> + }
> }
>
> unless ($sock->close) {
> @@ -634,7 +644,17 @@
> }
>
> # Run the command.
> - &{$command_table->{$cmd}} ($conn, $cmd, $rest);
> + # 2013-08-25 matfanjol
> + # Only for the LIST command I pass also the
> empty_list_if_list_a behavior
> + if ( $cmd eq 'LIST' )
> + {
> + &{$command_table->{$cmd}} ($conn, $cmd, $rest,
> +
> $self->{_server_behavior}{empty_list_if_list_a});
> + }
> + else
> + {
> + &{$command_table->{$cmd}} ($conn, $cmd, $rest);
> + }
> }
> } else { # Father
> close $socket;
> Index: src/init.c
> ===================================================================
> --- src/init.c.orig 2012-07-08 11:35:36.000000000 +0200
> +++ src/init.c 2013-08-23 14:33:24.000000000 +0200
> @@ -213,6 +213,7 @@
> { "mirror", NULL, cmd_spec_mirror },
> { "netrc", &opt.netrc, cmd_boolean },
> { "noclobber", &opt.noclobber, cmd_boolean },
> + { "nolista", &opt.nolista, cmd_boolean },
> { "noparent", &opt.no_parent, cmd_boolean },
> { "noproxy", &opt.no_proxy, cmd_vector },
> { "numtries", &opt.ntry, cmd_number_inf },/*
> deprecated*/
> Index: src/main.c
> ===================================================================
> --- src/main.c.orig 2012-07-07 10:26:21.000000000 +0200
> +++ src/main.c 2013-08-23 14:23:30.000000000 +0200
> @@ -234,6 +234,7 @@
> { "mirror", 'm', OPT_BOOLEAN, "mirror", -1 },
> { "no", 'n', OPT__NO, NULL, required_argument },
> { "no-clobber", 0, OPT_BOOLEAN, "noclobber", -1 },
> + { "no-list-a", 0, OPT_BOOLEAN, "nolista", -1 },
> { "no-parent", 0, OPT_BOOLEAN, "noparent", -1 },
> { "output-document", 'O', OPT_VALUE, "outputdocument", -1 },
> { "output-file", 'o', OPT_VALUE, "logfile", -1 },
> @@ -662,6 +663,8 @@
> N_("\
> --no-glob turn off FTP file name globbing.\n"),
> N_("\
> + --no-list-a disable \"LIST -a\" command and use just
> LIST.\n"),
> + N_("\
> --no-passive-ftp disable the \"passive\" transfer mode.\n"),
> N_("\
> --preserve-permissions preserve remote file permissions.\n"),
> Index: src/options.h
> ===================================================================
> --- src/options.h.orig 2012-06-06 13:42:10.000000000 +0200
> +++ src/options.h 2013-08-23 14:27:29.000000000 +0200
> @@ -280,6 +280,8 @@
> bool show_all_dns_entries; /* Show all the DNS entries when resolving a
> name. */
> bool report_bps; /*Output bandwidth in bits format*/
> + bool nolista; /* Avoid the "LIST -a" command in FTP.
> + "LIST" will be used. */
> };
>
> extern struct options opt;
> Index: tests/run-px
> ===================================================================
> --- tests/run-px.orig 2013-08-25 12:08:09.719417558 +0200
> +++ tests/run-px 2013-08-25 12:08:40.000000000 +0200
> @@ -35,6 +35,7 @@
> 'Test-ftp-iri-fallback.px',
> 'Test-ftp-iri-recursive.px',
> 'Test-ftp-iri-disabled.px',
> + 'Test-ftp-no-list-a.px',
> 'Test-HTTP-Content-Disposition-1.px',
> 'Test-HTTP-Content-Disposition-2.px',
> 'Test-HTTP-Content-Disposition.px',
> Index: tests/Test-ftp-no-list-a.px
> ===================================================================
> --- /dev/null 2013-08-25 06:34:23.227937278 +0200
> +++ tests/Test-ftp-no-list-a.px 2013-08-25 12:14:05.000000000 +0200
> @@ -0,0 +1,54 @@
> +#!/usr/bin/env perl
> +
> +use strict;
> +use warnings;
> +
> +use FTPTest;
> +
> +
> +###############################################################################
> +
> +my $afile = <<EOF;
> +Some text.
> +EOF
> +
> +my $bfile = <<EOF;
> +Some more text.
> +EOF
> +
> +$afile =~ s/\n/\r\n/g;
> +$bfile =~ s/\n/\r\n/g;
> +
> +# code, msg, headers, content
> +my %urls = (
> + '/afile.txt' => {
> + content => $afile,
> + },
> + '/bfile.txt' => {
> + content => $bfile,
> + },
> +);
> +
> +my $cmdline = $WgetTest::WGETPATH . " --no-directories --recursive --level=1
> --accept \"?file.txt\" --no-list-a ftp://localhost:{{port}}/";
> +
> +my $expected_error_code = 0;
> +
> +my %expected_downloaded_files = (
> + 'afile.txt' => {
> + content => $afile,
> + },
> + 'bfile.txt' => {
> + content => $bfile,
> + },
> +);
> +
> +###############################################################################
> +
> +my $the_test = FTPTest->new (name => "Test-ftp-bad-list",
> + input => \%urls,
> + cmdline => $cmdline,
> + errcode => $expected_error_code,
> + output => \%expected_downloaded_files,
> + server_behavior => {empty_list_if_list_a => 1});
> +exit $the_test->run();
> +
> Index: doc/wget.texi
> ===================================================================
> --- doc/wget.texi.orig 2012-08-04 10:41:52.000000000 +0200
> +++ doc/wget.texi 2013-08-24 17:10:57.000000000 +0200
> @@ -876,7 +876,7 @@
> actions of one.
>
> @cindex proxy
> address@hidden --no-proxy
> address@hidden --no-proxy
> Don't use proxies, even if the appropriate @code{*_proxy} environment
> variable is defined.
>
> @@ -977,9 +977,9 @@
> whose encoding does not match the one used locally.
>
> @cindex IPv6
> address@hidden -4
> address@hidden -4
> @itemx --inet4-only
> address@hidden -6
> address@hidden -6
> @itemx --inet6-only
> Force connecting to IPv4 or IPv6 addresses. With @samp{--inet4-only}
> or @samp{-4}, Wget will only connect to IPv4 hosts, ignoring AAAA
> @@ -1764,6 +1764,16 @@
> system-specific. This is why it currently works only with Unix @sc{ftp}
> servers (and the ones emulating Unix @code{ls} output).
>
> address@hidden list ftp
> address@hidden --no-list-a
> +When a directory list is required within a @sc{ftp} connection WGet sends
> +the @code{LIST -a} command. If this command fails, WGet tries again with
> +the @code{LIST} command.
> +Some systems don't work well with @code{LIST -a} command (i.e. the command
> +doesn't fail, but the list is empty). In those cases it is possible to use
> +the @samp{--no-list-a} option that tells WGet to use just the @code{LIST}
> +command.
> +
> @cindex passive ftp
> @item --no-passive-ftp
> Disable the use of the @dfn{passive} FTP transfer mode. Passive FTP
> @@ -3094,7 +3104,7 @@
> Change setting of passive @sc{ftp}, equivalent to the
> @samp{--passive-ftp} option.
>
> address@hidden password = @var{string}
> address@hidden password = @var{string}
> Specify password @var{string} for both @sc{ftp} and @sc{http} file
> retrieval.
> This command can be overridden using the @samp{ftp_password} and
> @samp{http_password} command for @sc{ftp} and @sc{http} respectively.