Re: [Wp-mirror-list] [Xmldatadumps-l] ftpmirror.your.org - IPv4 and IPv6 give different results (fwd)

[wp mirror]

Dear Guy,

Excellent!  I too have 69GB under /var/lib/mediawiki/images.

Sincerely Yours,
Kent

On 11/18/13, Guy Castagnoli <address@hidden> wrote:
> Dear Kent,
>
> 2) Update wp-mirror
>
> Have downloaded the most recent version of wp-mirror, then ran wp-mirror
> --restore-default and wp-mirror --mirror is now running successfully
> 'out-of-the-box'..
>
> As a matter of feedback, I noticed one thing, and that is, that after the
> restore to default, the /var directory remained far from empty: df -h /var
> still gave 72GB, after showing  110+GB before the restore to default, and it
> turned out that 69GB of the 72GB went to the credit of /var/lib/mediawiki .
> In the mean time, after about 1 hour of running, i.e., about 5GB into
> commonswiki-20131106-image.sql.gz)  du -hs /var/lib/mediawiki gives 74GB..
> I'm assuming that the 69GB are from images that, according to the manual,
> --restore-default does not delete..
>
> Regards,
>
> Guy
>
> -----Original Message-----
> From: wp mirror [mailto:address@hidden
> Sent: 18 November 2013 08:38
> To: Guy Castagnoli
> Cc: address@hidden; wpmirrordev
> Subject: Re: [Xmldatadumps-l] ftpmirror.your.org - IPv4 and IPv6 give
> different results (fwd)
>
> Dear Guy,
>
> Thanks for your input on curl vs. wget.
>
> 1) Redirect
>
> I have decided to stay with curl for now.  As per your suggestion,
> wp-mirror now invokes curl with the `--location' option in order to
> follow the redirect.  The updated wp-mirror runs correctly on my
> platform.
>
> 2) Update wp-mirror
>
> Please download and install wp-mirror to get the update.
>
> (shell)$ wget
> http://download.savannah.gnu.org/releases/wp-mirror/wp-mirror_0.6-1_all.deb
> (shell)$ sha1sum wp-mirror_0.6-1_all.deb
> 3c63016c90e6f57c95cc5d20a3001d57f7617728  wp-mirror_0.6-1_all.deb
> (root-shell)# dpkg --install wp-mirror_0.6-1_all.deb
> (root-shell)# wp-mirror --mirror
>
> Note that mirrors of <http://download.savannah.gnu.org> can take a few
> hours to sync.  So you may wish to verify the downloaded package with
> `sha1sum'.
>
> Sincerely Yours,
> Kent
>
> On 11/17/13, Dr. Kent L. Miller <address@hidden> wrote:
>> Dear Kent,
>>
>>
>>
>> 1) *wikimedia-site-xdump*
>>
>>
>>
>> thanks for the fixing this/having this looked.. Tried the curl tests
>> again.
>> With --ipv6 my installation doesn't work at all yet because I haven't
>> activated ipv6 yet. I guess it's time for me do so some time soon..
>>
>>
>>
>> However "curl -L" now downloads the "rsync-dirlist-last-1-good.txt" file
>> correctly, as does wget. However, by default wget writes the file to
>> disk,
>> and to get the behavior of "curl -L" it would need "wget -q -O -", that
>> is,
>> it needs some options, albeit different ones, as well..
>>
>>
>>
>> regards,
>>
>>
>>
>> Guy
>>
>>
>>        -----Original message-----
>>        From: Dr. Kent L. Miller <address@hidden>
>>        Sent: Sat 16-11-2013 23:01
>>        Subject: Re: [Xmldatadumps-l] ftpmirror.your.org - IPv4 and IPv6
>>        give different results (fwd)
>>        To: Guy Castagnoli <address@hidden>;
>>        CC: wpmirrordev <address@hidden>; wp-mirror-list
>>        <address@hidden>;
>>        Dear Guy,
>>
>>        1) *wikimedia-site-xdump*
>>
>>        The HTTP/1.1 301 redirect with IPv4 is now fixed at <your.org>
>>
>>        However, the error did spoil your out-of-box experience.  So I
>>        am thinking
>>        about replacing `curl' with `wget' which automatically follows
>>        redirects.
>>
>>        Sincerely Yours,
>>        Kent
>>
>>        ---------- Forwarded message ----------
>>        Date: Sat, 16 Nov 2013 14:53:49 -0600
>>        From: Kevin Day <address@hidden>
>>        To: Dr. Kent L. Miller <address@hidden>
>>        Subject: Re: [Xmldatadumps-l] ftpmirror.your.org - IPv4 and IPv6
>>        give different
>>            results
>>
>>        Nice catch!
>>
>>        That should be fixed now, and sending a 301 redirect on both
>>        ipv4 and ipv6.
>>
>>        Several of the pages on the “dumps” collection have hardcoded
>>        links that require them to be in the / (root) directory, so they
>>        require their own hostname.
>>
>>        — Kevin
>>
>>        On Nov 16, 2013, at 2:32 PM, Dr. Kent L. Miller
>>        <address@hidden> wrote:
>>
>>        > Dear Ariel,
>>        >
>>        > When I compare the following two commands
>>        >
>>        > (shell)$ curl --ipv4 --verbose
>>        http://ftpmirror.your.org/pub/wikimedia/dumps/
>>        > (shell)$ curl --ipv6 --verbose
>>        http://ftpmirror.your.org/pub/wikimedia/dumps/
>>        >
>>        > The first reports "HTTP/1.1 301 Moved Permanently"
>>        > The second reports "HTTP/1.1 200 OK"
>>        >
>>        > Has anyone else noticed this?
>>        >
>>        > Sincerely Yours,
>>        > Kent
>>        >
>>        >
>>        >
>>
>> +------------------------------------------------------------------------+
>>        > pub  1024D/359E5142 2008-09-01       GPG key available on
>>        pgpkeys.mit.edu
>>        >     Key fingerprint = 8D4F 4485 7F7D 5406 230C  9749 B821 2572
>>        359E 5142
>>        > uid  Dr. Kent L. Miller <address@hidden>
>>        >
>>
>> +------------------------------------------------------------------------+
>>        >
>>        >
>>        > _______________________________________________
>>        > Xmldatadumps-l mailing list
>>        > address@hidden
>>        > https://lists.wikimedia.org/mailman/listinfo/xmldatadumps-l
>>
>>
>
>