[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Wp-mirror-list] Attempting to mirror on laptop
From: |
wp mirror |
Subject: |
[Wp-mirror-list] Attempting to mirror on laptop |
Date: |
Wed, 15 Aug 2012 08:21:34 -0400 |
Dear Benjamin,
0) Preamble
Thank you for providing output from `wp-mirror --text'. When
WP-MIRROR runs in monitor mode (--text, --screen, or --gui) it
collects state information by forking shell processes that run MySQL
as well as certain BASH commands. This suggests that MySQL problems
have been solved, and that file system permissions are in order.
Thank you for the chown error messages. It appears that WP-MIRROR is
not locating the image files that were downloaded when the i-chunks
were processed. You should have over 50,000 images occupying about
40G. So where are the image files?
1) images
Downloaded image files should be stored in a directory tree under
/var/lib/mediawiki/images/[0-9a-f]/. Resized image files should be
stored under /var/lib/mediawiki/images/thumb/. Mathematical
expressions should have been converted to PNG image files and stored
under /var/lib/mediawiki/images/math/.
To that end, please let me see your output for the following commands:
root-shell> ls -l /var/lib/mediawiki/images/
total 6235
drwxr-xr-x 18 www-data www-data 432 Nov 27 2011 0
drwxr-xr-x 18 www-data www-data 432 Nov 27 2011 1
drwxr-xr-x 18 www-data www-data 432 Nov 27 2011 2
drwxr-xr-x 18 www-data www-data 432 Nov 27 2011 3
drwxr-xr-x 18 www-data www-data 432 Nov 27 2011 4
drwxr-xr-x 18 www-data www-data 432 Nov 27 2011 5
drwxr-xr-x 18 www-data www-data 432 Nov 27 2011 6
drwxr-xr-x 18 www-data www-data 432 Nov 27 2011 7
drwxr-xr-x 18 www-data www-data 432 Nov 27 2011 8
drwxr-xr-x 18 www-data www-data 432 Nov 27 2011 9
drwxr-xr-x 18 www-data www-data 432 Nov 27 2011 a
drwxr-xr-x 18 www-data www-data 432 Nov 27 2011 b
drwxr-xr-x 18 www-data www-data 432 Nov 27 2011 bad-images
drwxr-xr-x 18 www-data www-data 432 Nov 27 2011 c
drwxr-xr-x 18 www-data www-data 432 Nov 27 2011 d
-rw-r--r-- 1 root root 1507005 Aug 12 03:09 download.log
drwxr-xr-x 18 www-data www-data 432 Nov 27 2011 e
-rw-r--r-- 1 root root 278194 Aug 12 03:06 exists.log
drwxr-xr-x 18 www-data www-data 432 Nov 27 2011 f
-rw-r--r-- 1 root root 4527367 Aug 12 03:10 failed.log
drwxr-xr-x 18 www-data www-data 432 Nov 27 2011 math
drwxrwxrwx 2 www-data www-data 48 Nov 27 2011 temp
drwxrwxrwx 18 www-data www-data 432 Nov 29 2011 thumb
drwxrwxrwx 2 www-data www-data 48 Aug 12 09:54 tmp
drwxr-xr-x 2 www-data www-data 62936 Aug 14 23:37 wp-mirror
root-shell> du --human --total /var/lib/mediawiki/images/[0-9a-f] | tail -n 1
45G total
root-shell> du --human --total /var/lib/mediawiki/images/thumb | tail -n 1
1.1G total
root-shell> du --human --total /var/lib/mediawiki/images/math | tail -n 1
16M total
2) work files
WP-MIRROR stores its work files (dump files, x-chunks, i-chunks, etc.)
under the directory /var/lib/mediawiki/images/wp-mirror/. I would
like to see if files are there, and if the i-chunks are properly
formed.
To that end, please let me see your output for the following commands.
root-shell> du --human --total /var/lib/mediawiki/images/wp-mirror/ | tail -n 1
1.5G total
root-shell> ls
/var/lib/mediawiki/images/wp-mirror/simplewiki-latest-md5sums.txt*
| wc -l
2
root-shell> ls
/var/lib/mediawiki/images/wp-mirror/simplewiki-20120805-pages-articles*
| wc -l
315
root-shell> ls /var/lib/mediawiki/images/wp-mirror/[0-9a-f]
ls: cannot access /var/lib/mediawiki/images/wp-mirror/[0-9a-f]: No
such file or directory
root-shell> cat
/var/lib/mediawiki/images/wp-mirror/simplewiki-20120805-pages-articles-p000000000-c000001000.sh
| head -n 33
#!/bin/sh
IMAGE=
OUTPUT=
IMAGEPATH=http://upload.wikimedia.org/wikipedia/nil/
COMMONSPATH=http://upload.wikimedia.org/wikipedia/commons/
/bin/mkdir -p $OUTPUT./thumb
/bin/chmod 777 $OUTPUT./thumb
/bin/mkdir -p $OUTPUT./temp
/bin/chmod 777 $OUTPUT./temp
/bin/mkdir -p $OUTPUT./tmp
/bin/chmod 777 $OUTPUT./tmp
if [ -a $IMAGE./c/c7/PB050006.JPG ]; then
echo c/c7/PB050006.JPG already exists >> exists.log
else
curl --retry 0 -f -O $COMMONSPATH./c/c7/PB050006.JPG
if [ -a $IMAGE./PB050006.JPG ]; then
/bin/mkdir -p $OUTPUT./c/c7/
/bin/mv ./PB050006.JPG $OUTPUT./c/c7/
echo ./c/c7/PB050006.JPG downloaded >> download.log
else
curl --retry 0 -f -O $IMAGEPATH./c/c7/PB050006.JPG
if [ -a $IMAGE./PB050006.JPG ]; then
/bin/mkdir -p $OUTPUT./c/c7/
/bin/mv ./PB050006.JPG $OUTPUT./c/c7/
echo ./c/c7/PB050006.JPG downloaded >> download.log
else
echo ./c/c7/PB050006.JPG failed >> failed.log
fi
fi
fi
3) MediaWiki
Please let me know if you can browse <simple.mediawiki.site>.
To that end, please open a web browser to:
<http://simple.mediawiki.site/index.php/Main_Page>
Please let me know if you see images on this page:
<http://simple.mediawiki.site/index.php/Dinosaur>
Please let me know if you see nicely formatted math symbols on this page:
<http://simple.mediawiki.site/index.php/Pythagorean_theorem>
Sincerely Yours,
Kent
- [Wp-mirror-list] Attempting to mirror on laptop, wp mirror, 2012/08/05
- [Wp-mirror-list] Attempting to mirror on laptop, wp mirror, 2012/08/07
- [Wp-mirror-list] Attempting to mirror on laptop, Dr. Kent L. Miller, 2012/08/07
- [Wp-mirror-list] Attempting to mirror on laptop, wp mirror, 2012/08/08
- [Wp-mirror-list] Attempting to mirror on laptop, wp mirror, 2012/08/09
- [Wp-mirror-list] Attempting to mirror on laptop, Dr. Kent L. Miller, 2012/08/13
- [Wp-mirror-list] Attempting to mirror on laptop,
wp mirror <=
- [Wp-mirror-list] Attempting to mirror on laptop, Dr. Kent L. Miller, 2012/08/15
- [Wp-mirror-list] Attempting to mirror on laptop, wp mirror, 2012/08/17
- [Wp-mirror-list] Attempting to mirror on laptop, Dr. Kent L. Miller, 2012/08/19
- [Wp-mirror-list] Attempting to mirror on laptop, wp mirror, 2012/08/19
- [Wp-mirror-list] Attempting to mirror on laptop, wp mirror, 2012/08/20