Re: [Wp-mirror-list] WP-MIRROR 0.7 feedback

[Luiz Augusto]

On Fri, May 30, 2014 at 5:15 AM, wp mirror <address@hidden> wrote:

[...]

0) Bug report

I have not been able to reproduce the bug. That said, I set up `kvm' with my own script, rather than use VirtualBox. So I have not actually reproduced the environment in which you tested WP-MIRROR.

Days ago I've tried again to mirror cawikisource,glwikisource using WP-MIRROR 0.7.3 and the same issue appeared. Don't know if this will help you, but I've uploaded the entire wp-mirror.log to

https://drive.google.com/file/d/0B0ByxZKX30zycjFPdmNXMXhPa3M/edit?usp=sharing

Maybe adding some kind of check+grant permissions to wikiadmin/wikiuser before some very specific steps solves it?

1) MediaWiki Extensions

[...]

The variety of namespace IDs has me concerned. [...]

Has anyone compiled a table of (language-code, namespace ID pair) that I could use? Is anyone managing the assignment of namespace IDs?

 

Custom namespaces IDs are an issue on all Wikimedia wikis. Wikisource and ProofreadPage extension are only the ones that mostly relies on it, but certanily you will get some kind of issues/bugs even on Wikipedia mirroring.

BTW on all Wikisource subdomains "Page" and "Index" are alias to the corresponding local namespace. You can quickly test it browsing to pages as 

<https://en.wikisource.org/wiki/Page:Key>

<https://de.wikisource.org/wiki/Page:Key>

<https://en.wikisource.org/wiki/Index:Key>

<https://pt.wikisource.org/wiki/Index:Key>

Or even better, you can browse in the live config files at <http://noc.wikimedia.org/conf/highlight.php?file=InitialiseSettings.php>.

I've found wgMetaNamespace, wgMetaNamespaceTalk, wgNamespaceAliases, wgNamespacesWithSubpages, wgProofreadPageNamespaceIds related to namespaces in general, but possibly there is more setting strings related to it.

Maybe this kind of custom settings will be scriptable to get at time of mirroring or even can be placed in some sort of wp-mirror-advconfig deb package?

 

2) Special wikis

You can mirror `betawikiversity' and `sourceswiki' with the following commands:

[...]

I tested both mirroring and browsing before writing this e-mail.

Many thanks!

 

3) XML data dumps

[...]

I assume that when you ask for the `allpages' data dump, you are referring to the `pages-meta-current' data dump. This feature request can be implemented easily, and will appear in the next release (v0.7.4). Implementing `pages-meta-history' would be a little more effort.

 

Yep, I was reffering to pages-meta-current sets.

I don't think that pages-meta-history will be relevant to mirroring efforts, specially because edit history metadata are available on stub-meta-history,stub-meta-current,stub-articles sets (at some time adding support for -stub- may be interesting, in order to keep some historical data even the original wiki get's down forever, but this isn't going to happen on the near future)

Luiz