Re: Fwd: Google Code shutting down

[Han-Wen Nienhuys]

On Fri, Mar 13, 2015 at 4:40 PM, Trevor Daniels <address@hidden> wrote:
>> have a look at
>>
>> https://code.google.com/p/support-tools/wiki/IssueExporterTool
>
> That is quite useful, thanks.  I've exported the LilyPond Issues
> DB to a JSON file on my laptop.  5Mb.  It seems complete as far
> as the text goes, but images do not seem to be included, as least
> for the one issue I've inspected in detail (Issue 4005).  All
> the JSON file says for the image is:
>
>             "attachments" : [ {
>               "attachmentId" : "40050007000",
>               "fileName" : "Screenshot.png",
>               "fileSize" : 137527,
>               "mimetype" : "image/png"
>             } ]
>
> but I don't see any attachments anywhere in the download.

I'll send mail to the folks handling this to mention this, but you can
do it manually:

You can extract the attachments from the HTML sources:


wget --quiet -O-
'https://code.google.com/p/lilypond/issues/detail?id=4316'|grep
/attach>att.txt

$ grep 2-19-16.png att.txt|head -1
 <a 
href="//lilypond.googlecode.com/issues/attachment?aid=43160000000&amp;name=atest-2-19-16.png&amp;token=ABZ6GAcfLZCKQxStq2tURaM_f0RSDPjJFw%3A1426334299992">

this is the URL, minus the protocol part. For some reason, the & in
the URL has been escaped.



$ wget $(echo  
'https://lilypond.googlecode.com/issues/attachment?aid=43160000000&amp;name=atest-2-19-16.png&amp;token=ABZ6GAeDaPOMP8ZuFedDBpB7Z3xy01aM0g%3A1426334055154&amp;inline=1'
| sed 's/&amp;/\&/g')
--2015-03-14 13:00:10--
https://lilypond.googlecode.com/issues/attachment?aid=43160000000&name=atest-2-19-16.png&token=ABZ6GAeDaPOMP8ZuFedDBpB7Z3xy01aM0g%3A1426334055154&inline=1
Resolving lilypond.googlecode.com (lilypond.googlecode.com)...
74.125.136.82, 2a00:1450:4013:c01::52
Connecting to lilypond.googlecode.com
(lilypond.googlecode.com)|74.125.136.82|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1749 (1.7K) [image/png]
Saving to: 
‘attachment?aid=43160000000&name=atest-2-19-16.png&token=ABZ6GAeDaPOMP8ZuFedDBpB7Z3xy01aM0g%3A1426334055154&inline=1’

attachment?aid=43160000000&name=atest-2-19-
100%[=============================================================================================>]
  1.71K  --.-KB/s   in 0s

2015-03-14 13:00:11 (24.3 MB/s) -
‘attachment?aid=43160000000&name=atest-2-19-16.png&token=ABZ6GAeDaPOMP8ZuFedDBpB7Z3xy01aM0g%3A1426334055154&inline=1’
saved [1749/1749]



You can loop over the issues (4000-something), extract a ID => URL
mapping and then download them. If you do this, you may need to rate
limit this so you don't get anti-robot captchas.

> Trevor

-- 
Han-Wen Nienhuys - address@hidden - http://www.xs4all.nl/~hanwen