[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [Bug-wget] wget and collapsible lists?
From: |
Giuseppe Scrivano |
Subject: |
Re: [Bug-wget] wget and collapsible lists? |
Date: |
Thu, 05 Aug 2010 13:51:07 +0200 |
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/23.1 (gnu/linux) |
Johnny <address@hidden> writes:
> I am trying to fetch a complete set of pdf docs, whereof some are
> "hidden" in a collapsible list; if you visit the site you must expand
> the list to get the docs. Usind wget, I cannot get all the files (the
> top level files downloads, but not the rest).
>
> This is what I tried:
> wget -r -H -l 3 -A pdf
> https://www.ukap1000application.com/doc_pdf_library.aspx
I get a different page if I spoof the user-agent.
$ wget -O- -q https://www.ukap1000application.com/doc_pdf_library.aspx \
| wc -c
36152
$ wget -q -O- \
--user-agent "Mozilla/5.0 (rv:1.9.2.8) Gecko/20100803 Foo/3.6.8" \
https://www.ukap1000application.com/doc_pdf_library.aspx | wc -c
174706
Try to do the same with your command.
Cheers,
Giuseppe