Re: [Sks-devel] Request: Install an efficient robots.txt file

[Tobias Frei]

Hi robots.txt fan,

I just wondered what happened if I removed robots.txt from my server, to reduce the whole discussion to absurdity. :)

Best regards,
ToBeFree

On Thu, Jun 22, 2017, 13:08 Kristian Fiskerstrand <address@hidden> wrote:

On 06/22/2017 10:40 AM, robots.txt fan wrote:
> Kristian, you have responded to this thread, I believe you manage the first one on the list. Is there a reason why only /status is blocked and not /pks?
>
> https://sks-keyservers.net (blocks /status, but not /pks)

The real reason is that /pks didn't exist when the robots.txt file was
created, so I've [added it now], granted more for site resource
management reasons than privacy reasons.

>From a privacy perspective robots.txt doesn't make sense, the data is
already public, bad actors ignore robots.txt and crawl the site just the
same; and the full data set is available and part of regular workflow
for bootstrapping own servers.

References:
[added it now]
https://git.sumptuouscapital.com/?p=sks-keyservers-pool.git;a=commit;h=b98e7522990961541165dfc23781a45a1a5e05a9

--
----------------------------
Kristian Fiskerstrand
Blog: https://blog.sumptuouscapital.com
Twitter: @krifisk
----------------------------
Public OpenPGP keyblock at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3
----------------------------
"Whatever you do in life, surround yourself with smart people who'll
argue with you."
John Wooden

_______________________________________________
Sks-devel mailing list
address@hidden
https://lists.nongnu.org/mailman/listinfo/sks-devel