[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
bug#52338: Crawler bots are downloading substitutes
From: |
Mathieu Othacehe |
Subject: |
bug#52338: Crawler bots are downloading substitutes |
Date: |
Thu, 09 Dec 2021 14:27:38 +0100 |
User-agent: |
Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) |
Hello Leo,
> + (nginx-location-configuration
> + (uri "/robots.txt")
> + (body
> + (list
> + "add_header Content-Type text/plain;"
> + "return 200 \"User-agent: *\nDisallow: /nar/\n\";"))))))
Nice, the bots are also accessing the Cuirass web interface, do you
think it would be possible to extend this snippet to prevent it?
Thanks,
Mathieu
- bug#52338: Crawler bots are downloading substitutes, Leo Famulari, 2021/12/06
- bug#52338: [maintenance] hydra: berlin: Create robots.txt., Leo Famulari, 2021/12/06
- bug#52338: Crawler bots are downloading substitutes,
Mathieu Othacehe <=
- bug#52338: Crawler bots are downloading substitutes, Tobias Geerinckx-Rice, 2021/12/09
- bug#52338: Crawler bots are downloading substitutes, Leo Famulari, 2021/12/10
- bug#52338: Crawler bots are downloading substitutes, Tobias Geerinckx-Rice, 2021/12/10
- bug#52338: Crawler bots are downloading substitutes, Mathieu Othacehe, 2021/12/11
- bug#52338: Crawler bots are downloading substitutes, Mathieu Othacehe, 2021/12/19
bug#52338: Crawler bots are downloading substitutes, Mark H Weaver, 2021/12/10