Unified Diff: static/robots.txt
Issue 29673562:
Disallow / (all) for BUbiNG and bingbot (Closed)
Patch Set:
Created Jan. 18, 2018, 9:39 a.m.
Use n/p to move between diff chunks;
N/P to move between comments.
« no previous file with comments
|
« no previous file
|
no next file »
|
no next file with comments »
Expand Comments ('e') | Collapse Comments ('c') | Hide Comments ('s')
Expand Comments ('e') | Collapse Comments ('c') | Hide Comments ('s')
| Index: static/robots.txt |
| =================================================================== |
| --- a/static/robots.txt |
| +++ b/static/robots.txt |
| @@ -5,6 +5,8 @@ |
| User-agent: puf |
| User-agent: MSIECrawler |
| User-agent: Nutch |
| +User-agent: BUbiNG |
| +User-agent: bingbot |
| Disallow: / |
| User-agent: * |
« no previous file with comments
|
« no previous file
|
no next file »
|
no next file with comments »
