Add Wikipedia article.
authorOleksandr Gavenko <gavenkoa@gmail.com>
Sun, 29 Jan 2017 22:04:52 +0200
changeset 2073 1abbd5a7db80
parent 2072 72921b56230b
child 2074 6210b92c65d2
Add Wikipedia article.
web-site.rst
--- a/web-site.rst	Sun Jan 29 21:56:25 2017 +0200
+++ b/web-site.rst	Sun Jan 29 22:04:52 2017 +0200
@@ -41,14 +41,17 @@
 
 See:
 
-  http://www.robotstxt.org/
-                home page
-  http://www.robotstxt.org/robotstxt.html
-                About /robots.txt
-  http://www.robotstxt.org/faq.html
-                Frequently Asked Questions
-  http://googlewebmastercentral.blogspot.com/2008/06/improving-on-robots-exclusion-protocol.html
-                Improving on Robots Exclusion Protocol
+http://www.robotstxt.org/
+  Page provides description for robots.txt usual practice and discussion about
+  possible standardization efforts.
+http://www.robotstxt.org/robotstxt.html
+  About /robots.txt
+http://www.robotstxt.org/faq.html
+  Frequently Asked Questions.
+https://en.wikipedia.org/wiki/Robots_exclusion_standard
+  Wikipedia article on robots.txt.
+http://googlewebmastercentral.blogspot.com/2008/06/improving-on-robots-exclusion-protocol.html
+  Improving on Robots Exclusion Protocol.
 
 Sitemap
 =======