web-seo.rst
author Oleksandr Gavenko <gavenkoa@gmail.com>
Mon, 30 Jan 2017 00:44:30 +0200
changeset 2075 ccaa2f364422
parent 2010 cd153030eca9
child 2106 09eb514c7b80
permissions -rw-r--r--
Improve page formatting.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
1334
9bf0d5a1f0cf Include common header with quick links.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1218
diff changeset
     1
.. -*- coding: utf-8; -*-
932
b65ec623c6d1 WEB SEO.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
     2
2010
cd153030eca9 SEO tips
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2009
diff changeset
     3
======================================
cd153030eca9 SEO tips
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2009
diff changeset
     4
 WEB SEO (search engine optimisation)
cd153030eca9 SEO tips
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2009
diff changeset
     5
======================================
932
b65ec623c6d1 WEB SEO.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
     6
.. contents::
1905
fba288d59662 Include only local subsections into TOC. This prevent duplication of
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1334
diff changeset
     7
   :local:
932
b65ec623c6d1 WEB SEO.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
     8
2007
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
     9
General search engines
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
    10
======================
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
    11
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
    12
Major search engines are:
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
    13
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
    14
* https://google.com from Google Co.
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
    15
* https://bing.com from Microsoft Co.
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
    16
* https://yahoo.com or https://search.yahoo.com/ from Yahoo Co and it uses Bing
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
    17
  engine.
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
    18
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
    19
Major local search engine are:
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
    20
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
    21
* https://yandex.com from Yandex Co that have large market on Russia, Ukraine, Belarus.
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
    22
2008
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
    23
Search engine webmaster tools
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
    24
=============================
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
    25
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
    26
https://www.google.com/webmasters/tools/home
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
    27
  Google webmaster tools.
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
    28
https://www.bing.com/webmaster/
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
    29
  Bind webmaster tools.
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
    30
https://webmaster.yandex.ru/
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
    31
  Yandex webmaster tools.
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
    32
2007
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
    33
SEO tips
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
    34
========
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
    35
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
    36
* Use pain text rather than images, CSS, JavaScript, Flash, SilverLite or other
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
    37
  technologies for important content.
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
    38
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
    39
* Make sure your site work without with CSS, JavaScript, Flash, Java, etc
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
    40
  disabled. Check your site with screen reader or text browser (lynx, w3m).
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
    41
2008
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
    42
* Build valid HTML pages. Validate against W3C and other tools. Make site mobile
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
    43
  friendly and use the mobile friendly test.
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
    44
2007
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
    45
* Avoid HTML frames and iframes.
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
    46
2008
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
    47
* Market your site so other sites link to yours. The more valuable and relevant
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
    48
  site point to you the more rank you get!
2007
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
    49
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
    50
* Tend to focus on single topic per page.
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
    51
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
    52
* Avoid cryptic URLs. Craft human attractive URLs as URL words directly can be
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
    53
  indexed. Use dashes instead of spaces.
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
    54
2009
33db8b550960 SERP structure
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2008
diff changeset
    55
* Avoid duplication. Shutdown mirrors. Use ``<link rel="canonical" ...>`` tag to
33db8b550960 SERP structure
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2008
diff changeset
    56
  point to page on main version from mirrors or page variants. Avoid page
33db8b550960 SERP structure
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2008
diff changeset
    57
  variants (like embedding sid into URL). Use::
2007
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
    58
2009
33db8b550960 SERP structure
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2008
diff changeset
    59
    <link rel="prev" ...>
33db8b550960 SERP structure
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2008
diff changeset
    60
    <link rel="next" ...>
33db8b550960 SERP structure
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2008
diff changeset
    61
33db8b550960 SERP structure
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2008
diff changeset
    62
  to gather paginated content.
33db8b550960 SERP structure
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2008
diff changeset
    63
33db8b550960 SERP structure
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2008
diff changeset
    64
* Tags ``title``, ``h1`` / ``h2`` / ``h3`` and::
2008
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
    65
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
    66
    <meta name="description" content="...">
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
    67
    <meta name="keywords" content="...">
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
    68
2009
33db8b550960 SERP structure
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2008
diff changeset
    69
  have great impact. As usual make sure them have unique content across the web
33db8b550960 SERP structure
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2008
diff changeset
    70
  and your pages.
2008
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
    71
2010
cd153030eca9 SEO tips
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2009
diff changeset
    72
* Use keywrds for internal ``a`` links.
cd153030eca9 SEO tips
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2009
diff changeset
    73
2008
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
    74
* Add ``rel="nofollow"`` to spam or money making links.
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
    75
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
    76
* Mark up site using microformats: ARIA, schema.org.
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
    77
2010
cd153030eca9 SEO tips
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2009
diff changeset
    78
* Keep navigational structure (global and hierarchical) easy to crawl.
cd153030eca9 SEO tips
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2009
diff changeset
    79
2008
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
    80
* Submit sitemap to search engine webmaster tools.
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
    81
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
    82
* Ensure a fast load time and absent of HTTP errors.
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
    83
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
    84
* Make site sharable across social media.
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
    85
2010
cd153030eca9 SEO tips
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2009
diff changeset
    86
* Use redirects when changes site structure. This usually move page value to new
cd153030eca9 SEO tips
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2009
diff changeset
    87
  page.
cd153030eca9 SEO tips
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2009
diff changeset
    88
cd153030eca9 SEO tips
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2009
diff changeset
    89
* Produce new content frequently. That make crawlers visit your site more
cd153030eca9 SEO tips
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2009
diff changeset
    90
  frequently.
cd153030eca9 SEO tips
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2009
diff changeset
    91
2008
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
    92
https://support.google.com/webmasters/answer/40349
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
    93
  Steps to a Google-friendly site.
2007
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
    94
https://support.google.com/webmasters/answer/139066
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
    95
  Use canonical URLs
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
    96
https://en.wikipedia.org/wiki/Canonical_link_element
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
    97
  Wiki article.
2009
33db8b550960 SERP structure
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2008
diff changeset
    98
https://support.google.com/webmasters/answer/1663744
33db8b550960 SERP structure
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2008
diff changeset
    99
  Indicate paginated content.
33db8b550960 SERP structure
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2008
diff changeset
   100
https://support.google.com/webmasters/answer/76329
33db8b550960 SERP structure
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2008
diff changeset
   101
  Keep a simple URL structure.
2010
cd153030eca9 SEO tips
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2009
diff changeset
   102
https://www.bing.com/webmaster/help/webmaster-guidelines-30fba23a
cd153030eca9 SEO tips
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2009
diff changeset
   103
  Bing Webmaster Guidelines.
cd153030eca9 SEO tips
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2009
diff changeset
   104
https://yandex.ru/support/webmaster/recommendations/intro.xml
cd153030eca9 SEO tips
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2009
diff changeset
   105
  Рекомендации по созданию сайтов от Yandex.
cd153030eca9 SEO tips
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2009
diff changeset
   106
https://support.google.com/webmasters/answer/35769
cd153030eca9 SEO tips
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2009
diff changeset
   107
  Google Webmaster Guidelines.
2007
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   108
2008
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
   109
http://webmasters.stackexchange.com/questions/22877/seo-where-do-i-start/
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
   110
http://webmasters.stackexchange.com/questions/10281/seo-tips-and-tricks-for-web-developers
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
   111
http://webmasters.stackexchange.com/questions/2/what-are-the-best-ways-to-increase-a-sites-position-in-google
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
   112
http://webmasters.stackexchange.com/questions/237/basic-seo-optimization
5919a8903b3f Search engine webmaster tools.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2007
diff changeset
   113
2009
33db8b550960 SERP structure
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2008
diff changeset
   114
SERP structure
33db8b550960 SERP structure
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2008
diff changeset
   115
==============
33db8b550960 SERP structure
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2008
diff changeset
   116
33db8b550960 SERP structure
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2008
diff changeset
   117
All popular search providers use ``title`` tag content as text for main link to
33db8b550960 SERP structure
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2008
diff changeset
   118
result entry and ``meta name="description"`` or relevant page piece for
33db8b550960 SERP structure
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2008
diff changeset
   119
description.
33db8b550960 SERP structure
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2008
diff changeset
   120
2010
cd153030eca9 SEO tips
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2009
diff changeset
   121
Also ``title`` is used in browser window title bar and as bookmark title.
cd153030eca9 SEO tips
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2009
diff changeset
   122
2009
33db8b550960 SERP structure
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2008
diff changeset
   123
https://en.wikipedia.org/wiki/Search_engine_results_page
33db8b550960 SERP structure
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2008
diff changeset
   124
  Wikiperia article.
2010
cd153030eca9 SEO tips
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2009
diff changeset
   125
https://www.w3.org/QA/Tips/good-titles.html
cd153030eca9 SEO tips
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2009
diff changeset
   126
  <title>: the most important element of a quality Web page.
cd153030eca9 SEO tips
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2009
diff changeset
   127
https://www.w3.org/QA/Tips/Use_h1_for_Title
cd153030eca9 SEO tips
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2009
diff changeset
   128
  Use <h1> for top-level heading.
2009
33db8b550960 SERP structure
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 2008
diff changeset
   129
2007
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   130
Informing search engine about site structure
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   131
============================================
1217
3940dace0d88 Informing search engine about site structure.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1032
diff changeset
   132
::
3940dace0d88 Informing search engine about site structure.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1032
diff changeset
   133
3940dace0d88 Informing search engine about site structure.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1032
diff changeset
   134
  http://www.example.org/sitemap.xml
3940dace0d88 Informing search engine about site structure.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1032
diff changeset
   135
  http://www.example.org/robots.txt
3940dace0d88 Informing search engine about site structure.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1032
diff changeset
   136
2007
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   137
robots.txt
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   138
==========
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   139
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   140
Start line with ``#`` to commet it out.
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   141
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   142
Disable all robots (that respect ``robots.txt``)::
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   143
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   144
  User-agent: *
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   145
  Disallow: /
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   146
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   147
To allow all robots complete access::
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   148
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   149
  User-agent: *
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   150
  Disallow:
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   151
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   152
To allow a single robot::
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   153
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   154
  User-agent: Google
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   155
  Disallow:
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   156
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   157
  User-agent: *
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   158
  Disallow: /
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   159
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   160
All valuable search engines support ``Allow`` and ``Sitemap`` tags::
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   161
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   162
  User-agent: *
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   163
  Allow: /
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   164
  Sitemap: http://example.com/sitemap1.xml
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   165
  Sitemap: http://example.com/my_sitemap2.xml
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   166
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   167
The longer path the higher priority of ``Allow`` and ``Disallow`` tags.
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   168
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   169
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   170
http://www.robotstxt.org/orig.html
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   171
  Original specification. Only ``User-agent`` and ``Disallow`` tags supported.
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   172
http://robotstxt.org.ru/
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   173
  Russian followup to http://www.robotstxt.org
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   174
https://developers.google.com/webmasters/control-crawl-index/docs/robots_txt
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   175
  Google extension specification.
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   176
https://yandex.ru/support/webmaster/controlling-robot/robots-txt.xml
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   177
  Yandex extension specification.
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   178
http://en.wikipedia.org/wiki/Robots.txt
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   179
  Wiki article.
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   180
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   181
meta robots
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   182
===========
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   183
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   184
Instruction to not index current page::
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   185
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   186
  <meta name="robots" content="noindex">
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   187
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   188
Instruction to not follow links on current page::
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   189
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   190
  <meta name="robots" content="nofollow">
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   191
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   192
Pair combinations::
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   193
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   194
  <meta name="robots" content="noindex, nofollow">
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   195
  <meta name="robots" content="noindex, follow">
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   196
  <meta name="robots" content="index, nofollow">
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   197
  <meta name="robots" content="index, follow">
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   198
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   199
rel nofollow attribute in link
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   200
==============================
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   201
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   202
To prevent spam on forums and blogs or to including commertial links::
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   203
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   204
  <a href="http://spam.com/sale/20-off" rel="nofollow">You win 100$!!</a>
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   205
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   206
https://support.google.com/webmasters/answer/96569
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   207
  Use rel="nofollow" for specific links.
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   208
https://support.google.com/webmasters/answer/81749
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   209
  Ways to Prevent Comment Spam.
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   210
https://en.wikipedia.org/wiki/Nofollow
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   211
  Wiki article.
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   212
https://codex.wordpress.org/Nofollow
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   213
  Wordpress suggestions.
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   214
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   215
sitemap
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   216
=======
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   217
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   218
* http://en.wikipedia.org/wiki/Sitemaps
ebd98c98304b SEO tips. robots.txt. rel nofollow attribute in link.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1923
diff changeset
   219
* http://www.sitemaps.org/
1217
3940dace0d88 Informing search engine about site structure.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 1032
diff changeset
   220
932
b65ec623c6d1 WEB SEO.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   221
Alexa.
b65ec623c6d1 WEB SEO.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   222
======
b65ec623c6d1 WEB SEO.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   223
b65ec623c6d1 WEB SEO.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   224
Statistics (most useful Top Search Queries):
b65ec623c6d1 WEB SEO.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   225
b65ec623c6d1 WEB SEO.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   226
  http://www.alexa.com/siteinfo/DOMAIN
b65ec623c6d1 WEB SEO.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   227
b65ec623c6d1 WEB SEO.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   228
Google.
b65ec623c6d1 WEB SEO.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   229
=======
b65ec623c6d1 WEB SEO.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   230
970
04ce91d377dd http://www.google.com/trends
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 956
diff changeset
   231
  http://www.google.com/trends
04ce91d377dd http://www.google.com/trends
Oleksandr Gavenko <gavenkoa@gmail.com>
parents: 956
diff changeset
   232
932
b65ec623c6d1 WEB SEO.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   233
Yahoo.
b65ec623c6d1 WEB SEO.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   234
======
b65ec623c6d1 WEB SEO.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   235
b65ec623c6d1 WEB SEO.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   236
Give backlink (this page redirect to siteexplorer.search.yahoo.com):
b65ec623c6d1 WEB SEO.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   237
b65ec623c6d1 WEB SEO.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   238
  linkdomain:DOMAIN -site:DOMAIN
b65ec623c6d1 WEB SEO.
Oleksandr Gavenko <gavenkoa@gmail.com>
parents:
diff changeset
   239