One of the reasons why a site can lose its positions and the amount of traffic – is an increasing number of duplicates on the site. They can appear as a result of specific work of CMS, or as a result of desire to get maximum traffic from the search by increasing the number of sample pages on your web site, and because of an intentional or an unintentional placement of links by the third parties on your site from other resources.
The problem with duplicates is very similar to the problem of search of the canonical address of the page by search analyzers. In several cases robot can identify canonical address, if for example the order of parameters in dynamic URL was changed:
Is practically the same as:
But in most cases, especially when using the NC, canonical page is difficult to define, that is why complete duplicates and partial duplicates get indexed.
What is interesting, for Yandex duplicates are less intimidating even on the search results pages (that are partial duplicates of each other), it can generate good traffic, but attitude of Google to the double is more critical (because of the war against MFA and site templates ).
Main Methods of Duplicate Search on the Site
The methods with the help of which you can quickly find duplicated pages on your site are described below. Use them periodically.
1. Google Webmaster
Go to panel Google for web masters, find the Optimization section – HTML Optimization. On this page you can see a number of duplicated pages with description and TITLE.
In such way you can find absolute duplicates, but unfortunately, you cannot find the partial duplicates that have unique, but sample titles.
2. Xenu Program
XenuLinkSleuth – one of famous programs for optimizers that allows you to conduct technical audit of the web site and to find similar titles as well (if for example, you do not have access to Google Web Master).
To read more detailed information, follow this link and read the article. Simply scan the site, sort out the results according to their titles, and search for visual similarities in titles. This method also has its drawbacks – with its help you cannot find the partial duplicates.
3. Search Results
Search results can show you not only the site itself, but also search engine attitude to it.. IN order to find duplicates in Google, you have to make a special request. Its components are: site:mysite.com – it will show you pages of the mysite.ru that are indexed by Google( general index).
site:mysite.com/& – shows you pages that take part in search conducted by search engines ( main index). In such way you can define less informative pages and partial duplicates that do not take part in search and may prevent pages of the main index from ranking higher.
While searching do not forget to follow the link: repeat search, including omitted results. If the results were not enough to see the objective picture. For example, site:amberexpert.com -site:amberexpert.com/&
Now, when you have found all the duplicates, you can bravely delete them , having corrected the site engine and having added tag rel=canonical to the site titles.