Shimon Sandler

Finding Duplicate Content with Free Tools

Checking for duplicate content is part of the process of analyzing a website. The most common types of duplicate content to look for is in Title tags, Meta descriptions, product descriptions, dynamic URLs, and Canonicalization.
Fighting off the multi-headed monster of duplicate content
Many websites reuse the same Title tag for their entire site. That’s duplicate content.

Many ecommerce sites resell products from manufacturers will often use the product descriptions provided by the manufacturer. Hence, an abundance of duplicate content.

Top 6 Duplicate Content Mistakes:
1. Circular Navigation – multiple paths though a website.
2. Print-Friendly Pages – Fix: block search engine’s from printer friendly pages.
3. Inconsistent Linking
4. Product Only Pages – with nothing differentiating them from other pages.
5. Transparent Serving
6. Bad Cloaking

There are various ways to check for Duplicate Content. You can do a manual check, and/or use tools to check for dupe content. There are a ton of fee-based SEO tools out there that bundle a duplicate content tool into their suite of tools. However for this post, I’m just going to stick with just the free tools.

Free Tools

Copyscape will check to see if any other site contains the same text as your page. This is great to see all the other ecommerce sites that have the same product description.

Yahoo Explorer is a great free tool to find duplicate content. Make sure you see the results for Pages. Then, Export results to TSV. Open in Excel.

The spreadsheet will contain the TITLE tag, URL, SIZE, and FORMAT. Sort the data by descending or ascending for the Title tag. This will enable you to quickly scan the document for batches of duplicate content.

Xenu Link Sleuth is a popular free tool. This is an application download. Besides providing the same data as Yahoo Explorer, the Xenu Link Sleuth can detect broken links. Then, export to Excel. Sort data as needed.

Google Webmaster Tools. Go to Diagnostics, HTML suggestions, Duplicate title tags – next screen is a list, with a hyperlink to “Download this table”.

Additional Reading:
How to find your website Duplicate Content Issues
How to Remedy Duplicate Content
Google Dupe Content
Site Explorer is a First Step
Duplicate Content Issues

Photo credit: kajojak

Be Sociable, Share!


  1. I always use CopyScape, but here`s an interesting little factoid for you. One client of mine has a e commerce website with multiple product pages. At one point all of his pages were ranking well (both PR and in the SERPs) except for one page. We couldn`t figure it out! I ran the page through CopyScape and it came back with dup content. I asked him to rewrite the content, explaining it was the reason for the penalty. He explained to me that the content came directly from the product manufacturer and it was a requirement to include in order to sell the product. Long story short, he had to contact the manufacturer to request to use unique content, but as soon as he changes it, tah dah! The page was ranking within weeks. Something to keep in mind with reseller clients! My client never even thought twice about the manufacturer`s product description reading as dup content for the search engines!

  2. I prefer to use PlagiarismDetect when checking plagiarism.

  3. I am very careful about duplicate content. Quite interesting about the commerce store above. Wouldn’t that be indicative of most e-commerce sites then…meaning the description of the products?

  4. I really watch out on creating a tag, I sometimes tend to forget that I already used them, or duplicate content. i’m careful now.

  5. Generally I use copyscape to detect plag!

  6. I always use CopyScape &I think CopyScape is the best for duplicate content.

  7. Nice tool copyscape i am daily checking the post before posting and after posting it, to check whether some body is copying my content

    Nice Post


  8. i use also copyscape to detect plag

  9. Thanks for your post. It’s very helpful for a newbie like me :).
    As for balise title, how can we count the number of duplicate titles of a website?

  10. I use copyscape too. But i need a free tool to do it.

  11. Thanks for the post guys,

    I have recently had to deal with content plag, the worst thing about it, is the Blain-tent copying and disregard for other peoples efforts.

    The other site still hasn’t responded to my emails!

  12. i use copyscape and plagiarismdetect

    anyway thanks for the information you posted here..

  13. I found this article very helpful. Thanks for sharing this information.

  14. I wanted to know what tools I can use to find out if someone copied my contents. Google pointed me to this. Thank you so much for the tools you listed. They’re very helpful!

  15. Copyscape is best tool to check duplicate content. Ans unique content is best for our site to be ranked top in SERP.

  16. I too have this problem with my competitors copying my content and then rating better than I do. How do I stop this from happening?

  17. Thanks for posting this information. My sites have been hit by Google’s recent changes and it seems that duplicate content is one of the main reasons. I’m now going through all of my sites and rewriting the content individually, so that none of them duplicate each other and ensuring each has a 301 redirect from www to non-www!

Speak Your Mind