Comments on: Fixing duplicate content (and no, I’m not going to talk about pagination)

By: richardbaxterseo

richardbaxterseo — Mon, 09 Feb 2009 18:01:37 +0000

@Mike – you need to speak to g1SMD (you should be able to contact him at SEOmoz)

By: Mike

Mike — Thu, 08 Jan 2009 14:57:02 +0000

I am in the process of moving a site from one host to another. My new server has apache userdir turned on so my site is visible on

exampleserver.com/~mysite
and
mysite.com

I have had a problem before with this sort of setup where google discovered both ways into the site and penalised me for duplicate content.

Unfortunately I cannot turn Userdir off.

Is there a way of preventing this sort of leak with htaccess or robots.txt?

By: EnhakyEffiff

EnhakyEffiff — Sun, 28 Sep 2008 17:36:39 +0000

omg.. good work, dude

By: richardbaxterseo

richardbaxterseo — Tue, 08 Jul 2008 17:42:50 +0000

Hey Tbone, I think it does matter. The other problem you have is Google’s extremely aggressive discovery methods. They’re definitely using toolbar data and http refers for discovery so it’s very hard to tell what’s going on without a really full investigation.

By: Tbone Malone

Tbone Malone — Tue, 08 Jul 2008 15:23:22 +0000

Nice post.

Regarding javascript. Recently on a fairly trusted site we used javascript on a link as part of some A/B testing. Thinking Google wouldn’t index the “B” page. Low and behold they indexed it within two days. This was the only link pointing to this new page. It wasn’t an “onclick” event though. Don’t know if that would matter.

By: Richard Baxter

Richard Baxter — Tue, 08 Jul 2008 12:17:03 +0000

You can't get much more obvious than the javascript link back to my homepage and Google hasn't discovered the url yet. See? I happen to have seen a lot of action with onclick events and I've seen no evidence to support your theory

By: Gab

Gab — Tue, 08 Jul 2008 02:58:24 +0000

With Google executing JS on trusted sites, especially where the url is obvious in the js, I wouldn’t trust onclick to avoid dupe content. my untested 2 pence.

By: Tom

Tom — Fri, 04 Jul 2008 07:47:33 +0000

Another good post. Another way to fix this (though slightly more technical) is to 301 redirect from your tracking URLs to the canonical version. This will still let you track using a lot of analytics packages.

If that doesn’t work for you, you could conditionally 301 redirect for google bot (this is what amazon does)