Tackling Duplicate Content on Mobile Websites

Resizing and reshaping content for smaller browsers

A PHP work-around for responsive design

Today I want to talk about how to optimise your website for mobile browsers.
With the ever increasing use of mobile devices for accessing the internet, it would be foolish to overlook how your website performs on mobile browsers. There are many things to take into account when considering how best to present the content of your website to mobile users, not least that the screen size is drastically smaller, and navigation is often touchscreen based.

Some of the main things that need to be taken into account are that any navigation must be large and well spaced, and text should be presented in a large, clear format.

The Subdomain Solution (Bad)

Many websites have chosen to deal with the issue by detecting the user-agent of the users browser and sending the mobile visitors to a specific mobile subdomain (typically “m.domain.com”) where the information is presented in a format more fitting for mobile browsers.
The sad thing about this approach is that it really goes against the idea of Duplicate Content where it has been established that any given piece of content should have one single URL.
To address this problem we could do a few things such as canonicalising the pages to the URL of the page that is not on the mobile subdomain.
For example:

m.domain.com/article.html
is the mobile version of
www.domain.com/article.html

It would make sense to set the mobile version of the article as NOINDEX and add the following canoncial tag in the head of the page:
<link rel=”canonical” href=”http://www.domain.com/article.html” />
In this case we’ve made efforts to preserve any link equity that is pointed to the mobile version of the site, with the canonical tag, and prevent the mobile version from being listed in the Google search results.

Perhaps an easier move would be to block the entire subdomain in the robots.txt file of your website. Provided this was done before Google crawled the subdomain it would prevent the mobile version of your website from posing any duplicate content issues.
This would be the easiest to implement, however in this case you loose all link equity that is pointed at the mobile version. Sadly, neither of these options seem to be employed by most websites, including many big media companies.

For example:
Reuters
BBC
The Guardian

If we take a look at the BBC, for example, taking the first result we find in the m.bbc.co.uk subdomain:
http://m.bbc.co.uk/news/10519300
and compare it with the non mobile version:
http://www.bbc.co.uk/news/10519300

We can clearly see that both pages contain identical content.
The mobile version is not set as “NOINDEX”
No canonical tags are presented on the mobile page
The non mobile version does, thankfully, have a canonical tag :)
(This opens a whole new question as to why Google is indexing pages with duplicate content from the same domain when it is already aware of a canonicalised version, but that is a story for another day ;) )

Wait, are you saying the BBC has a huge duplicate content problem?
Potentially, Yes.
Currently totaling “About 7,910 results” according to Google.
Reuters comes in a little higher at “About 805,000 results”.

If we take some of the content of the Reuters article and search it on Google we can clearly see that both versions of the website are shown in the results:

Reuters duplicate content

So as we have established, this approach is something of a cluster-fuck when it comes to duplicate content. But what is perhaps even less forgivable is how this affects user experience.

For example, if I browse an article on my smartphone and decided to share it, either on a social network or by email, then the shared URL will be that of the mobile sub-domain.

Lets say I am keeping abreast of the latest Japanese blowfish news on my smartphone. I choose to share this article with someone and send them the URL I am currently reading: http://mobile.reuters.com/article/oddlyEnoughNews/idUSBRE83202P20120403

Go ahead and visit that link from your non mobile device.
What you get is the mobile optimised content filling your full-size desktop browser!

What you would need to do here is check the user-agent of each visitor to the mobile subdomain then redirect them to the full version if they are not using a mobile device.

This approach would also theoretically fix any duplicate content issues because the googlebot would always be redirected to the full version of the content if it tried to access the mobile version.
In this case link equity would also be preserved, for the most part, by a 301 redirect.

But still, this still feels a little over-engineered and where possible I always shy away from using 301 redirects. Many SEO's regard 301 redirects as the silver bullet that cures a multitude of ailments. If you look at best practice for 301 redirects they should be used only when content has previously existed on a URL and now permanently exists on a new URL. In addition, adding another redirect to the pile makes it more likely that your URL will go through a bunch of redirects before ending up at the right place.
For example:

m.domain.com/article/abc/ >>>
m.domain.com/article/abc (loose the trailing slash) >>>
www.domain.com/article/abc (non mobile version) >>>
www.domain.com/article/abc-updated (updated version of the article)

Ok, so this is a bit of a "worse case scenario" example but by no means unimaginable.
At each redirect, the link equity passed forward to the article is reduced somewhat, you piss off the googlebot sending it through a chain of redirects, tie up more server resources, and overall it just looks unprofessional and dumb.

Surely we can do this a little better.

The Dynamic CSS Solution (Better!)

Surely a better solution would be to show both mobile and non-mobile visitors the same URL.

iphone website

danclarkie.co.uk as seen on an iPhone.


If we can do this then we automatically fix any duplicate content problems, we retain full link equity of any links pointing to the mobile version, and save putting another bloody 301 redirect out there in the universe.

Seems to make sense to me :)

So here’s what I suggest.

Obviously we want to present mobile friendly content to mobile browsers and still present the full power content to non mobile browsers, but we can easily do this without sending them to different URLs.

The simple plan is:

1: Detect the User-Agent
2: Deliver the relevant CSS file.
3: Profit.

This way we retain exactly the same source code for the page (aside from the different CSS file name) and deliver our content under one single URL, optimised for whichever type of device you are using.
Great success!

Firstly we will need to make a CSS file that is optimised for mobile devices. Some things to bear in mind here are, again, the reduced size of the mobile screen and the touch-screen navigation.

It is usually a good idea to make the font very big and well spaced, this can be achieved with the CSS attributes line-height and font-size
For example:

#content, #content input, #content textarea {
color: #333;
line-height: 40px;
font-size: 40px;
}

You may also need to hide a bunch of stuff that causes display errors, for this we can use the trusty old "display:none;" attribute. Whilst this tag always starts to upset some lesser informed ultra white hat SEOs, when used properly, as in this case, it presents no risk. Remember this CSS is only shown to mobile browsers and one can very easily justify that you are hiding certain, less important, attributes in the interest of user experience and making your content more accessible.

There isn't really a better way to do this than working with a smartphone and testing changes in the CSS live on the device, tweaking it until it looks perfect.

Now you should have 2 CSS files, one for mobile browsers, and your original, non mobile optimised, CSS file.
If we go ahead and upload them to the server, naming them something like "style.css" and "mstyle.css" then we are almost done.

Next we still need to perform a little user-agent detection at the top of each page. If you are using WordPress or a similar CMS then this can easily be added to the header file of your current template.

<link rel="stylesheet" type="text/css" media="all" href="<?php 
if (preg_match('/(up.browser|up.link|mmp|symbian|smartphone|midp|wap|phone|android)/i', strtolower($_SERVER['HTTP_USER_AGENT']))) {
echo "**LINK TO MOBILE CSS FILE**";
}
else{
echo "**LINK TO NON MOBILE CSS FILE**";
} 
?>" />

What we’re doing here is checking the user-agent on the fly and delivering the correct CSS file to them based on their device.

Test it out on your smartphone and computer, if everything has worked you should see the mobile optimised page on your smartphone and the classic page on your computer.

No need to set up a bunch of complex canonical tags, NOINDEX tags, 301 redirects, etc.

We’ve instantly made your content much easier to consume on mobile devices without creating any duplicate content issues or affecting the style of your site for non mobile users.

Above all else, delivering good content in a way that is best for your users should be the main focus of any SEO effort ;)

Until next time, Happy Easter :)