This an minimal, read-only version of the original Stop Junk Mail website.

Home Blogs Diary 2015 03

The DMA's exclusive deal with Google

15th March 2015

The other day someone contacted me to say that the Direct Marketing Association had quietly ditched its Your Choice Preference Scheme. This doesn't seem to be the case – what seems to have caused confusion is that they fail to mention the existence of the scheme on their page about preference services. You'd almost expect this to be the case; it's a dummy scheme anyway.

While searching the domain for strings such as "your choice" and "junk mail" I did find that the DMA website tells all search engines not to index the website – with the exception of Google. If you use DuckDuckGo, for instance, you get results like this:

A screenshot showing no results for a search for 'junk mail' on the domain ''.
The DMA won't allow us see a description.

You get similar results if you use other alternatives for Google. The reason these search engines aren't allowed to display results is that the DMA website instructs them not to do so using a robot.txt file:

User-agent: Googlebot
Allow: /
Disallow /user/kerry-mailbird-co-uk

User-agent: *
Disallow: /
Disallow /user/kerry-mailbird-co-uk

The first three lines tell the Googlebot that it's allowed to index all pages on the site, apart from the page /user/kerry-mailbird-co-uk. The second block tells all other search engines that they're not allowed to index any page and that they're also not allowed to index the page /user/kerry-mailbird-co-uk. (The last line is obviously redundant as the preceding rule already stated that nothing may be indexed – apparently it's rather important that Kerry Holden is hidden from view ;)

I suspect the DMA only wanted to prevent the website is listed on sites such as the Wayback Machine – it wouldn't be the first time they're trying to get rid of old and perhaps embarrassing content. If so they could have simply told their robot not to index the site:

User-agent: ia_archiver
Disallow: /

Or maybe they feel that people should only use Google?

Last updated: 
15th March 2015

Add new comment

Please prove that you're human (and not a spambot). Refresh the page if the challenge is too difficult.