Say you want to sell hot air. How would you go about it?

Your best option is to organise a door-drop campaign. At least, that's how one marketing expert interprets a recent piece of junk mail research published by the British Population Survey (BPS). [Hyperlink to marketingweek.co.uk removed in May 2012 as the linked page no longer exists – JB]

"Direct marketers should accept that their work might be unloved but it is effective", is his conclusion. Why? Because the statistics tell him that of all marketing channels unsolicited leaflets have the highest response rate; no less than 7% of householders will respond to your leaflet! If you would advertise hot air on television, radio or the internet you'd get a response rate of just 4%. "It underlines the giant strides made [by marketers] in targeting and analytics [sic] that puts mail through the letterboxes of the potentially more receptive."

Hmmm… is the BPS seriously asserting that, on average, leaflets generate a response rate of 7%? Addressed junk mail achieves response rates of 1 to 3%. Surely it can't be right that unaddressed mail gets respond rates of 7%?

Looking at the questions respondents were asked it quickly becomes clear that the figure of 7% actually isn't a response rate as we know it. To measure a response rate I could give 100 people a leaflet and count how many of them rush to my shop after reading it. If seven people would do so I got a response rate of 7%. But that's not what the BPS has done; the researchers simply asked respondents if they had responded to any leaflet they had received during the last few weeks. What the research tells us, in other words, is that 7% of people taking part in the survey responded to at least one of an unspecified number of leaflets they received in a period of a few weeks. The response rate is not 7/100 but 7/X, with X being the total number of leaflets received by respondents.

If you try to calculate the actual response rate the research turns out to be quite embarrassing for marketing gurus. The calculation is tricky as we need to take into account the total number of leaflets the average respondent received and the number of leaflets they responded to; both are not measured in the BPS's research. And, what on earth is "a few weeks"? Two? Three? Five? Still, even if we're generous and accept 'two' as a minimum definition of a few; assume that the people who did respond to leaflets did so twice (i.e. they responded to two rather than just one leaflet); and if we combine this with data from the Direct Marketing Association about the total volume of unaddressed junk mail we do get to a response rate that looks more natural:

• Annually, 9,300,000,000 leaflets are delivered to 25,800,000 addresses
• The average household receives (9,300,000,000 / 25,800,000 =) 360.5 leaflets per annum
• That makes (360.5 / 52 =) 6.9 leaflets per week
• That's (2 x 6.9 =) 13.8 leaflets in our two-week period
• 100 respondents would receive 100 x 13.8 = 1,380 leaflets in total
• Seven respondents responded to two leaflets: 14/1,380 = 1.0%

Yep, 1%. That's a figure closer to what you would expect, innit?