How accurate are mailing statistics?
This (paraphrased) question came in today from a client:
CiviMail is reporting that over 50 recipients clicked the link to register for our event, but this seems very unlikely. How reliable are these numbers?
It’s a smart question. Truth is, the "Click-through" numbers can be artificially high, for reasons beyond your (or your CRM’s) control.
How could this be?
One reason is that the company that hosts your recipients’ email accounts may do things that make it look like the recipient clicked one or more links in the email.
This is true regardless of what system you might use to send the email — CiviCRM, your own gmail account, some other newsletter software, whatever.
Why would they do that?
The usual rationale here is that the email hosting company wants to block email messages that contain links to known fraud or otherwise malicious content.
So, they'll actually open every single link in the email message, before it even gets to the recipient's in-box.
(For example, several of my clients’ recipients appear to have clicked on all 11 links in a given mailing, but we agree that’s just unlikely.)
Can’t CiviCRM figure that out and adjust the statistics?
Unfortunately, there's no easy way to know if the link was "opened" by the actual human recipient or by the email company's scanning software.
This has the effect of artificially inflating the click tracking in your mailing reports.
FWIW, there are some ideas floating around out there "on the web" about how to deal with this kind of problem, but these ideas don't have a lot of traction yet.
Most mass-mailing solutions (including CiviCRM) will need work before they're able to implement such ideas to help get more accurate numbers in the face of this kind of "pre-scanning" activity.
So these numbers are just useless?
Not entirely. Because not all email hosting services are doing this kind of thing, the click-through rates on your emails are still good as relative indicators:
if one of your mailings shows a click-through rate that’s relatively lower or higher than your other mailings, that probably indicates that more or fewer real people clicked those links.
But as an absolute indicator of “who clicked what”, yes, these numbers are less useful than you might have thought.
To sum up:
It's a problem that lots of other people are talking about (not just in CiviCRM) but no one has a solid solution yet.
As a result, meanwhile, you should assume that the click-through numbers are artificially inflated.
To quote one blogger on this topic, "Having misleading statistics is just something we have to deal with as a marketer."
More info:
For reference, here's a (rather long) blog post from civicrm.org that tries to explain all of this. It might be interesting if you have the time.
All the best,
A.