When someone is staring you in the face and telling you something that is blatantly untrue, how do you figure out whether they’re lying to you on purpose, or merely not able to grasp the reality behind what they’re saying? How do you decide whether they’re liars or fools?
This article of lies from the Kent And Medway Safety Camera Partnership caught my eye recently. In it they make the remarkable claim that since the partnership’s inception, they have saved 37 people from death or serious injury. That’s a lot of people. I found myself wondering “how did they work that out, then?”, and sure enough, they’re more than willing to explain:
“Figures relate to the Partnership’s first 15 months of operation from July 2002 to September 2003 inclusive. In the 3 years prior to July 2002 there were 189 KSI’s (killed or seriously injured) at all fixed and mobile sites, which equates to 79 in a fifteen-month period. In the last 15 months that the Partnership has been operating there have been 42 people killed or seriously injured which means 37 lives have been saved.”
So… how many flaws can you spot in their reasoning? I can see 3, but I’m willing to listen to any comments about any I’ve missed.
Firstly, 2 samples?! By taking just two samples, the number of KSIs in the 3 years leading up to July 2002, and the number of KSIs in the 15 months prior to July 2002, they have reached this conclusion? I wonder if the person behind that logic uses it in everyday life… I’d hate to see their bank balance…
Secondly… 3 years… 189 KSIs in total, therefore 79 KSIs every 15 months, right? Not necessarily. That’s not what the figures say, no matter how much you’d like to believe it. How do we know there wasn’t a trend in those three years anyway? What if those 189 were made up of 90-odd in the first year, 60-odd in the second, and 30-odd in the third. If the trend continued (we assume all trends continue forever, no matter how small our range of samples… see above), then in year 4 there should have been NO KSIs AT ALL. Doesn’t that mean the cameras have KILLED OR SERIOUSLY INJURED
39 42 PEOPLE? Well, no. Not really. The figures don’t say there was a trend like that. But the point is they don’t say there wasn’t one either.
Thirdly, ‘regression to the mean’. Basically a geeky statistical term which means ‘things getting back to normal’. The problem with the logic here, is that the cameras are necessarily placed at sites which have had more accidents than average (something like 4 KSIs in 3 years I believe). So that’s a high point in the site’s history. In the year’s following, the accident rate goes back down to the site’s average rate. The important thing to see here is that the reduction would have happened whether the camera was installed or not. The statistics used to justify the installation of the cameras in the first place are dodgy at best, but to use the fact that the rates decrease after installation as proof that they work is ridiculous, or just plain lies.
Various people are getting bored of me ranting about this, I know. I’m sorry, but for some reason it really gets my goat. I’m not thrilled at being lied to by a public body. I don’t like the way that they make wild claims in big simple sounding press releases that, to be fair, most of the population will swallow without a second thought. Oh and don’t get me started on why they’re suddenly called safety cameras… they’re SPEED cameras, dammit…
Update: Yes, I can make mistakes too… their stats don’t say that the cameras have killed 42 people, not 39 as I originally said.