The original paper is here (predictably the DM don't bother to link to it), and here is the university's press release which has been pretty slavishly copied to create the article. The summary is, the researchers combined crime data from one source (the United Nations Office on Drugs and Crime), with survey data on people's attitudes to religion and other things (from World Values Surveys and European Value Surveys). They found that the belief in heaven and hell is a strong predictor of crime rates at a country-wide level, whilst rates of particular religious beliefs were not.
My main problem with the press release, and consequently the newspaper article, is this sentence:
The finding surfaced from a comprehensive analysis of 26 years of data involving 143,197 people in 67 countries.The implication of this seems to be that the research involves following 143,000 people for 26 years, and seeing both whether they are religious and whether they commit crimes. In fact the number refers to how many people were asked about their religion. This is a nice large sample size, so they probably have pretty accurate information about religiosity in the relevant countries.
However the crime data is taken from a completely different source! The UN numbers are based on national statistics, which are, of course, collected in completely different ways in different places, and at varying levels of accuracy. I would be very wary of comparing rates of rape or burglary between countries based on this methodology, let alone with variables from other surveys.
The second point which occurs to me is that the crime data effectively covers all the people in these countries, so one might as well say that we have "data involving [several billion] people in 67 countries." In fact there isn't really any data on individuals which answers the question being posed, so our sample size is, at best, 67. The sentence quoted above misleads us into thinking the evidence is much stronger than it really is.
Anyway, if you believe the numbers, you might legitimately conclude that rates of belief in hell are correlated with crime rates amongst the countries surveyed. This is shown in the paper's graph, reproduced here:
[Note to all those who present data: don't try to label all 67 points on a graph like this, it gets very confusing.]
What we would like, but do not have, is information at the individual level about whether those people who commit crimes and those who don't believe in hell are the same people. It is quite possible that if you group individuals in some way (such as by country), and only look at the group averages, you get completely different relationships between variables. This is called the ecological fallacy.
To see this, imagine what would happen if the US was treated as 50 separate states, instead of one country, and that all the states occupied pretty much the same point on the graph (the US is the yellow dot at about (11%, 0.8)); this would increase the slope of the best fit line, making the relationship seem more important. Conversely, suppose we did this to Tanzania (the blue dot on the left), dividing it into its 26 regions; then the relationship would seem a lot weaker. Combining the Latin American countries would also make the relationship weaker, because many of them have both high crime rates and high belief in heaven relative compared to hell.
Now it might seem very arbitrary and silly to divide Tanzania up in this way, or to combine Latin America, but the point is that countries are somewhat arbitrary units in the context of the relationship between religion and crime too. Some countries are very homogenous, some very diverse; some groups of countries are very culturally similar, and at other times in history would have been considered a single country (e.g. Norway and Sweden); some countries are much more populous than others (China vs. Guatemala).
Even if we could see a relationship at the level of individuals, this would be much stronger evidence than the country-wide averages, but would still only be a correlation. It could be that people stop believing in hell if they see lots of crime, or if they see people committing crimes and getting away with it. It could be that other variables such as prosperity or unmeasured cultural similarities are important. The authors try to account for these possibilities, but there's only so much you can do.
The original paper carefully mentions both these problems:
[I]t will be important to examine these real-world effects at the level of the individual. The present findings tie rates of belief at the societal level to national crime rates... It is also possible, however, that an intervening variable or variables are at work at the societal level. ... The direct causal explanation is most closely in line with the experimental findings, but it could well be that both the direct and indirect mechanisms are at work. To assess individual-level effects simultaneously with societal-level effects, it will be necessary to collect data with both national crime rates and individual tendencies toward immoral behavior.The press release, however, only mentions the causation problem and not the ecological one, and the newspaper articles mention neither. The Daily Mail's article largely avoids the trap of mentioning individual level phrases (apart from the headline). But because it contains no caveat that (i) country-wide averages may not reflect individual level traits, or (ii) correlation does not imply causation, the overall impression given is that if an individual believes in hell this causes him to be less likely to commit crimes.
Other headlines I've seen include "People who believe in redemption commit more crimes" (Real Bollywood), "Belief In Hell Lowers Crime Rate, According To International Study" (Huffington Post) and "Belief in Hell Keeps Crime Rates Down, According to New Study" (Christian Post), all of which screw up on one or both of these points.
To be clear, I don't have a problem with doing this kind of analysis, but it only constitutes very weak evidence of a direct relationship between crime and belief in hell. It's probably not worthy of a newspaper article at all, but the desire of universities for self-publicity is strong, and therefore the pressure on researchers to get press attention is strong too.
Reference:
Shariff, A and Rhemtulla, M. - Divergent Effects of Beliefs in Heaven and Hell on National Crime Rates, PLoS ONE 7(6), 2012.
No comments:
Post a Comment