The Noisy Non-Revival

 A few months ago, when great fuss was being made about a YouGov survey for the Bible Society that showed that there was a revival - especially among young people - in churches, the Church Mouse said that it didn't seem to correlate with experience.

When other criticism of the methodology also came in,  and people pointed out that the survey didn't agree with the Good Old C of E's Statistics for Mission, defences such as that of Dr Rob Edward-Symmons on the Premier website came out.

A bit of an industry appeared, celebrating the new rush of converts into the churches. There were conferences. Some people said how much it matched with their experience. 

The Bible Society has just pulled the survey, and removed its FAQs from its website. YouGov has admitted that the methodology was sound.

Which is a great shame. It would have been lovely to see all these people who were allegedly newly arrived in church.

But there's two lessons. One is that some people suffered from confirmation bias. Any church leader experiencing growth would have been shouting out "WE ARE PART OF THE QUIET REVIVAL". Any church leader experiencing decline would have kept quiet and wondered why God was punishing them or what was wrong with them. So the noise makes it sound like there's a quiet revival.

And the other is - you're only as good as the data you use. Many years ago now, a major retailer was having problems with planning for their "returns" - Internet sales that were then sent back. Every week, tens or over hundreds of thousands of items were reported to be in the supply chain from the stores back to the distribution centre. And yet every week only a few thousand actually turned up. They kept planning for and fearing the avalanche, which would overwhelm the available labour and space for processing returns. It never happened. Eventually, someone wondered where the discrepancy came from. And found there was a system issue - stores could report they were a returning an item, but then discover they could sell it instead. The avalanche of returned items - which would have filled up every store warehouse and every lorry in the supply chain -  simply didn't exist. The items weren't in the supply chain, often they weren't even in the stores. Sometimes they were walking down the street on their new owners.

The moral is - if your data is surprising, there's a good chance it's wrong. If your data is good news you weren't expecting, definitely go and check it again. If there's other data showing completely different evidence - compare the two and find out why.

It's the same reason in a way that one injury or death due to a vaccination makes so much more impact than millions of people being better protected. People shout about the dramatic, not about the undramatic.

Data is good. It can help you make good decisions. Just make sure it's right.

Now get out and worship God like you're supposed to. That, at least, hasn't been withdrawn.