I trade in numbers, and am passionate about them. From crime rates to the climate, we need them to describe and make sense of the world around us. But I’ve also learned to be very cautious in their company. They don’t just help us to interpret the world, they can be powerful enough to change it too—and not always for the better.
A focus on one number, a shock balance of trade deficit that was published three days before a general election, is said to have done for Harold Wilson’s premiership in 1970. A particular measure of public borrowing caused the British government to bring in the IMF to bail it out in 1976. More recently, a net immigration figure in June 2016 played a role in the EU referendum decision, as of course did the most famous number of recent years: the £350m a week that we supposedly send to Europe.
Numbers, then, can and do disrupt the course of political history—that’s real power. With that power comes danger, especially where numbers are arbitrary or misleading, which is—in fact—what all those examples of politically powerful numbers were. That alarming trade deficit, released in the run-up to the 1970 general election, was puffed up by a one-off purchase of two jumbo jets, not by any underlying economic problem. The borrowing figures that led directly to the IMF intervention of 1976 were later revised down. Net immigration figures can be driven as much by the numbers of retired Brits moving to the Costa del Sol as they are by the thing that many voters actually worried about, the numbers of migrants coming to the UK for work. As for £350m a week, the less said the better.
Numbers don’t even have to be wrong to mislead. They just have to distract attention from other things that do matter, but can’t be counted or aren’t included in one particular figure. We go through fashions of fetishising particular measures—trade deficits in the 1970s, money supply in the 1980s, public sector borrowing more recently—to the exclusion of all else, crowding important information out of the debate.
Don’t get me wrong—the great power of numbers is often to the good. Huge extra spending on the health service was driven by knowledge of how many people were waiting unconscionable periods of time for an operation, information that had to be carefully counted. Investment in green energy has been driven by seeing what has happened to global temperatures. The government is, rightly, trying to deal with dreadful productivity growth—and only because the numbers tell us there is a problem.
Indispensable, powerful, valuable, dangerous. Which is which? How do we know which numbers should catalyse action and which should be ignored? How do we avoid falling under the spell of the wrong or misleading ones? And when should alarm bells ring? How do we spot a figure that is concealing more than it reveals, or a number which is likely to command disproportionate attention?
x + y = rhetoric
Numbers acquire rhetorical power when they make something that otherwise sounds hazy and general sound concrete and specific. Whether or not you accept the McKinsey principle—if you can’t measure it, you can’t manage it—talking in terms of measurable things certainly makes it easy for you to sound like you know what you’re doing.By talking about something that can be counted, it is easy to persuade the casual listener—who might well be busy, and only half paying attention—that you are saying something solid. And where you are speaking to an audience that doesn’t have the inclination or the energy to interrogate your numbers, they can shield your arguments, however flaky they may be.
Politicians know this intuitively, which is why they spew out figures in every interview, confident they won’t be challenged. Lobbyists do too, which is why the special pleading of every industry and interest is these days entered along with some absurd estimate of its “value to the British economy.” The civil servants who get this stuff are usually savvy enough to file it where it belongs, but it works just often enough to make it worthwhile. Tell the government your sector “is worth £100bn to the economy,” and instead of thinking about how much more worth could be created by setting your workers or factories to something more useful, a nervous minister will instinctively think “who wants to put £100bn at risk?”
By repeating daft numbers of that sort over time, you can, occasionally, make them sink in, seem familiar, and then gradually more plausible. If you want to lodge a statistic—be it true or untrue—in the public mind, then repetition works.
Very often, however, voters endure a blizzard of competing statistics. The 2015 election was typical: every imaginable estimate was thrown at us of how much better or worse off we had become. Each party used different time periods, different income measures, different inflation measures, different just about everything. Usually the numbers weren’t out-and-out false; some were individually informative. Together, however, they created nothing but statistical fog.
To penetrate that fog, a number needs to be clear, concrete, specific, repeated with certainty and clarity. That’s why £350m a week worked so well. The amount of money we send to the EU is a concept everyone can understand. It was clear, simple, repeated. It framed the debate. Distressingly for those of us who care about these things its complete lack of truthfulness seemed not to matter. No amount of quibbling from “experts,” even from the UK’s statistical watchdog, could cut through. Estimates of long-run economic costs of leaving the EU were, by contrast, less effective because they were, well, estimates. Despite being a better guide to the actual consequences of withdrawal, they looked more made up. They didn’t speak of a real thing in the here and now.
Pleasing number = popular official
Much less obvious than the power of numbers on the campaign trail is the force they can exert over public policy behind closed doors. I’ve seen it first hand. A minister will cling on to a convenient number as an entertainer clings to their favourite catchphrase.Some time ago, when I was in the Department for Education, the way ministers bandied around numbers purporting to show how the UK had moved up or down some international league table, was frankly embarrassing. Movements in such rankings can often owe more to what’s going on in other countries than your own. What’s more, comparing education outcomes—unlike, say, football teams—across different countries in a single league table is hard, partly subjective and prone to measurement error. While there is agreement across borders about what counts as a goal, that can’t be assumed when it comes to adequacy or excellence in literacy. But ministers regarded such numbers—alone—as adequate evidence to support policy decisions. The meticulous people producing them were often horrified to see the way in which they were actually used.
Numbers that are hard to pin down are especially prone to abuse. You can claim a policy will cost less than it actually will, or a tax will raise more than it will, but eventually there is a reality check—the money comes in (or goes out) the door as claimed, or it doesn’t. The Office for Budget Responsibility adds extra scrutiny to these sorts of numbers.
But what about a claim that the benefits from a new rail line will be five times the cost of building it, or the benefits of a new regulation twice the cost to companies of implementing it? Here you are in much murkier territory. All manner of assumptions—discount rates, passenger numbers many decades out, hazy “spillover benefits”—are involved. All of these things, however hazy, are important. That isn’t the problem. The problem is that you can push and prod these sorts of numbers until they become congenial. An official who can produce a congenial number becomes a congenial official.
The real value of doing such cost-benefit analyses doesn’t lie in the number that spills out at the end, but on the understanding that comes from the structured evidence gathering process that produces the number. But guess which is easier to focus on?
Things can go wrong even without anyone inside government wanting to cook the figures. Sometimes a number can be a great gauge of what’s going on until it begins to command attention, at which point it goes awry. As with quantum particles, numbers can be changed by the mere fact of being observed.
The simplest examples here concern crude public sector targets. Passes at some particular exam grade might measure school quality pretty well until they are used as the benchmark for general success. At that point, rather suddenly, schools might start to flatter their performance by throwing all their efforts at children who are close to that grade while disregarding the rest. The target distorts behaviour rather than measuring quality.
The same sort of thing can easily happen when hospitals start targeting particular measures of waiting times in A&E, or times spent on waiting lists. Government departments are especially prone to playing the numbers game. Targets for the number of apprentices, amount of foreign aid money spent, number of immigrants, end up driving policies which ignore quality, value for money, collateral damage and, all too often, plain common sense.
Take the current net immigration target. In the first place it’s a bizarre target being the difference between numbers leaving and entering the country. If thousands of highly-trained British engineers were to emigrate to ply their trade elsewhere, that would be a triumph. Second, as a bald numerical target it takes no account of who is entering the country. Reducing the number of foreign doctors coming in to work in the NHS “helps.” Third, there are all sorts of fuzzy edges around who counts as a migrant. A student who is only here in term time, and who always planned to leave after two or three years, would very likely not meet the commonsensical definition of a migrant for many of the people who are most worried about immigration.
The problem goes even deeper, however. Sometimes a number’s relationship with reality can break down without anyone consciously manipulating it. Back in the 1970s, the economist Charles Goodhart formulated the law that “any statistical regularity will tend to collapse once pressure is placed upon it for control purposes.” If everyone knows, for example, that the government is targeting some particular exchange rate, this will affect expectations and behaviour in financial markets, which in turn will warp not only the exchange rate itself, but also its relationship to those other things—like inflation or employment—which the target was originally intended to get a grip on. At this point, even the most enthusiastic of number-crunchers might be tempted to give up.
Principles of sound arithmetic
There is, however, no need to despair. Numbers can still do invaluable work so long as we know how to set about them. No magic tricks are required to avoid the pitfalls; often there isn’t even any need for special mathematical training. Stick to three simple principles, and you can see off most of the problems.The first is to serve all your figures with a generous dose of humility: be clear about their limitations. Numbers are often complex constructions. GDP is the classic example. It summarises a vast quantity of information in a single statistic, a process that inevitably lends itself to a spurious degree of precision. It measures many of the things it does include with error, while there is much that we care about in an economy that it doesn’t include at all.
It remains useful—arguably the best summary statistic we have—but it needs handling with care. It is not the sole benchmark of economic performance, and should always be read alongside other numbers on earnings, unemployment, inequality, health and much else besides.
Even with far simpler statistics, measurement error can distort things: so be alert to it. Crime figures suggest rising rates of rape may actually be about improved reporting. We need to get these numbers right, and improved reporting is a good thing, but we should not take these numbers as evidence that the world is becoming a worse or less safe place. Keep in mind that most numbers are fallible guides, and be humble about what you claim on the back of them. That way, you remain in charge.
A second rule, in painting by numbers, is to paint with an appropriately broad brush. Don’t imagine that by adding ever more decimal places you are conveying anything important; it is more likely you will be confusing yourself and everyone else. Politicians giving speeches and journalists writing headlines will hanker for precision: President Lyndon Johnson is said to have berated some poor economist offering him a range of forecasts “ranges are for cattle, give me a number.” But precision is not the same as accuracy.
Precise numbers may sound solid—this is the rate of economic growth this quarter, or the impact of eating cabbage on your life expectancy—but that doesn’t make them certain at all. By contrast, some of the qualitative conclusions scientists and economists are most confident about—that all else being equal, the UK will grow slower outside the single market, or that the world is warming as a result of man-made climate change—are discussed as if they were clouded in doubt. Although not quite 100 per cent certain, these sort of claims aren’t far off. The exact impacts, on the other hand, are highly uncertain. In public debate, the inevitable arguments about the exact number somehow end up clouding the solid general conclusion.
Precise numbers, then, can drive us to unwarranted certainty, and the lack of them to an unwarranted sense of ignorance. Better to banish the false allure of precision, and concentrate on the all-important thing: being broadly accurate.
Closely related is the third and final principle for handling numbers. When asked if economists needed to be mathematicians, Keynes replied “No.” What they needed instead, he said, was “a sense of proportion.” The real problems arise when that deserts us, and we get hung up on the latest bump up and down, and lose sight of the bigger picture. We may not be able to measure real income growth very precisely, and we may go wrong in judging whether it has edged up or down in the last few months. We can, however, be sure that across the last several years, real incomes have fared very badly indeed.
A world of rolling 24-hour news exacerbates the dangers. It loses track of the important numbers—that paint the big picture—in a blur of shiny new ones. It pours energy into understanding why this quarter’s GDP growth was 0.1 per cent more than forecast, and forget that growth over the last decade has been the worst in more than 70 years.
Without a sense of proportion, and a willingness to take the long view, blips in the data—such as sampling variation, reversions to the mean—will get mistaken for new trends. When you install a speed camera at an accident blackspot you can easily be fooled into believing it has been effective because the number of accidents falls. But this may be no more than a return to a normal number after the unusually bad year which led you to judge it a blackspot.
Without a sense of proportion, you will also be susceptible to perhaps the single most widespread misuse of numbers—the use of proportionate changes without any reference to the base. What do I mean by that? Well, when you hear that eating a certain food “doubles your risk” of some form of cancer you are likely to be scared. But what the typical scare story in the press omits is the baseline risk. If my risk is increased from one in a thousand to two in a thousand that’s a doubling, but not one to worry too much about. An increase in risk from 10 per cent to 11 per cent is only a 10 per cent increase in risk but is, or should be, ten times as worrying.
This is one member of a wider family of errors, involving the muddling of relative and absolute numbers. Sorry to go back to Brexit, but it again provides the perfect example of misuse. The UK is often said to be more important to the EU than the EU is to us because “they export more to us than we do to them.” Now it is true that exports from other EU countries amount to £80bn more than our exports to them. But that does not make us more important to them than they are to us.
The economy of the rest of the EU is about five times bigger than is the UK’s economy. So their exports to us would need to be five times bigger than our exports to them to make us a more important market for them than they are for us. That £80bn absolute difference is in fact 30 per cent in relative terms, not the requisite 400 per cent. Trade between the UK and the EU is, quite obviously, much more important for the UK than it is for the rest of the EU.
With numbers, and politicians, like these the only route for us—the public—is to be vigilant; to recognise the pitfalls and be sceptical about claims made; to be aware of the problems of league tables, mean reversion, lack of baseline data and the rest. That’s a big ask though. What we really need is a media which can do much of that for us by not publicising obviously dodgy numbers and offering an intelligent critique of those it does publicise. With one or two honourable and very specific exceptions our media signally fails in this duty.
If this all sounds terribly negative, and perhaps rather odd coming from someone who spends his life pouring numbers into the public domain, then consider this. That’s why I and my colleagues at the Institute for Fiscal Studies do what we do: try to interpret, interrogate, and explain the important data in as clear and objective way as possible. Many others in universities, think tanks and charities do the same, and so, away from the public eye, do noble officials who try and ensure that ministers are not bewitched by a single eye-catching number, but instead make their decisions on the basis of the full statistical picture.
Expert interpreters of the numbers are, without doubt, very sorely needed. But merely by paying attention, and keeping their eye on the big picture every thoughtful citizen can do their bit to protect themselves from being crunched up by the numbers.
