There is no easy way to measure inflation—but some ways are better than others

The digital era and globalisation have wrought unprecedented changes in the UK economy, with an increasing variety of goods and services being purchased, often online, by UK consumers. In response, the Office for National Statistics has been busy ensuring our measures of inflation reflect these changes, and that we better understand their impact on different households.

Measuring inflation is complex and we’ve often encountered a range of challenges, which have required innovative solutions, but we are pressing ahead and have reached some firm conclusions.

Traditionally, the majority of price quotes are gathered manually once a month. This means people going into retail stores across the country armed with tablet computers. We plan over the next few years to replace many of these quotes with scanner data (collected as products are sold) and prices scraped from retailers’ websites. These new data sources will give us many more price points and better reflect the changing buying patterns of households, including the relative popularity of different products.

Recently, we’ve also been developing inflation rates for different household groups. They show, perhaps unsurprisingly, that different types of households face different rates of inflation.

For example, our latest results based on our headline measures of inflation suggest that, since March 2017, higher income households have faced consistently higher inflation rates than lower income households: this was largely driven by spending on transport and on recreation and culture. However, this goes against the longer-term trend where lower income households have faced higher rates of inflation.

Our headline consumer prices measures, which include the Consumer Prices Index (CPI) and CPI plus owner occupiers’ housing costs (CPIH), for the most part reflect the change in price of acquiring goods and services—in other words, we record the advertised price for an apple or a new car. However, this doesn’t always reflect the actual costs experienced by consumers from month to month; for example, when they are committed to repayment plans such as when they are financing a car.

So, we have been developing a new set of measures called the Household Costs Indices (HCIs), which aim to reflect the changing prices and costs as experienced by different groups of households. We are already measuring student loans based on when repayments are made rather than when tuition fees are charged, and we also capture mortgage interest costs, which are excluded from other measures of inflation, such as CPI and CPIH.

The HCIs also differ in the weight given to items in the basket of goods and services. Our headline inflation measures have a weight that is relative to total UK expenditure on that item. However, recent analysis suggested that this results in a CPI skewed towards high-spending households. For the HCIs, we instead give items a weight which reflects the spending of the average household.

Our latest results show that, on a HCIs basis, over the longer-term lower income households have on average experienced stronger monthly increases in costs and prices than higher income households.

We also found that retired households have faced greater price and costs rises than non-retired households. However, it is important to view these findings in context: incomes outgrew costs for retired households, whereas for non-retired households incomes and costs have kept broadly in line.

Alongside this work, we’ve been improving how we measure price changes that affect businesses, making sure that technological advances are properly accounted for. We have also been working on improving our methods for measuring import and export prices, so we can better measure underlying trade patterns.

It hasn’t all been plain sailing when it comes to improving our inflation measures. We continue to produce the older Retail Prices Index (RPI) measure, despite its clear shortcomings, as we are mandated to by parliament. For a variety of reasons, the RPI usually overstates the rate of inflation, consistently giving a higher rate than our headline inflation measures. Following a public consultation a few years ago, we decided to continue publishing the RPI due to its widespread use but to encourage people to move to better quality inflation measures. Despite this, it continues to be used in a wide array of public and private sector contracts.

That decision was made with the best intentions. We hoped that by highlighting its shortcomings we could encourage its users on to better measures. However, over time it became clear that this policy was not working, and users were not moving away from the RPI as we would have liked. A recent House of Lords Economic Affairs Committee inquiry highlighted that the strategy was not working, with RPI use remaining widespread. In March, David Norgrove, chair of the UK Statistics Authority, wrote to the then chancellor of the exchequer requesting his consent to bring the methods of RPI into line with CPIH. The new chancellor recently agreed to consult on implementing this change between 2025 and 2030.

This proposed change would put us on a sounder footing to provide a set of consumer price statistics produced according to international statistical standards, and which meet the diverse needs for price measurement, providing deeper insight into many aspects of inflation and how they affect different households.