It used to be said that a penny saved was a penny earned, but rising prices helped push out the penny years ago, and with inflation now running at well over seven per cent, it’s eating into every nickel and dime.

So while inflation is clearly happening, accurately capturing in a single number all the various price changes across the Canadian economy is much less straightforward. 

“It’s a very, very tough concept to measure, particularly in more or less real time,” said Derek Holt, head of capital markets economics at Scotiabank. 

Getting the inflation number right is important because so much is based on it, including Bank of Canada rate decisions and cost-of-living adjustments for pensions and some social payments, as well as being widely used as a reference for wages, rent, and child support payments. 

There have been numerous method changes to make it more accurate since the government first started to track inflation in 1914 as war in Europe, record-high immigration and land speculation were driving up prices, but the data still attracts a fair amount of skepticism.

The Bank of Canada and others have noted that Canadians often think inflation, as measured by the consumer price index (CPI), is running hotter than official data, but it’s not just average consumers who find it hard to trust the numbers.

Holt wrote a report in December about the “fake data” Statistics Canada was providing for its inflation rate that is “plagued by uncertainties," while businessman turned blogger Frank Giustra wrote in a Toronto Star column in January about the “fantasy math” of U.S. inflation data produced by an agency using “deliberate obfuscation” to keep the number low. 

The Bank of Canada relies so much on the data that it has periodically checked in to try and gauge for itself just how accurate the number is, and the most recent answer, going back close to a decade, is that CPI probably overstates inflation by about 0.5 per cent. 

The sources of the potential error in creating the number point to just how tricky it is to keep on top of tracking what price increases the average consumer faces.

A key source of error is the substitution effect, where consumers will switch products when their normal choice gets too expensive, such as from sirloin to ground beef. The CPI will still be measuring how much the price of sirloin has changed, but the shopper has already moved on.

The list of more than 500 goods and services that Statistics Canada tracks stays fixed month to month so that there can be a clear comparison of products, said Taylor Mitchell, a senior economist at the federal agency.

“We only compare, month to month, like products and we ensure that they really are truly like."

That includes correcting for product weights and quantities to account for any shrinkflation, she said.

To minimize the problem of shifting consumer habits, which changed especially fast during the pandemic, Statistics Canada moved to updating its product list every year. The change, which took effect with May's 7.7 per cent inflation read, also allows the agency to update how much each product influences the overall number, with things that people spend more on having a bigger effect.

Statistics Canada also used to gather in person about half of the 93,000 prices it collects monthly, but the pandemic helped accelerate a shift to using receipt data provided by retailers, as well as more prices sourced online and by phone. The increased data helps the agency know what the most popular products are, said Mitchell.

“That's really the goal, identify the products that consumers are actually buying.”

Over the years that’s meant dropping products like anvils, coal, canned strawberries and DVD rentals from the list (though bowling fees have still made the cut), while adding electricity, automobiles, and cannabis. 

Other sources of potential error can be trickier to fix, such as how much to account for changes in quality. 

While complicated, especially for things like computers and cars, there are established ways to do it, said Elizabeth Abraham, chief economist at Statistics Canada’s consumer prices division. 

“These are the standards that are applied across all statistical agencies across the world.”

But even with standards there’s room for interpretation.

A key criticism of Holt’s was that Statistics Canada wasn’t separately measuring used vehicle prices that had shot up during the pandemic.

Starting in May the agency did start to add them, but Holt pointed out that it is using JD Power data that showed used car prices up about 25 per cent in March from a year earlier, while Blackbook figured they were up 39 per cent. Statistics Canada also decided to use a wider age of car than the U.S., and gave them a smaller influence on the overall inflation number than the U.S. 

Just how Statistics Canada decides how to make sure it’s comparing similar used vehicles in terms of quality is also still a “black box,” said Holt. 

“Nobody outside of the agency can judge how they did it.”

Canada also treats housing differently than the U.S., he said, with the U.S. figuring how much a homeowner would have to spend to rent their house, while Statistics Canada tries to figure out how much it costs to maintain a home. 

Because it’s trying to measure consumption, the agency doesn’t factor in home price increases that are seen as an investment, but it does count mortgage interest costs, as well as the price of latex caulking tubes.

“There will always be these multiple questions about methodological differences even across countries with core standards,” said Holt. 

But while there will always be some room for error in the measure, the perception of inflation being off by consumers could be attributed to a much simpler answer, which is that the consumer price index is trying to capture what the average Canadian will see, said Mitchell.