Averting statistical tragedies

NEW HAVEN — In a July 2020 article for Brookings Papers on Economic Activity, Tristan Reed and I showed that, contrary to expectations, COVID-19 deaths per capita were much lower in poorer countries than in richer ones. Readers immediately countered that this finding must be due to mismeasurement or a lack of data for these countries. Our result has since withstood scrutiny and the test of time, but the initial response was revealing: Statistics originating in developing countries tend to be met with suspicion (and often are dismissed outright).

Is this bias justified? In a recent paper for the Journal of Economic Perspectives, “Why is Growth in Developing Countries so Hard to Measure?” my co-authors and I find that it is not. Notwithstanding a few highly publicized cases of data manipulation, growth estimates from developing countries are as reliable as those from advanced economies, on average.

The traditional approach in the economics literature is to look for correlation between estimates obtained using different data sources. Employing this method, we compared estimates based on three distinct sources: the System of National Accounts (SNA), household survey data, and newly available satellite data (mainly on nighttime light, and occasionally on vegetation).

Such comparisons show that the differences in average growth rates across the three data sources are small, typically around 1.5 percentage points or less. Though gaps of this magnitude might be considered large for high-income countries (where annual growth rates have recently been in the 3-4 percent range), they are relatively modest for many fast-growing developing economies. An average error margin of about 1.5 percentage points does not seem grave in view of the uncertainty surrounding these estimates.

Moreover, a new database assembled by the International Monetary Fund and the World Bank shows no evidence that SNA data from low-income countries are systematically manipulated for political reasons. Interestingly, it is middle-income countries that seem more problematic, suggesting that politically motivated manipulation may be more feasible above some threshold level of statistical capacity and sophistication. To a certain extent, these findings are a reason for optimism, because they show that statistics generated in developing countries are indeed meaningful, and that it would be ill-informed to dismiss them out of hand.

Nonetheless, poorer countries could obviously benefit from greater statistical capacity. As Shanta Devarajan of the World Bank argued in an influential 2013 article, low-income countries, especially in Africa, suffer from a “statistical tragedy.” Owing to a lack of resources for data collection, management, and dissemination, and to an absence of coordination among relevant agencies and stakeholders, policymakers in many low-income countries must resort to using old data and outdated methods.

The question is how developing economies can increase their statistical capacity. Given the high costs and uncertainty associated with such investments, they are unlikely to be pursued at a time when governments are already under fiscal pressure from the COVID-19 crisis.

Fortunately, there is a more feasible approach: Harness technology and use new data sources (in combination with traditional sources) to alleviate resource constraints. An explosion of new and publicly available data—from web scraping, Google searches, digital transactions, mobile-phone metadata, social-media usage, and satellite data—has helped researchers estimate important economic variables at a lower cost.

Such data sources proved especially useful during the pandemic, allowing economists to obtain much-needed information about poverty, inflation, businesses’ prospects, and people’s well-being—all in real time. And because these data can be obtained faster and at much lower cost than with traditional methods (such as door-to-door in-person surveys), they are a source of hope for resource-starved developing countries.

It may seem naïve to appeal for better data in the midst of a pandemic, when many low-income countries have still not managed to secure life-saving vaccines for their populations. Yet one of the biggest risks of the pandemic is that it will cause important development agendas to be neglected or postponed indefinitely. Just as policymakers must insist on additional measures to empower women and increase investments in human capital, so must they pursue greater statistical capacity. We cannot improve what we cannot measure.

—Project Syndicate

* * *

Pinelopi Koujianou Goldberg, a former World Bank Group chief economist and editor-in-chief of the American Economic Review, is Professor of Economics at Yale University.