We see what we measure

This year’s World Metrology Day is themed around the global food system. Beyond the need for metrics that can capture complexity, this is an opportunity to reflect more broadly on the quantitative paradigm that largely underlies food systems research.

The World Metrology Day, celebrated on 20 May 2023, was dedicated to measurements supporting the global food system. The choice couldn’t be timelier. According to the Food and Agriculture Organization of the United Nations¹, the world is moving backwards in its efforts to end hunger; advancing depends on robust and verifiable metrics that can help us map out food systems’ cracks, allocate resources efficiently, appraise policy impact, and monitor progress towards nutrition and sustainability goals.

Yet, the incessant quest for more and better measurement must be critically assessed. Technicalities tend to become centre stage and end up determining the questions we ask and putting constraints on the universe of answers we consider. Big data, artificial intelligence and other recent developments in information technology have opened up so many possibilities for easier, cheaper and more accurate data compilation that it is tempting to focus on their applications and lose sight of broader questions.

In contrast to conventional disciplines, food systems research acknowledges the multidimensionality of food and the interlinkages among its environmental, social, cultural, economic and nutritional aspects. This holistic approach requires indicators able to capture food systems’ complexity more fully. Besides conceptual and epistemological intricacies, however, the need to develop such indicators without posing prohibitive costs or technical hurdles remains a challenge, as evidenced by countries’ monitoring, reporting and verification efforts in the context of the United Nations Sustainable Development Goals (SDGs)².

Even the most encompassing analytical frameworks employed to explore synergies and trade-offs across SDG objectives have blind spots, considering only measurable variables for which data are available. Latent variables may be embedded in model calibration factors and statistical error terms, but it’s impossible to know exactly what they refer to or how they relate to other factors.

Metrics designed to capture complexity, such as composite indices, carry a lot of information — that is, data that have been processed and made sense of according to a specific purpose, narrative or world view. Even simpler parameters such as prices aren’t deprived of a narrative, as the discussion on the true cost of food — and the fact that food prices do not account for their impact on human and environmental health — so well illustrates. In fact, even the way data collection is performed carries a narrative; previous work has shown that assumptions built into official agricultural databases have reinforced binary and simplistic framings³ in the USA, for instance. Decisions on the type of data to collect and the information that can be derived from them are mediated by varying stances on food security, development pathways and food sustainability strategies that must be openly and clearly stated. As metrics often shape the targets, indicators and thresholds assumed desirable for food systems transformation, metric design should be participatory also for legitimacy and ethics reasons.

The limitations of quantitative measurements must be recognized. Some things simply cannot be measured on a scale unless they are simplified to a point that renders results meaningless. Examples span what economists would call ‘irrational’ behaviour, the value attributed to food, and notions of sovereignty, equity and justice. In such cases, qualitative investigation is paramount, and the fields of sociology, anthropology, human geography and political economy should gain more prominence. Still, specific knowledge systems are disregarded in scientific research for not fitting existing standards⁴, resulting in exclusive, contextually inappropriate and ineffective policy solutions. Efforts to bring together scientists from different disciplines are commendable but might not add much if they force qualitative knowledge into a quantitative frame.

Food systems research is problem oriented and policy relevant, falling under the scope of ‘post-normal science’ as it deals with uncertain facts, disputed values, high stakes and urgent decisions⁵. Unsurprisingly, the solution to the current food system crisis is technological as well as institutional, cultural, political and behavioural. While there’s no doubt that food system metrics should evolve towards greater complexity and interdisciplinarity, being mindful of the limits and biases of quantitative measurement is just as important to move forward.