New Metrics
Groundbreaking Cabot Study Reveals Shortcomings of Conventional Sustainability Metrics

Conventional vs. Context-Based Metrics

The Cabot study classified two types of metrics as conventional: absolute and relative metrics. A third type was described as context-based metrics (CBMs).

Absolute metrics express gross impacts by an organization during a defined period of time, such as total greenhouse gases (GHG) emitted or total water consumed in a year. Relative metrics express performance in terms of some other variable of interest, such as GHG emitted or water consumed per unit of production, per dollar of sales, or per full-time-equivalent employees. Relative metrics are sometimes also referred to as intensity metrics.

Context-based metrics differ from absolute and relative metrics in that they express performance relative to contextually relevant social or environmental thresholds. CBMs dictate, for example, that GHG emissions be measured against limits in the Earth's carrying capacity to absorb them without putting the climate system at risk; that water consumption be measured against the finite limits of renewable supplies; and that other impacts be measured against thresholds of their own kinds — including social ones, too.

Importantly for measurement and management purposes, such limits or thresholds are not just factual or empirical, but normative as well. In other words, they constitute norms or standards of performance that organizations can voluntarily embrace as a basis for assessing and managing their own sustainability.

Here it should be clear that only context-based metrics take real social and environmental limits explicitly into account - absolute and relative metrics do not. Thus, when it comes to assessing the true sustainability performance of organizations, CBMs provide the most rigorous, literal, and authentic measures of performance.

Still, absolute and relative metrics dominate the sustainability measurement and reporting landscape today, as if the measures they provide are robust indicators of sustainability performance. But are they?

Context-Based GHG Metrics

There are currently several context-based greenhouse gas (GHG) metrics in use by businesses today, including the CSI metric developed by BT and Jørgen Randers of the BI Norwegian Business School; C-FACT, developed by Autodesk; and a derivative of C-FACT developed by EMC. A similar tool developed for community use by the Donella Meadows Institute also exists.

All of these CBMs are perhaps best described as climate change mitigation metrics, since the thresholds they employ represent emissions-reduction targets that must be achieved to in order to reverse climate change and restore GHG concentrations in the atmosphere to prescribed levels (e.g., 350 parts per million, or ppm, for CO2).

The climate change mitigation metric used at Cabot is one we developed at the Center for Sustainable Organizations. Similar to the other metrics noted above, our metric identifies a company's "fair share" allocation of permissible CO2 emissions by calculating the size of its contribution to GDP, just as we did for water assessments some time ago, and are now doing, as well, for solid wastes. More importantly, like the others, it also specifies targets for year-over-year reductions in accordance with a GHG stabilization scenario taken from the science of climate modeling.

In the case of Cabot, the stabilization scenario we used is known as WRE350, which prescribes a long-term trajectory of emissions reductions required to reverse climate change and restore atmospheric CO2 concentrations to safe levels (i.e., to 350 ppm). We first pioneered this approach at Ben & Jerry's in 2007.

We started by calculating Cabot's actual CO2 emissions per dollar of contribution to GDP for a baseline year (2005). We then compared results for the six following years (2006-2011), after adjusting for inflation and real growth in GDP, to minimum reduction thresholds for what such emissions would have to be, per WRE350, in order to be sustainable -- in other words, for Cabot to fairly and proportionately contribute to the reversal of climate change and help restore CO2 concentrations to prescribed levels.

Results of Analysis

In all six years, measures were taken of Cabot's CO2 emissions at its four manufacturing plants and corporate offices using the three types of metrics defined above (see Figure 1). Absolute measures were expressed in terms of total CO2 emissions; relative measures were expressed in terms of kilograms of CO2 emissions per thousand pounds of product produced; and context-based measures were expressed in terms of the ratio of actual versus normative emissions per dollar of contribution to GDP.

Among the significant observations and conclusions we reached were the following:

1. As reflected in its context-based scores, Cabot's emissions were sustainable in all six years, no matter what was happening at the same time from an absolute or relative perspective;
2. In only three of the six years ('06, '09 and '11) did performance from both an absolute and relative perspective agree, directionally, with Cabot's context-based scores;
3. In two of the six years ('07 and '10), performance from both an absolute and relative perspective completely contradicted Cabot's context-based scores. Conventional metrics in those years sent precisely the wrong signals about the direction of Cabot's actual sustainability performance;
4. Last, in one year ('08), the absolute and relative scores contradicted one another; in all other years they agreed. In that year, however, only the absolute score agreed, directionally, with the context-based score. The relative score, by contrast, indicated an ostensibly favorable change in emissions intensity, even as Cabot's true sustainability performance that year (per its CBM score) actually fell back a bit.

The results summarized above strongly suggest that conventional metrics only occasionally, if not randomly, correlate with the true sustainability performance of organizations. This is the most significant insight to emerge from our study. In light of it, it should also be clear that the use of such metrics can and does often lead to false positives and/or negatives when it comes to reporting the true sustainability performance of organizations. Yet they are the predominant style of metrics in use today!

This problem pertains not only to greenhouse gas emissions, of course, but to social and environmental impacts of all kinds. Indeed, the use of context-free metrics to describe the sustainability performance of organizations is as pervasive as it is perverse. This study demonstrates that to truly understand sustainability performance, context-based metrics must become the new, best practice in sustainability management.