New Metrics

We Need to Choose Better Intensity Metrics; Here's How

The availability and measurability of data is critical in determining the right metric. Quantifying ‘social good,’ for example, is nebulous; if the metrics we select give us no ability to measurably improve them, then our work is for naught.

Brand Provided | By Yoram Bernet | Published 1 year ago | About a 11 minute read

Image: Vidal Balielo Jr

This article was prompted by a recent discussion I had with a colleague about the right intensity metric to use for quantifying greenhouse gas (GHG) emissions in the healthcare industry. In particular, I was questioning the use of facility area as the denominator. I’ve been working in the sustainability industry for some time now; and I’ve come to believe that organizations often choose suboptimal intensity metrics — particularly, intensity denominators.

What’s an intensity metric?

The amount of greenhouse gas (GHG) emissions generated by an organization is an important KPI when evaluating that organization's sustainability performance. Total energy consumed is a closely related KPI. Other KPIs common in the sustainability industry include amounts of waste generated or water consumed. These indicators are considered absolute indicators. In the interest of demonstrating social responsibility, most organizations seek to reduce these KPIs.

Absolute KPIs have an important role, but they are also problematic. Consider that most organizations are compelled to show growth over time — reducing KPIs in the form of absolute waste or consumption poses a dilemma for these organizations (For the purpose of this conversation, we’ll ignore the inherent conflict in pursuing both indefinite growth and sustainability simultaneously). Regardless of whether a particular organization is growing or not, there will be natural fluctuations in any organization’s activity due to market fluctuations or other business cycles. To accommodate these fluctuations, it’s common practice for organizations to quantify sustainability performance using intensity indicators or intensity metrics. Such intensity metrics introduce a denominator to express efficiency. For example, a manufacturing organization might use an intensity metric in the form of:

The concept of emissions per widget manufactured as a key sustainability indicator makes sense for a manufacturing organization. However, going back to the premise for this article, what makes “facility area” the right denominator for intensity metrics that express sustainability performance in healthcare?

We’ll proceed with our discussion of intensity metrics under the assumption that using GHG emissions or other forms of waste generated or energy consumed as a numerator is non-controversial. We’ll focus on the denominator, instead. Despite this focus, it’s important to remember that the intensity metrics under discussion must be considered holistically.

What is meant by efficiency?

We stated above that intensity denominators are used to express efficiency. A common definition of efficiency is:

maximum productivity with minimum waste

Going back to the sample intensity metric for the manufacturing organization, we see that waste is expressed in the numerator while productivity is expressed in the denominator. As such, it makes sense that waste is expressed as GHG emissions and productivity as “number of widgets manufactured.” It also makes sense that we would seek to minimize the intensity metric that is waste/productivity.

Going back again to the healthcare example, how does “facility area” express productivity? Is that the right denominator? More generally, is productivity the right denominator?

This is where it gets interesting.

The purpose of intensity metrics

Let’s be clear on the utility of intensity metrics. We’ve established that our intensity metric is intended to quantify efficiency. At the most abstract level, we’re measuring efficiency so that we can drive change to improve outcomes. Many organizations set targets (such as Science Based Targets) for various intensity metrics and work hard to achieve them. While the goal of improving outcomes or improving efficiency is arguably an objective goal, the specific metrics that we seek to optimize are subjective and subject to debate.

Stakeholders and perspectives

The first criteria that we should consider in selecting intensity metrics are the stakeholders that are seeking to quantify performance and the organizations for which performance is being quantified. Different stakeholders of different types of organizations will look at efficiency from different perspectives.

The profit perspective

Executives and board members of for-profit organizations may seek to maximize profit. One definition of “profit” is:

a ﬁnancial gain — especially the difference between the amount earned and the amount spent in buying, operating, or producing something

In this case, a denominator that expresses productivity is ideal because, according to the deﬁnition, proﬁt is maximized when productivity is maximized. Amount spent is a proxy for GHG emissions or energy expended (either of which make up the numerator).

The social good perspective

Directors of social justice organizations will likely seek to optimize their organization’s efficiency in generating social good. For a non-profit healthcare organization, an ideal denominator that serves the social good might be “patient health.”

In the case of manufacturing organizations, it’s unlikely that most stakeholders will be directly motivated to maximize social good. Even if there were stakeholders seeking to do so, it’s hard to imagine a denominator metric for a manufacturing organization that would align with social good. However, given that GHG emissions is a common numerator for intensity metrics; social good is served by minimizing the numerator, regardless of the type of organization.

Similarly, healthcare organizations may be for-profit organizations and as such, profit is an appropriate denominator. It’s safe to say that a single organization may have multiple intensity metrics that stakeholders seek to optimize.

Denominator type vs denominator metric

We’ve considered two different denominators for different stakeholder types: profit and social good. These are metric types rather than specific metrics.

To illustrate, the concept of profit is somewhat abstract; but for the most part, we expect to measure profit in financial terms. Thus, for a denominator type of profit, the specific denominator metric might be revenue. Specific metrics for denominators of type ‘social good’ are less obvious.

So, to determine the appropriate intensity denominator we must consider:

The organization being measured
The stakeholder that is measuring
The denominator type that represents productivity for the organization and the stakeholder

Only then can we determine a specific denominator metric.

Alignment of denominator metric with desired outcome

As we work our way down the decision tree to choose a denominator metric, it’s important to confirm that we remain aligned with the ultimate outcome desired. In the abstract, we’re seeking an intensity metric that quantifies efficiency in the form of a cost/benefit ratio. We’re committed to expressing the numerator (cost) as GHG emissions or other forms of waste or energy consumed. Depending on the organization and the stakeholder, it’s relatively straightforward to choose a denominator type that aligns with benefit. In general, it will be harder to choose a specific denominator metric that aligns with benefit.

For a denominator of type profit, this is not a problem; we previously suggested that ‘revenues’ might be a well-aligned metric. Another metric that corresponds to profit, but is slightly less directly correlated, might be ‘number of widgets manufactured’ (for a manufacturing organization).

On the other hand, while ‘patient health’ might be a type of denominator that corresponds well to the benefit sought for a healthcare organization, what would the specific denominator metric be? Would it be the number of patients ‘healed’? Would it be some quantification of the average ‘wellbeing’ reported by a patient population? These metrics are clearly well aligned with the benefit we’re seeking, but they are elusive to quantify.

It could be argued that ‘facility area’ is an easily quantified, specific metric that aligns with patient health. Healthcare facilities with more area are able, in theory, to treat more patients. However, the line from area to patient health (let alone, benefit) is tenuous.

Availability of measurable data

One of the reasons that ‘area’ is such a common denominator metric across organization types is that it’s easily measured — as is ‘revenues.’ It’s much more difficult to collect measurable data representing patient health or wellbeing.

The availability and measurability of data quantifying a metric is an important criterion to consider in determining the right metric.

Controllability and motivation

The next considerations are the controllability of the metric and executives’ motivation around changing the metric. Remember that the reason we’re discussing intensity metrics to begin with is because we’re trying to drive business decisions and change — to improve the cost/benefit ratio of our operations. We may be setting targets that we expect to be able to meet. If the metrics we select leave us with no ability or motivation to improve them, then our work is for naught.

Going back to the ‘area’ metric, this is a metric that executives can often control. Many knowledge industries (such as consulting firms or accounting firms) operate multiple office spaces and tend to increase or decrease office area as business needs warrant. To be clear, we don’t expect that executives would be motivated to increase area for the sake of meeting intensity targets. However, area serves as an indirect indicator of

staffing levels; so, executives would likely be motivated to staff efficiently based on business needs.

While our focus here is on the denominator of our intensity metric, it’s the efficiency as a whole that we’re trying to improve. Thus, while we might question the value in increasing area per se; the value in decreasing emissions per area is obvious. We’d expect executives to adjust area in response to business needs while simultaneously looking for methods to decrease emissions per area.

Unintended consequences

As we consider metrics that correspond less directly with outcome, we must beware of unintended consequences. Continuing with the example above, a bad actor could conceivably choose to lease warehouse space that is unconditioned (meaning, not needing energy to heat or cool; and therefore; not increasing the numerator or cost part of the intensity metric), solely for the purpose of reducing the overall value of an efficiency metric for which they are accountable.

Granularity

Often, it is easy to find a denominator intensity metric that is directly correlated with outcome if we are granular enough. For example, we could define an intensity metric that is:

This intensity metric is especially interesting in the context of our healthcare example. To illustrate, imagine that we carve out facility area corresponding exclusively to operating theaters. Cooling operating rooms, for example, aligns well with the assumed benefit of surgical healing; and much of the cost (in the form of energy consumed and emissions generated) of operating rooms is expended on cooling energy. In this case, a hospital executive would be motivated to seek efficient methods of cooling operating facilities.

We see that, in this case, defining our intensity metric at fine-enough granularity results in a metric that is well-aligned to our cost/benefit efficiency consideration — which is easy enough to measure and that we can expect executives to be motivated to reduce. While these kinds of fine-granularity metrics can be attractive, they are not without peril.

Peril of metrics that are too granular

It turns out that fugitive emissions from anesthesia gases can account for as much as 50 percent of perioperative department emissions. As such, cooling-related emissions costs — while important — can be a distraction from more productive areas of focus. The organizations and operations that we’re working with are complex systems. The subsystems that make them up can interact with each other in unexpected ways, which argues for looking at organizations more holistically. After all, we should be seeking the overall wellbeing of a population and not the maximum number of surgeries in optimizing the cost/benefit ratio of healthcare.

Summary

We’ve not identified the optimal sustainability intensity metric for use in healthcare or any other industry. We’ve not even defined a rigid methodology for identifying an intensity metric. But we’ve seen that it might make sense to have more than one intensity metric — whether that’s in the interest of serving multiple stakeholders or examining a complex system to get more directly correlated metrics.

Instead, we’ve discussed a sort of rubric — a set of considerations to help us reach a decision regarding the ‘right’ intensity metrics to use for the purpose of evaluating organizations. We’ve decided that the intensity numerator is a metric that quantifies GHG emissions, or other waste or energy consumed. So, the rubric should be used largely to arrive at a denominator metric (recognizing that the intensity ratio must be considered holistically).

Considerations

We list those considerations here — recognizing that we might generally apply these considerations in the order listed; but more often, we will be moving between them iteratively:

The organization being measured — What’s the organization? What are the organizational boundaries?
The stakeholders that are measuring — Who are the stakeholders? What are the various stakeholders’ interests? Are any aligned? Consider defining different intensity metrics for different stakeholders.
The denominator type — What type of metric represents productivity for the organization for the different stakeholders?
Denominator metrics — By now, it should be possible to converge on a short list of candidate metrics.
Alignment — How well do each of the candidate metrics align with the stakeholders’ interests?
Data availability — Is data available for each of the metrics? Is it quantitative? Quantifiable?
Controllability — Are the metrics subject to control? Can executives succeed by making business decisions that will impact the metrics?
Motivation — Will executives be motivated to change the metrics? What unintended consequences might arise?
Granularity — Are the metrics fine-grain enough to correlate with the outcomes? Or are they too granular — obscuring the overall outcomes desired or threatening to interact in ways that are counterproductive?

While this article is not conclusive, I hope to have offered a rubric that the reader finds useful. In subsequent articles, I’ll offer more specifics. I’ll also offer a mechanism for quantifying high-level, system-wide intensity metrics for organizations that produce a mix of goods and/or services.