Marketing and Comms
Benchmarks vs. Actual Data: Balancing Accuracy, Effort in GHG Reporting

Due to a growth in demand for sustainability reporting data in recent years, many organizations have begun quantifying and reporting on their greenhouse gas emissions (GHGs). Whether companies aim to confidently address stakeholder questions, identify data-backed opportunities for operational improvement or position themselves as a sustainability leader, GHG inventories are gaining importance in both the corporate and public spheres. But the methodologies for preparing these reports vary and present reporters with significant decisions. 

First, organizations must choose a reporting framework. This involves selecting a organizational boundary, understanding and identifying which emissions-producing activities fall within the boundary, and determining which type of activity data will drive the emissions results. Seemingly simple, activity data often presents complex challenges.

Gathering actual utility consumption data — such as bills quantifying kilowatt hours of electricity consumed at a particular facility — is always the preferred approach. Not only is it the most accurate reflection of an organization’s consumption, but it is also the most actionable; it can be used to drive business decisions. However, as many reporters begin to seek this information from their providers, they often find that actual data is incomplete, suspect — or, as is frequently the case for leased facilities, simply unavailable. 

In the absence of actual data, many turn to energy-use intensity (EUI) benchmarking figures — which approximate energy usage per building area — to estimate activity data. Reporters using this method enjoy complete, defensible results but are unable to use their inventory to understand their specific behaviors or measure emissions reductions from operational changes. Such benchmark data is hardly actionable.

This can leave reporters feeling they must choose between two suboptimal approaches.

Common inventory pitfalls

Facing precisely this dilemma, one of our clients elected to generate two emissions inventories for the same time period — one using EUI benchmarks and the other using actual data — to determine which method would yield more dependable results. Surprisingly, emissions produced by the benchmark-driven report nearly doubled those of the actual data-driven one. Comparing the inventories side-by-side revealed sources of errors and limitations faced by many reporters. Below, we identify the three most common discrepancies and discuss how to resolve them.

• Missing facilities: GHG inventories typically begin by identifying all the facilities the company operates, listing the facilities’ locations, and surveying them to understand which emissions-producing activities occur at each. This key step serves as a foundation from which subsequent emissions inventory calculations follow. Many organizations — particularly large, multinational companies — fail to capture all facilities. Exclusions of this nature inadvertently understate reported emissions. Depending on the size, function and location of these facilities, these gaps can have significant consequences.

• Some facilities will provide unreliable data or no data: Many of the organizations we work with do not own their facilities. When they attempt to acquire actual utility data for these facilities, they find their lessor is either unable or unwilling to provide it. Many lessors have not submetered each office within a building, and simply include the cost of utilities with rent. Even for submetered facilities, property management companies frequently are not accustomed to tracking and providing this information to their tenants. Sometimes, they’ll retain the most recent bill, but be unable to provide actual data to adequately cover the entire reporting period.

• Benchmarks won’t reflect actual behaviors: Any EUI benchmark will inherently somewhat understate or overstate energy consumption. Facilities with outdated equipment or high occupancy rates may consume more energy than predicted by benchmarks; and conversely, those that have adopted energy-efficiency initiatives or are only infrequently operated will consume less energy than indicated by benchmarks. The ratio of conditioned to unconditioned (ex: large closets, unfinished basements, infrequently used meeting rooms) space of any given facility also may deviate from the ratios of the facility set used in benchmarking figures’ underlying calculations. Further, it’s harder to find EUIs for some countries than it is for others. Authoritative sources of benchmarks exist for the US and the UK; but they can be harder to source for smaller countries — and error may be introduced if benchmarks are applied to a geography that wasn’t included in the underlying benchmark data set.

Solutions

What can you do? Follow the steps below to understand how to adopt what we consider to be a hybrid methodology that employs as much actual data as possible, evaluates the quality of that data prior to inclusion, and fills in all data gaps with authoritative benchmarks.

• Step 1: Capture all facilities. Make sure each facility operated by your organization is included in your inventory. Understanding the location (including climate region), size, and principle building function all significantly impact energy use and emissions. Interview regional managers and examine company real estate databases to ensure there are no exclusions. Whether actual utility data is collected for that facility or that facility’s area is used to approximate energy consumption, properly accounting for all facilities avoids material gaps and creates a reliable framework for the rest of the inventory.

• Step 2: Seek actual data first. Unlike benchmarking figures, actual data is actionable — meaning it can be used to drive and measure changes in behavior. In pursuit of this actionability, survey property managers and speak to lessors to understand for which facilities’ actual data will be available. Explain the period for which you’re seeking data and clarify that you’ll need consumption totals — such as kilowatt hours of electricity or therms of natural gas — and not just the cost of these services. Aim to obtain actual data for every facility possible — particularly large facilities, facilities that you know use considerable amounts of energy, and facilities over which your organization can exercise significant operational control. To streamline future inventories, ask your contact to provide this data on an ongoing basis. This greater temporal granularity will give you the opportunity to continually monitor progress and make decisions in real time. Tracking and analyzing this data in a cloud-based sustainability tool will further facilitate this process by identifying outliers, highlighting gaps and extrapolating any missing data.

• Step 3: Wherever actual data is provided, evaluate its quality. Prior to including actual data in an emissions inventory, evaluate its quality to make sure it is both complete and defensible. Complete means the data reflects usage from the entire reporting period. Next, confirm the actual data is defensible by determining an outlier threshold and comparing the data to a benchmark such as CBECS. Our client applied a factor of three: If data was less than one-third of the benchmark, or three-times higher than the benchmark, that data was deemed unreliable. Data defensibility can also be checked by performing approximate calculations for expected energy use for that facility, based on building use and occupancy. For example, a typical desktop computer uses about 100 watts of electricity. If it runs for eight hours per day, 250 days per year, it would consume roughly 200 kilowatt hours per year. If ten employees share this office, each using only a desktop computer and not using energy for lighting or other purposes, we could anticipate 2,000 kWh for the year. But, if this facility reports 3,000 kilowatt hours for the year, and the facility supports fifty employees and a variety of activities, something is amiss. Ensuring actual data passes both completeness and defensibility checks will give you confidence that you can trust the data you’re using.

• Step 4: Wherever actual data is absent, use an EUI benchmark. Use energy use benchmarks to fill in the gaps for the remaining facilities for which actual data is unavailable or deemed indefensible. Benchmarking figures, sourced from a reliable agency such as CBECS, are a credible and commonly used method that avoids understating emissions. Whenever possible, choose benchmarks from authorities that offer EUIs based on granular categories that impact energy use. For example, a benchmark that describes energy use in an office building in a marine climate will yield more specific and accurate results than a broad, country-average benchmarking figure that does not differentiate by climate region or facility type.

• Step 5: Survey each year to identify whether anything has changed. With contacts established and this hybrid methodology in place, subsequent emissions inventories should require less communication and discovery. Each year, repeat the process of evaluating actual data quality and ensure your benchmarking figures and their source activity (typically building area) are up-to-date.

Benefits of the hybrid method

This approach offers three critical benefits: It eliminates data gaps, reduces tedious data maintenance work; and most importantly, focuses on actionability of data.

• No gaps: Using either a benchmark or actual activity data for each facility operated by your organizations ensures you have a complete and dependable base from which you can compare future inventory results.

• Limits data maintenance: Adopting a hybrid approach that compartmentalizes facilities by activity data type limits data maintenance for subsequent inventories. If your facility contact has identified that they are unable to provide actual utility bills for your GHG inventories, you know for future years you only need to understand whether that facility’s benchmarking data (i.e. its area) has shifted. This is especially relevant for small facilities that have minimal impact or over which your organization has limited operational control to begin with. This reduces the tedious tasks of identifying who to speak to and clarifying needs to ascertain whether data will even be available.

• More actionability: Most critically, this hybrid approach empowers reporters to direct their focus and effort towards where they can have the most impact: large, important facilities for which actual data is available. Firstly, this method makes it easy to identify outliers. Since every facility is accounted for, and all actual data has been compared to a benchmark, it's easy to identify which facilities are performing better or worse than expected. Reporters can seek to understand key behaviors or best practices from efficient facilities, and implement those learnings at other locations. Inefficient offices can be targets for sustainability projects and emissions-reduction initiatives. Finally, it’s possible to define relevant and data-backed targets, suitable as science-based targets, for the entire organization and monitor progress towards them over time with confidence.

Conclusion

Actual utility data and energy-use intensity benchmarks each present reporters with limitations and advantages in greenhouse gas reporting. In pursuit of actionability, many reporters aim to use actual data but encounter difficulties in obtaining it. Benchmarks yield complete — but inactionable — results. Instead of choosing one or the other, reporters can adopt the hybrid methodology described above to balance accuracy, effort and actionability — and produce a defensible emissions inventory. The results of this inventory can be used to monitor year-over-year change in a consistent manner, streamline data collection and communication with data providers; and most critically, empower reporters to use data to make decisions.