AI picked the winner. Who set the rules?

An AI-ranked design decision is only as sound as the priorities behind it, and sustainability professionals are well placed to ask what got left out.

Imagine a product team comparing two designs: a sealed device and a modular alternative that can be repaired. An AI-supported assessment ranks the sealed version first. It meets the manufacturing cost target and scores well on reliability. The recommendation arrives with a neat explanation. The meeting can move on.

Then someone asks whether the comparison included repair costs and product lifetime.

This is an illustrative scenario, but the question is familiar. Before we celebrate a faster decision, we need to understand what the business asked the system to value. A ranking can look conclusive while the priorities behind it remain unresolved.

For sustainability professionals, this creates a demanding task: examining the recommendation while working with colleagues who may already feel relieved, vindicated or under pressure to proceed.

What did we ask it to value?

In our scenario, manufacturing cost is a visible input. The potential benefits of repairability will influence the ranking only if the assessment includes relevant consequences after the sale. Those benefits still depend on assumptions: will customers repair the product, can parts be supplied, and does extending its life reduce environmental impacts across the lifecycle?

Engineering might reasonably prioritise reliability. Procurement needs supply continuity. Service teams need a design they can support. Sustainability brings consequences that may sit outside the immediate budget or planning period.

Someone must decide how these considerations enter the assessment. Mandatory requirements should remain explicit constraints; other priorities need an agreed basis for comparison. A compelling explanation cannot settle a disagreement about what matters.

In their 2019 paper, Problem Formulation and Fairness, Samir Passi and Solon Barocas examined how a corporate data-science team translated business objectives into problems that could be modelled. Drawing on six months of fieldwork, they found that defining and operationalising those problems involved negotiation and discretionary choices, including the selection of target variables and proxies.

Once those choices have been made, a second question arises: does the explanation help colleagues assess the recommendation, or make them more willing to accept it?

Research by Gagan Bansal and colleagues, presented at CHI 2021, examined AI-assisted decisions across three experimental tasks involving review sentiment and logical reasoning. The researchers found that explanations increased acceptance of AI recommendations, including incorrect ones. However, explanations did not significantly improve human-AI team accuracy compared with simply displaying the AI’s confidence.

When a sound ambition meets insufficient evidence

Agreeing on the criteria still leaves another challenge: whether the evidence is strong enough to compare the options.

In a consumer-product project I worked on, we proposed introducing recycled material into selected components. The environmental rationale was to reduce the use of virgin material. Existing customer research mentioned interest on recycled content at a broad level, but provided insufficient validation for this particular change. Technical questions about the new material and product specifications also required further work.

The proposal was postponed.

Here, the competing considerations were tangible: reducing virgin-material use, establishing technical suitability and demonstrating relevance to customers. General interest in sustainability could not answer the specific product question. The available evidence left too much unresolved to proceed.

AI's application to this example is hypothetical. In a similar situation, an AI-supported assessment would need to preserve that uncertainty. A broad consumer preference should remain separate from evidence about a particular component change. Technical suitability would still require validation. Producing a more polished business case would leave those gaps intact.

The emotional challenge for a sustainability professional is understandable. Postponement can feel like losing momentum on an important commitment. Yet recognising a colleague's unresolved technical concern gives us a specific problem to address. Treating it as resistance risks leaving the evidence gap untouched.

What are colleagues being asked to risk?

Return to the hypothetical design meeting. The engineering lead may be accountable for failures in use. A procurement colleague may face a launch deadline. The sustainability professional may feel responsible for consequences that otherwise receive little attention.

These are possible pressures, not diagnoses of colleagues' motives. They suggest a useful question: what would each person need to know before supporting a different choice?

Questioning a recommendation can also become uncomfortable when someone has invested time or credibility in producing it. A request to revisit assumptions may sound like a challenge to competence. Sustainability professionals can reduce that personal tension by making the question precise: “Would the ranking change if we included service costs over the expected product lifetime?”

Amy Edmondson’s 1999 study of 51 work teams within one manufacturing company found that team psychological safety was associated with learning behaviour. The study described psychological safety as a shared belief that the team is safe for interpersonal risk-taking.

In practice, the person chairing the meeting can ask each function to identify one assumption that needs checking before the recommendation is accepted. This gives questioning a legitimate place in the process, including when the result favours sustainability's preferred option.

What would change our minds?

A useful next step is to examine the conditions under which another option becomes preferable. In the repairability scenario, the team could compare different repair uptake assumptions and service costs, using validated calculations and appropriate lifecycle evidence. A language model could help organise the questions and summarise supplied evidence, with sources checked by the relevant specialists.

If the ranking changes under plausible assumptions, decision-makers should see that sensitivity. If critical information is missing, the next action may be a test or targeted customer research. Teams can agree on the evidence required and who will obtain it before revisiting the choice.

Sustainability professionals have a valuable role in connecting these questions across functions. We can ask where a cost moves, who carries a risk and which consequence has disappeared outside the assessment's boundaries. We must also accept answers that challenge our own preferred solution.

Before approving an AI-supported recommendation, record the criteria, unresolved assumptions and accountable decision-maker. After implementation, compare actual results with expectations, including reliability, service demand and environmental performance where relevant.

In our hypothetical case, AI picked the sealed design because the assessment prioritised manufacturing cost and reliability. Including repair costs and product lifetime might change the result. Before accepting the winner, the team needs to agree whether those rules reflect what the business wants the product to achieve.

Get the report

Upcoming Events

Wednesday, November 18, 2026
Beyond the Pilot: What It Really Takes to Scale Food Loss and Waste Solutions
Webinar
Sponsored by CARE
More Information

December 2-3, 2026
SB Member Network: Selling Sustainability and Shifting Consumer Demand and Behavior December Member Meeting
Member Event
More Information

February 2-3, 2027
Enabling the Good Life Summit
US Event
More Information

Related Stories

Now Every Business will have to Examine the Ethics of Artificial Intelligence ARTIFICIAL INTELLIGENCE FOR SUSTAINABILITY
Now Every Business will have to Examine the Ethics of Artificial Intelligence