At the Uncertainty Project, we highlight tools and techniques for strategic decision making that are either thought-provoking, applicable, or both!

Every other week we package up our learnings and share them with this newsletter as we build the Uncertainty Project!

In case you missed it, last week we teed up today’s post with ‘From data-driven to decision-driven’ It’s not a pre-requisite for this post, but a worthy read nonetheless!

This week

• 🔮 Topic: The 'data-driven' mindset feeds our dangerous craving for certainty

• 🛠️ New Technique: Cynefin Framework - Chris Butler gives an overview of Cynefin, a sensemaking framework that aids decision making by identifying environmental domains and understanding their dynamics

• 🛠️ New Technique: Causal Loop Diagrams - Modeling dynamic relationships and feedback loops in systems, enhancing understanding and decision-making in complex organizational environments.

The 'data-driven' mindset feeds our dangerous craving for certainty

Last week we introduced a concept that isn’t new (mentioned as early as 2010) but, it’s coming back with force - decision-driven analysis over data-driven decision making.

Within the last couple of years (led by Cassie Kozyrkov, Lorien Pratt, Stefano Puntoni, Bart De Langhe, and others), this perspective has surfaced from the decision intelligence community - with roots in AI/ML, not just frustrated ‘business people’.

But the argument isn’t an either/or ultimatum, it’s cautioning that there’s danger in starting with data - especially without interrogation or any defined purpose.

The pursuit of information is often in service of confirmation bias, not learning or understanding - but this isn’t done maliciously. It’s a natural, self-reinforcing tendency to confirm our existing beliefs and feel certain in our decisions.

Certainty is an emotional state that we crave - the alternative, of course, is uncertainty and ambiguity, which is viscerally uncomfortable to stomach.

A self-reinforcing cocktail of cognitive biases provides us with this sense of certainty:

• Confirmation Bias: We tend to search for, interpret, and remember information in a way that confirms our pre-existing beliefs and ignore contradictory evidence. This bias can make us more susceptible to believing false information if it aligns with what we already think.

• Ambiguity aversion: If a piece of false information provides a sense of certainty or closure, there's a chance that the brain might prefer or accept it over a more ambiguous, albeit accurate, piece of information.

• Need for cognitive closure: When faced with an event or outcome, our brains seek causes. This can lead to the invention or acceptance of false causes if no clear, true cause is evident.

• Simplicity bias: accurate information is more complex than false or oversimplified versions. The brain tends to prefer simpler stories or explanations because they require less cognitive effort to process.

• Groupthink: Group consensus can provide a feeling of certainty and comfort, even if the consensus is based on misinformation (e.g. collective illusions)

• Cognitive Dissonance: ‘Uncomfortable truths’ are difficult to accept, so we opt to discount them or explain them away.

• Illusory Causation: Our tendency to perceive a causal relationship between two events that are actually unrelated, simply because they are frequently paired together or occur close in time. This leads people to infer cause-and-effect relationships where none exist.

The output of these mechanisms is the feeling of certainty - and that’s what they’re optimized for. Without them, we’d be cognitively overloaded and paralyzed.

But we need these cognitive mechanisms to drive ‘strong convictions, loosely held’ - this sweet spot between unfounded overconfidence (illusory certainty) and the paralyzing fear of uncertainty.

The idea that perfect information is out there is an empty pursuit - it’s chasing the illusion of certainty.

Towards a ‘Decision-driven’ mindset

A decision-driven mindset, as proposed through decision intelligence, treats the decision as the focal point instead of data. It directly combats the streetlight effect - to start with the purpose, then shine a light instead of starting with the light and searching for purpose.

The means to measure shouldn’t drive our problem space, the problem space should drive our investment in what to measure.

As data calcifies our beliefs and assumptions, we’re more likely to interpret new information in ways that entrench us in these existing beliefs and assumptions - not challenge them.

Illusory certainty may very well be the current epidemic of modern strategy - which is likely why decision intelligence, as a combination of modern data science and decision science, presents a compelling evolution of the data-driven mindset.

So is data important? Of course! Hopefully, no one reads this post and thinks the observation is otherwise - it’s about the application, not the data itself. As Douglas Hubbard explains in his book, How to Measure Anything, there are three reasons to care about measurement:

• When it informs key decisions

• When it has its own market value and could be sold to other parties for a profit

• Entertainment, research, or to satisfy a curiosity

‘Proving we’re right’ and ‘supporting an argument’ didn’t make the list. Informing a decision is a pre-decision point activity even though post-decision justification is often masked as ‘informing a decision’ - often surfacing words like “because…”, “certain”, “says”, “know”, and “will”.

A decision-driven mindset leads with:

• Questions, not answers

• Opportunities, not solutions

• Challenge, not consensus

• Dialog, not arguments

• ‘I don’t know…’, not ‘I know…’

Data is misused, miscommunicated, misinterpreted, or worse - manipulated. It seems in many organizations, data-driven has become a sort of dogma. It’s used as a filibuster to delay progress or to discredit dissenters by masking opinions as fact.

But data is not factual - and sometimes it’s useful. It requires human interpretation and judgment.

Data is often weaponized instead of applied. It feeds confirmation instead of curiosity and is used for convincing instead of suggesting or challenging.

Data is an ingredient, not the truth - it must be interrogated - but our desire to feel certain is stronger than any desire for understanding. To paraphrase the economist Mervyn King, observations are of little value without understanding the process that gave rise to the observation.

Have thoughts or comments? Let us know!

✨ Highlights (Interesting reads):

• Cassie Kozyrkov, Introduction to Decision Intelligence: “Strategies based on pure mathematical rationality without a qualitative understanding of decision-making and human behavior can be pretty naïve and tend to underperform relative to those based on joint mastery of the quantitative and qualitative sides.”

• Lorien Pratt, Link: How Decision Intelligence Connects Data, Actions, and Outcomes for a Better World: “Many decisions made by large organizations, and society as a whole, have reached a level of complexity that has outgrown the capabilities of informal decision making processes. The stakes have become too high, and the game is now played too fast for relying on intuition and luck alone. We need a system that gives us the best chance of winning.”

• Daniel Hill, Data Science at Meta & previously Expedia, Your strategy should drive your data, not the other way round: How the philosophy of science interacts with data science practice and business strategy

• FLUX Review, load-bearing Beliefs (we talk about beliefs quite a bit and thought this was a great term for deeply entrenched beliefs): “Then there are load-bearing beliefs. These beliefs are foundational, and changing them can have significant consequences. They might include convictions like what defines our mission or things that define our identity. These are beliefs that are so well connected in your overall web of beliefs that changing them is painful.”

• Alicia Juarero, Think Safe-fail to thrive under conditions of uncertainty: Alicia Juarero talks about the importance of temporal context and feedback when it comes to understanding dynamic processes, draws parallels to resilience in nature rather than stability, and explores how managers can be catalysts who allow for exploration of the ‘state space’ and evolvability.