At the Uncertainty Project, we highlight tools and techniques for strategic decision making that are either thought-provoking, applicable, or both!

Every other week we package up our learnings and share them with this newsletter as we build the Uncertainty Project!

In case you missed it, we posted over the weekend about unraveling uncertainty and complexity.

This week:

• 🔮 Topic: From Data-driven to Decision-driven (Part 1)

• 🛠️ New technique: Simple Rules - Complex systems benefit from "Simple Rules" to guide decisions, emphasizing flexibility, simplicity, and decentralized authority

• 👍👎 Give us feedback! We want to hear how we can make the uncertainty project better - both through this newsletter and through the content we cover on the site. Let us know here!

From Data-driven to Decision-driven

Part 1 of 2

We’ve all heard about ‘data-driven decision making’. The past decade has seen a race for more and more data - as if we just had perfect information, we’d make the right decision. This has often been described as being data-rich, but insight poor.

Taking it to the extreme, it seems paradoxical.

If achieving perfect value-capture from data were the secret sauce to quality decision making, then in theory, incumbents with coffers of data would make all the right moves. It would be like playing chess against AlphaZero.

Of course, this isn’t the case. Intuitively, we know it’s more than some kind of information arbitrage.

In radically uncertain environments we interpret novel situations, establish ‘what’s going on here’, construct a cohesive narrative with incomplete information, and make decisions.

This is the essence of complexity economics - it’s adaptive, not optimizing. ‘Adaptive’ is survival in scenarios where the scope problems are not well framed, information is imperfect, and the variables change constantly (most of which we can’t observe or measure)

The goal in this environment isn’t to somehow know the decision will be right (this is outcome bias), it’s to create the primordial soup for what’s ‘right’ to emerge.

Data is an ingredient, but data alone will not drive decision making without perception and conviction. And conversely, perception and conviction are dangerous and error-prone without data. (unfounded over-confidence)

Cassie Kozyrkov, Chief Decision Scientist at Google, articulates this point well:

So what does it mean to be decision-driven?

The concept of ‘decision-driven’ has been thrown around for a little over a decade. Interestingly, an article in HBR about decision-driven organizations popped up in 2010 (written by folks at Bain & Company) - around the same time we see one of McKinsey’s ‘landmark articles’ on the advent of ‘Big Data’.

That same year, Olivier Sibony, who later wrote Noise with Daniel Kahneman, was leading research at McKinsey revealing the importance of dialogue in tandem with analysis.

From there, through the feeding frenzy of big data and analytics, it’s almost a decade of silence until an MIT Sloan article, Leading With Decision-Driven Data Analytics, resurfaces the term.

In the article, Bart de Langhe and Stefano Puntoni broadly define being ‘decision-driven’ as focusing on framing the right questions.

To summarize their principles, being decision-driven is characterized by:

• Framing questions over finding ‘answers’

• Gathering enough information to decide is more important than complete information

• Exploring unknowns instead of optimizing knowns - avoiding pre-mature convergence by going “wide first, then narrow’

• Identifying data blindspots (data that could impact a decision isn’t available)

• Recognizing historical data may not model future events

Similarly, economists Mervyn King and John Kay talk about the dangers of relying on data and models alone when navigating in environments of radical uncertainty where, “all models are wrong, but some are useful”.

They go on to say:

King is no stronger to uncertainty - he was instrumental in navigating the financial crisis as the governor of the Bank of England. In his book with John Kay, Radical Uncertainty, they provide multiple examples of when a data-driven approach relied on models as predictors of the future, but failed.

In complex systems, we’d be foolish to believe we can account for all (or even most) variables, and as observed through the work of Donella Meadows and others, we can assume that how we believe we’ll impact a system is often wrong - or even completely opposite.

A decision-driven approach leans heavily on abductive and inductive reasoning. It relies on trusted, often competing judgments backed by evidence (often different perspectives on the same data) to create a picture of what’s going on, what it means, and bet on the best opportunities to reveal more information - or drive some kind of desired change.

The path from data-driven to decision-driven

This problem is not about finding answers in the data - the last decade has seen strides in data access, reporting, and discoverability. With AI, answers to questions that used to seemingly require a clunky dashboard will be a prompt away.

The problem seems to be in navigating the unknowns, asking the right questions, and fostering emergence - not chasing targets.

In the next post, we’re covering a few interesting studies that surface potential problems with a purely data-driven mindset and the advantages of ‘decision-driven’:

• How self-reinforcing biases use data to entrench false beliefs

• How teams can often ‘weaponize’ research in support of less cohesive narratives

• How goal blindness can keep us too ‘on track’ and ignore opportunistic situations that can arise in uncertainty

• How data enables our addiction to certainty

✨ Highlights (Interesting reads):

• Dave Snowden’s recent talks on Estuarine Mapping and Constructor Theory are incredibly interesting 👀 - here’s a shorter clip where he gives a succinct overview of the concepts and a longer discussion he has with Bryce Hoffman and Marcus Dimbleby that’s well worth the time.

• The FLUX Review: to Build long-term, you have to remember long-term. “Collective memory is key for the growth and longevity of an organization. This memory is where organizational learning is stored. It enables groups to build and improve over time. Organizations with a robust collective memory can learn from the past. This allows them to act better in the present and plan for the future.”

• OrgDev covered decision architecture among a few other interesting models and frameworks - you can read John’s original post on Decision Architecture here

• Cassie Kozyrkov, Chief Decision Scientist at Google, introduces the concept of ‘Decision Intelligence’ and talks about data vs decisions from a data science perspective. “Data are beautiful, but it’s decisions that are important. It’s through our decisions — our actions — that we affect the world around us.”

And since you made it all the way to the bottom of this post, we’re testing out a tool that helps teams manage, prioritize, and communicate decisions more effectively. It’s totally free, but we’re looking for some feedback! If you want to try it out, hop on the waitlist!

• 👀 The Decision Board - A free tool based on the Uncertainty Project

  (we’re only bringing on a few folks at a time, but giving priority to TUP members)