- The Uncertainty Project
- Posts
- đź Prioritization and Uncertainty
đź Prioritization and Uncertainty
Thinking about the Value of Information
Good Morning!
At the Uncertainty Project, we explore models and techniques for managing uncertainty, decision making, and strategy. Every week we package up our learnings and share them with the 1,800+ leaders like you that read this newsletter!
In case you missed it, last week we talked about Bayesian thinking and product risks.
![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,format=auto,onerror=redirect,quality=80/uploads/asset/file/3f00e682-0df4-4a22-9912-b05f6b55bd7d/image__213_.png?t=1707534621)
![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,format=auto,onerror=redirect,quality=80/uploads/asset/file/114b6d0f-96bd-46df-8878-72a6de92a09b/image.png?t=1716307211)
Weâre thrilled to announce a partnership with the Alliance for Decision Education! Co-founded by Annie Duke, the Alliance for Decision Education is an educational nonprofit dedicated to the belief that better decisions lead to better lives and a better society. The Alliance is creating a national movement to ensure Decision Education becomes a critical part of every middle and high school studentâs learning experience. Hereâs the full announcement.
Weâre incredibly excited about their mission and weâll be reaching out to the community with opportunities to get involved, but if youâre interested in getting involved, reply to this email and weâre happy to get you more information!
Upcoming Talk
đ„ Upcoming Live Talk: Nick Drage, a Fractional Strategist at Path Dependence and a âprofessional devilâs advocateâ, will be talking about âGaming in Businessâ. Nick has a background in cyber security and running exercises like wargaming to simulate and prepare for future scenarios. Sign up for the talk on May 29th!
![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,format=auto,onerror=redirect,quality=80/uploads/asset/file/3f00e682-0df4-4a22-9912-b05f6b55bd7d/image__213_.png?t=1707534621)
Prioritization and Uncertainty
Why does the conversation always come back to prioritization?
Any discussion of ways of working with teams, with organizations, with portfolio leaders⊠we end up looking for that magic heuristic that can help us âseparate the wheat from the chaffâ.
And leaders hear this concern all the time, from their teams and peers:
âIâm confused on the priorities.â
âWe have too many top priorities!â
âThose might be your priorities, but theyâre not mineâŠâ
âOur priorities keep changing!â
But it makes sense. Because when we talk about priorities, we are recognizing some basic needs across operations:
A need for focus⊠for good strategic alignment that helps us coordinate our large efforts
A need to deal with finite constraints⊠since there is always âmore to doâ than time and resources with which we can do it
A need for speed⊠to emphasize flow or throughput, by placing limits on the work-in-progress
Solving for these needs requires decisions about priority. And like all decisions, these will drive the allocation of resources. What weâre doing vs. what weâre not going to do. Thatâs why it can get so heated. And also why we canât avoid it.
But decisions made in the presence of uncertainty are harder. More complicated. More frustrating to make and to communicate. So we need better approaches, when faced with significant uncertainty when making prioritization decisions.
Prioritization is about taking a set of options or opportunities and sorting them out by importance and urgency. It usually drives the sequence with which you attack the options. Because if itâs the most important and the most urgent, then you probably want to do it next, right?
But youâll need a way to decide what constitutes importance. That will obviously depend on your context. And even urgency can depend a bit on context, but the key thing there is just to remember to talk about it alongside importance.
The comparison amongst the options will be relative. Relative to each other, which we sometimes refer to as opportunity costs: âI think this thing is important (and urgent) and doing it next wonât starve resources from a better opportunity.â
But how sure are we of this? This is where conversations can get difficult.
Options or opportunities all start out as a twinkle in someoneâs eye. They are just a raw and rough idea. Weâre not supposed to be sure of anything about it, at this point. Just that it seems like it might have some promise.
Usually, we allow for a refinement of ideas via some kind of âintakeâ mechanism - a proto-backlog of stuff that weâre still molding into something we can prioritize. But hereâs the thing:
That molding, or refining, of our understanding never really stops. Thatâs the learning that we are doing all through execution, and which doesnât end until value is realized.
That molding, or refining, or learning isnât free. We spend resources on that. And there is opportunity cost on that. So that choice is part of this âhow to make decisions on prioritiesâ conversation.
Is yields questions about prioritization like:
How should I balance time spent on ideation and refinement vs. pursuing prioritized opportunities?
When does the amount of time spent refining an idea become too much time?
When does the importance and urgency of collecting more information (for refining a new idea) get prioritized over the more mature ideas? Or the ones we are already working on?
To think about these, weâll need to expand our concept of âvalueâ of outcomes and the outputs that get us there (which is what makes an opportunity important and urgent) to include the value of information itself.
If we can expand our (usually economic) model for value to include a concept of the value of information, then we might be able to compare (and prioritize) work-delivering-outputs with work-getting-information. And we know that work-getting-information is usually the cheaper way to reduce uncertainty.
Why should we consider different approaches to prioritization when facing uncertainty?
When uncertainty is low, there is a lower risk involved with committing resources to execute on our opportunities. Weâre likely seeing a straight line from the opportunity to some desired outcome, and feeling confident in our ability to get there. We can put a high confidence level in the expected value of the outcome, or even a version that factors in urgency (e.g. cost-of-delay).
When uncertainty is high, then there is a higher risk involved with committing resources to execute on our opportunities. Maybe we talk in terms of âhypothesesâ for these opportunities. Hopefully, there is enough psychological safety to admit that weâre âonly 40% confidentâ that this proposed output could yield that desired outcome. We can talk about risk.
âRisk is any uncertainty that matters.â
When we are able to drive a dialog about (and even quantify) risk, then our ability to prioritize under conditions of uncertainty gets better. For example, a product manager might survey the uncertainty across their opportunities and start to organize them around feasibility risk, usability risk, value risk, and business viability risk. These expose the assumptions that form the foundation of our opinions about importance and urgency.
When we consider making a decision to prioritize âOpportunity Xâ, itâs a fair question to ask whether we understand X well enough to make the decision now. We may need to compare X against âOpportunity Yâ which is in progress, and (at this point) much better understood than X. Our intuition or hunch or conviction in X might be strong, but itâs not going to be a fair fight head-to-head (for resources), when Y has the upper hand on certainty. And remember, decision makers looooove certainty (we all do).
So how can we decide when to green-light a little more information gathering (i.e. discovery, research, or just study) on X?
Here weâll turn to a concept from Douglas Hubbardâs definition of the Value of Information, from âHow to Measure Anythingâ.
Letâs say that âOpportunity Xâ was a proposed project that would run a marketing campaign (this is Hubbardâs example). In our over-simplified world here, letâs say that the campaign will either be a complete success or a complete failure (check out the book for more real-world examples).
We draw our focus to the Expected Opportunity Loss, which, for an alternative, is âthe cost if we chose that path and it turns out to be wrong.â So weâll start by stating our confidence that this path of doing the campaign is right (60%). Then weâll explore the expectations of impact if weâre right (usually this is all that gets discussed in a project review!) and the expectations of impact if weâre wrong (... cricketsâŠ). We also cover the alternative of âdoing nothingâ where the impact here is zero (âŠsometimes itâs not).
Variable | Campaign Works | Campaign Fails |
Chance of Outcome | 60% | 40% |
Impact if campaign is approved | + $40M | - $5M |
Impact if campaign is rejected | $0 | $0 |
The Expected Opportunity Loss for the alternative where we approve the campaign is:
EOL if approved: $5M x 40% = $2M
For the alternative where we reject the proposal it is:
EOL if rejected: $40M x 60% = $24M
So for this decision, the overall Expected Opportunity Loss is the sum of these two, or $26M.
âEOL exists because you are uncertain about the possibility of negative consequences of your decision. If you could reduce the uncertainty, the EOL would be reduced. In regard to making business decisions, that is what a measurement is really for.â
This is our opening to extend our economic model to consider the value of new information (what he is referring to above as âmeasurementâ). Just like we have traditionally approved projects or initiatives when the return on investment (or value of the desired outcomes) exceeds some threshold, we should consider investing in obtaining (or just waiting for) a bit more information to emerge, when the EOL on a decision is above some preset threshold. Essentially, weâd be saying, âWeâre not comfortable making this call right now, the stakes are too high.â
Compare this to how you might be handling this situation today. A leader comes in with a proposal to invest $2M to yield $20M (they say) over 3 years. They express 70% confidence in this assertion and have dutifully logged a couple risks. Where is the discussion of the coast if it fails? And the comparison to other opportunities through this lens?
Todayâs prioritization approaches usually donât venture into this area of opportunity losses. Like a rerun of âItâs Always Sunny in the Portfolio Review.â
Hubbard defines the Value of New Information as the value of the reduction in EOL it creates:
Expected Value of New Information (EVI) = EOL(before) - EOL(after)
This gives us something to compare the Cost of Information against, as we mold our ideas from a twinkle to a promising opportunity to a funded project or initiative.
Source: âHow To Measure Anythingâ, Douglas W. Hubbard.
The curves suggest that your bang-for buck is very high when you first start molding.
Or as Hubbard puts it, âIf you know almost nothing, almost anything will tell you something.â
Itâs only if you continue too far, seeking perfect certainty, that the value diminishes. Our innate desire for certainty can drive us bankrupt, if we let it.
Itâs also good to remember that our complex environments - where many moving parts and stakeholder interests meets significant uncertainty - is not a place where we can approach prioritization as an optimization problem. Instead, this is a place for âsatisficingâ.
âBecause real-world optimization, with or without computers, is impossible, the real economic actor is in fact a âsatisficerâ, a person who accepts âgood enoughâ alternatives, not because less is preferred to more, but because there is no alternative.â
So the purpose of prioritization is:
To communicate importance (and urgency) across the organization
To âsatisficeâ in the presence of resource constraints
To create an environment in which teams can thrive (i.e. with managed WIP)
What form should a prioritization exercise produce? What is the right context and conditions?
An item is prioritized relative to others in a set. Start by identifying the set. Is it a backlog? A portfolio? A set of businesses? A set of opportunities? Can you set constraints on its size?
The set should be time-bound. Since our concepts of importance and urgency are relative to time, make the set be time-bound (e.g. opportunities for a specific quarter, or for a fiscal year).
The decision-maker for the prioritization should be at the right level in the organization. They should be accountable for the results and the risks for the set as a whole. If Iâm advocating for one opportunity over the others, and I donât have any incentives for the success of the others (only mine, as champion), Iâm not the right person to prioritize it.
A prioritized set of items, bounded by time, is the concept of a timebox. Use it to stabilize your prioritization choices to protect the teams from debilitating churn. Large efforts cross-cutting departments need stable timeboxes over longer durations (e.g. stable for 6 months) than small efforts done by teams (e,g, plans for a month or a sprint).
Priorities are decisions, and itâs okay if they need to change (they will, or at least should, occasionally. Thatâs what a pivot isâŠ)
âHave strong opinions, loosely held.â
But hold them long enough to protect the teams that are executing. Pursue the new information, when the value is there, but when the information leads to changes in priorities, drive those changes to be synchronized to timeboxes, to manage the impact of the changes. Not in an email the next Monday morning.
So as Barry OâReilly has said, what we need is a âsystematic approach to managing the risk of planned work, by gathering information to reduce uncertainty.â Hereâs an idea of a modified Eisenhower Matrix that could help shift the prioritization focus to inclusion in a set (e.g. timebox) and also make the uncertainty more visible. This assumes that there is enough uncertainty floating around to warrant describing the opportunities as âhypothesesâ.
Modified Eisenhower Matrix
Overlay a timebox and a resource-constraint âcut-lineâ on the familiar Eisenhower matrix. Within each quadrant (or set), shift items left or right based on their uncertainty, relative to the others in the quadrant. Look at the value of information to shift items slightly leftward, over the next quarter. Make confidence levels visible to support EOL-based analysis to justify discovery, research, or some deeper study of external conditions. The color codes could suggest these assessments: Is the hypothesis ready for planning (green)? Does the EOL warrant some discovery first (yellow)? Or is it still a twinkle, but with strong agreement that it is a very important and urgent twinkle (clear)?
The active hypotheses could each get a deep dive into their assumptions and risks. You canât manage what you canât see, and this could help shift the management focus to better approaches, given the presence of uncertainty.
Modified Assumption Map
Would this be a better approach? Not sure yet, still need to try it out. But it might be an effective way to cut prioritization down to whatâs essential:
Communicating strategy - whatâs in, whatâs out
Aligning the focus - whatâs now, whatâs next
Decisions to help allocate resources - finite sized quadrants
Acknowledging uncertainty - reducing uncertainty has value, too
Can we avoid prioritization altogether? No, but we can try to minimize the effort we spend on it. We can aim to shift focus and energy from prioritization exercises to learning exercises, where learning truly encompasses both the forming of ideas and the execution of opportunities.
Prioritization under conditions of uncertainty should feel different. It needs a risk-mindset. We should be comfortable talking about when we need to âbuy informationâ and how to effectively purchase it. It recognizes that we will be managing bets: which bet is best? When do they âhitâ? How big or small? This is what we will ultimately be prioritizing. And when we can make them small, that naturally leads to agility and adaptability - exactly what we need.
Prioritization is âfocus managementâ for the organization, which is a communication job. Good communication needs common language and shared understanding of context. Make the uncertainty more visible, such that when needs change (and then opportunities change in expected value), you can steer resources, to drive learning, to reduce risk profiles.
And that kinda sounds like a foreign language, doesnât it?
![](https://media.beehiiv.com/cdn-cgi/image/fit=scale-down,format=auto,onerror=redirect,quality=80/uploads/asset/file/3f00e682-0df4-4a22-9912-b05f6b55bd7d/image__213_.png?t=1707534621)
How was this week's post?We'd love to know what you think! (click one) |
Reply