What is the chance that Ben Johnston is guilty after having failed the first test? Greater than 90%, right? Wrong.
By Kent Burkholder
Decision tree analysis has been around for years. It’s rare to find someone who’s never heard of it. Yet from my experience, the tool is vastly underutilized in many companies. Why?
Garbage in, Garbage out
In pushing back on probabilistic analysis people sometimes mutter, “Garbage in, garbage out.” Industry’s track record in assessing the risks and uncertainties has not been stellar and, if we can’t trust the inputs, the outputs must be equally suspect. So goes the argument.
It doesn’t have to be this way. Research has shown that by using expert interviewing techniques to consider a broader range of influences and possible outcomes, our estimates become much better. Like anything, if we practice the techniques, and track and review the results, we improve.
It’s So Subjective
It is. But I would also argue that practically every number in every economic evaluation is subjective, including the future tax and royalty rates (just ask the E&P companies who operate in Alberta!). Somehow we are OK when we put our “best estimate” of all the forecasts into an NPV evaluation but, as soon as we put ranges to those estimates, they become “subjective” (in a bad way). All the numbers are subjective and I mean that in good way, in that the numbers reflect the integration of the relevant, objective data with our years of industry experience and knowhow, combined with our judgment of what could happen in the future.
Gut Feel
Every stakeholder brings a different perspective to a given project, borne of their individual experiences, natural biases, and role within the project. Everyone has a different gut feel how each uncertainty will play out: reserves, capital costs, or whatever. As such, it is difficult to collectively agree on THE ONE scenario to represent all the potential outcomes for the project.
Probabilistic analysis is not burdened by this problem. When we agree the range that might be seen for each key input (uncertainty), the ranges ultimately reflect all the relevant opinions on the team, optimistic and pessimistic. Research has shown that uncertainty assessments become much better when we consider multiple points of view. It’s a great team tool.
Weird Science
Sometimes our intuition on probabilities is quite good. Sometimes it’s not. Consider the following example… Ben Johnston was the first to cross the finish line in the 100 meter sprint in the 1988 Olympic Games. The next day he failed the drug test and was disqualified. Given the following assumptions:
Let’s walk through the math to figure it out. First assume that 1000 athletes attend the games. Based on our estimates, how many of the 1000 take performance enhancing drugs? 10%, or 100 people. 900 are clean. If we test all the 100 drug takers, how many will get caught? 98%, or 98 people out of 100. If we test all the 900 clean athletes, how many will be falsely accused? 7% (100%-93%), or 63 athletes. If we tested all 1000 athletes, how many would fail the first test? 98+63, or 161. Therefore given that any randomly selected athlete fails the first test, what is the chance that they are truly taking performance enhancing drugs? 98 / 161, or 60.9%.
The math is not complicated. By writing it down and walking through it logically, we have a check on whether our brain is playing tricks on us. As a team, we are documenting our assumptions for all to see and allowing multiple sets of eyes to look for flaws in our logic.
What if?
Ultimately we are assessing uncertainties to determine how they might influence our decisions, to answer those “what if” questions that gnaw at decision makers. We already run sensitivities to understand how the value might change if prices are low or high but what happens when all of those uncertainties are changing at the same time? Decision trees can be used to test a much wider view of the entire decision space than straight sensitivity analysis and they do so in a structured way.
Consider the diagrams below. The decision tree shows the nine combinations that result from three inputs of both price and reserves. These same combinations are shown as the orange diamonds in a two-dimensional “map” representation to the right. The blue background space is made up of the myriads of potential combinations of price and reserves. The orange diamonds are the points that we have selected to evaluate. You can see that the diamonds cover the decision space quite reasonably… and with only nine points!
Click to enlarge image
Simple Does It, Sometimes
Decision trees can add value even with little hard data. Discussions amongst experts even around simple trees can yield surprising insights that might be masked in a larger evaluation. As such, it’s a great idea to try to dissect a problem as much as you can into components to better understand how each contributes to the overall decision problem. Take the decision tree below, as an example. A team was trying to decide how best to design an upcoming development well. They were concerned about the potential for a water zone below the main oil producing zone and they anticipated needing to decide where to perforate the well bore. If the water zone existed and vertical permeability in the reservoir rock was high (i.e., water flowed easily upwards), they would avoid the water by perforating only the top of the oil interval, keeping as far away from the water as they could. If the vertical permeability was low (i.e., water couldn’t flow upwards), they would perforate the whole oil interval and get the best rates and recovery.
They mapped the problem out in the simple Oracle tree below. With a brief conversation around the probabilities and potential values, they realized that perforating the top of the interval did not erode that much value even when water was not going to be a problem. Besides, they could always go back into the well later to optimize the perforations.
Click to enlarge image
Real Options
Because of their simple structure and simple math, decision trees are generally considered to be easy to understand and interpret. A short explanation probability and risk-weighted value is all that’s required and, once complete, the audience follows along and contributes.
Consider the tree below. We are debating piloting a new enhanced oil recovery (EOR) technology, the results of which help us decide whether or not we would opt for a full-field implementation. The key uncertainty is recovery factor (RF). The tree is structured in time according to how we anticipate the decisions to be made and uncertainties resolved:
We first decide whether or not to run the pilot;
If we pilot, we’ll get the pilot results, and update our interpretation of how effective the technology might be in a full-field implementation;
We’ll then decide whether or not to commercialize; and
If we commercialize, we’ll get the actual results.
When we interpret RF=25% from the pilot and we decide to commercialize, there remains a 26% chance of a highly negative result. Are we comfortable with this? Probably not. Is there something we could do to limit this exposure or improve the interpretation reliability? Possibly. We might extend or expand the pilot. These insights fall out of decision tree analysis just by walking down the branches and answering those “what if” questions. Invariably we uncover even better strategies than we considered initially.
Add Water and Serve
As an industry, we already run NPV analysis on our key investment decisions, complete with sensitivities. To layer decision trees on top of our current evaluations is not a big leap. Teams will gain greater clarity on the options available and the ranges of outcomes, decision makers will gain greater clarity on the risks and rewards, and ultimately we’ll all make better decisions.
0 comments:
Post a Comment