Data gives businesses increased power to make winning decisions. But, good data can still lead to bad business decisions. Leaders can draw incorrect conclusions when confirmatory rather than exploratory data analysis occurs.
Any organization can experience confirmatory data analysis or confirmation bias come reporting time. Confirmation bias is the tendency to seek out, favor, and interpret data so that it confirms one’s preexisting beliefs or ideas. As a company leader, you need to be aware of this and work to avoid it.
Source: Scott Adams, Dilbert.com, 2012
Confirmation bias
If you expect to discover specific results when analyzing your data, you will find examples to prove the results you expected. You will also emphasize data that supports your expectations and downplay or ignore data that opposes those expectations. This action is called confirmation bias and can lead to incorrect data analysis.
“The human understanding when it has once adopted an opinion (either as being the received opinion or as being agreeable to itself) draws all things else to support and agree with it. And though there be a greater number and weight of instances to be found on the other side, yet these it either neglects and despises…”
-Francis Bacon
Once a person adopted an idea or belief, Francis Bacon believed that confirmation bias affected his judgment. The bias would be so strong that even if more evidence existed which opposed his beliefs, he would cling to the examples that supported his beliefs and ignore those that didn’t.
For most of us, confirmation bias isn’t a conscious choice. That’s why it’s critical to realize that even the most neutral individuals can be affected by confirmation bias when making decisions.
Avoiding confirmation bias in data exploration
As a leader, how can you avoid confirmation bias and encourage your team to do the same when interpreting company data?
A Forbes article explains how billionaire investor Warren Buffett avoids confirmation bias when making decisions. Buffett recognizes personal confirmation bias can sway his decisions and seeks out opinions that contradict his own. To combat his personal confirmation bias, Buffett invited one of his known critics, hedge fund trader Doug Kass, to participate in the Berkshire Hathaway annual meeting in Omaha. On top of being a Buffett critic, Kass bet against Berkshire Hathaway stock at the time by shorting it. While some might be afraid to have a naysayer at such an important meeting, Buffett welcomed the clarity Kass’ words and point of view could bring to Buffett’s decision making.
Like Buffett, here are a few ways to combat confirmation bias in data exploration:
- Be aware of confirmation bias when reviewing data and drawing conclusions based on your findings.
- Review data with curiosity about what you can discover rather than a conviction about what it will tell you (exploratory analysis).
- Seek out evidence to disprove your hypothesis when interpreting data and drawing conclusions.
- Assign someone to play devil’s advocate when reviewing your findings and seek to understand their disagreements.
Source: Scott Adams, Dilbert.com, 1999
Other types of bias to avoid in decision making
Confirmation bias is not the only bias preventing leaders or teams from accurately interpreting company data. There are other things that can cause data to be misinterpreted if you’re not aware of and work to avoid them.
OUTLIERS
If you’re attempting to create a predictive model based off of your data, outliers can significantly skew the results leading to an unrealistic picture of what you should expect to achieve in the future.
If you’re reviewing specific data, like leads per week, and the data contains a few questionable outliers, try calculating the average of your data with outliers included as well as the average without the outliers included. This will show you the difference the outliers are causing.
If the impact of the outliers significantly skews your numbers, you may want to disregard them in your analysis so you can make decisions on correct, unskewed assumptions. Regardless, you should look into what created the outliers to see if they reveal a potential opportunity or simple coincidence.
SELECTION BIAS
Selection bias occurs when the data you will review is not chosen at random. While there are times to exclude specific data, like when dealing with outliers, choosing or excluding data for a specific reason can create an inaccurate representation of the results your business is achieving.
For example, if you know you generated more leads during a specific month or quarter and choose to only show that quarter as a progress update on lead generation for the year, you could be succumbing to selection bias.
RUSH-TO-SOLVE BIAS
At times, you just want a quick answer so you can make a decision and move on to the next item that requires your attention. This habit can cause you to only review limited data without digging as deeply into the data as you should. In both instances, you might miss the data’s full story.
It’s worth noting that at times you may feel pressured to make a quick decision. In such situations, try delaying the decision so you have more time to review necessary data. If you can’t delay, make the best decision possible based on the data you have. But be aware that limiting your analysis can increase the risk of an undesirable outcome.
AVAILABILITY BIAS
If you often make decisions based off of the data that is most readily available and don’t dig in further to get the full picture, you are falling prey to availability bias. Availability bias can be linked to rush-to-solve bias. If a decision isn’t urgent, dig into your data further to ensure you’re getting the full story from it and not just a piece of the story.
ANCHOR BIAS
Anchor bias often occurs when someone is unfamiliar with a subject or encounters new concepts, so she “anchors” her responses to the first presented number or piece of information.
A Forbes article about overcoming biases shares techniques to overcome anchor bias which include slowing down, reflecting, and asking more questions.
If you want to make better decisions as a leader when your team presents data about a topic you are unfamiliar with, set aside extra time to ask them questions about the data and the topic and allow for discussion so you can better understand the topic.
Making unbiased decisions is half of the battle when trying to run a successful company. Without the right tools, data collection, exploration, and review can be hard and add to the difficulty of running a company well. With the right approach and the right business intelligence tool, you can make better decisions faster with the data you have.
