What is data-driven decision making? Step-by-step guide in 2024

Most successful organizations treat data-driven decision making as a primary objective and pursue it with religious zeal. However, data-driven decision making, the steps leading to it and how AI is changing it are not well-defined.

In an AI-first company, data-driven decision making means

• Strategic decisions are made by a diverse group including executives that rely on a sufficiently comprehensive, high quality set of information. By definition, a bad strategic decision should be important enough to lead to failure of the company.
• There will be operational decisions with significant business implications where out-of-the-box solutions do not produce satisfactory results. These are generated by custom-built machine learning models. These models can be built by running data science competitions, in-house teams or by working with industry leaders.
• Most other operational decisions are handled by continuously learning machine learning models, which produce explainable decisions. Operational decisions are frequent (once a week or more frequently) and not critical (a single mistake is unlikely to lead to failure of the company).
• Operational decisions which can not be automated with good accuracy are delegated to humans.
  • If data is lacking, opinion based decisions are made.
  • If data exists and has been analyzed, decision maker relies on analysis.
  • If data exists but is not analyzed yet, cost of analysis determines whether an opinion or data based decision will be made.

Before settling on this framework for decision making for modern corporations, we need to identify how we can evaluate different decision making models. However, if you like you can directly skip to the sections that interest you:

Why is it important?

Even though it is obvious that data-driven decision making is important, it would be a shame if we were not being data-driven about proving it. Though quantitative data about data-driven decision making is hard to come by, there is significant evidence that

• volume of data generated is increasing. Every day we create 2.5 quintillion bytes of data according to Domo’s analysis in 2017. 90% of the data in the world today has been created in the last two years.
• companies are investing more in data and machine learning technology to get insights from that data.

A relevant survey, was conducted by New Vantage Partners about Big Data investments of companies. Results highlight that businesses are trying to get an edge over each other by implementing Big Data across the company. Here are the figures in the survey:

• Percentage of companies investing more than $500M in Big Data/AI tech has increased from 13% (2018) to 21.1% (2019).
• Percentage of companies investing between $50M and $500M has increased from 27% (2018) to 34% (2019).
• Percentage of companies investing less than $50M has decreased from 60% (2018) to 45% (2019).

How to assess different decision making models?

Decision making has 3 KPIs: Speed, quality, auditability. Lack of any of these can lead to criticial failures.

• Slow decision making cripples organizations and allows competitors to gain market share.
• Bad decisions lead to failure. A bad decision, no matter how well executed, is unlikely to generate value. For example, Motorola’s decision to continue investing in Iridium satellites which cost billions of dollars resulted in a big write-off for the company. Great execution could not change the outcome as mobile communication was providing a cheaper and higher quality alternative to satellite phones by the time Iridium phones were launched.
• If decisions can not be audited, they are hard to improve. Decision quality is difficult to measure at the time of the decision but can be estimated later by considering other alternatives and current market conditions. Such postmortem analysis should lead to better decisions

What are different decision making models?

Decision making models starting from the lowest level of sophistication are:

1- Opinion based

While it provides fast decisions, decision quality can be low which is dangerous for strategic decisions. However, it can be applied for fast decisions making in cases where:

• High quality data is not available
• Cost of analysis is more than the value of a good decision

2- Driven by limited amount of data

Organizations with data quality or availability issues are likely to rely on limited amounts of data to make decisions. This is potentially the most dangerous decision making model as decision makers could use limited data to support their opinions, leading to widely supported but opinion based decisions.

A significant share of consulting projects fall under this category where consultants could be incentivized to find limited data points to support opinion based hypotheses.

Decision quality can be better than opinion based decision making if decision makers are open to new information, take into account psychological biases, vary their approach to making prediction (inside view vs outside view) and look for answers rather than searching for data to support their opinion.

3- Data-driven decision making via manual analysis

For this model, data needs to be available, high quality and there needs to be consensus about the correctness of data. It is the preferable approach for important decisions where timing is not critical and where cost of automation is prohibitive compared to its benefits.

To enable this, companies need to:

• manage their data efficiently and effectively
• ensure the quality of their data
• rely on powerful analytics/business intelligence tools

4- Automated data-driven decision making

Data is available and high quality and there is consensus about the correctness of data and it has been processed so decisions are made automatically. Based on outcomes, the decision making model continuously improves itself over time. However, audit is challenging when reasons for decisions are not provided.

To achieve this, companies need to apply the latest machine learning and AI approaches to build automated, operational models based on their data.

5- Automated data-driven explainable decision making

In addition to automated data-driven decision making, explainable machine learning models are used to make decisions. Explanations help audit decisions and continuously improve the underlying model via manual interventions on top of automated continuous learning.

Why I wrote this?

I spent most of my professional career at McKinsey where data-driven decision making is one of the core principles. It was a great principle but quite challenging to achieve as a consultant without access to internal data. Given my current experience with AI, I wanted to think how we can better define data driven decision making.

How can your company become a data-driven enterprise in the age of AI?

1. Companies need to achieve digital transformation to ensure that they have digital processes. For example, we have a detailed guide on using RPA to digitize and automate repetitive processes. Digital transformation enables the company to generate data which will be used to build models and operationalize decision making. Data needs to be high quality and accessible to enable further transformation.
2. AI transformation can start as processes are digitized. High value processes where mature out-0f-the-box solutions exist are the first areas for AI transformation.
3. As companies deploy more machine learning/AI based solutions, there will be areas where out-of the box solutions are inadequate. Companies need to start deploying custom AI solutions to automate decision making in such areas.
4. As models are more frequently used in decision making, companies need to ensure that they are explainable and free from bias.
5. Along this journey, management needs to embrace data driven decision making as well. Conscious trade-offs need to be made about the effort for data analytics vs the quality of the decision. This cultural change is not a separate step but should be developed as the company’s analytical capabilities mature. A company with mature digital processes can afford to make more of its decisions in a data driven way vs a company with offline processes since it takes significant effort to analyze results of such processes in detail.

After these steps, the company will have achieved the level of maturity described at the beginning of the article. The exact steps to get there will depend on the current maturity of the company. Companies can work with various partners to get support in this journey:

• digital transformation consulting companies to identify the dx roadmap and implement key initiatives
• analytics consultants to set up the analytics framework of the enterprise which allows critical decisions to be completed with a mix of automated and manual analysis
• data science / AI consultants to operationalize company’s data by building custom models that autonomously make business decisions such as processing loans or claims
• Data science competition organizers to help companies leverage wisdom of crowds to build effective models at limited cost

If you have relevant data to solve your machine learning problems but need support from partners in building machine learning models, we can help:

If you have more questions, we are happy to help: