Otto Dettmer/Ikon Images
Imagine that, as an operations manager at a global retailer, you have invested heavily in data science. Your data science team has built sophisticated models that produce demand predictions for the company’s products with impressive accuracy. Yet, when the CEO, seeking to increase profit, asks you how much inventory should be held and where, the answers are not clear. The models predict demand but do not tell management what to do with those predictions. After investing millions of dollars in data science solutions, the company realizes it has many insights about demand but few decisions to act upon.
This is not a unique scenario. Studies suggest that 60% to 85% of data science projects end in failure.1 The reason is not usually bad algorithms or incompetent data scientists. More often, there is a disconnect between the models that the data scientists produce and the information that business managers actually need. Often, the data scientist does not speak the language of business, and management does not speak the language of the data scientist.2 Data scientists are trained to focus on prediction and accuracy metrics, whereas managers’ focus is on making choices that improve business metrics such as cost, market share, or profit. As a result of this communication gap, models often look elegant on paper but sit unused in practice. As one pair of analytics experts put it, “The number one reason data science projects fail is because businesspeople and data scientists, individually and collectively, don’t understand the real business problem at hand.”3 The focus on the actual business decision is missing.
This is where decision science comes in. Unlike data science, which focuses on modeling the data, decision science models the decisions that business managers must make. This aligns the data model analysis with the business goals. By ensuring that the models are directly related to the business problem, your team can avoid relying only on “convenient data” and tie the model metrics directly to business outcomes.
For business leaders, the message is clear: If your analytics investments are not delivering ROI, stop modeling just your data and start modeling your decisions. Here is what that looks like in practice.
Decision Science Versus Data Science
Decision science focuses on improving decision-making using data: As the name suggests, the emphasis is on decisions rather than on data. In a recent study, we examined academic programs and job ads in data science and decision science, with the goal of determining how their skills differ and/or complement each other.4 We examined required coursework in both data science and decision science master’s degree programs and looked at the skills listed in job ads for each role.
We found that data science places a greater emphasis on skills related to coding, prediction, and machine learning. In contrast, decision science focuses more on directly modeling decisions using data, logic, and mathematics. This skill is known as decision modeling. Decision science, therefore, tends to focus more on problem framing, problem-solving, and business domain knowledge. Decision science relies on data science for predictions and on other data-based relationships as inputs to decision models that ultimately inform and drive decision-making. Your organization needs both skill sets.
The bread and butter of data science is prediction and machine learning. As a previous study of ours confirmed, data scientists are trained with a focus on modeling data, not on decision modeling. As Bipin Chadha, senior vice president of data science at CSAA Insurance Group, said in a presentation at Machine Learning Week 2024, “Many data scientists mistakenly think predictions are decisions.” Indeed, this is the crux of the disconnect between data scientists and management.
This data science disconnect — focusing on the data rather than on the decision to be made — can result in teams solving irrelevant problems and/or using the wrong metrics. This often leads to management rejecting solutions.5
Models do not create value until managers use them. And managers use them only when the analysis reflects the decisions they face. Shifting the focus from data models to decision models compels analysts to engage directly with leadership, sparking conversations that uncover root causes, align priorities, and build trust in the recommendations.
How Decision Models Can Help
When a data science team models an organizational decision, not just the data, it forces communication between the analytics team and management, mitigating a big hurdle to project success. Here is how decision models make a difference.
Decision Models Strengthen Problem Framing
Managers expect the data science team to generate ROI by solving problems and providing the intelligence needed to make better decisions. However, managers often do not have the time to truly understand the root cause of the problem. Managers tend to experience the symptoms of some deeper problem.
A good decision scientist is like a good medical doctor: They must ask a lot of questions about symptoms to navigate to the real underlying problem. Decision modeling forces this discussion and reinforces that the most critical step is identifying the true problem that can then be modeled. The problem must be properly framed before it can be solved.
For example, consider the case of forecasting demand for 50 products over the next four quarters. Your data scientists build a set of 50 highly accurate machine learning forecasting models that include internal and external features. The data science team reports to management that, based on the model validation holdout samples, the mean absolute percentage error (MAPE) on the models ranges from 6% to 9%. The data scientists have done the job that they were trained to do.
As a manager, you may or may not know what MAPE is, but you know that solving the decision problem you face requires accurate demand forecasts. But the decisions you need to make are more complex than that: how much of each product to produce, and how much to hold in inventory to maximize the contribution to profit and fixed costs. Management’s metric is clear: dollars.
The real decision problem, then, is not one of forecast accuracy, though the decision problem is dependent on forecast accuracy. It is something like the following: Maximize contribution to profit and fixed costs while following our business rules.
To model contribution to profit and fixed costs, you will need estimates of the per-unit contribution for each of your products, the opportunity costs of unmet demand, and the cost of holding inventory. Business rules will have to include limited production and storage capacity, material availability, and labor availability. And you will need the demand estimates from the forecasting models because they will inform you about how much you can expect to sell. Importantly, the costs of producing too much and too little often differ significantly, and focusing purely on model accuracy glosses over this reality by treating each error equally.
To model this decision, the decision scientist’s approach would, perhaps, be to use an optimization model in combination with scenarios or simulati
