Why Does AI Fail?: 4 Reasons for AI Project Failure in 2024

Artificial intelligence and machine learning are changing industries and how businesses function with hundreds of use cases. However, developing and successfully implementing an AI project to business processes pose significant challenges to companies:

• 70% of companies report minimal or no impact from AI.
• 87% of data science projects never make it into production.

Whether it’s a self-driving car crash, a biased algorithm, or a breakdown in a customer service chatbot, failures in deployed AI systems can have serious consequences and raise important ethical and societal questions. By identifying and addressing the underlying issues, companies can mitigate the risks associated with AI and ensure that it is used in a way that is safe, ethical, and responsible.

In this article, we’ll run through four common reasons for the high rates of AI project failures and explore real-world examples of AI failures.

1. Unclear business objectives

Artificial intelligence is a powerful technology but implementing it without a well-defined business problem and clear business goals is not enough to achieve success. Instead of starting from the solution for an indefinite business problem, companies must start by determining and defining business problems and then decide whether an AI techniques and tools would help solve them.

Besides, measuring the costs and potential benefits of an AI project is challenging because:

• Developing an AI project and and building/training an AI model is experimental in nature and may require a long trial-and-error process.
• AI models try to solve probabilistic business problems which means the outcomes may not be the same for each use case.

A well-defined business goal can provide a clear idea of whether AI is the right tool or whether there are alternative tools or methods to solve the problem at hand. This can save companies from unnecessary costs.

2. Poor data quality

Data is the key resource of every AI project. Businesses need to develop a data governance strategy to ensure the availability, quality, integrity, and security of the data they will use in their project. Working with outdated, insufficient, or biased data can lead to garbage-in-garbage-out situations, failure of the project, and wasting business resources.

The performance of AI tools and deep learning algorithms deployed in response to COVID-19 pandemic is a good example of the importance of data quality in AI projects. Researchers tested hundreds of AI tools developed for diagnosing COVID-19 or predicting patients’ risk from data such as medical images and concluded that none of them are suitable for clinical use. Most of the problems were linked to data quality issues such as unknown sources and mislabeling:

• Many models used a dataset of healthy chest scans of children as examples of non-COVID cases. In the end, the AI learned to identify kids, not COVID cases.
• In some cases, the AI used text fonts that hospitals used to label the scans as a predictor of COVID risk.
• In some other cases, machine learning models used chest scans that were taken while lying down for some patients and standing up for others. Since a lying patient is more likely to be ill, the AI learned to predict patients’ risk from their positions.

Before embarking on an AI project, companies should ensure that they have sufficient and relevant data from reliable sources which represent their business operations, have correct labels, and are suitable for the AI tool deployed. Otherwise, AI tools can produce erroneous outcomes and can be dangerous if used in decision-making, as shown in the examples below.

Data collection experts can help your business if you do not have good quality data readily available.

You can check our data-driven list of data collection/harvesting services to find the option that best suits your project needs.

For more in-depth knowledge on data collection, feel free to download our whitepaper:

3. Lack of collaboration between teams

Having a data science team working in isolation on an AI project is not a recipe for success. Building a successful AI project requires collaboration between data scientists, data engineers, IT professionals, designers, and line of business professionals. Creating a collaborative technical environment would help companies to:

• Ensure that the output of the AI project will be well-integrated into their overall technological architecture
• Standardize the AI development process 
• Share learnings and experience, develop best practices
• Deploy AI solutions at scale

There are sets of practices known as DataOps and MLOps to bridge the gap between different teams and operationalize AI systems at scale. Moreover, establishing a federated AI Center of Excellence (CoE) where data scientists from different business domains can work together can improve collaboration.

4. Lack of talent

According to a 2019 survey the top challenge for adopting AI for businesses is the lack of skilled data science professionals. Creating a talented data science team can be costly and time-consuming due to this skill shortage. Without a team with proper training and business domain expertise, companies should not expect to accomplish much with their AI initiative.

Businesses must analyze the costs and benefits of creating in-house data science teams. For more on the advantages and disadvantages of building in-house teams and outsourcing, check our articles on in-house AI and ML outsourcing. According to your business objectives and the scale of your operations, initially, outsourcing can be a more cost-effective alternative to implement AI applications.

What are some examples of AI project failures?

IBM Watson for Oncology

A well-known example of an AI project failure is IBM’s partnership with The University of Texas M.D. Anderson Cancer Center to develop IBM Watson for Oncology to improve cancer care. According to StatNews, internal IBM documents show that Watson frequently gave erroneous cancer treatment advice, such as giving bleeding drugs for a patient with severe bleeding. Watson’s training data contained a small number of hypothetical cancer patient data rather than real patient data. According to a report by the University of Texas System Administration, the project cost was $62 million for M.D. Anderson without an achievement.

Amazon’s AI recruiting tool

Amazon’s AI recruiting tool that discriminated against women is another popular example of AI fail. The tool was trained on a dataset containing mostly resumes from male candidates and it interpreted that women candidates are less preferable.

Racial and sexual discrimination in facial recognition tools

AI researchers found that commercial facial recognition technologies, such as those of IBM’s, Microsoft’s, and Amazon’s, performed poorly on dark-skinned females and well on light-skinned men (Figure 1).

Self-driving cars driving in the opposite lane

Researchers demonstrated they they can mislead a Tesla car’s computer vision system into driving in the opposite lane by placing small stickers on the road.

Other

• Canadian tech startup Element AI had faced difficulties in getting its products to the market due to high operational costs with minimal revenue. According to an interview with the CEO of Element AI Jean-Francois Gagné, in 2019, business partners that don’t use their data efficiently and the lack of existing infrastructure to scale an AI model were some of the reasons for these difficulties.
• You can check our article on examples of chatbot and conversational AI failures.

If you want to get started with AI in your business, feel free to check our lists of:

• AI Platforms
• AI Consultants
• AI Development Services

If you have a specific business challenge that is suitable for AI, we can help you find the right vendor to overcome that challenge: