Why are so many AI projects failing when they don’t need to be? Various sources are quoting AI project failure rates upwards of 70-80%. At Cognilytica, we have seen thousands of AI projects ourselves and the most of the reasons for these failures can be traced to 10 key reasons. One of those reasons is due to the inappropriate and insufficient way in which those AI projects are being run.
In our conversations with many Fortune 1000 Chief Data Officers, heads of data science and AI, we have been surprised by the responses about how they are running their data-centric projects. Certainly we are not surprised to hear that Agile methodology reigns supreme in all IT shops. More surprisingly is that heads of data science are using elementary-school level Scientific Method as the way in which they are running their multi-million dollar data and AI projects. Really! However, as much as Agile is a best practice for running functionality-driven application and systems projects, it is not a best practice for data-centric projects. Data iterations and model iteration needs are completely different from the needs for traditional app development.
What is even more surprising is that data-centric methodologies have been around since the late 1990s and only now are organizations waking up to the fact that the reasons why their AI projects are failing is not because of technology insufficiency or lack of AI skills, but rather because organizations are skipping necessary steps, shortcutting data lifecycle operations, misaligning business needs and data capabilities, and other aspects of what are easily addressed in data-centric methodologies.
Data-centric methodologies such as CPMAI, which are in turn based on decades-proven methodologies such as CRISP-DM don’t replace Agile, but rather add another necessary tool in the toolbelt of best practices for running AI projects. We dive more deeply into this subject in our latest AI Failure Series podcast. Click here to listen to the latest podcast on this topic and change your AI failures into successes!