Artificial intelligence (AI) applications give organizations the opportunity to reduce employee workloads, free up time for more complex tasks, reduce human error, help humans make faster decisions, and improve customer service and customer engagement. However, many organizations don’t realize the benefits that AI can provide. According to some statistics, over 80% of AI projects fail to provide any real benefit or positive ROI. The key to avoiding these failures is to use established best practices for AI & Data project management.

With all the talent, resources, and effort being put into AI projects, the failure rate should not be so significantly high. Indeed, there’s substantial good technology and talent that knows how to put technology to use for AI applications. The problem stems from the lack of following best practices for managing AI projects. Project managers are misapplying AI to the wrong problems, missing significant data quality and quantity problems, taking too long on project iterations, and jumping the gun when it comes to AI implementation and operationalization.  In order for AI projects to be successful, AI and data project managers need to follow established best practices for data and AI project management. 

Key Challenges for AI Projects

Despite the fact that AI has been around for many decades, AI has not yet reached significant levels of adoption outside of narrow applications in computer vision (CV), natural language processing (NLP), and aspects of predictive analytics and pattern and anomaly detection. And even in these areas, many CV and NLP implementations are seeing results that don’t meet required levels of performance return. 

Understanding the problem and business value

The cornerstone to any AI project is the need to understand and identify what problem the organization needs to solve, and whether or not AI is the best solution to that problem. Demands from upper management, external pressures, or teams feeling pressured with timing, push teams to rush into AI projects prematurely. Without a firm grounding in how AI will be used in the real world and how it will add value to the organization, project managers end up wasting a lot of time, money, and resources building a useless AI system. 

People often don’t realize the limitations and risks of an AI system and the fact that AI might not be the best solution to your problem. There are many credible alternatives to AI that could provide similar benefits from rules-based systems, non-intelligent forms of automation, and even better use of human labor and skill. If you can use automation, program your way to the solution, or use people to approach a problem more quickly and cheaper than the AI alternatives, those non-AI solutions will be a better fit.

Many AI projects fail because they fail to deliver their promised benefits or returns. Before you embark on an AI project, an organization needs to make sure that they perform a cost/benefit analysis to make sure your return is worth it. What are the criteria for project success? What are the total costs for the AI project, not just during development and first operationalization, but across the whole AI lifecycle?

Lack of sufficient quantity and quality of data

Data is at the heart of making AI work, so it should come as no surprise that AI and ML systems need enough good quality data to “learn”. In general, a large volume of data is needed, especially for supervised learning approaches, in order to properly train the AI or ML system. The amount of data needed may vary depending on your project, the algorithm you’re using, and other factors such as in house versus third party data. For example, neural nets need a lot of data to be trained while decision trees or ​​k-means clustering don’t need as much data to still produce high quality results.

Far too often, organizations jump right into AI projects without first addressing and accessing their data including figuring out the amount of data they have, where these data sources are coming from whether it be internal or external third-party sources of data, requirements for data access, the need to augment existing data including data labeling, and other crucial factors and questions that should be addressed before beginning the project. 

Another critical factor that leads to AI project failure is poor quality of data. The quality of the data that is being fed to the AI system is extremely important for the system to learn and create an accurate model. The adage “garbage in is garbage out” applies in a significant way with AI projects. Data Preparation is a critical step for AI projects to make sure it’s in a usable state. AI projects need to devote significant time and resources to cleaning, transforming, and manipulating data, making any modifications if needed of third-party data, deciding if human-involved data annotation and manipulation (“labeling”) is needed, and performing  any additional data augmentation steps as needed. This is one of the key established best practices for AI & data project management.

Underestimating time and cost of the data component of AI projects

Organizations often underestimate the amount of time and resources it takes to really run AI projects. Far too often projects get started without first addressing data needs and accessibility. Once they get to that step in the process they are often stalled by lack of access to needed data, needing to send data out for labeling, or internal quarreling over access. It’s critical to obtain the data needed to fuel AI projects. Since AI requires a data-centric approach, not having the sufficient resources or time to work with data will result in poor AI project outcomes. Organizations need to carefully consider if they are able to take the time and money to provide their projects with enough data and good quality data. 

Leveraging Established Best Practices for AI & Big Data Project Management

Project managers and teams that want to avoid key challenges for AI projects outlined should focus on getting up to speed not only on emerging AI best practices, but established big data best practices that have been developed over the past few decades for project success. The complexity of AI and advanced big data projects is often underestimated. This gives managers the misconception that AI and big data can be utilized to achieve great goals fairly easily without much planning. They quickly realize projects will under-deliver, fail to show ROI, or apply AI to a problem that could have been solved with a different (cheaper, less risky) solution. 

Fortunately, there exist proven best practices for AI and big data project management methodology that helps teams manage AI projects for success. The industry’s leading vendor-neutral best practice for AI project management is the Cognitive Project Management for AI (CPMAI) approach. The CPMAI methodology is a vendor neutral step by step approach for AI, ML, and advanced big data projects. This methodology for AI project is built upon the well-established, data centric CRISP-DM, and incorporates best-practices agile and iterative approaches for short sprints for projects. CPMAI provides an advanced big data and AI project methodology, approach, process and  plan template that provides the steps for AI, big data, machine learning projects to follow for project success. CPMAI is the fastest growing certification for running and managing AI projects, with 220% annual growth rate, providing not just established best practices for AI & data project management, but also fundamental training and education for AI, machine learning, and data science project management courses that organizations need for alignment. 

There are six main phases of CPMAI, illustratrate below, provide guidance for each stage of AI development and continued project iteration to make sure your project is successful. Each of these CPMAI phases are iterative with each other and allows for progression backwards or forwards during AI project development depending on the need and challenges met in the real world and provide a solid ML project structure to follow. 

Source: Cognilytica

CPMAI Phase I: Business Understanding

The first step of CPMAI methodology for any AI project is  gathering an understanding of the business requirements and understanding the business needs. After all, if you’re not solving a real business problem for your organization then why are you even doing the project at all? In phase one of your project iteration you should focus on understanding the project objectives and requirements from a business perspective, then converting this knowledge into an AI and ML problem definition and a preliminary plan designed to achieve the objectives. 

CPMAI Phase II: Data Understanding

The second phase of CPMAI methodology for any AI project is understanding your data. The most important part here is understanding what data is required to address the business problem, whether or not that data is available, and what format(s) your data is in. Data is what fuels your AI projects so you should make sure you have a firm understanding of your data before getting too far along in your project.

CPMAI Phase III: Data Preparation 

The third step of the CPMAI approach to AI project management is Data Preparation. Once you have figured out what problem you are solving and what data you have, next you need to make sure the data you have is usable for your project. In this step you need to do tasks such as data cleansing, data aggregation, data augmentation, data labeling, data normalization, data transformation and any other activities for data of structured, unstructured, and semi-structured nature. 

CPMAI Phase IV: Model Development 

Once we have a business understanding that lends to a data understanding and the data prepared with pipelines in place, we can finally get to AI model development. In CPMAI, the fourth step of your AI project is the creation and development of machine learning models and supporting AI system artifacts. This includes model technique selection and application, model training, model hyperparameter setting and adjustment, model validation, ensemble model development and testing, algorithm selection, and model optimization. By the time you are ready to build your very first model you’ve already determined the business needs, the data requirements, and gotten the data in the right format and quality. If you haven’t, then you need to revisit these steps before moving forward.

CPMAI Phase V: Model Evaluation 

Developing the model is not the end point but rather the midpoint in a CPMAI iteration. Once a model is created, it needs to be evaluated to make sure it performs according to the business requirements and other factors set in the previous steps of a machine learning project. In this fifth phase of CPMAI, you are now ready for model evaluation. From an AI perspective this includes model metric evaluation, model precision and accuracy, determination of false positive and negative rates, key performance indicator metrics, model performance metrics, model quality measurements, and a determination as to whether or not the model is suitable for meeting the goals or whether earlier phases should be iterated upon to reach those goals.

Phase VI: Model Operationalization

The sixth step of each AI project lifecycle iteration is putting the model you just created into operation and monitoring, managing, and iterating those AI systems to keep the relevant and providing business value as defined in the first phase. CPMAI Phase VI makes sure to address model versioning and iteration, model deployment, model monitoring, model staging in development and production environments, and other aspects of getting the model in a position to provide value to meet the stated purpose. 

Putting Established Best Practices for AI & Big Data Project Management Into Action

Artificial Intelligence project management requires focusing on not just tools, technology, and talent, but also the techniques and methodologies to make the AI system deliver on promised value. CPMAI as the established best practice for AI and data project management provides a robust, iterative approach to AI and data science project management that helps you address the key AI specific considerations in the right order and at the right level, providing the technique and supporting materials including a workbook that helps guide CPMAI phase implementation of your progress that will increase your chances of project success. Cognilytica provides CPMAI project management training and  AI project management certification that individuals and organizations can complete in a short amount of time to enhance their skills, careers, and benefit to the organization.  Learn more about CPMAI and join the thousands of other professionals who earned their Cognilytica CPMAI Certification in the AI project management course and data project management certification.