Home CPMAI™ Methodology

CPMAI™ Methodology

by rschmelzer
CPMAI™ Methodology

CPMAI: Methodology for Implementing AI Projects Successfully

  • Data-Centric

  • AI-Focused

  • Best Practices Based

  • Iterative and Agile

  • Vendor Neutral

  • Focused on the Right Tasks for AI Operational Success

Organizations of all sizes are implementing AI, ML, and cognitive technology projects for a wide range of reasons in a disparate array of industries and customer sectors. Some AI efforts are focused on development of intelligent devices and vehicles. Other efforts are internally-focused enterprise predictive analytics, fraud management, or other process-oriented activities that aim to provide an additional layer of insight or automation on top of existing data and tooling. Yet other initiatives are focused on conversational interface and application development that are meant to be distributed across an array of devices and systems. And others have AI & ML project development goals for public or private sector applications that differ in more significant ways than these.

Despite all these AI project differences, the goals of these efforts are the same: the development and application of cognitive technologies that leverage the emerging capabilities of machine learning and associated approaches to meet a range of important needs. Yet, existing methodologies that are either application development-centric or enterprise architecture focused or rooted in hardware or software development approaches face significant challenges when faced with the unique lifecycle requirements of AI projects. What is needed is a project management methodology that takes into account the various data-centric needs of AI while also keeping in mind the application-focused uses of the models and other artifacts produced during an AI lifecycle.

The Cognitive Project Management for AI Methodology: Best Practices Methodology for AI & Cognitive Technology Projects

Cognilytica has worked with dozens of organizations and through hundreds of real-world implementations to develop a methodology optimized for the deliver of in-production, high value, low risk AI projects. The Cognitive Project Management for AI (CPMAI™) Methodology leverages decades of real-world methodology experience for running big data projects combined with the latest best practices expertise learned from running real-world AI projects.

The Core Elements of CPMAI

CPMAI extends the well-known CRISP-DM methodology with AI and ML specific documents, processes, and tasks. The CPMAI methodology also incorporates the latest practices in Agile Methodologies and adds additional DataOps activities that aim to make CPMAI data-first, AI-relevant, highly iterative, and focused on the right tasks for operational success. At its core the CPMAI methodology is composed of six primary phases :

Phase I: AI-Relevant Business Understanding

    • Key Elements of this Phase
      • What are the core business requirements and objectives?
      • Why does this project need AI?
      • How would we meet those needs with AI?
      • How can we succeed and not fail in bad ways?
    • AI-Relevant Considerations:
      • What is the business objective here?
      • Does this business objective require a Cognitive solution?
      • What are the cognitive parts and what are not?
      • AI Go / No Go Assessment
      • Which of the seven patterns does this require?
      • What are the defined “success” criteria for the project?
      • How can the project be staged into Agile / iterative sprints?
      • What is the general way that the Cognitive solution will be used in production environment?
      • What are the transparency requirements?
      • What are the ethical / responsibility considerations?
      • Is there adequate AI and data knowledge / experience?
      • Failure tolerance: Acceptable TP / FP / TN / FN
      • Critical failure modes
    • Methodology Documents & Artifacts
      • Determine Business Objectives
      • Assessment of Business Situation
      • Cognitive Requirements
      • AI Go / No Go
      • AI Pattern Identification
      • Cognitive Parts of Project Identification
      • Non-Cognitive Parts of Project Identification
      • Sprint / Agile Release Planning
      • Production Environment
      • AI Project Goals
      • AI Project Success Criteria
      • Transparency Requirements
      • Responsible AI
      • AI Skills Assessment
      • AI Failure Modes
      • Confusion Matrix Acceptable Values
      • Production of Project Plan

Phase II: AI-Relevant Data Understanding

    • Key Elements of this Phase
      • What are the core data requirements and objectives?
      • Where are the AI-relevant data sources?
      • What type of data is available for AI projects to meet business objectives?
      • What is the quality of the available data?
    • AI-Relevant Considerations
      • Where are the sources of the data for training?
      • Is data in structured, unstructured, or semi-structured format?
      • What is the current quantity and quality of training data?
      • Is there enough well-labeled, clean data for testing and validation?
      • Can you make use of pre-trained models?
      • Is there a difference between training and operational data?
      • Where is the location of operational and training data?
      • What data do models on edge devices need to have access to?
      • Is data acquisition needed from cameras or sensors / devices?
    • Methodology Documents & Artifacts
      • Collecting initial data
      • Data description report
      • Data exploration report
      • Data quality report
      • Data Source Formats
      • Training Data Identification
      • Training Data Quality Report
      • Test Data Identification
      • Test Data Quality
      • Pre-trained Model Identification
      • Extensions of Pre-Trained Models
      • Edge Model Data Needs

Phase III: AI-Relevant Data Preparation

    • Key Elements of this Phase
      • Get the data into a condition where it can be used to address business requirements
      • Selection, ingestion, and combination of data
      • Cleansing, labeling, and data preparation
      • Data formatting, augmentation, construction of data
    • AI-Relevant Considerations
      • What features need enhancement and engineering?
      • How can we prune and optimize data to make modeling more accurate and effective?
      • Provide transparency for methods of data selection, pruning, feature enhancement
      • Creation of pipelines to deal with model iteration
      • Feature augmentation and data transformation
      • Data normalization
    • Methodology Documents & Artifacts
      • Dataset description
      • Rationale for dataset inclusion / exclusion
      • Data cleansing report
      • Derived attributes and generated records
      • Merged and Reformatted data
      • Data Labeling Method and report
      • Data Augmentation
      • Data Anonymization
      • Data Normalization
      • Data De-Noising

Phase IV: Machine Learning Data Modeling

    • Key Elements of this Phase
      • Select Modeling Techniques
      • Algorithm selection
      • Modeling Assumptions
      • Generate Test Design
      • Model development and iteration
      • Hyperparameter optimization method
      • Model assessment
      • Model iteration
    • AI-Relevant Considerations
      • Algorithm selection
      • Ensemble methods usage and approach
      • Model evaluation methods
      • Model tuning / Hyperparameter Optimization
      • Model training approach and scaling
      • Model re-training approach
      • Pre-trained model extension
      • Considerations for Supervised, Unsupervised, and Reinforcement approaches
    • Methodology Documents & Artifacts
      • Modeling assumptions
      • Test design
      • Algorithm Selection
      • Hyperparameter Optimization iterations
      • Ensemble Methods
      • Model Validation
      • Model Training
      • Pre-trained Model Usage
      • Transfer Learning usage
      • AutoML Usage
      • Model description
      • Model assessment

Phase V: AI Model evaluation

    • Key Elements of this Phase
      • Is the trained model actually learning?
      • Does the model really work against validated data?
      • Is the model right and wrong in acceptable ways?
    • AI-Relevant Considerations
      • Training learning curves
      • Confusion matrix evaluation considerations
      • Model validation
      • Real-world model testing and validation
      • Model iteration and improvement
      • Performance vs. Accuracy
      • Business KPI Evaluation
    • Methodology Documents & Artifacts
      • Results evaluation
      • Model approval
      • Process review
      • Learning curves report
      • Validation configuration
      • Validation report
      • Confusion matrix & Scores
      • Lift, ROC, AUC reports
      • Hyperparameter adjustments
      • KPI Evaluation
      • Model iteration

Phase VI: AI Model Operationalization

    • Key Elements of this Phase
      • Putting AI models into operation
      • Inference phase of AI
      • Model deployment configuration
      • Real-world model monitoring and management
    • AI-Relevant Considerations
      • AI operationalization considerations
      • Edge vs. Server vs. Cloud AI model operationalization
      • Continuous AI model iteration
      • Continuous model management and evaluation
      • Model extension
      • Model performance evaluation
      • Model scaffolding and external code requirements
    • Methodology Documents & Artifacts
      • Deployment plan
      • Monitoring and maintenance plan
      • Final model report
      • Model operationalization approaches
      • Model scaffolding requirements
      • Model governance framework
      • Model maintenance

The above represents at a high level all the activities, procedures, and documents that are needed to make sure that AI projects proceed with highest quality and reduced risk. The specifics of the CPMAI are provided in the form of a CPMAI workbook which is provided to those who have successfully completed a CPMAI training and certification, as described in more detail below.

Seven Patterns of AI

One of the core components of the CPMAI methodology is the use of the Seven Patterns of AI to identify how established approaches to particular AI and cognitive technology patterns can be applied to specific project requirements. Much of the effort of simplifying and providing high-quality repeatability for AI projects stems from leveraging one or more of the Seven Patterns and building iterative, agile projects based on these patterns. Learn more about the Seven Patterns of the AI in our writing, podcasts, and more.

CPMAI Workbook

The CPMAI Methodology is detailed in the CPMAI™ Workbook, which is provided to those who have successfully completed a CPMAI training and certification. It is also provided to select CPMAI partners and engagement clients who are putting AI projects into place. If you’re interested in accessing the CPMAI workbook to see greater details of the methodology, please email us at info@cognilytica.com, or better yet, sign up to attend our in-person or online CPMAI training and certification so we can show you first hand how to apply this methodology to your specific projects.

CPMAI Training & Certification

The best way to incorporate CPMAI into your AI projects is to enroll in a CPMAI training and certification. The full bootcamp takes only three days in person, or can be attended in live-taught, online training in a few short weeks. The training course involves not only learning about the CPMAI methodology, but also foundations in AI, AI use cases and patterns, big data approaches to AI, AI algorithms and approaches, applying CPMAI methodologies, AI solutions and technologies, and responsible and ethical approaches to AI. The course includes hands-on instruction, group discussions, and exercises that help to reinforce training. Upon completion, you will receive a certificate, a pin, and our endorsement of your completion of the course for use in your resume and personal profiles. Federal government employees can also earn Continuing Learning Points (CLPs) towards their career requirements.

To learn more about the training we offer and additional ways of deepening your AI education, please see our training page here. Cognilytica offers a range of different training offered in different delivery modes that are the most convenient for you.

This website uses cookies to improve your experience. We'll assume you're ok with this, but you can opt-out if you wish. Accept