The Problem With Short Cuts in Getting Data AI-Ready

Michael Campion, Ph.D.

‍

Recently, I came across an article discussing how organizations can go about getting their data Artificial Intelligence (AI)-ready (i.e., ready for model training and general use by AI systems developed within the workplace). The key suggestions were as follows, and, on the surface, they seemed to make sense. However, below the surface, a number of troubling issues are overlooked. First, organizations need to create order out of chaos by integrating fragmented data accumulated through a variety of methods and located across myriad departments and devices (i.e., synchronization and integration). Second, information pipelines need to be built to ensure data flows properly among models being developed (i.e., Automation). Third, mechanisms for auditing data need to be built to reduce security risks and threats (i.e., Security). Finally, data needs to be delivered at the time when it is most valuable (i.e., Speed).

The Issues Associated with the Application of this Methodology

There are a number of problems associated with attempts at massive overhauls such as the one listed above. I’ve outlined a number of examples below.

Data Quality Issues

• Incompleteness/Missing Data: Many datasets have gaps, missing values, or partial records, leading to inaccurate predictions from AI models.
• Inaccuracy/Errors: Data may contain incorrect information, typos, or outdated entries. AI models trained on, and using, inaccurate data will produce unreliable outcomes.
• Inconsistency: Data from different sources or departments often uses varied formats, naming conventions, or structures, making it difficult to integrate and unify.
• Noisy Data: Irrelevant, duplicate, or redundant information can negatively impact model performance.
• Sparsity: Insufficient data points for certain categories or features can lead to biased or limited model training.

Data Silos and Integration Challenges

• Fragmented Data Sources: Organizations often have data spread across numerous disparate systems (e.g., CRM, ERP, spreadsheets, legacy databases), making it difficult to get a holistic view of how data fit together, and whether patterns identified are, indeed, meaningful.
• Lack of Interoperability: Integrating data from these diverse systems, especially those with different formats and structures, requires significant effort and can lead to data conflicts and loss of data integrity.
• Manual Processes: Many organizations still rely on manual data export and merging, which is time-consuming, prone to errors, and hinders real-time insights.

Data Volume and Scalability

• Massive Data Volumes: The sheer volume of data in modern organizations can overwhelm existing infrastructure and traditional data processing methods.
• Infrastructure Limitations: Many organizations lack the scalable infrastructure, computing power, and storage necessary to handle large-scale AI deployments.
• Real-time Processing: AI models often require real-time data ingestion and processing, which many legacy systems are not equipped to provide.

Data Governance and Management

• Lack of Clear Ownership: Without designated data stewards and clear accountability for data quality and compliance, issues can easily proliferate.
• Undefined Policies: Organizations may lack clear policies and procedures for data collection, storage, usage, and retention, especially for AI purposes.
• Data Lineage and Traceability: It can be difficult to track the origin, transformations, and usage of data across various pipelines, which is crucial for debugging, auditing, and ensuring transparency.
• Version Control: Data changes frequently, and without proper versioning, it’s hard to reproduce results or debug issues in AI models.
• Metadata Management: The absence of comprehensive metadata (data about data) makes it challenging to discover, understand, and use data effectively for AI training.

Data Privacy, Security, and Compliance

• Sensitive Data: AI training datasets often contain personally identifiable information (PII), financial details, or other sensitive corporate data.
• Regulatory Compliance: Evolving data privacy regulations (e.g., GDPR, CCPA) impose strict requirements on data handling, requiring organizations to ensure data security, consent management, and auditability.
• Bias and Fairness: AI models can amplify biases present in the training data, leading to unfair or discriminatory outcomes. Identifying and mitigating these biases is a significant ethical and technical challenge.
• Data Security: Protecting AI training data from unauthorized access, breaches, and malicious attacks (like data poisoning) is paramount.

Lack of Context and Understanding

• Data without Context: Even accurate data can be useless for AI if it lacks the necessary context (metadata, business definitions, relationships) for the AI system to understand its meaning.
• Domain Expertise: Bridging the gap between data scientists and domain experts is crucial to ensure the data is understood and prepared in a way that aligns with business objectives.

Skill Gaps and Organizational Culture

• Talent Shortages: There’s a significant lack of skilled professionals in data science, machine learning, and data engineering who can effectively prepare data for AI.
• Resistance to Change: Employees may resist new AI technologies due to fear of job displacement or a lack of understanding.
• Siloed Teams: Poor communication and collaboration between traditional business intelligence teams and AI/data science teams can hinder progress.
• In essence, AI models are only as good as the data they are trained on. Without a robust data strategy that prioritizes quality, governance, and accessibility, organizations risk investing heavily in AI initiatives that fail to deliver meaningful value.

In summary, the road to readying your data for AI training use is one fraught with obstacles that make most existing organizational data difficult to use. Further, this can lead organizations down a difficult and costly road to realizing this.

Finding a Middle Ground

One might ask then, how do I find a realistic, cost-effective way to begin to incorporate AI and realize the potential efficiencies associated therein?

One option is to iterate this process. First, begin with a plan with clear goals. This plan should ensure that there is a clear business need for AI. It sounds obvious, but throwing AI at a problem is usually not the solution. Furthermore, it is a costly endeavor that can upend a business and cost jobs. Second, start small. It is important to ensure the plan proves itself at a smaller scale before attempting to scale it. Third, play the devil’s advocate when it comes to data inclusion. There should be a clear argument for data inclusion rather than data exclusion. Finally, when unsure about data integrity/quality, consider beginning to gather new data with its specific AI use case in mind. This may mean throwing out a lot of old, existing data, but you may just be taking out the trash!

‍