AI adoption is surging while data quality is plummeting, new report finds

Generative AI adoption surges amid data challenges: The rapid growth of generative AI in enterprise settings is accompanied by significant hurdles in data management and quality assurance, according to Appen’s 2024 State of AI Report.

• Generative AI adoption increased by 17% in 2024, with expanded use in IT operations, manufacturing, and R&D sectors.
• Companies are facing a 10% year-over-year increase in bottlenecks related to sourcing, cleaning, and labeling data for AI systems.
• The demand for high-quality, accurate, diverse, and properly labeled data tailored to specific AI use cases is growing as AI models tackle more complex problems.

Enterprise AI deployments face setbacks: Despite the growth in generative AI adoption, there’s a concerning trend in the overall deployment and return on investment of AI projects across enterprises.

• There has been an 8.1% drop in AI projects reaching deployment since 2021.
• Deployed AI projects showing meaningful ROI have decreased by 9.4% in the same period.
• These declines are attributed to the increasing complexity of AI models and more ambitious AI initiatives undertaken by companies.

Data quality concerns intensify: The report highlights a critical issue in the AI landscape: the declining quality of data used for training and evaluating AI models.

• Data accuracy has dropped by nearly 9% since 2021, raising concerns about the reliability of AI systems.
• To address this, 86% of companies are retraining or updating their models at least quarterly.
• 90% of businesses rely on external data sources for training and evaluation, emphasizing the importance of diverse data inputs.

Data management emerges as a primary challenge: The increasing complexity of AI projects is exacerbating data-related bottlenecks, making data management a central concern for organizations.

• Data management has become the leading challenge for AI projects in 2024.
• Companies are focusing on developing long-term strategies to ensure data accuracy, consistency, and diversity.
• The shift towards custom data collection for training AI models reflects the need for more specialized and high-quality data sets.

Human-in-the-loop approaches gain prominence: As AI systems become more sophisticated, the role of human oversight in machine learning processes is becoming increasingly crucial.

• 80% of respondents emphasize the importance of human-in-the-loop machine learning for their AI projects.
• Human involvement is seen as essential for ethical AI development and mitigating bias in AI systems.
• This approach is particularly critical for generative AI to prevent harmful or biased outputs.

Expert insights on AI trends: Si Chen, Head of Strategy at Appen, provides context on the report’s findings and the evolving AI landscape.

• Chen notes that while generative AI is driving innovation, it’s also creating new challenges in data management and quality assurance.
• He emphasizes the need for companies to focus on data quality and ethical considerations as they scale their AI initiatives.
• Chen suggests that the decline in AI deployment and ROI might be temporary as companies adjust to more complex AI projects.

Implications for the future of AI in business: The report’s findings suggest a period of adjustment as enterprises grapple with the complexities of advanced AI technologies.

• The challenges in data quality and management highlight the need for improved data governance strategies and tools.
• The emphasis on human-in-the-loop approaches indicates a shift towards more responsible and ethical AI development practices.
• As companies refine their approaches to data management and AI deployment, we may see a rebound in AI project success rates and ROI in the coming years.