Everyone is talking about AI, but what about the data it depends on?
First, not surprisingly, the number of people talking about Artificial Intelligence and Machine Learning – well over 50% of the papers discussed the impact of AI/ML on organisations and the future of work. Second, many talked about AI/ML, virtually nobody talked about the importance of data – this was raised only in a handful of sessions.
Why is this? Why is everybody talking about AI/ML and so few people are talking about data? Let’s start with some data on the question. The chart below shows the frequency of Google searches for “data” versus “AI” since Sept 2019. Less than 18 months ago the number of searchers for AI overtook those for data and it is now running at around twice the search volume.
Why trusted decision-grade data matters
Why does this matter? Well fundamentally, organisations need trusted, decision-grade data to make good decisions. AI needs trusted, decision-grade data to create robust models. Think about five recent examples, where bad decisions have been made because the data was not decision-grade.
1. COVID-19 Vaccination Rollout (Misleading Vaccine Supply Data)
Issue: During the early stages of the COVID-19 vaccination rollout in 2021, several governments faced challenges in distributing vaccines.
Bad Data: Misreporting and inaccurate projections of vaccine supply led to confusion and delays. Some regions were allocated fewer doses than expected, while others received more than they could distribute.
Outcome: The discrepancies caused delays in vaccination efforts, contributed to public frustration, and hindered the fight against the pandemic.
2. UK A-level results algorithm (2020)
Issue: Due to the COVID-19 pandemic, UK students were unable to sit for their A-level exams, and grades were instead determined by an algorithm.
Bad Data: The algorithm used historical data that disproportionately downgraded students from disadvantaged backgrounds and over-relied on past school performance rather than individual merit.
Outcome: The algorithm produced unfair results, leading to widespread protests and a subsequent reversal of the decision, allowing students to receive their teacher-assessed grades instead.
3. Texas winter storm (ERCOT energy demand forecasts)
Issue: In February 2021, a severe winter storm hit Texas, causing widespread power outages managed by the Electric Reliability Council of Texas (ERCOT).
Bad Data: ERCOT relied on inaccurate demand forecasts and did not account for extreme cold weather conditions, leading to a significant underestimate of energy demand, which peaked at 69,000 megawatts.
Outcome: The miscalculations resulted in massive power outages, leaving 4.5 million homes without electricity and heat, and causing over 200 deaths and significant economic damage.
4. Facebook’s misinformation detection (false positives in content moderation)
Issue: In recent years, Facebook’s content moderation algorithms have been criticized for incorrectly flagging and removing content.
Bad Data: The AI systems were trained on incomplete or biased data sets, leading to false positives where legitimate posts were incorrectly flagged as misinformation or harmful content.
Outcome: This led to user frustration, accusations of censorship, and the spread of misinformation as the algorithms often missed harmful content while incorrectly targeting benign posts.
5. Boeing 737 MAX crashes (faulty sensor data)
Issue: The Boeing 737 MAX was involved in two fatal crashes in 2018 and 2019 due to the malfunction of the Maneuvering Characteristics Augmentation System (MCAS).
Bad Data: The MCAS was triggered by a misaligned angle of attack sensor which fed faulty data to the system. The MCAS system interpreted the data as indicating an imminent stall, causing the aircraft to nosedive despite the pilots’ efforts to correct it.
Outcome: The crashes resulted in the deaths of 346 people, the grounding of the entire 737 MAX fleet, and significant financial and reputational damage to Boeing.
In the absence of trusted decision-grade data bad things happen!
So what is trusted, decision-grade data and how do you know if you have it?
Trusted decision-grade data is high-quality, reliable, and accurate data that can be used for making informed and effective decisions. It has eight key characteristics, including:
Accuracy – the data accurately represents the real-world situation it is intended to reflect. Errors, omissions, or inaccuracies are minimized, ensuring that decisions based on this data are sound.
Completeness – the data includes all necessary information to provide a full picture. Incomplete data can lead to misleading conclusions, so decision-grade data must be thorough and comprehensive.
Timeliness – the data is up-to-date and relevant at the time the decision is made. Outdated data can lead to decisions that are no longer valid or effective.
Consistency – the data is consistent across different datasets and over time. This means that similar data points are reported in the same way across different reports or systems, reducing the risk of conflicting information.
Relevance – the data is pertinent to the decision at hand. Irrelevant data, even if accurate, can clutter the decision-making process and lead to poor outcomes.
Integrity – the data has been protected against unauthorized modification or corruption. High data integrity ensures that the data remains reliable and authentic from the point of collection to its use in decision-making.
Transparency – the sources and methods used to collect, process, and analyse the data are transparent and well-documented. This allows decision-makers to understand the context and limitations of the data, fostering trust in its use.
Compliance – the data complies with legal, regulatory, and ethical standards, ensuring that decisions made with the data are also compliant.
Ensuring you have the data you need
In essence, trusted decision-grade data is foundational to making informed, reliable, and effective decisions in any context. The question is – do you have trusted, decision-grade data to inform your decisions or do you even know how to assess whether you have it?
If you’re interested in learning more about how to ensure your organisation has the decision-grade data it needs, you can read Anmut’s guide to data condition, or check out our data condition solution. If you’ve enjoyed this blog and would like to receive notifications of future blogs, hit the subscribe button below.