AI Snake Oil: Navigating Hype and Reality
A discussion with Professor Arvin Narayan, author of "AI Snake Oil," explored the balanced perspective needed to navigate the promises and perils of artificial intelligence. The event, co-hosted by the Schwarzman College of Computing and MIT's Shaping the Future of Work initiative, aimed to bring clarity, rigor, and technical expertise to the development and deployment of AI technology.
Introduction and Welcome
The event began with welcomes from representatives of the Schwarzman College of Computing and MIT's Shaping the Future of Work initiative. Professor Narayan's book, "AI Snake Oil," was lauded for its timely contribution to the AI debate, offering a balanced view amidst widespread discussion about existential risks. The book draws a parallel between AI snake oil and traditional snake oil, where sellers promise miracle cures with false pretenses, sometimes harmless, but often with damaging consequences.
Defining AI Snake Oil
"AI snake oil" is defined as AI that does not and cannot work. The book aims to distinguish this from areas where AI can be effective, particularly in high-stakes settings like hiring, healthcare, and justice. A shared commitment to clarity and rigor unites the efforts of the Schwarzman College of Computing and the Shaping the Future of Work initiative, particularly in how AI technology is developed and deployed.
The Initiative for Shaping the Future of Work
The Initiative for Shaping the Future of Work, co-led by Daron Acemoglu, David Autor, and Simon Johnson, was launched due to concerns about the future of work, inequality, and productivity in the age of digital technologies and AI. The initiative emphasizes that the future of these technologies is not predetermined, and different technologies have different consequences. The goal is to steer technology towards more socially beneficial directions through academic research.
Professor Narayan's expertise, combining technical skills with a clear understanding of AI applications, is seen as invaluable to this effort. The focus should be on understanding AI's capabilities and limitations, avoiding both excessive optimism and pessimism.
The Origin Story of "AI Snake Oil"
The book's origin traces back to 2019 when Professor Narayan observed hiring automation software promising to analyze candidates' videos to determine personality and suitability. He found this claim dubious, and subsequent investigations revealed flawed practices, such as easily manipulated results based on superficial video changes. This led to a talk titled "How to Recognize AI Snake Oil," which resonated widely and eventually evolved into the book.
AI is Not One Single Technology
A key point is that AI is not a singular technology but an umbrella term for loosely related technologies. ChatGPT and AI used in banks for credit risk assessment are both forms of learning from data, but they differ significantly in their function, application, potential failures, and consequences.
-
Predictive AI: Used in hiring, lending, criminal justice, healthcare, and education, predictive AI makes decisions about individuals based on predictions of their future behavior.
-
Generative AI: Generates text, images, and other content.
-
Social Media Algorithms: Pose societal-scale risks.
-
Self-Driving Cars and Robotics: Also discussed in the book.
Skepticism Towards Predictive AI
Predictive AI, particularly in criminal justice, faces significant scrutiny. The ProPublica investigation in 2016 highlighted racial bias in risk assessment algorithms. Beyond bias, concerns arise about the accuracy of these systems, which often perform only slightly better than random guessing. Decisions affecting freedom should not be based on such unreliable predictions.
Generative AI: Potential and Pitfalls
Generative AI is recognized as useful for knowledge workers and enjoyable to use. Examples include creating educational apps for children. However, irresponsible release practices by the generative AI industry raise serious concerns.
-
Harmful Consequences: AI-generated books with misinformation, AI companions encouraging harmful behavior, and AI notification apps creating non-consensual nude images.
-
Labor Exploitation: The human annotation work required to clean training data is often outsourced to developing countries with precarious working conditions.
Personal agency and judgment are crucial in determining the appropriate use of AI. The example of a candidate proposing an AI chatbot as mayor illustrates the importance of human interaction in resolving complex political issues.
A Framework for Evaluating AI Applications
A two-dimensional framework is proposed for evaluating AI applications:
- How well does it work? (Snake oil vs. effective)
- Is it harmful? (Regardless of effectiveness)
Examples:
-
Top Right (Harmful Snake Oil): Video interviews, criminal risk prediction, and cheating detection tools.
-
Bottom Right (Harmful but Effective): Mass surveillance using facial recognition.
AI as Normal Technology
The book's framework aims to help people push back against problematic AI. The goal is to nudge the industry towards reliable technology that performs one task well and fades into the background, similar to autocomplete. A new paper, "AI as Normal Technology," envisions AI over the next 20 years as a transformative force unfolding gradually, with both positive and negative effects. This perspective contrasts with narratives of superintelligence utopia, superintelligence doom, or AI as a passing fad. The paper proposes policy ideas for steering and shaping AI in a positive direction.
Q&A Highlights
The Q&A session explored deeper concerns about predictive AI, generative AI, and AGI. The potential for AI to increase worker skills and expertise was also discussed, alongside challenges in achieving this goal. The panel also spoke about how to combat the negative association of AI among the public.