A gradual disempowerment reading list
I recently completed BlueDot Impact’s AGI Strategy Course. (I recommend it.) Through the weekly discussions, I went down the rabbit hole of gradual disempowerment. I find many AI threats compelling, but gradual disempowerment is the risk where my background perhaps offers the most insight.
I find an interesting tension in this topic. I have written in the past about how more decisions should be handed to AI (e.g. here, here and here). Since Paul Meehl’s Clinical Versus Statistical Prediction in 1954, a large body of research has shown that statistical prediction can outperform human judgement in some domains. We should be capturing those benefits.
However, I can see an endpoint where this doesn’t end well. Competitive pressures between companies and states could drive the adoption of AI decision-making to the point where humans are completely out of the loop. The result is a disempowered humanity. This handover magnifies risks from misaligned AI or rogue actors.
As a follow-up to the BlueDot course, I have pulled together the gradual disempowerment reading list below.
The list reflects my worldview, so later entries may not appear in standard AI safety reading lists. It contains core readings on gradual disempowerment, plus adjacent work on automation, human control, and deskilling that helps explain how humans might lose meaningful influence.
I’ll add more papers as I find them.
The core thesis
Christiano (2019) What failure looks like: A precursor to the gradual disempowerment thesis. Humanity goes “out with a whimper” as human reasoning ceases to be able to compete with AI.
Brynjolfsson (2022) The Turing Trap: The Promise & Peril of Human-Like Artificial Intelligence: Human-like AI can push development toward substitution rather than augmentation, reducing human bargaining power.
Critch and Russell (2023) TASRA: a Taxonomy and Analysis of Societal-Scale Risks from AI: See Risk Type I Diffusion of Responsibility.
Kulveit et al. (2025) Gradual Disempowerment: Systemic Existential Risks from Incremental AI Development: The anchor paper.
Chakravorty and Erwan (2025) Gradual Disempowerment Summary: BlueDot Impact’s summary of the Kulveit et al. paper. A quick way to get across it.
Drago and Laine (2025) The Intelligence Curse (pdf): If states and companies don’t need people, they won’t care about them. Chapters 2 and 3 are particularly relevant to the gradual disempowerment thesis.
Fenwick (2025) Gradual disempowerment: A summary of the gradual disempowerment thesis on 80,000 Hours. Includes links to other readings.
Davidson (2025) Thoughts on Gradual Disempowerment: Some rough notes testing the edges of the thesis.
Sharma et al. (2026a) Who’s in Charge? Disempowerment Patterns in Real-World LLM Usage: Claude conversations show patterns of user autonomy being undermined. Disempowerment potential appears to be increasing. See also the related blog post by Sharma et al. (2026b).
Wiblin (2025) #234 – David Duvenaud on why ‘aligned AI’ could still kill democracy: Robert Wiblin interviews David Duvenaud, one of the authors of Kulveit et al. (2025).
Automation bias and the erosion of human capability
Despite all the talk about people becoming deskilled as AI takes over, there isn’t a lot of great literature concerning the latest AI. However, there is a rich, older literature on automation that we can draw on.
Bainbridge (1983) Ironies of automation (ungated pdf): When you automate the routine and leave the exceptions to the human, the operator has fewer opportunities to practise in preparation for those exceptional circumstances.
Endsley and Kiris (1995) The Out-of-the-Loop Performance Problem and Level of Control in Automation (ungated pdf): Operators “out of the loop” have a harder time detecting errors and stepping in when automation errs than those who manually perform the same tasks.
Parasuraman and Riley (1997) Humans and Automation: Use, Misuse, Disuse, Abuse (ungated pdf): A classic paper on automation. The use, misuse, disuse, abuse split helpful is helpful.
Skitka et al. (1999) Does automation bias decision-making? (ungated pdf): Participants given an imperfect decision aid did worse than those given none.
Parasuraman and Manzey (2010) Complacency and Bias in Human Use of Automation: An Attentional Integration (ungated pdf): Review of automation complacency and automation bias.
Barr et al. (2015) The brain in your pocket: Evidence that Smartphones are used to supplant thinking: People offload thinking to the device.
Sarkar et al. (2024) When Copilot Becomes Autopilot: Generative AI’s Critical Risk to Knowledge Work and a Critical Solution: A research agenda for AI as a critic or provocateur.
Lee et al. (2025) The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers: Higher confidence in generative AI is associated with less critical thinking.
Krook (2025) When Autonomy Breaks: The Hidden Existential Risk of AI (ungated pdf): Humans may lose skills such as critical thinking, decision-making and social care in an AGI world. This paper puts some of the above references into context, arguing why the deskilling matters.
Organisations working on (or at least noting) the risk of gradual disempowerment
AI Objectives Institute: Gradual disempowerment is one of their streams of work. Includes a coauthor of the gradual disempowerment paper.
The Alignment of Complex Systems Research Group: Home to one of the two lead authors of the gradual disempowerment paper, and another member of the research team.
Centre for AI Safety: “As AI becomes more capable, businesses will likely replace more types of human labor with AI, potentially triggering mass unemployment. If major aspects of society are automated, this risks human enfeeblement as we cede control of civilization to AI.”
Secure AI Future: “The concern of human enfeeblement arises in a scenario where AI becomes so efficient and pervasive in performing tasks that humans traditionally handled, leading to a societal dependence on AI for even the most basic activities.”