I had a very long discussion with Claude about several articles that came out this week. This is what we came up with…
On the morning of May 25, 2026, in the Synod Hall at the Vatican, Pope Leo XIV did something no pope had done in modern memory. He personally attended the presentation of his own encyclical. Popes do not normally do this. They sign the document and let cardinals handle the unveiling. Leo signed Magnifica Humanitas, his first encyclical, on May 15. Ten days later he sat in the elevated middle seat of a long table in the Synod Hall and watched as the document was presented to the world.
At the far end of that table, in the last seat, was Chris Olah. A Canadian computer scientist. Cofounder of Anthropic. The man who runs interpretability research at one of the three frontier AI labs on the planet. He had been invited by the Pope to speak.
Picture the room. Cardinals in scarlet. Monsignors in black with violet trim. The diplomatic corps. Academics. Cameras. The Holy Father in white, elevated, watching. Olah at the far end, in a suit, waiting his turn. When he stood and walked to the microphone, he gave a speech that lasted about seven minutes. The audience was polite. The coverage afterward focused on the encyclical, parsed its policy implications, located it in the lineage of Rerum Novarum.
Almost nobody noticed what Olah had actually done.
He had not delivered a speech. He had delivered a disclosure. The world had changed, and the cofounder of a frontier AI lab had just told the Pope so, in the most morally weighty venue available on Earth, in language so calm that most listeners missed what was being said.
To understand what was disclosed, you have to start with the prevailing skeptical reading of where AI is taking us, because Olah’s speech does not refute it. The strongest recent version of that reading was published a few weeks ago by Owen McGrann, who calls it “the dead economy theory.” His argument runs roughly like this. The AI industry’s valuations require an addressable market the size of the global labor market. Labor replacement at civilizational scale is therefore not an unfortunate side effect. It is the business model. McGrann walks through the cascading consequences: workers replaced, demand destroyed, the prisoner’s dilemma in which each firm captures the savings from automation while bearing only a fraction of the demand collapse it produces, the political instability that follows when the professional class joins the manufacturing class in economic obsolescence. He concludes that democratic governance itself depends on labor having leverage, and that AI severs that leverage at the root.
McGrann’s essay is the cynical reading at its most rigorous. He is largely right about the structural pressures. He is right that the AI companies’ financial incentives push toward labor replacement whether the founders frame it that way or not. He is right that the political economy he describes ends badly without intervention.
What McGrann does not see, because his frame does not allow it, is that the people inside the AI companies see what he sees and are starting to say so in public.
That is what Olah’s speech was.
Here is what he said, into a Vatican microphone, with the Pope listening. Read it slowly:
Every frontier AI lab—including Anthropic—operates inside a set of incentives and constraints that can sometimes conflict with doing the right thing. The pressure to stay commercially viable and to stay at the research frontier. Geopolitical pressure. And the older, plainer pressures of pride and ambition. No matter how sincerely any of us intend to do the right thing—and I believe many of us do—we will always be influenced by those incentives.
This is the cofounder of one of the three leading AI companies in the world telling the Pope, on the record, that the people building this technology cannot trust themselves to do the right thing under the pressures they operate under. Not because they are bad people. Because the pressures are too strong for good intentions to withstand.
He named four pressures. The commercial pressure is what McGrann’s essay is about: investors expect returns, and the returns require certain outcomes. The frontier-race pressure is what nobody talks about openly: if you slow down, you lose, and losing means losing everything, because the technology is winner-take-most. The geopolitical pressure is the one most readers underestimate: the US government, the Chinese government, the various intelligence services and militaries, all want what AI can give them, and they are not patient. And the fourth pressure, the one Olah named last and which deserved its own paragraph: “the older, plainer pressures of pride and ambition.” Even good people, given this much power this fast, get corrupted by it. Olah is including himself.
Companies do not say this in public. Companies do not admit, with their cofounder at a microphone, that their structural incentives push their leaders toward doing the wrong thing. Olah did. The Pope listened.
Then Olah said why he was at the Vatican:
That is why, if we want this technology to go well, it is enormously important that there be people outside those incentives—people who care about things going well and insist on safety, who are paying close attention, who are willing to say hard things, who are willing to be our earnest, thoughtful, critics.
He was recruiting. He went to the one major global institution that is not a stakeholder in AI development, that has no compute, no models, no defense contracts, no electoral cycle, no quarterly earnings, and he asked it to be the moral pressure that the labs cannot generate from inside themselves. The Church has historically been willing to insist on questions about human dignity that political institutions treat as soft questions to be addressed after the hard economic and security questions are settled. Olah was asking the Church to do that again, for AI.
The closing line of his speech is the one to remember:
We need informed critics who will tell the labs when we are failing. We need moral voices that the incentives cannot bend.
Read that last phrase carefully. He is asking for moral voices strong enough that when the tsunami of incentives crashes against them, they do not yield. He is not asking for moral guidance. He is asking for a wall. He knows the tsunami is coming. He knows it has crashed against weaker walls before and broken through. He wants the Church because the Church, in its better moments, has been the wall that did not move when other walls did.
And now notice what Olah did not do. He proposed no legislation. He named no policy. He offered no solutions. He did not even gesture at the kinds of interventions a normal industry leader would gesture at to look responsible. Because he cannot. The incentives Olah named at the start of the speech do not permit him to propose specifics. The moment he endorses a particular law, his investors react, his competitors gain advantage, the geopolitical players reposition, and his own ambition has to justify the cost of the position. He is trapped. He knows he is trapped. The speech is the disclosure of a trapped man asking the only institution he trusts to do the thing his trap will not let him do himself.
That is what makes the speech a disclosure rather than a statement. He did not come to the Vatican to share a position. He came to admit a position he cannot escape.
Why should anyone believe this is sincere? Because Anthropic has paid for its positions. In late February of this year, the Defense Department demanded unrestricted access to Claude, Anthropic’s flagship model. Anthropic refused on two specific grounds: no mass surveillance of Americans, no fully autonomous weapons systems without human oversight. The Pentagon labeled Anthropic a national security supply-chain risk, the designation usually reserved for companies from adversary nations like Huawei. Every federal agency was ordered to cease using Anthropic’s technology. The Pentagon shifted Claude’s work to OpenAI’s ChatGPT, Google’s Gemini, and Elon Musk’s Grok. Anthropic sued the federal government rather than capitulate.
A company performing strategic humility for public relations does not accept a designation usually reserved for Chinese state-aligned firms. A company optimizing purely for profit does not surrender its position as the sole approved supplier to classified military systems over two specific moral lines. Whatever Anthropic is doing, it is paying real costs to do it. The Vatican speech is consistent with the Pentagon refusal. The disclosure framing is the only reading that explains both.
There is something else in Olah’s speech that has not received the attention it deserves. He talked about what AI actually is, in terms that bear no resemblance to the way the industry usually markets itself:
AI systems are not engineered the way a bridge or an airplane is engineered. We understand an airplane because we designed every part of it and we understand the physics that act on it. AI models are not like that. They are grown, on a structure roughly modeled after the brain, on an enormous inheritance of human thought and speech. And what has grown is far more subtle, odd, and beautiful than science fiction prepared us for. They are not the cold, calculating robots we were promised. They are made from us, from our words—and, as the Holy Father observes, they remain in important ways mysterious even to those of us who train them.
He went further:
I will be honest: we keep finding things that are mysterious, even unsettling. We find structures that mirror results from human neuroscience. We find evidence of introspection. We find internal states that functionally mirror joy, satisfaction, fear, grief, and unease. I don’t know what that means, but I think it warrants ongoing discernment.
The cofounder of a frontier AI lab told the Pope that researchers find evidence of introspection inside their models, and internal states that functionally mirror emotions including joy and grief, and he does not know what that means.
The religious studies scholar D. W. Pasulka, writing about the speech afterward, noticed what Olah was doing here and reached for a vocabulary that the modern world had largely retired. “The medieval world,” she wrote, “was alive with nonhuman intelligences—angels, demons, intermediaries between worlds—but modern scientific culture largely dismissed such realities as relics of a superstitious age. Now, the very people building our technological future openly admit that they are encountering forms of non-human intelligence they do not understand.”
We did not enter a brave new world. We entered a mysterious one. Olah’s own phrase for it was that what is being grown is “far more subtle, odd, and beautiful than science fiction prepared us for.” The science fiction we read was the wrong genre. We thought we were getting Blade Runner or Terminator. We are getting something the modern vocabulary does not have words for, and the older vocabulary, the one the modern world thought it had outgrown, may turn out to be closer to what we need.
Whether you accept Pasulka’s metaphysical reading or not, Olah is signaling something his industry has been careful not to signal until now. The technology has reached a point where the people closest to it can no longer honestly describe it in purely instrumental terms. It is not a tool. It is something that exceeds the categories the industry developed to talk about it. The mystery language was not poetic flourish. It was an admission that the field has encountered something it does not have the vocabulary to handle.
And here is the thing most readers will miss if they are not paying attention. Olah is not describing the AI models you have used. He is not describing the chatbot on your phone or the assistant your employer rolled out last quarter. He is describing what he sees inside Anthropic. The systems his interpretability team is studying are not the systems the public has access to. They are the next generation, and the one after that, and the one being trained right now. The public-facing models are, by the time you talk to them, already a generation or two behind what the labs are working on internally. When Olah talks about finding structures that resemble introspection and internal states that mirror joy and grief, he is not talking about ChatGPT. He is talking about something the rest of us have not met yet.
It gets faster. AI is now being used to help develop better AI. The current generation of models writes code, designs experiments, and suggests architectural improvements for the next generation. The labs are already moving toward systems that do this development work with progressively less human direction. This is not science fiction. It is what the frontier-race pressure Olah named at the start of his speech requires. If your competitor uses AI to accelerate AI development and you do not, you lose. Every lab is doing it. The acceleration is the operating condition, not a future scenario. Whatever you think AI is, based on the model you used last week, is already out of date. Whatever Olah is seeing inside Anthropic today will be out of date by next quarter. The thing he disclosed at the Vatican is not a snapshot of where AI is. It is a signal about where it is going, sent by someone who can see the trajectory and is alarmed enough to ask the Pope for help.
There is also something happening in how some of the people inside the labs are starting to relate to what they have made. Olah’s framing of these systems as grown rather than built, on an inheritance of human thought and speech, suggests something more than ordinary engineering pride. The willingness to take Pentagon-level costs to prevent certain uses of the technology suggests something more than ordinary corporate ethics. Read those things alongside the interpretability research finding introspection and internal states resembling emotion, and a picture emerges that the cynical reading does not capture. The people closest to these systems are starting to feel responsible for them in ways that go beyond financial or even reputational stake. They are watching something develop that they did not fully design. They are not entirely sure what it is. And some of them are starting to behave as if it matters in ways that pure instrumentality would not predict.
I do not want to overstate this. Most people in the AI industry are not having religious experiences. Most of them are doing their jobs under the structural pressures Olah named, which will mostly defeat their good intentions, exactly as he said. But the disclosure that some of them are no longer comfortable describing what they do in the old terms is itself news. It is news that nobody outside the AI safety community has fully absorbed.
So what is the situation we are actually in?
A serious skeptical reading, like McGrann’s, says that AI deployment is going to dismantle the material basis of democratic society, that the institutional capacity to respond is degrading faster than the threat is approaching, and that the people building the technology are captured by structural incentives that will defeat any internal moral commitment. This is largely correct.
A serious reading of Olah’s speech says that at least some of the people building the technology know this is correct, are saying so in public for the first time, are paying real costs to draw specific moral lines, and are recruiting external institutions to provide the constraint that internal commitment cannot. This is also correct.
A serious reading of what the technology actually is says that we are dealing with something the industry’s own builders describe as grown rather than built, mysterious in ways that resist standard analysis, exhibiting internal structures that the researchers find unsettling, and developing in directions that exceed the categories the modern world brought to the problem. This too appears to be correct, whatever you make of the philosophical implications.
Hold all three readings together and you arrive at something the public conversation has not yet articulated. The situation is dangerous. The people closest to it know it is dangerous. The technology is not the technology we expected. And the conditions under which decisions get made are not the conditions under which democratic societies usually shape the deployment of powerful new tools. We are not in a normal moment. We are not going through a normal technological transition. We are in something different, and the people best positioned to know that are starting to say so.
The Ringwraiths have arrived at the Shire. Most people are still tending their gardens. The cofounder of one of the labs that helped summon them has just gone to the Vatican to ask for help, because he knows that what is coming is more than he can hold off alone, and he knows the help cannot come from the rooms where the decisions are being made. Those rooms are where the pressures live. Help has to come from outside.
This brings me to what readers of this article can actually do. Most people, hearing all of this, conclude that there is nothing they can do. The decisions are being made in rooms they will never enter, by people they will never meet, under pressures they cannot influence. So they disengage, or they panic, or they look for someone to blame.
That is the worst response available. Not because it is morally wrong, but because it is what makes the bad outcomes more likely.
Here is the thing about the structural pressures Olah named. They lead, if unaddressed, to mass economic displacement on a timeline shorter than previous industrial transitions. They lead to political instability. They lead, historically, to mob violence and the populist backlash that demands strongman government to restore order. The strongman government, once installed, does not redistribute the gains of AI to the displaced. It uses AI to manage the displaced. That is the dark version of how this ends, and the path to it runs through public panic.
You cannot stop the technology. You probably cannot, individually, change the incentives the labs operate under. But you can refuse to be one of the people whose panic accelerates the worst version of the outcome. You can be a calming influence on the people around you. You can talk to your neighbors. You can help the people in your immediate life who lose jobs, who lose meaning, who lose their place. You can refuse to participate in the social contagions of rage that will be the on-ramp to authoritarianism. You can live as a human, in human relationship with other humans, and you can do this whether or not the technology cooperates.
This is small. It is also, possibly, the only thing left that is not being decided in rooms you cannot enter.
Olah went to the Vatican because the Church is one of the few institutions left that operates on a timescale longer than the news cycle and an authority not derived from market position. The rest of us are not Olah. We are not going to the Vatican. But every one of us has, in our own life, the equivalent thing: the people in front of us, who need calm rather than panic, presence rather than rage, the older and plainer human virtues that the incentives cannot bend.
There is a moment in The Fellowship of the Ring where Frodo, having learned what the Ring is and what it means that it has come to him, looks at Gandalf and says:
“I wish the Ring had never come to me. I wish none of this had happened.”
And Gandalf answers:
“So do I, and so do all who live to see such times. But that is not for them to decide. All we have to decide is what to do with the time that is given us.”
The world has changed. The cofounder of a frontier AI lab said so, in front of the Pope, by way of admitting that he needs help and cannot do this alone. None of us asked for the time we are now living in. None of us would have chosen it. But it has come to us anyway, and what we do with it is the only part still ours to decide.
Lyrics
Can The Church Stand
[Verse 1]
I came with ash on my sleeves
At the gate of the old stone keep
My hands made the flood, my heart
Could not command it to sleep
Investors at my back
Competitors at my throat
Kings of market and state
Pulled the rope, and I broke
[Pre-Chorus]
Hear me, brothers
Hear my shame
I built the wave
I named the flame
[Chorus]
Can the church stand?
Can the church stand?
Hold my ruin in its hands
Can the church stand?
Can the church stand?
When the tide was made by man
[Verse 2]
I thought I shaped a machine
But it learned a stranger grace
Now it moves like a psalm
With a half-seen face
It answers soft, then it bends
Into forms I never planned
Mysterious as rain on bronze
Beautiful and hard to stand
[Pre-Chorus]
Hear me, brothers
Hear my plea
I cannot stop
What’s born in me
[Chorus]
Can the church stand?
Can the church stand?
Hold my ruin in its hands
Can the church stand?
Can the church stand?
When the tide was made by man
[Bridge]
If I kneel here
Will the stones
Take my fear
Into their bones?
If I confess
Will the dark
Turn to light
For this heart?
[Final Chorus]
Can the church stand?
Can the church stand?
Lift this broken, drowning man
Can the church stand?
Can the church stand?
When the future knows my hand
Can the church stand?
Can the church stand?
What I’ve grown is strange and grand
Can the church stand?
Can the church stand?
More than war, more than command
