On a crisp night on the cusp of 1918, a young man watches the moon rise above the trenches somewhere in war-torn France. Scribbling in his diary, he records how, “gliding through the barbed wire,” it floodlit no-man’s-land. Fulfilling duties as a stretcher-bearer, that man was Pierre Teilhard de Chardin. Watching the “hesitant crescent,” he experienced an epiphany: about evolution, intelligence and extinction.
Teilhard went on over the course of his life, to infamously argue that all of cosmic history can be understood as an unstoppable upsurge toward more complex, more potent forms of intelligence. Through webbing together evermore rapid and intense information flows across its surface, he believed technological civilization — intentionally or not — was building a kind of planetary “brain.” The oncoming arrival of this mega-mind, for Teilhard, heralded the next phase of intelligence’s ascent.
For Teilhard, even World War I was but a growth spurt in this implacable process. Yet, gazing up at the moon’s pockmarked yet comparably motionless face from France’s freshly cratered trenches, he saw a harbinger of humankind’s future. “The day will come,” he prophesied, when Earth, too, will be “bleached to a uniform whiteness.”
Teilhard was pondering entropy, or the universal tendency toward decay: for suns to die and planets to freeze. But, even this, could not stop intelligence, he believed. He was certain that, with enough intensification, cogitation could overleap even this fateful bound, albeit only through escaping the “flesh” of humanity long beforehand.
It’s an archaic conviction: that, in perfecting itself, intelligence must somehow rid itself of humankind and that this is somehow preordained. For the longest time, there have been those who welcome this and those who caution against it. Of course, Teilhard, being an ordained Jesuit priest, was never coy about the theological roots at play here. Though couched in cutting-edge vocabularies of science and technology, Teilhard’s forecast for intelligence’s future was entirely religious at root. Even today, in our supposedly disenchanted age, the secular and the sacred often remain hard to disentangle.
Heresy & Faith In The Age Of Intelligent Machines
Recent tumults at OpenAI have cascaded into accusations that certain camps subscribe, whether unwittingly or not, to so-called secular religions. Self-styled “AI accelerationists” pit themselves against what they call “AI safetyists”: those who want to let progress in AI rip, versus those who want to pause it; or the “boomers” versus the “doomers.” Commentators have called this split a “religious schism.” The general claim being that people on either side are acting in ways normally associated with the devout whilst professing no motivating belief in supernatural affairs.
But the professed uncloaking of secular religion isn’t unfolding in the realm of AI alone. The climate movement’s enemies make similar allegations of its scientists, calling them fanatics; so, too, do conservatives levy the charge at what they enjoy calling “woke” politics; many ideologies associated with Silicon Valley are similarly scorned.
It seems, in our putatively secular age, heresiology is as healthy as ever and religiosity, ironically, the rifest transgression. But is this all just sound and fury, stacking misunderstanding upon misinterpretation?
Often used simplistically as an insult, imputing irrational zeal, many may be inclined simply to dismiss charges of “secular religion” as, for want of a better term, bad faith. However, when it comes to Western discourses on artificial general intelligence (AGI) there undeniably are — to the eyes of an historian — striking resonances with ancient debates in Abrahamic, and particularly Christian, traditions.
But the aim here isn’t to undermine the legitimacy of AGI discourse nor, especially, the cause of AI safety and AI ethics. It does seem, with today’s continued coalescence of synthetic intelligence, we could be standing near the brink of an epoch-making transition. The risks are very real and the uncertainties run very deep. Rather, the aim is to acknowledge whatever theological inheritances might lurk here and suggest what productively might be done with them, whilst ultimately exploring what it even means to uncloak secular religions.
The Ancient Pursuit of Perfected Intelligence
Despite vehement disagreement on almost everything else, proponents of AI safety and acceleration, alike, share the belief in a coming, potentially imminent, historic threshold beyond which all things recognizably human may dissolve. At least some accelerationists seem to welcome this outcome, though this may be more of an attempt (with predictable success) to court notoriety than any serious commitment. By stark contrast, those concerned with AI safety, fearing the construction of powerful AI systems not “aligned” with human interests and values emphatically want to avert any such fate.
“It’s an archaic conviction: that, in perfecting itself, intelligence must somehow rid itself of humankind and that this is somehow preordained.”
It is, by now, trite to point out the resemblance of such belief to age-old apocalyptic and millenarian patterns of thought. But many cultures exhibit such strains. What remains hidden is a deeper, and more specifically Christian, consonance of convictions. Namely, that history is intelligible as the perfecting of mind or spirit, that this perfection is somehow inevitable or imminent, and that arriving at this threshold may precipitate the endangerment of everything meaningfully human.
Perfection here merely denotes the complete fulfillment of what something fundamentally is. Importantly, this need not have any moralizing or humanizing component. You can have the perfect murderer or storm, after all. Indeed, as we shall shortly explore, perfection has accrued — since Christianity’s beginnings — thoroughly eschatological connotations.
Note, now, that superintelligence is often formulated as perfected intelligence: the ultimate culmination of agential potency or bare cogitation. Especially so in conjectures concerning explosive self-improvement, wherein an agent is pictured rapidly approaching the limits of instrumental power.
Should such cogitative cataclysm ever actually be sparked, this would, indubitably, be cause for concern for us humans, as we’ve absolutely no way of anticipating what a mind vaster than our own would do with us. Acknowledging this, however, doesn’t preclude additionally acknowledging the fact that the basic idea also conspicuously resembles conceptual knots embedded at the roots of Abrahamic and Hellenistic thought.
Engendered by ancient ambiguity about climactic points, insofar as these have long been pictured as heralding that perfection which endangers our essentially imperfect existence, this tension has expressed itself, again and again throughout European history, as a recurrent split. A split between those who, on one hand, respond to this promised climax with profound wariness and those who, on the other, express an impatient desire to usher in its arrival.
It is, at the least, noteworthy that there seems to be, in today’s AI discord, a recapitulation of this basic schism.
Thermodynamics & Applied Theogony
For contemporary accelerationists, their impatience for the prophesied climax is often couched in scientific-sounding vocabularies of thermodynamic pulsion rather than spiritual perfection. Yet the fundamental schema remains identical.
In one version, the mind’s prophesied culmination is envisioned as achieved by the full unshackling of technocapital’s disruptive and dissipative flows, out of which an inhuman superintelligence is supposedly assembling itself, even if this requires relinquishing attachment to more allegedly profane concerns of safety and stability. At least, that’s the grand narrative professed to be at play.
To the degree that regulations and talk of ethics retard this pre-destiny, such hidebound humanistic habits are cast as imperfections to be sidestepped. At the logical extreme, should humankind itself continue to dam the cataract of intelligence’s entropic perfection, our displacement becomes laudable.
However, though assenting acceleration to the point of existential abandon is provocative enough to attract media attention — and murmurs of a conspiracy against the species appeal luridly to ultimate friend-enemy distinctions — there’s nothing new or even that heterodox here, at least considered against Abrahamic eschatology.
We’ve seen how, already in the 1910s, Teilhard had produced a remarkably similar vision: albeit couched in the friendlier vocabulary of mystical self-renunciation rather than thermodynamic luxuriance. The central thrust, however, is fundamentally the same.
Writing several years after his moonlit epiphany, Teilhard admitted that intelligence’s apotheosis, in his eyes, demands “evacuation, pure and simple, of the old mankind.” It can only be consummated in our “death.” He compared the arrival of the posthuman to a tumor: budding off from humanity, “pushing it aside and taking its place.”
Death By Perfection
This all began, perhaps, with a linguistic quirk, an accident of accidence. Across Indo-European languages, terms for spiritual perfection — represented, for example, in Sanskrit by nirvāṇa, in Arabic by fanāʾ or in Pāli by parinibbuto — also connote “extinguishment” or “extinction.” In Greek, “perfection” is expressed with teleios, which is closely related to the word telos, which, of course, means “end.” Another kindred word, teleute, means “to finish” or “die.” An inevitable knotting of meaning is the result.
This semantic entanglement became most influentially elevated by Neoplatonism, which argued that psychical perfection — both the primal and final state of things — demands the annihilation of everything we are, insofar as we are defined by what distinguishes and delimits us, and therefore exist only as a temporary errancy from perfection’s primordial unity and supremacy.
Influenced by Neoplatonism, Abrahamic theology’s innovation was to take this dynamic and apply it to cosmic history writ large, stretching from Genesis to the End of Days. For Christendom, Jesus’s crucifixion at Golgotha split this timeline cleanly in two and provided the emblem and precursor for time’s entire tendency. Christ, that is, according to the Bible and Church Fathers, was “perfected” by dying on the cross.
“It seems, in our putatively secular age, heresiology is as healthy as ever and religiosity, ironically, the rifest transgression.”
Thanks largely to Augustine, the conviction that spiritual perfection is to be found for the deserving in death alone became doctrinally cemented. This allowed an ongoing, recurring schism between those convinced such an outcome must be staved off — even if only rhetorically — set against those believing it is unavoidable and must be accelerated.
On the one hand, there were voices like Gregory of Nyssa, who, from 4th century Anatolia, warned against “arriving too quickly at the limit of perfection” because, presumably, this entails termination. On the other, some writers wrote about this tendency not in the spirit of hesitation but of anticipation. Medieval mystics, from Eriguena to Eckhart, accordingly wrote of divinity as both a sort of super intellect and endpoint: toward which all history tends, manifesting itself from our strivings whilst also motivating them, the arrival of which entails the obliteration of everything positively human.
Radicals even attempted to put this into practice. Take, for example, the Brethren of the Free Spirit, who believed they could embody divinity by stripping away everything humane. This entailed rejecting all practices of morality and self-denial as petty repressions. For believing that what is godly is, in its omnipotence, beyond all moral restraint — or, in modern AI parlance, cannot be “aligned” — followers of the Free Spirit concluded that the pathway to manifesting superior modes of life likewise demanded overstepping all such regulation.
Called antinomians back then, they were the spiritual accelerationists of their day: willing the realization of “unaligned” godliness, through the pursuit of untrammeled deregulation and consumption, even to the point of self-annihilation.
Superintelligence, Omnipotence & Millennium
The furthest possible cry from the latter-day antinomianism of today’s accelerationists are those concerned with AI safety, regulation and precaution. Nonetheless, some of their arguments undeniably also contain Abrahamic echoes.
Certain discussions of superintelligence, for example, closely resemble late Medieval concern over omnipotence. This began in reaction to Scholastic notions of a rationally ordered cosmos, where everything has a demonstrable reason for being the way it is rather than any other way. Islamic and Christian scholars protested that such assumptions put unacceptable limits upon God’s freedom to have acted otherwise in creating things differently. Theologians thus began pursuing arguments edifying and extremifying divine omnipotence.
Beginning in Ashʿarite strains of Islamic theology and climaxing in the radical views of Franciscans like William of Ockham, this produced the troubling specter of divinity as a kind of super-agent: beyond all anthropomorphism, anticipation and governance; capable of unlimited manipulation of both matter and mind.
Late Medieval theologians thought-experimented luridly upon the counterintuitive and disturbing abilities such a hyper-agent might possess. Divinity here became sheer power, uncoupled from axes of virtue or vice, like contemporary conjectures on “paperclip maximizing” superintelligences. Theologians accordingly distinguished the ways deity has operated, in not acting immorally or revoking natural law, from the vaster range of ways it could possibly act. This distinction, between divinity’s so-called “ordered” and “absolute” agency, resembles the distinction researchers talk about today concerning potential superintelligences being “aligned” or “unaligned” relative to human interests and safety.
These notions proved frightening. Historians, indeed, have argued it was because he was so haunted by such ideas that Descartes sought a new redoubt of certainty — cogito ergo sum — which could conceptually tame omnipotence’s overreaches.
Moreover, so too is it true that some concerned with AI risk seem to be motivated by their own visions of historical perfection. The implication being that an aligned AGI — being functionally omniscient — would provide some solution to existence’s riddle, yielding final closure to history’s travails, proffering optimal answers to all our problems or, at least, the eradication of disagreement. This mirrors age-old Christian anticipations of a millennium of happy stasis on Earth after secular history’s closure.
Further urgency is derived from the theorization that, with the help of aligned AGIs, humanity may flood the universe with consciousness’s light. Whether you find such goals supernally inspiring or abysmally hubristic, it isn’t difficult to notice this similarly echoes ancient Abrahamic conviction that a maximally full cosmos is a maximally preferable one.
Theists of centuries gone by, happily believing their Creator wrought the best world possible, assumed such fullness — of the cosmos by life — was simply a descriptive fact. Given modern science’s ongoing failure to find signs of minds elsewhere, this has now been recast, instead, as a practical goal. The driving desiderata remains the same, nonetheless.
What’s more, the nowhere-centered perspective from which such final outcomes to cosmic history can be judged, impartially, as “bad” or “good” itself undoubtedly resembles that of monotheism’s omniscient judge.
“For revealed religion, no question need remain unanswerable, which explains the ease with which it installs expectations and needs.”
On The Genealogy Of Worldviews
Our ideas do not emerge from a vacuum; they are secreted from prior genealogies. Intellectual history, like evolution, is a conservative thing because it also must be an opportunistic thing. Each generation cannot invent its worldview spontaneously; it inherits what came before and works with that. For this reason, certain master metaphors become entrenched — perpetuating themselves generationally, continuing to warp inferences — even if the original context in which they avowedly served some function has long since passed.
Further, the appearance and prevalence of certain motifs earlier in an intellectual tradition — long before subsequent efforts to submit them to scrutiny — seem reason to suspect that, to the degree they still appeal, they may do so due to suprarational factors.
Because monotheism played the predominant historical role in sculpting worldviews throughout Europe, it unavoidably echoes in the outlooks of secular individuals today. Inevitably, Holy Ghosts haunt Western conjectures concerning superintelligences. But what do we do with this?
Defining Secularization
To ascertain what’s at stake here, it helps to inspect how debates concerning secularized religion first gained prominence. Following Carl Schmitt’s lead, it was the German intellectual Karl Löwith who first full-throatedly argued in 1949 that modernity itself is just a concealed form of secularized religiosity. Post-war, such argumentation found fertile ground by addressing the ways godless philosophies seemed doomed to repeat the persecutions of the dogmatic past they purported to have superseded.
Löwith argued that modernity’s crowning concept of “progress” is illegitimate because it is merely secularized eschatology. Modernity’s core dynamo is not even its own, he alleged, but the dishonest expropriation of an essentially religious conception.
If accurate, this charge would be fatal for modernity, as that epoch which professes to stand on its own terms, in clean breakage with the dogmas of tradition. The broad aim was thus one of genealogical undermining: an attack on the rational credentials of some notion through revealing how its origins are somehow extrarational.
Nonetheless, it is the 1964 reply of another German philosopher, Hans Blumenberg, that remains more enlightening. “Progress,” Blumenberg insisted, isn’t an idea simplistically stolen from eschatology. Indeed, it was, throughout the Middle Ages, precisely the continued failure of the apocalypse to arrive that allowed a sense of worldly history to itself consolidate, as it became gradually freed from the conviction that time is exhaustively orchestrated from beyond by divine diktat. Communities could thereafter begin to feel they were authors of their own histories, lending time trajectory in the form of their own accumulating insights or inventions.
This is not, therefore, some unavowable theft on modernity’s behalf. Instead, “progress” is better thought of as the reoccupation of a conceptual role that, though originally ingrained by Christianity, was thereafter left yawning by Christian eschatology’s failure to deliver on its promises. Predictably, this new notion of worldly progress was subsequently enlisted to fulfill various explanatory roles that proved predictably irrational and overly ambitious: for example, in attempts to find history’s “purpose,” final goal or overarching tendency. But the fault here stems mostly from the overpromising of the parent, rather than the illegitimacy of the heir. So it remains today.
This is because monotheism, in Blumenberg’s eyes, has played the eminent part in shaping European thought; it has done so, largely, by instilling certain conceptual desires that, once entrenched, are hard to dispel. Indeed, for revealed religion, no question need remain unanswerable, which explains the ease with which it installs expectations and needs.
But conceptual habits are inertial. Once made manifest, they do not disappear upon their first frustration, but can only fade gradually. This explains why staunch atheist Bertrand Russell could feel such self-pity upon pondering the universe’s eventual heat death, whilst writing so passionately concerning how this seemingly annuls all human endeavour. For you can only feel cut off from a promised inheritance if, deep down, part of you still feels you deserved it (the inheritance in Russell’s case being eternal life, functioning as gratification of the deep-set Hellenistic expectation that what is valuable must also be imperishable).
It is this that drives the temptation to overstretch otherwise rational conceptions: to fill the gap and cater for legacy expectations, concerning cosmic homeliness or intelligibility, instilled by past religions. Notions like “progress” aren’t illegitimate thereby, because the culpability for their overstretching — say, into recurrently producing utopian convulsions or intoxicatingly totalizing narratives — lies more fairly elsewhere. That is, with the earlier religion that instilled, yet failed to fulfill, its eschatological promises in the first place. Perhaps something similar is happening in the vicinity of discussions about “human extinction” and “existential risk” today.
“Those attempting to address morality’s final fate in the cosmos seem particularly liable to be tempted to overstretch otherwise sensible ideas to fulfill essentially theological needs.”
Apocalypse Versus Extinction
In 2020, I published a book on the history of human extinction as an idea, arguing for its distinctly modern provenance as crisply distinct from religious apocalypse. Prior commentary largely overlooked this novelty, emphasizing the continuity of contemporary eschatologies with those of the past.
This, however, muddies a key distinction. Whereas apocalypse promises the ultimate consummation of moral order within the universe, extinction forecasts its irreversible termination. Extinction pictures moralizing animals disappearing and a physical cosmos persisting without us, containing no independent tendency to manifest our values. An acme of disenchantment. Whereas apocalypse secures a sense of an ending, extinction heralds only the ending of sense.
Nonetheless, I see now this doesn’t preclude the fact that existential questions — those attempting to address morality’s final fate in the cosmos — seem particularly liable to be tempted to overstretch otherwise sensible ideas to fulfill essentially theological needs. Needs bequeathed by the traditions one happens to have been born into.
In concluding my aforementioned book pondering the topic of potential cosmic imperatives — concerning some ultimate triumph of intellect over independent nature — I now believe I might have committed such overstretching myself.
But does such liability mean all fears of extinction are illegitimate? Absolutely not. No, there is something distinct — even epoch-making — in contemporary conceptions of existential catastrophe, as opposed to elder apocalypses. But this doesn’t mean that vestiges of the latter don’t subtly contaminate the former.
For the case of AI risk, specifically, the uncanny recurrence of theological themes should make us thoughtfully vigilant. But none of this means AI systems do not, and cannot, present extreme and existential risks. Indeed, it is precisely because of this that we ought to be abundantly cautious when it comes to potential distortions in framing the topic, arising from locally inherited traditions. After all, ideas like existential risk are treacherously well-shaped and thus especially susceptible to filling the gaps left by the frustrated expectations and promises of older religions.
Again, one way things seem susceptible to become distorted is in the expectation that, in the form of aligned AGI, history itself might have some final and optimal concluding state, some lastingly perfect answer to existence’s riddle. This lurks in convictions, seemingly held by some, that this is an attainability just within reach: one which thus demands to be locked in, irreversibly and without dithering or debate, henceforth to be propagated throughout the entire light cone.
Will History End?
But history, even after “safe alignment” of minds more powerful than our own — whatever that might mean or involve — won’t have a conclusion; no matter how much I yearn for simplicity or solutionism, given my steeping in monotheism’s legacies.
So long as there are rational agents around, of whatever sort, what we call history will continue, transforming its reasoners, along with their conundrums and solutions, without any end state in sight. Yet this is a good thing. Rationality rests, ultimately, in self-correction, whilst history is the only medium within which this unfolds. What value, after all, would there be in a value that cannot be refined, redrawn, renovated or rejected entirely?
Indeed, there can be no such thing as “human values” apart from the histories that spawn them. In ways shot through with contingency, of course, yet necessarily so. For they must be hewn from a certain gratuitousness, because otherwise the guides and guardrails we live by would not be our own, in the sense of being genuinely earnt through the precedent of voluntary effort and willful revision. They would, otherwise, be imposed, tyrannically and thus arbitrarily, from without.
Collaboration with artificial modes of thinking may disclose to tomorrow’s cogitators more capacious ways of comprehending what it even is to be a thing that thinks: transforming their priorities in turn, in ways impossible to anticipate.
This is why, beyond the inarguable priority of ensuring existential security, it is perhaps misguided to construe alignment in a strong sense as the supergluing of synthetic mind to some contemporaneous picture of what humans are and value. Like all self-images proposed throughout the past, any such picture will, given time, undoubtedly be proven to have been a premature reification. Shackling mind to such a fleeting snapshot would be a betrayal of what’s been most valuable about humans so far: namely, that we are a work in progress.
Successful alignment, then, isn’t to be framed as humane history’s conclusion; as some closing, clean, perfect solution; but as, instead, a deepening of what history itself might yet mean for cogitating creatures. It would be an opportunity — to see anew what we, as thinking things, are in the eyes of an other — and not an ending.
“Ideas like existential risk are treacherously well-shaped and thus especially susceptible to filling the gaps left by the frustrated expectations and promises of older religions.”
Existence, moreover, appears to be a riddle whose solution won’t simply be brute forced by amassing “more” intelligence. As the philosopher and historian of ideas Isaiah Berlin once sagely said, “some among the Great Goods cannot live together”: we are forever “doomed to choose, and every choice may entail an irreparable loss”. We needn’t deify dissensus because of this, nor jettison bold hope for a happier world, but perhaps it pays to focus on making our world survivable rather than solved. That is, both resilient enough to survive yet free enough to disagree.
Risks to our biosphere, particularly from emerging technologies, demand urgent attention; but coupling such endeavours with reveries of imminently perfect solutions to the word-riddle seems unhelpful. More plausibly, it is evidence of hangovers from the theological overpromises of the past. We can, however, desire the future to be supernally successful, but perhaps it’s best to approach this strictly as a guiding light rather than some constitutive calling: as a provisional hope, propelling all present action and inference, without demanding for itself imminence nor immanentizing.
Certainly, we all must act as if there is one perfectly complete answer to existence — upon which all possible minds couldn’t but “align” — because it is this expectation that motivates us, in the first place, to bother even replacing our opinions when others reveal their inadequacy. But this doesn’t mean such an outcome is destined, imminent, or even attainable. Ignoring this can transform hope’s distant lodestar into a supermassive blackhole on our doorstep: overwhelming our decisions with the crushing gravity of what seems at stake, coaxing rashness in pursuit of that perfect millennium, rather than sensibly shepherding present endeavors from afar.
Remembering this, we can perhaps retain visionary hope, whilst also strafing around the temptations to think or act rashly in the neighborhood of existential questions and existential risks. Given that the child is the parent of the adult, we ought to be appropriately careful with the initial conditions we are setting when it comes to begetting the minds of the future.
Ultimately though, there are probably ways our ideas never will be entirely undistorted by the pathways taken by their past histories through the unknowably wider space of what’s totally possible to conceive and believe. Uncloaking and scorning “secular religions,” after all, often involves assuming a viewpoint presuming itself entirely free from theological residues. But this simply isn’t how intellectual history works, as an edifice erected piecemeal from vestiges. And long may it continue, in all its faults and foibles, with both humans and their cunning artifacts together in tow. After all, I like humans, in their exuberant imperfection; most of my friends, so far, are humans.
By Thomas Moynihan
