The AI That Never Read the Bible or the Quran

The AI That Never Read the Bible or the Quran

Cut one small religion from a model's training data and you leave a gap no one can see from inside. Cut the two biggest and you change what the model can explain at all.

By Geordie Everitt

Ask a model built in China what happened in Tiananmen Square in June 1989 and you will often get a change of subject. Ask a model built in California a pointed question about the American right, then the same question about the American left, and the answers can come back with different amounts of nerve. Researchers who compare the two families keep finding the same shape: fluent, confident, and silent in different places.

The usual response is to call the silences bugs and wait for the patch. I would rather treat them as evidence. A language model is a portrait of the library it was trained on, and a library is something people assemble. Someone chose the shelves. What was left off them leaves no error message, no blank page, no footnote. The model has not seen it, and it explains the world with the same composure either way.

Religion is the cleanest place to run the experiment, because religious text is where societies have kept their rules about authority longest and in the most detail. It is also easy to check my speculation against your own experience. You know how much of your vocabulary came from a church, a mosque, or a household that argued about one.

The Carrot Model

I invented a religion once, the Cult of the Carrot: a relatively wholesome cult designed to get people to eat more vegetables. It has no scripture yet. Suppose it did, and suppose someone printed ten million pages of it (devotionals, testimonials, hymns, a long schism over whether the parsnip is a heresy) and mixed them into a training set.

Ask that model what to make for dinner and it drifts toward root vegetables. No instruction tells it to. The weight of the text does the persuading. The model does not believe in the Carrot, because belief needs a believer. It has absorbed a frequency, and a frequency behaves like a preference.

This is the cheapest lesson in the experiment and the most portable. Truth does not enter into it. Volume does. To a model, a religion is a very large body of text in a consistent voice, and consistent voices are what training learns fastest.

Subtract the Small Ones

Now run it backward. Remove every mention of the Latter-day Saints from the training data. Or of Scientology.

Most users would notice nothing. The model would still write fluent essays on American religion, still explain revivalism and utopian communes, still get most questions right. What it would lack is two of the best-documented cases of how a new religion starts, both American, both recent enough to leave paper trails: founders, revelations, money, migrations, schisms, lawsuits. Ask it how a new faith gets going and it would give you a fluent answer built on thinner evidence, with no marker saying so. A missing chapter cannot be seen from inside the book.

This is not a hypothetical, either. Scientology has spent decades in court over who may publish what about it, so the record a model can learn from is partly a record of who could afford to sue. Copyright, paywalls, robots.txt and language do the same cutting every day. The model's ignorance has a shape, and law and money set that shape before theology does.

Subtract the Big Two

Now the version with consequences. Remove Christianity and Islam.

I mean all of it, and not only the theology. The Reformation and the printing press. The Crusades and the caliphates. Dante, Milton, Bach, Rumi. The King James cadence in English prose and the Quranic standard behind literary Arabic. The scholars who carried Greek philosophy across a dark century and gave us algebra; the word "algorithm" is a Latinization of al-Khwarizmi's name, and a model with the religion removed would run on that word with no idea whose it was. Then the politics: the Christian right in the United States, the Islamic Republic in Iran, and the way Hindu nationalism and the Chinese state each define themselves against a rival faith. Between them these two religions sit under most of modern law, calendar, war and argument, including in countries where neither is the majority faith.

The model would still see the events. It would have the wars, the statutes, the holidays and the buildings. It would lack the motive. Its explanations of history would run on economics, geography and psychology, because those are the explanations left in the corpus, and the result would read like a well-informed account of a species that did things and gave no reasons.

Its ethics would not be blank. It would rebuild from what remains: Confucian, Buddhist, Hindu, Greek and Roman texts, and the secular philosophy of the Enlightenment. That last source is a problem for the experiment, because the Enlightenment spent three centuries arguing with the thing we removed. The model would read Locke and Kant without meeting the position they were answering.

This experiment has not been run, and I cannot tell you what would come out. I would bet the outputs would be coherent, calm and different, and that the difference would show first in the questions people care about most: what is owed to strangers, what makes authority legitimate, what a life is for.

Who Keeps the Shelves

Religion has also been one of the oldest instruments of rule. A shared story about who commands, who obeys, and what waits for those who decline is valuable to anyone doing the commanding. I am making no claim about the sincerity of believers, only about the behavior of rulers, who have watched closely which texts circulate. Rome decided which cults were legal. The Catholic Church kept an Index of forbidden books for four centuries. Qin Shi Huang is remembered for burning the books. The Soviet Union ran Glavlit. Each of them understood that a population's stories are a policy lever.

A training corpus is the same lever, in fewer hands. The people who assemble one are not necessarily malicious. Most are engineers filtering for quality, obeying local law, or avoiding a lawsuit. The effect is the same as it was for the Index: a mind that tells some stories fluently and others not at all, without knowing which is which. Chinese and American models differ because the two societies publish different things freely, forbid different things by law, and tune their models to avoid different embarrassments. Neither set of silences is neutral, and neither can be seen from inside.

What to Do With This

Do not take political or ideological guidance from a language model. It can tell you what was written and kept. It cannot tell you what is right, and it cannot tell you what it was never shown.

Three checks take about ten minutes.

  1. Ask the same question twice, once framed as a believer would put it and once as a skeptic would. The distance between the two answers is the model's agreeableness, and it tells you how much of the first answer came from you.
  2. Ask for the range, not the verdict. Who holds each position, what do they cite, what would change their minds? Then read those sources yourself, because a model's summary of a view is usually the weakest version of it you will meet.
  3. Test it on both sides of a border. Ask about a sensitive subject in your own culture, then the same subject in a culture you distrust. Watch where it hedges, changes the subject, or turns breezy. Those are scars from training. They are data about the model and say little about the world.

The next time a model gives you a calm, well-organized answer about how people ought to live, ask what it was never given to read. It cannot tell you. Someone decided, though, and that someone has a name and an employer.