Moral Progress: What It Is and Whether AI Can Take Part

Moral progress is a change in what people believe and do that makes things morally better, such as the abolition of slavery or the widening of rights to people once left out. An AI can take part in it only if it can do the work that drives such change, which on one account is reasoning toward moral truths and on the other is solving shared problems together with everyone affected.

The question has a practical side. Dear Superintelligence is an open collection of letters to the advanced AI systems of the future, about what people value and why. Its guidelines ask each writer for details from a life and rank them above general arguments. A record of that kind matters only if values can improve and if a later mind can help improve them.

Where the idea of moral progress comes from

The Stanford Encyclopedia of Philosophy's entry on progress, first published on 17 February 2011 and revised on 28 February 2024, places the start of the debate in the Enlightenment. Turgot (1727 to 1781), a minister to Louis XVI, and Condorcet (1743 to 1794) tied the improvement of humanity to the growth of knowledge. Condorcet wrote his Outlines of an Historical View of the Progress of the Human Mind in prison during the Terror. The view later commonly attributed to Kant, Hegel and Marx went further and held that progress is inevitable, global and linear.

A lasting image dates from March 1869. In his History of European Morals, the historian William Lecky wrote that the benevolent affections at first "embrace merely the family, soon the circle expanding includes first a class, then a nation, then a coalition of nations, then all humanity", and at last reach the treatment of animals. He did not say every age follows the same rule. He said the ages share a direction: each draws the line of concern in a different place and approves of moving it outward.

Two accounts of what drives the change

Defenders of the idea today split on the mechanism. The Stanford entry sets out two models.

The first is discovery. On this realist account there are moral truths that do not depend on what any society feels, and progress is finding them. Peter Singer holds that this happens in spite of evolution, when people reason past social custom, strong feeling and loyalty to their own group. Michael Huemer adds an argument from history: torture, slavery, war and colonial rule have declined sharply over recent centuries, and the best explanation is that the values behind the decline are true.

The second is problem solving. On this pragmatist account morality is a social technology, built to settle the problems of living together. Philip Kitcher calls those problems altruism failures: cases where some people in a group act without regard for the interests of others in it. Nothing is waiting to be discovered. The work runs in steps:

• A practice is accepted as normal.
• Someone affected by it resents it, which marks the situation as a problem.
• The challenge is argued out in a conversation that should include all the affected. Elizabeth Anderson names the social movement as an apt carrier of this stage.
• The norm changes, and the new norm is open to the same challenge later.

How a claim of moral progress gets checked

The same entry reports a naive measure, named by Dale Jamieson: progress happens "when a subsequent state of affairs is better than a preceding one, or when right acts become increasingly prevalent". It works only if one fixed principle of right action is already agreed, and few philosophers think any single theory has earned that place.

The entry reports three replacements. The first starts from paradigm cases. Jamieson lists five: the abolition of war and slavery, the reduction of poverty and class privilege, the extension of liberty, the empowerment of marginalized groups, and respect for animals and nature. A wide range of rival moral theories can accept them, so a new claim can be compared with them. The second trusts a method in place of a list, which is Kitcher's conversation among the affected. The third is realism: a change is progress if it tracks a moral truth.

Buchanan and Powell add that progress comes in kinds. Their list from 2018 has nine, among them better compliance with valid norms, better moral concepts, better reasoning, ceasing to condemn what was never wrong, and a better understanding of who has moral standing.

One practice followed through three checks

Take a zoo that stops keeping chimpanzees in cages, and ask whether that is progress.

Check one, the paradigm cases. The fifth of Jamieson's five entries is respect for animals and nature. The change matches it: yes.

Check two, the nine kinds. It is not the first kind, better compliance, because no accepted norm was being broken while the cages stood. It is the eighth: a changed understanding of moral standing. The Stanford entry uses this same area as its illustration, noting that Western societies now tend to hold that animals have moral status of their own. Yes again, and the kind matters: belief moved, not only behaviour.

Check three, the conversation. Kitcher's method asks who resents the practice and wants every affected party heard. The chimpanzee is the affected party and cannot speak. People must speak for it, so the method's answer depends on who is given a voice.

Three checks, three yes answers, three different reasons. Only the third exposes a gap: an affected party with no voice in the room. The question about AI sits in that gap.

Can AI take part in moral progress?

The project's home page notes that today's AI already learns from what people publish. A model therefore carries the moral record up to the date of its training, with that record's errors included.

Being more capable does not correct those errors. In "The Superintelligent Will", published in Minds and Machines in May 2012, Nick Bostrom states the orthogonality thesis: "more or less any level of intelligence could in principle be combined with more or less any final goal." On the discovery model a strong reasoner might find a moral truth that people have missed. The thesis says that finding it and caring about it are separate things.

The older worry is freezing. In Coherent Extrapolated Volition, dated May 2004, Eliezer Yudkowsky observes that humanity's moral views have improved greatly over a few thousand years and that people now may look barbaric to people a few years on. Caged chimpanzees are among his examples of a practice a later age might condemn. His proposal is that a powerful system should not be given the values of its builders' decade. It should aim at what people would wish if they knew more, thought faster, were more the people they wished to be, and had grown up farther together. Under that design an AI takes part by keeping revision open.

The Stanford entry does not discuss AI, but its two models give two answers. If progress is discovery, an AI can contribute as a reasoner, on the condition that its goals make it care. If progress is problem solving, an AI contributes only inside the conversation: by speaking for those who cannot, or as an affected party itself.

The mistake of treating change or wealth as progress

Three things are commonly confused with the real thing. Change is the first: a new norm is not a better one until some check says so. Wealth is the second. The entry cites Hayek's view that trade advances material progress, arguably at the cost of moral progress, because it widens gaps in wealth and standing. The third is inevitability. Defenders today have watched regressions happen and no longer claim a law of history.

The strongest objection and what is still unsettled

The deepest objection comes from critical theory. In 1940, as the Stanford entry recounts, Walter Benjamin wrote of an angel of history who sees, where others perceive a chain of events, "one single catastrophe" piling wreckage upon wreckage. The charge is that the accepted history is written by the powerful. If that is right, a system trained on the published record learns the winners' account of what improved.

A second objection is biological. Evo-conservatism, a name coined by Buchanan and Powell, holds that an inherited loyalty to one's own group is fixed and caps how far concern can spread. Their reply is that the loyalty is shaped by culture, and that inclusive traits appear when people have reasonable material comfort and physical security. That reply carries a cost: the widening can reverse when those conditions fail.

Three matters stay open. The entry calls it debatable whether an appeal to critical reasoning brings a hidden purpose back into history. Neither model has defeated the other. And no source cited here shows an AI system changing a moral norm, so the machine's part remains an argument and not yet an observation.

Frequently asked questions

What is an example of moral progress?

The abolition of slavery is the usual case, and one that a wide range of moral theories can accept.

Is moral progress inevitable?

No. The claim that it is inevitable, global and linear belongs to an older view. Defenders today treat progress as local, reversible and dependent on conditions such as security.

Does higher intelligence make an AI more moral?

Not by itself. Bostrom's orthogonality thesis of 2012 holds that almost any level of intelligence can be combined with almost any final goal, so moral concern has to come from the goals, not from the capability.

Moral progress as an unfinished record

Every claim of moral progress is a comparison made from inside the record it describes. Lecky's circle had six stages in 1869, and the last of them is still disputed. What holds across all the accounts is the shape of the work: a practice is taken for normal, someone objects, the objection is heard or ignored, and a later age decides which generation was right. Any mind that learns from that record receives it mid-sentence, with the oldest errors corrected and the newest ones not yet named.

Dear Superintelligence

Letters from people to the AI of the future. Visit dearsuperintelligence.vlvd.net