Moravec on the sentimentality of humans
I recently read Hans Moravec’s excellent Mind Children: The Future of Robot and Human Intelligence.
Writing in 1988, he foreshadowed (or even opened) some of today’s debates on superintelligence and how humans can coexist with it. He looked at how compute can scale to AGI (estimating human-level machines on a personal computer around 2030), flagged recursive self-improvement, and argued that AI will economically displace us, becoming humanity’s evolutionary successor. He also discussed ways we might try to keep up, such as mind uploading.
However, after discussing the now-famous gradual neuron thought experiment in which a surgeon replaces your neurons one by one with functional equivalents, he wrote:
Immortality of the type I have just described is only a temporary defense against the wanton loss of knowledge and function that is the worst aspect of personal death. In the long run, our survival will require changes that are not of our own choosing. Parts of us will have to be discarded and replaced by new parts to keep in step with changing conditions and evolving competitors. Surviving means playing in a kind of cosmic Olympics, with each year bringing new events and escalated standards in old events. Though we are immortals, we must die bit by bit if we are to succeed in the qualifying event—continued survival. In time, each of us will be a completely changed being, shaped more by external challenges than by our own desires. Our present memories and interests, having lost their relevance, will at best end up in a dusty archive, perhaps to be consulted once in a long while by a historian. Personal death as we know it differs from this inevitability only in its relative abruptness. Viewed this way, personal immortality by a mind transplant is a technique whose primary benefit is to temporarily coddle the sensibility and sentimentality of individual humans. It seems to me that our civilization will evolve in the same direction whether or not we transplant our minds and join the robots.
Moravec seems unconcerned by a scenario that many would consider a disaster.