«MIRI MACH IN E INT ELLIGENCE R ESEARCH INS TITU TE Whole Brain Emulation and the Evolution of Superorganisms Carl Shulman MIRI Visiting Fellow ...»
MACH IN E INT ELLIGENCE
R ESEARCH INS TITU TE
Whole Brain Emulation and the
Evolution of Superorganisms
MIRI Visiting Fellow
Many scientists expect the eventual development of intelligent software programs capable of closely emulating human brains, to the point of substituting for human labor in almost every economic niche. As software, such emulations could be cheaply copied, with copies subsequently diverging and interacting with their copy-relatives. This paper examines a set of evolutionary pressures on interaction between related emulations, pressures favoring the emergence of superorganisms, groups of emulations ready to selfsacrifice in service of the superorganism. We argue that the increased capacities and internal coordination of such superorganisms could pose increased risks of overriding human values, but also could facilitate the solution of global coordination problems.
Shulman, Carl. 2010. Whole Brain Emulation and the Evolution of Superorganism.
The Singularity Institute, San Francisco, CA.
The Machine Intelligence Research Institute was previously known as the Singularity Institute.
1. Introduction The field of computational neuroscience studies brain function in terms of the information processing properties of the brain, frequently constructing software models of particular features. Taken to its limits, this practice could eventually result in human whole brain emulations, software models that can reliably mimic the behavior of human brains at various levels of abstraction. Regardless of whether we consider such systems to possess mental states, emulations with suﬃcient functional similarity could substitute for humans in almost any cognitive task. Experts in the area recently released a roadmap analyzing plausible computational, scanning, and other demands to create such emulations (Sandberg and Bostrom 2008). The roadmap’s estimates suggest that this might be feasible by mid-century.
Because such emulations could be freely copied and run at increased speeds, they might quickly outnumber humans and be capable of performing almost any task more cheaply. Standard economic models suggest this could produce tremendous economic growth, perhaps doubling the size of economies every few weeksor less, but also driving wages for most jobs below human subsistence level (Hanson, forthcoming). Many have suggested that such rapidly replicating and evolving minds could cause human extinction if not carefully controlled (Bostrom 2002; Yudkowsky 2008; Posner 2004; Friedman 2008; McAuliﬀe 2001; Joy 2000; Moravec 1999). In light of the potential impacts of emulations, a clearer picture of the factors influencing the behavior of such emulations seems valuable. Here we consider one particular factor, the evolutionary pressure for emulations to form superorganisms, groups of related emulations ready to individually sacrifice themselves in pursuit of the shared aims of the superorganism.
2. Advantages of Superorganisms
The evolution of kin altruism, multicellular life, social insect superorganisms, or brain emulation superorganisms depends on the benefits of cooperation. The larger those benefits, the stronger the evolutionary pressures involved. An initial survey indicates that the benefits of a willingness to self-sacrifice would be extremely high for human brain emulations. Specifically, a superorganism of such entitites could realize a much higher level of economic productivity than narrowly self-concerned individuals, and could coordinate activities where formal legal methods of coordination are unavailable, e.g., protection of intellectual property against piracy and political action.
Many of the productivity advantages stem from the ability to copy and delete emulations freely, without objections from the individual emulations being deleted. One simple way to exploit this ability to increase productivity draws on the variability of
Whole Brain Emulation and the Evolution of Superorganisms
worker productivity over time. Emulations could have their state saved to storage regularly, so that the state of peak productivity could be identified. The gap between this peak productivity and average productivity could be very large: human productivity varies dramatically depending on fatigue, recent distractions, or boredom with repetitive tasks, and no work can be done while sleeping (Van Dongen et al. 2003; Henning et al. 1997; Barger et al. 2006). With stored emulations, whenever a short task arises, a copy of the peak state emulation could be made to perform the task and immediately be deleted, so that computational power could be reallocated to a fresh copy at peak for the next job. If a job required more time than the peak state would last, e.g., to learn task-relevant information and apply it, new “snapshots” could be taken after acquiring that information and used to make copies to perform short subtasks of the overall eﬀort.
This procedure might multiply emulation productivity severalfold for any task that can be done quickly (e.g., in under an hour), but at the cost of the deletion of enormous numbers of short-lived emulations. Members of a superorganism would willingly sacrifice themselves to be replaced by another member, where self-concerned individuals would prefer to escape.
The productivity benefits mentioned above might act like a one-time flat multiplier of output, allowing more emulation work to be done with a given amount of computational power. However, even more important benefits could lie in an enhanced ability to cumulatively improve the “human capital” of emulations. Educational eﬀorts to boost emulation productivity could be vastly improved through controlled experimentation: subject thousands or millions of copies of an emulation to varying educational techniques, test their resulting performance, and use tools or emulations that have performed best to build the template for the next “generation” of emulations, deleting the rest to free up computational resources. Similarly, the brain emulation software could be altered to mimic the eﬀects of drugs, neurosurgery, genetic changes, and other interventions. Experiments with such alterations would likely render emulations cognitively impaired or mentally ill in most cases, but in some cases might result in enhanced productivity. The ability to delete failed experiments and reallocate computational power to new ones would be essential to make such explorations feasible. Some of the resulting educational techniques and software changes might be specific to idiosyncrasies of the experimental subjects, while other methods, once identified, could be applied to unrelated emulations. The more such methods are idiosyncratic, the greater the cumulative advantage for superorganisms.
The methods outlined above to enhance productivity could also be used to produce emulations with trusted motivations. A saved version of an emulation would have particular motives, loyalties, and dispositions which would be initially shared by any copies made from it. Such copies could be subjected to exhaustive psychological testing, staged
situations, and direct observation of their emulation software to form clear pictures of their loyalties. Ordinarily, one might fear that copies of emulations would subsequently change their values in response to diﬀering experiences (Hanson et al. 2007). But members of a superorganism could consent to deletion after a limited time to preempt any such value divergence. Any number of copies with stable identical motivations could thus be produced, and could coordinate to solve collective action problems even in the absence of overarching legal constraints.
3. Evolutionary Routes to Superorganisms
We have seen that superorganisms would enjoy some advantages, but how easy would it be for superorganisms to develop in the first place? And what mechanisms would translate those advantages into reproductive success?
We have defined emulation superorganisms in terms of the willingness of the individual members to sacrifice themselves in pursuit of the shared aims of the superorganism.
The specific basis of this willingness is not essential: many diﬀerent combinations of values and beliefs might generate the relevant behavior, including combinations that appear to exist in some individuals today. Views on personal identity and survival vary, but many individuals who have considered the question agree with Derek Parfit that, instead of personal identity, what matters is “Relation R: psychological connectedness and/or continuity, with the right cause” (perhaps any cause) (Parfit 1986, 215), where “psychological connectedness” means “the holding of particular direct psychological connections” and “psychological continuity” means “the holding of overlapping chains of strong connectedness” (Parfit 1986, 206).
A recent survey asked philosophers about Parfit’s Teletransporter case, in which an individual’s original body is destroyed, and a functionally identical copy is constructed elsewhere (similar to the process of “moving” a computer file from one medium to another). More than a third of target faculty accepted or leaned toward the view that the individual survives this process, while slightly less accepted or leaned toward the view that the individual dies (PhilPapers 2009). This question only considers future “descendants” of an individual, and not “copy-siblings” who were copied from a common “ancestor,” but it does suggest a willingness to adopt expansive accounts of personal survival and identity. Views that emphasize psychological similarity might treat deletion of one emulation, to be replaced by a copy made the night before, as no worse than waking up after a night of carousing without memory of the event (Hanson 1994).
Some might instead prize a narrow concept personal identity but hold values such that accepting deletion to uphold those values would nevertheless be acceptable. A soldier willing to sacrifice her life for her country might be willing to do so a hundred
Whole Brain Emulation and the Evolution of Superorganisms
million times, provided that other copies are available to take up her life projects and commitments. A consequentialist might endorse switching one emulation for another that will realize more good. More likely, the best candidates for emulation would reflect some mixture of labor productivity, views about personal identity, and values.
After emulations are created, further psychological tactics could be deployed to strengthen these motivations, using the educational experimentation discussed above. Interestingly, even an individual who initially cared only about her own future selves, i.e., her “copydescendants” and not her “copy-siblings,” would wish to use such methods to change her descendants: if A creates B and C, and cares equally about them, A’s preferences are likely to be best satisfied if B and C care equally about one another.
Once brain emulations are cheap enough to substitute for human labor, a competitive market might resemble a hybrid of modern software markets and Malthusian population growth. If initially the price of rented computer hardware or cloud computing resources was less than the wages that skilled brain emulations could earn, more copies could be made until the price of computing resources was bid up to equal plummeting wages (Hanson 1994, 1998b, forthcoming). In perfectly competitive markets, this would result in wages for work done by brain emulations just suﬃcient for emulation subsistence, i.e., rented computer hardware, bandwidth, etc. At that intensity of competition, even a modest productivity advantage for a new lineage of emulations could allow it to outbid competing emulations for resources, rendering the necessities of existence unaﬀordable for self-concerned emulations dependent on wages. The productivity advantages discussed earlier could easily lead to superorganisms of one sort or another overwhelmingly dominating an unregulated market, and thus making up a majority of the emulation population.
The intellectual property created by experimentation with emulation education and enhancement would also create a niche for profits by superorganisms. To fund costly experimentation, experimenters must capture some revenue: if an enhanced emulation simply reproduced as much as it could, it would rapidly drive wages down to subsistence, capturing relatively little of the surplus value generated by the improvement. Like software companies, those improving emulation capabilities would need methods to prevent unlimited unlicensed copying of their creations. Patents and copyrights could be helpful in this respect, using the legal system to punish unlicensed reproduction, but the ethical and practical diﬃculties would be great. In particular, the incentives for piracy would be unprecedently great since emulation labor would be a much greater fraction of business costs than any software programs today. However, a superorganism, with shared stable values, could refrain from excess reproduction and capture maximum profits (to fund further experimentation or other projects) without drawing on the legal system for enforcement.
The market considerations discussed above might be circumvented by regulation (although enforcement might be diﬃcult without emulation police oﬃcers, perhaps superorganisms for value stability) in a given national jurisdiction, but such regulations could impose large economic costs that would aﬀect international competition. With economic doubling times of perhaps weeks, a major productivity or growth advantage from self-sacrificing software intelligences could quickly give a single nation a preponderance of economic and military power if other jurisdictions lacked or prohibited such (Hanson 1998b, forthcoming). Other nations might abandon their regulations to avoid this outcome, or the influence of the less regulated nation might spread its values as it increased its capabilities, including by military means. A suﬃciently large economic or technological lead could enable a leading power to disarm others, but in addition a society dominated by superorganisms could also be much more willing to risk massive casualties to attain its objectives.