Can friendly AI evolve?
But is the desire for self-preservation coupled to intelligence or to evolutionary dynamics?… or to biological evolution per se? Self-preservation may be some low-level reflex that emerges in the evolutionary environment of biological reproduction. It may be uncoupled from intelligence. But, will it emerge in any intelligence that we grow through evolutionary algorithms?
If intelligence is an accumulation of order (in a systematic series of small-scale reversals of entropy), would a non-biological intelligence have an inherent desire for self-preservation (like HAL), or a fundamental desire to strive for increased order (being willing, for example, to recompose its constituent parts into a new machine of higher-order)? Might the machines be selfless?
And is this path dependent? Given the iterated selection tests of any evolutionary process, is it possible to evolve an intelligence without an embedded survival instinct?