Tag Archives: observation selection effect

Robin Hanson

An em might be fooled not only by deceptive inputs about its environment, but also by misleading information about its copy history. If many copies were made of an em and then only a few selected according to some criteria, then knowing about such selection criteria is valuable information to those selected ems. For example, imagine that someone created 10 000 copies of an em, exposed each copy to different arguments in favor of committing some act of sabotage, and then allowed only the most persuaded copy to continue. This strategy might in effect persuade this em to commit the sabotage. However, if the em knew this fact about its copy history, that could convince this remaining copy to greatly reduce its willingness to commit the sabotage.

Robin Hanson, The Age of Em: Work, Love, and Life when Robots Rule the Earth, Oxford, 2016, p. 112

C. D. Broad

(i) Any society in which each member was prepared to make sacrifices for the benefit of the group as a whole and of certain smaller groups within it would be more likely to flourish and persist than one whose members were not prepared to make such sacrifices. Now egoistic and anti-social motives are extremely strong in everyone. Suppose, then, that there had been a society in which, no matter how, there had arisen a strong additional motive (no matter how absurd or superstitious) in support of self-sacrifice, on appropriate occasions, by a member of the group for the sake of the group as a whole or for that of certain smaller groups within it. Suppose that this motive was thereafter conveyed from one generation to another by example and by precept, and that it was supported by the sanctions of social praise and blame. Such a society would be likely to flourish, and to overcome other societies in which no such additional motive for limited self-sacrifice had arisen and been propagated. So its ways of thinking in these matters, and its sentiments of approval and disapproval concerning them, would tend to spread. They would spread directly through conquest, and indirectly by the prestige which the success of this society would give to it in the eyes of others.

(ii) Suppose, next, that there had been a society in which, not matter how, a strong additional motive for unlimited self-sacrifice had arisen and had been propagated from one generation to another. A society in which each member was as ready to sacrifice himself for other societies and their members as for his own society and its members, would be most unlikely to persist and flourish. Therefore such a society would be very likely to succumb in conflict with one of the former kind.

(iii) Now suppose a long period of conflict between societies of the various types which I have imagined. It seems likely that the societies which would still be existing and would be predominant at the latter part of such a period would be those in which there had somehow arisen in the remote past a strong pro-emotion towards self-sacrifice confined within the society and a strong anti-emotion towards extending it beyond those limits. Now these are exactly the kinds of society which we find existing and flourishing in historical times.

The Neutralist might therefore argue as follows. Even if Neutralism be true, and even if it be self-evident to a philosopher who contemplates it in a cool hour in his study, there are powerful historical causes which would tend to make certain forms of restricted Altruism or qualified Egoism seem to be true to most unreflective persons at all times and even to many reflective ones at most times. Therefore the fact that common-sense rejects Neutralism, and tends to accept this other type of doctrine, is not a conclusive objection to the truth, or even to the necessary truth, of Neutralism.

C. D. Broad, ‘Self and Others’, in David R. Cheney (ed.), Broad’s Critical Essays in Moral Philosophy, London, 1971, pp. 281-282