172. First let us postulate that the computer scientists

succeed in developing intelligent machines that can do all

things better than human beings can do them. In that case

presumably all work will be done by vast, highly organized

systems of machines and no human effort will be necessary.

Either of two cases might occur. The machines might be

permitted to make all of their own decisions without human

oversight, or else human control over the machines might

be retained.

173. If the machines are permitted to make all their own

decisions, we can't make any conjectures as to the results,

because it is impossible to guess how such machines might

behave. We only point out that the fate of the human race

would be at the mercy of the machines. It might be argued

that the human race would never be foolish enough to hand

over all the power to the machines. But we are suggesting

neither that the human race would voluntarily turn power

over to the machines nor that the machines would willfully

seize power. What we do suggest is that the human race

might easily permit itself to drift into a position of such

dependence on the machines that it would have no practical

choice but to accept all of the machines' decisions. As

society and the problems that face it become more and

more complex and machines become more and more

intelligent, people will let machines make more of their

decisions for them, simply because machine-made decisions

will bring better result than man-made ones. Eventually a

stage may be reached at which the decisions necessary to

keep the system running will be so complex that human

beings will be incapable of making them intelligently. At that stage the machines will be in effective control. People won't be able to just turn the machines off, because they will be so dependent on them that turning them off would amount to suicide.

174. On the other hand it is possible that human control

over the machines may be retained. In that case the

average man may have control over certain private

machines of his own, such as his car or his personal

computer, but control over large systems of machines will

be in the hands of a tiny elite—just as it is today, but with two differences. Due to improved techniques the elite will have greater control over the masses; and because human

work will no longer be necessary the masses will be

superfluous, a useless burden on the system. If the elite is

ruthless they may simply decide to exterminate the mass of

humanity. If they are humane they may use propaganda or

other psychological or biological techniques to reduce the

birth rate until the mass of humanity becomes extinct,

leaving the world to the elite. Or, if the elite consist of soft-hearted liberals, they may decide to play the role of good shepherds to the rest of the human race. They will see to it that everyone's physical needs are satisfied, that all children are raised under psychologically hygienic conditions, that everyone has a wholesome hobby to keep him busy, and that anyone who may become dissatisfied undergoes

"treatment" to cure his "problem." Of course, life will be so purposeless that people will have to be biologically or

psychologically engineered either to remove their need for

the power process or to make them "sublimate" their drive

for power into some harmless hobby. These engineered

human beings may be happy in such a society, but they

most certainly will not be free. They will have been reduced

to the status of domestic animals.

175. But suppose now that the computer scientists do not

succeed in developing artificial intelligence, so that human

work remains necessary. Even so, machines will take care of

more and more of the simpler tasks so that there will be an

increasing surplus of human workers at the lower levels of

ability. (We see this happening already. There are many

people who find it difficult or impossible to get work,

because for intellectual or psychological reasons they

cannot acquire the level of training necessary to make

themselves useful in the present system.) On those who are

employed, ever-increasing demands will be placed; They will

need more and more training, more and more ability, and

will have to be ever more reliable, conforming and docile,

because they will be more and more like cells of a giant

organism. Their tasks will be increasingly specialized so that their work will be, in a sense, out of touch with the real world, being concentrated on one tiny slice of reality. The system will have to use any means that it can, whether

psychological or biological, to engineer people to be docile, to have the abilities that the system requires and to "sublimate" their drive for power into some specialized task. But the statement that the people of such a society will have to be docile may require qualification. The society may find competitiveness useful, provided that ways are found of directing competitiveness into channels that serve that needs of the system. We can imagine a future society in which there is endless competition for positions of prestige and power. But no more than a very few people will ever reach the top, where the only real power is (see end of paragraph 163). Very repellent is a society in which a person can satisfy his needs for power only by pushing large numbers of other people out of the way and depriving them of their opportunity for power.

176. One can envision scenarios that incorporate aspects of

more than one of the possibilities that we have just

discussed. For instance, it may be that machines will take

over most of the work that is of real, practical importance,

but that human beings will be kept busy by being given

relatively unimportant work. It has been suggested, for

example, that a great development of the service industries

might provide work for human beings. Thus people will

spend their time shining each others shoes, driving each

other around in taxicabs, making handicrafts for one

another, waiting on each other's tables, etc. This seems to

us a thoroughly contemptible way for the human race to

end up, and we doubt that many people would find fulfilling

lives in such pointless busy-work. They would seek other,

dangerous outlets (drugs, crime, "cults," hate groups) unless they were biologically or psychologically engineered to

adapt to such a way of life.

177. Needless to say, the scenarios outlined above do not

exhaust all the possibilities. They only indicate the kinds of

outcomes that seem to us most likely. But we can envision

no plausible scenarios that are any more palatable than the

ones we've just described. It is overwhelmingly probable

that if the industrial-technological system survives the next

40 to 100 years, it will by that time have developed certain

general characteristics: Individuals (at least those of the

"bourgeois" type, who are integrated into the system and

make it run, and who therefore have all the power) will be

more dependent than ever on large organizations; they will

be more "socialized" than ever and their physical and

mental qualities to a significant extent (possibly to a very

great extent) will be those that are engineered into them

rather than being the results of chance (or of God's will, or whatever); and whatever may be left of wild nature will be reduced to remnants preserved for scientific study and kept

under the supervision and management of scientists (hence

it will no longer be truly wild). In the long run (say a few

centuries from now) it is likely that neither the human race

nor any other important organisms will exist as we know

them today, because once you start modifying organisms

through genetic engineering there is no reason to stop at

any particular point, so that the modifications will probably continue until man and other organisms have been utterly transformed.

(http://www.autoadmit.com/thread.php?thread_id=5886209&forum_id=2Reputation#50024219)