Originally posted on sciy.org by Rich Carlson on Mon 13 Apr 2009 10:25 PM PDT
Cybernetics Is An Antihumanism: Advanced Technologies and the Rebellion Against the Human Condition Foreword I
chose the topic of my contribution to our workshop after I discovered,
first with amazement, then with wonder, N. Katherine Hayles’s beautiful
book, How We Became Posthuman: Virtual Bodies in Cybernetics, Literature, and Informatics1.
Amazement because she and I worked on the same fairly confidential
corpus, in particular the proceedings of the Macy conferences, which
were the birthplace of cybernetics and, I have claimed, of cognitive
science, we celebrate the same heroes, in particular Warren McCulloch,
Heinz von Foerster and Francisco Varela, and, in spite of these shared
interests and passions, we apparently never heard of each other. She
and I live and work worlds and languages apart. The world is still far
from being a close-knit village. Wonder at realizing how from the same
corpus we could arrive at interpretations that, although compatible or
even complementary, are so richly diverse or even divergent. My book on the Macy conferences and the origins of cybernetics and cognitive science, Sur l’origine des sciences cognitives, was first published in French in 19852; a second and completely revised edition followed in 19943; the first English-language edition, an extensively revised and amplified version of the latter, came out in 20004.
It is with shame that I acknowledge that during all this time, I never
came across Ms. Hayles’ work, published in book form in 1999. It is
with great sadness that I realize that there is no longer any way that
I could ask my two great friends, Heinz von Foerster and Francisco
Varela, two men of communication, why they never put us in touch. The
Chilean neurophilosopher, Francisco Varela, was the cofounder of the
theory of autopoietic systems; he chose to come to France and work in
my research institution after he was expelled from his country. Heinz
von Foerster, a Viennese Jewish immigrant to the United States, after
serving as secretary to the Macy Conferences, went on to found what was
to be called second-order cybernetics. Francisco and Heinz play
important roles in the story that I tell in my book. The former passed
away in 2000; the latter in 2002. I miss them both terribly. My
book seeks to disabuse readers of a number of ideas that I consider
mistaken. Cybernetics calls to mind a series of familiar images that
turn out on closer inspection to be highly doubtful. As the etymology
of the word suggests, cybernetics is meant to signify control, mastery,
governance—in short, the philosophical project associated with
Descartes, who assigned mankind the mission of exercising dominion over
the world, and over mankind itself. Within the cybernetics movement,
this view was championed by Norbert Wiener—unsurprisingly, perhaps,
since it was Wiener who gave it its name. But this gives only a very
partial, if not superficial idea of what cybernetics was about,
notwithstanding that even a philosopher of such penetrating insight as
Heidegger was taken in by it. In my work, I have relied on the notion, due to Karl Popper, of a metaphysical research program,
which is to say a set of presuppositions about the structure of the
world that are neither testable nor empirically falsifiable, but
without which no science would be possible. For there is no science
that does not rest on a metaphysics, though typically it remains
concealed. It is the responsibility of the philosopher to uncover this
metaphysics, and then to subject it to criticism. What I have tried to
show is that cybernetics, far from being the apotheosis of Cartesian
humanism, as Heidegger supposed, actually represented a crucial moment
in its demystification, and indeed in its deconstruction. To borrow a
term that has been applied to the structuralist movement in the human
sciences, cybernetics constituted a decisive step in the rise of antihumanism. Consider,
for example, the way in which cybernetics conceived the relationship
between man and machine. The philosophers of consciousness were not
alone in being caught up in the trap set by a question such as “Will it
be possible one day to design a machine that thinks?†The
cybernetician’s answer, rather in the spirit of Molière, was: “Madame,
you pride yourself so on thinking. And yet, you are only a machine!â€
The aim of cognitive science always was—and still is today—the
mechanization of the mind, not the humanization of the machine. “Continentalâ€
political philosophy has yet to acknowledge the notion of posthumanism.
On the other hand, the notion of antihumanism has been debated for at
least four decades. My contribution will bear on the latter only. My
hope is that our workshop will enable us to explore the possible
connections between the two notions and, beyond, perhaps, bridge the
gap between two cultural worlds so far apart. 1. Heidegger’s Error I
will start with a classic question: can the idea that we have of the
human person, which is to say of ourselves, survive the forward march
of scientific discovery? It is a commonplace that from Copernicus to
molecular biology, and from Marx to Freud along the way, we have had
steadily to abandon our proud view of ourselves as occupying a special
place in the universe, and to admit that we are at the mercy of
determinisms that leave little room for what we have been accustomed to
consider our freedom and our reason. Is not cognitive science now in
the process of completing this process of disillusionment and
demystification by showing us that just where we believe we sense the
workings of a mind, there is only the firing of neural networks, no
different in principle than an ordinary electric circuit? The task in
which I have joined with many others, faced with reductive
interpretations of scientific advance of this sort, has been to defend
the values proper to the human person, or, to put it more bluntly, to
defend humanism against the excesses of science and technology. Heidegger
completely inverted this way of posing the problem. For him it was no
longer a question of defending humanism but rather of indicting it. As
for science and technology, or rather "technoscience" (an expression
meant to signify that science is subordinated to the practical ambition
of achieving mastery over the world through technology), far from
threatening human values, they are on Heidegger's view the most
striking manifestation of them. This dual reversal is so remarkable
that it deserves to be considered in some detail, even—or above all—in
a reflection on the place of cybernetics in the history of ideas, for
it is precisely cybernetics that found itself to be the principal
object of Heidegger's attack. In those places where
Heideggerian thought has been influential, it became impossible to
defend human values against the claims of science. This was
particularly true in France, where structuralism—and then
poststructuralism—reigned supreme over the intellectual landscape for
several decades before taking refuge in the literature departments of
American universities. Anchored in the thought of the three great
Germanic "masters of suspicion"—Marx, Nietzsche, and Freud—against a
common background of Heideggerianism, the human sciences à la française made antihumanism their watchword5,
loudly celebrating exactly what humanists dread: the death of man. This
unfortunate creature, or rather a certain image that man created of
himself, was reproached for being "metaphysical." With Heidegger,
"metaphysics" acquired a new and quite special sense, opposite to its
usual meaning. For positivists ever since Comte, the progress of
science had been seen as forcing the retreat of metaphysics; for
Heidegger, by contrast, technoscience represented the culmination of
metaphysics. And the height of metaphysics was nothing other than
cybernetics. Let us try to unravel this tangled skein. For
Heidegger, metaphysics is the search for an ultimate foundation for all
reality, for a "primary being" in relation to which all other beings
find their place and purpose. Where traditional metaphysics
("onto-theology") had placed God, modern metaphysics substituted man.
This is why modern metaphysics is fundamentally humanist, and humanism
fundamentally metaphysical. Man is a subject endowed with consciousness
and will: his features were described at the dawn of modernity in the
philosophy of Descartes and Leibniz. As a conscious being, he is
present and transparent to himself; as a willing being, he causes
things to happen as he intends. Subjectivity, both as theoretical
presence to oneself and as practical mastery over the world, occupies
center stage in this scheme—whence the Cartesian promise to make man
"master and possessor of nature." In the metaphysical conception of the
world, Heidegger holds, everything that exists is a slave to the
purposes of man; everything becomes an object of his will, fashionable
as a function of his ends and desires. The value of things depends
solely on their capacity to help man realize his essence, which is to
achieve mastery over being. It thus becomes clear why technoscience,
and cybernetics in particular, may be said to represent the completion
of metaphysics. To contemplative thought—thought that poses the
question of meaning and of Being, understood as the sudden appearance
of things, which escapes all attempts at grasping it—Heidegger opposes
"calculating" thought. This latter type is characteristic of all forms
of planning that seek to attain ends by taking circumstances into
account. Technoscience, insofar as it constructs mathematical models to
better establish its mastery over the causal organization of the world,
knows only calculating thought. Cybernetics is precisely that which
calculates—computes—in order to govern, in the nautical sense (Wiener
coined the term from the Greek xvbepvntns, meaning "steersman"): it is
indeed the height of metaphysics. Heidegger anticipated the
objection that would be brought against him: "Because we are speaking
against humanism people fear a defense of the inhuman and a
glorification of barbaric brutality. For what is more logical than that for somebody who negates humanism nothing remains but the affirmation of inhumanity?"6
Heidegger defended himself by attacking. Barbarism is not to be found
where one usually looks for it. The true barbarians are the ones who
are supposed to be humanists, who, in the name of the dignity that man
accords himself, leave behind them a world devastated by technology, a
desert in which no one can truly be said to dwell. Let us for
the sake of argument grant the justice of Heidegger's position. At once
an additional enigma presents itself. If for him cybernetics really
represented the apotheosis of metaphysical humanism, how are we to
explain the fact that the human sciences in France, whose postwar
development I have just said can be understood only against the
background of Heidegger's philosophy, availed themselves of the
conceptual toolkit of cybernetics in order to deconstruct the
metaphysics of subjectivity? How is it that these sciences, in their
utter determination to put man as subject to death, each seeking to
outdo the other's radicalism, should have found in cybernetics the
weapons for their assaults? From the beginning of the
1950s—which is to say, from the end of the first cybernetics—through
the 1960s and 1970s, when the second cybernetics was investigating
theories of self-organization and cognitivism was on the rise, the
enterprise of mechanizing the human world underwent a parallel
development on each side of the Atlantic. This common destiny was
rarely noticed, perhaps because the thought of any similarity seemed
almost absurd: whereas cognitive science claimed to be the avant-garde
of modern science, structuralism—followed by poststructuralism—covered
itself in a pretentious and often incomprehensible philosophical
jargon. What is more, it was too tempting to accuse French
deconstructionists of a fascination with mathematical concepts and
models that they hardly understood. But even if this way of looking at
the matter is not entirely unjustified, it only scratches the surface.
There were very good reasons, in fact, why the deconstruction of
metaphysical humanism found in cybernetics an ally of the first order. At
the beginning of the 1940s, a philosopher of consciousness such as
Sartre could write: "The inhuman is merely . . . the mechanical."7.
Structuralists hastened to adopt this definition as their own, while
reversing the value assigned to its terms. Doing Heidegger one better,
they made a great show of championing the inhuman—which is to say the
mechanical8.
Cybernetics, as it happened, was ready to hand, having come along at
just the right moment to demystify the voluntary and conscious subject.
The will? All its manifestations could apparently be simulated, and
therefore duplicated, by a simple negative feedback mechanism.
Consciousness? The “Cybernetics Groupâ€9
had examined the Freudian unconscious, whose existence was defended by
one of its members, Lawrence Kubie, and found it chimerical. If Kubie
often found himself the butt of his colleagues' jokes, it was not
because he was thought to be an enemy of human dignity. It was rather
because the postulation of a hidden entity, located in the substructure
of a purportedly conscious subject, manifesting itself only through
symptoms while yet being endowed with the essential attributes of the
subject (intentionality, desires, beliefs, presence to oneself, and so
on), seemed to the cyberneticians nothing more than a poor conjuring
trick aimed at keeping the structure of subjectivity intact. It
is remarkable that a few years later the French psychoanalyst Jacques
Lacan, along with the anthropologist Claude Lévi-Strauss and the
Marxist philosopher Louis Althusser (one of the founders of
structuralism), should have adopted the same critical attitude toward
Freud as cybernetics. The father of psychoanalysis had been led to
postulate an improbable "death wish"—"beyond the pleasure principle,"
as he put it—as if the subject actually desired the very thing that
made him suffer, by voluntarily and repeatedly placing himself in
situations from which he could only emerge battered and hurt. This
compulsion (Zwang) to repeat failure Freud called Wiederholungszwang, an expression translated by Lacan as "automatisme de répétition," which is to say the automatism
of repetition. In so doing he replaced the supposed unconscious death
wish with the senseless functioning of a machine, the unconscious
henceforth being identified with a cybernetic automaton. The alliance
of psychoanalysis and cybernetics was neither anecdotal nor fortuitous:
it corresponded to a radicalization of the critique of metaphysical
humanism. There was a deeper reason for the encounter between the French sciences de l'homme
and cybernetics, however. What structuralism sought to conceive—in the
anthropology of Lévi-Strauss, for example, and particularly in his
study of systems of exchange in traditional societies—was a subjectless
cognition, indeed cognition without mental content. Whence the project
of making "symbolic thought" a mechanism peculiar not to individual
brains but to "unconscious" linguistic structures that automatically
operate behind the back, as it were, of unfortunate human "subjects,"
who are no more than a sort of afterthought. "It thinks" was destined
to take the place once and for all of the Cartesian cogito. Now
cognition without a subject was exactly the unlikely configuration that
cybernetics seemed to have succeeded in conceiving. Here again, the
encounter between cybernetics and structuralism was in no way
accidental. It grew out of a new intellectual necessity whose sudden
emergence appears in retrospect as an exceptional moment in the history
of ideas. 2. The Self-Mechanized Mind It
is time to come back to our enigma, which now may be formulated as a
paradox. Was cybernetics the height of metaphysical humanism, as
Heidegger maintained, or was it the height of its deconstruction, as
certain of Heidegger's followers believe? To this question I believe it
is necessary to reply that cybernetics was both things at once, and
that this is what made it not only the root of cognitive science, which
finds itself faced with the same paradox, but also a turning point in
the history of human conceptions of humanity. The title I have given to
this section—the self-mechanized mind—appears to have the form of a
self-referential statement, not unlike those strange loops the
cyberneticians were so crazy about, especially the cyberneticians of
the second phase. But this is only an appearance: the mind that carries
out the mechanization and the one that is the object of it are two
distinct (albeit closely related) entities, like the two ends of a
seesaw, the one rising ever higher in the heavens of metaphysical
humanism as the other descends further into the depths of its
deconstruction. In mechanizing the mind, in treating it as an artifact,
the mind presumes to exercise power over this artifact to a degree that
no psychology claiming to be scientific has ever dreamed of attaining.
The mind can now hope not only to manipulate this mechanized version of
itself at will, but even to reproduce and manufacture it in accordance
with its own wishes and intentions. Accordingly, the technologies of
the mind, present and future, open up a vast continent upon which man
now has to impose norms if he wishes to give them meaning and purpose.
The human subject will therefore need to have recourse to a
supplementary endowment of will and conscience in order to determine,
not what he can do, but what he ought to do—or, rather, what he ought
not to do. These new technologies will require a whole ethics to be
elaborated, an ethics not less demanding than the one that is slowly
being devised today in order to control the rapid development and
unforeseen consequences of new biotechnologies. But to speak of ethics,
conscience, the will—is this not to speak of the triumph of the
subject? The connection between the mechanization of life and
the mechanization of the mind is plain. Even if the Cybernetics Group
snubbed biology, to the great displeasure of John von Neumann, it was
of course a cybernetic metaphor that enabled molecular biology to
formulate its central dogma: the genome operates like a computer
program. This metaphor is surely not less false than the analogous
metaphor that structures the cognitivist paradigm. The theory of
biological self-organization, first opposed to the cybernetic paradigm
during the Macy Conferences before later being adopted by the second
cybernetics as its principal model, furnished then—and still furnishes
today—decisive arguments against the legitimacy of identifying DNA with
a "genetic program." Nonetheless—and this is the crucial point—even
though this identification is profoundly illegitimate from both a
scientific and a philosophical point of view, its technological
consequences have been considerable. Today, as a result, man may be
inclined to believe that he is the master of his own genome. Never, one
is tempted to say, has he been so near to realizing the Cartesian
promise: he has become—or is close to becoming—the master and possessor
of all of nature, up to and including himself. Must we then
salute this as yet another masterpiece of metaphysical humanism? It
seems at first altogether astonishing, though after a moment's
reflection perfectly comprehensible, that a German philosopher
following in the tradition of Nietzsche and Heidegger, Peter
Sloterdijk, should have recently come forward, determined to take issue
with the liberal humanism of his country's philosophical establishment,
and boldly affirmed that the new biotechnologies sound the death knell
for the era of humanism. Unleashing a debate the like of which is
hardly imaginable in any other country, this philosopher ventured to
assert: "The domestication of man by man is the great unimagined
prospect in the face of which humanism has looked the other way from
antiquity until the present day." And to prophesy: But
why should this "superhuman" power of man over himself be seen, in
Nietzschean fashion, as representing the death of humanism rather than
its apotheosis? For man to be able, as subject, to exercise a power of
this sort over himself, it is first necessary that he be reduced to the
rank of an object, able to be reshaped to suit any purpose. No raising
up can occur without a concomitant lowering, and vice versa. Let
us come back to cybernetics and, beyond that, to cognitive science. We
need to consider more closely the paradox that an enterprise that sets
itself the task of naturalizing the mind should have as its spearhead a
discipline that calls itself artificial intelligence. To be sure, the
desired naturalization proceeds via mechanization. Nothing about this
is inconsistent with a conception of the world that treats nature as an
immense computational machine. Within this world man is just another
machine—no surprise there. But in the name of what, or of whom, will
man, thus artificialized, exercise his increased power over himself? In
the name of this very blind mechanism with which he is identified? In
the name of a meaning that he claims is mere appearance or phenomenon?
His will and capacity for choice are now left dangling over the abyss.
The attempt to restore mind to the natural world that gave birth to it
ends up exiling the mind from the world and from nature. This paradox
is typical of what the French sociologist Louis Dumont, in his
magisterial study of the genesis of modern individualism, called The
paradox of the naturalization of the mind attempted by cybernetics, and
today by cognitive science, then, is that the mind has been raised up
as a demigod in relation to itself. Many of the criticisms
brought against the materialism of cognitive science from the point of
view either of a philosophy of consciousness or a defense of humanism
miss this paradox. Concentrating their (often justified) attacks on the
weaknesses and naiveté of such a mechanist materialism, they fail to
see that it invalidates itself by placing the human subject outside of
the very world to which he is said to belong. The recent interest shown
by cognitive science in what it regards as the "mystery" of
consciousness seems bound to accentuate this blindness. 3. The Nanotechnological Dream Since my book was first published, I have
thought a great deal about the philosophical foundations of what is
called the NBIC Convergence—the convergence of nanotechnology,
biotechnology, information technology, and cognitive science—and about
the ethical implications of this development.12
Here I have found many of the same tensions, contradictions, paradoxes,
and confusions that I discerned first within cybernetics, and then
within cognitive science. But now the potential consequences are far
more serious, because we are not dealing with a theoretical matter, a
certain view of the world, but with an entire program for acting upon
nature and mankind. In searching for the underlying metaphysics
of this program, I did not have far to look. One of the first reports
of the National Science Foundation devoted to the subject, entitled
“Converging Technologies for Improving Human Performance,†summarizes
the credo of the movement in a sort of haiku: Note
that cognitive science plays the leading role in this division of
labor, that of thinker—not an insignificant detail, for it shows that
the metaphysics of NBIC Convergence is embedded in the work of
cognitive scientists. It comes as no surprise, then, that the
contradictions inherent in cognitive science should be found at the
heart of the metaphysics itself. One of the main themes of my
book is the confrontation between Norbert Wiener and John von Neumann,
Wiener embodying the ideas of control, mastery, and design, von Neumann
the ideas of complexity and self-organization. Cybernetics never
succeeded in resolving the tension, indeed the contradiction, between
these two perspectives; more specifically, it never managed to give a
satisfactory answer to the problems involved in realizing its ambition
of designing an autonomous, self-organizing machine.
Nanotechnology—whose wildest dream is to reconstruct the natural world
that has been given to us, atom by atom—is caught up in the same
contradiction. The most obvious element of the nanotechnological dream is to substitute for what François Jacob called bricolage, or the tinkering of biological evolution, a paradigm of design.
Damien Broderick, the Australian cultural theorist and popular science
writer, barely manages to conceal his contempt for the world that human
beings have inherited when he talks about the likelihood that
“nanosystems, designed by human minds, will bypass all this Darwinian
wandering, and leap straight to design success.â€14
One can hardly fail to note the irony that science, which in America
has had to engage in an epic struggle to root out every trace of
creationism (including its most recent avatar, “intelligent designâ€)
from public education, should now revert to a logic of design in the
form of the nanotechnology program—the only difference being that now
it is mankind that assumes the role of the demiurge. Philosophers,
faced with the ambition of emerging technologies to supersede nature
and life as the engineers of evolution, the designers of biological and
natural processes, may suppose that they are dealing with an old idea:
Descartes’ vision of science as the means by which man may become the
master and possessor of nature. Again, however, this is only part of a
larger and more complicated picture. As another influential visionary,
the American applied physicist Kevin Kelly, revealingly remarked, “It
took us a long time to realize that the power of a technology is
proportional to its inherent out-of-controlness, its inherent
ability to surprise and be generative. In fact, unless we can worry
about a technology, it is not revolutionary enough.â€15
With NanoBioConvergence, a novel conception of engineering has indeed
been introduced. The engineer, far from seeking mastery over nature, is
now meant to feel that his enterprise will be crowned by success only
to the extent that the system component he has created is capable of
surprising him. For whoever wishes ultimately to create a
self-organizing system—another word for life—is bound to attempt to
reproduce its essential property, namely, the ability to make something
that is radically new. In her masterful study of the perils facing mankind, The Human Condition (1958),
of which we are celebrating the fiftieth anniversary, Hannah Arendt
brought out the fundamental paradox of our age: whereas the power of
mankind to alter its environment goes on increasing under the stimulus
of technological progress, less and less do we find ourselves in a
position to control the consequences of our actions. I take the liberty
of giving a long quotation here whose pertinence to the subject at hand
cannot be exaggerated—keeping in mind, too, that these lines were
written fifty years ago: The
sorcerer’s apprentice myth must therefore be updated: it is neither by
error nor terror that mankind will be dispossessed of its own
creations, but by design—which henceforth is understood to signify not mastery, but non-mastery and out-of-controlness. 4. The Rebellion Against the Human Condition Arendt began the same, decidedly prescient book with the following words: The
nanotechnological dream that began to take shape only a few decades
after the utterance of Arendt’s prophesy amounts to exactly this revolt
against the finiteness, the mortality of the human condition. Human
life has an end, for it is promised to death. But not only do the
champions of NBIC Convergence oppose themselves to fate, by promising
immortality; they quarrel with the very fact that we are born. Their
revolt against the given is therefore something subtler and less
visible, something still more fundamental, than the revolt against
human mortality, for it rejects the notion that we should be brought
into the world for no reason. “Human beings are ashamed to have
been born instead of made.†Thus the German philosopher Günther Anders
(Arendt’s first husband and himself a student of Heidegger)
characterized the essence of the revolt against the given in his great
book, published in 1956, Die Antiquiertheit des Menschen--The Antiquatedness (or Obsolescence) of the Human Being.18
One cannot help recalling here another philosophical emotion: the
nausea described by Jean-Paul Sartre, that sense of forlornness that
takes hold of human beings when they realize that they are not the
foundation of their own being. The human condition is ultimately one of
freedom; but freedom, being absolute, runs up against the obstacle of
its own contingency, for we are free to choose anything except the
condition of being unfree. Discovering that we have been thrown
into the world without any reason, we feel abandoned. Sartre
acknowledged his debt to Günther Anders in expressing this idea by
means of a phrase that was to become famous: man is “to freedom
condemned.â€19 Freedom,
Sartre held, never ceases trying to “nihilate†that which resists it.
Mankind will therefore do everything it can to become its own maker; to
owe its freedom to no one but itself. But only things are what they
are; only things coincide with themselves. Freedom, on the other hand,
is a mode of being that never coincides with itself since it
ceaselessly projects itself into the future, desiring to be what it is
not. Self-coincidence is what freedom aspires to and cannot attain,
just as a moth is irresistibly attracted to the flame that will consume
it. A metaphysical self-made man, were such a being possible,
would paradoxically have lost his freedom, and indeed would no longer
be a man at all, since freedom necessarily entails the impossibility of
transforming itself into a thing. Thus Anders’ notion of “Promethean
shame†leads inexorably to the obsolescence of man. Had they
lived to see the dawn of the twenty-first century, Sartre and Anders
would have found this argument resoundingly confirmed in the shape of
the NBIC Convergence—a Promethean project if ever there was one. For
the aim of this distinctively metaphysical program is to place mankind
in the position of being the divine maker of the world, the demiurge,
while at the same time condemning human beings to see themselves as out
of date. At the heart of the nanotechnological dream we therefore
encounter a paradox that has been with us since the cybernetic chapter
in the philosophical history of cognitive science—an extraordinary
paradox arising from the convergence of opposites, whereby the
overweening ambition and pride of a certain scientific humanism leads
directly to the obsolescence of mankind. It is in the light, or perhaps
I should say the shadow, of this paradox that all “ethical†questions
touching on the engineering of mankind by mankind must be considered. 5. “Playing God†versus the Blurring of Fundamental Distinctions In 1964, Norbert Wiener published an odd book with the curious title God and Golem, Inc.: A Comment on Certain Points where Cybernetics Impinges on Religion. In it one finds this: The
rest of the book is devoted to mobilizing the resources of cybernetics
to show that these are false dichotomies and that, in truth, “machines
are very well able to make other machines in their own image.â€21 In
recent years, the enterprise of “making life from scratch†has been
organized as a formal scientific discipline under the seemingly
innocuous name of synthetic biology. In June 2007, the occasion of the
first Kavli Futures Symposium at the University of Greenland in
Ilulissat, leading researchers from around the world gathered to
announce the convergence of work in synthetic biology and
nanotechnology and to take stock of the most recent advances in the
manufacture of artificial cells. Their call for a global effort to
promote “the construction or redesign of biological systems components
that do not naturally exist†evoked memories of the statement that was
issued in Asilomar, California more than thirty years earlier, in 1975,
by the pioneers of biotechnology. Like their predecessors, the founders
of synthetic biology insisted not only on the splendid things they were
poised to achieve, but also on the dangers that might flow from them.
Accordingly, they invited society to prepare itself for the
consequences, while laying down rules of ethical conduct for themselves.22
We know what became of the charter drawn up at Asilomar. A few years
later, this attempt by scientists to regulate their own research had
fallen to pieces. The dynamics of technological advance and the greed
of the marketplace refused to suffer any limitation. Only a week
before the symposium in Ilulissat, a spokesman for the ETC Group, an
environmental lobby based in Ottawa that has expanded its campaign
against genetically modified foods to include emerging
nanotechnologies, greeted the announcement of a feat of genetic
engineering by the J. Craig Venter Institute in Rockville, Maryland
with the memorable words, “For the first time, God has competition.†In
the event, ETC had misinterpreted the nature of the achievement.23
But if the Ilulissat Statement is to be believed, the actual synthesis
of an organism equipped with an artificial genome (“a free-living
organism that can grow and replicateâ€) will become a reality in the
next few years. Whatever the actual timetable may turn out to be, the
process of fabricating DNA is now better understood with every passing
day, and the moment when it will be possible to create an artificial
cell using artificial DNA is surely not far off. The question arises, however, whether such an achievement will really amount to creating life.
In order to assert this much, one must suppose that between life and
non-life there is an absolute distinction, a critical threshold, so
that whoever crosses it will have shattered a taboo, like the prophet
Jeremiah and like Rabbi Löw of Prague in the Jewish tradition, who
dared to create an artificial man, a golem. In the view of its promoters and some of its admirers, notably the English physicist and science writer Philip Ball24,
synthetic biology has succeeded in demonstrating that no threshold of
this type exists: between the dust of the earth and the creature that
God formed from it, there is no break in continuity that permits us to
say (quoting Genesis 2:7) that He breathed into man’s
nostrils the breath of life. And even in the event that synthetic
biology should turn out to be incapable of fabricating an artificial
cell, these researchers contend, it would still have had the virtue of
depriving the prescientific notion of life of all consistency. It
is here, in the very particular logic that is characteristic of dreams,
that nanotechnology plays an important symbolic role. It is typically
defined by the scale of the phenomena over which it promises to exert
control—a scale that is described in very vague terms, since it extends
from a tenth of a nanometer25
to a tenth of a micron. Nevertheless, over this entire gamut, the
essential distinction between life and non-life loses all meaning. It
is meaningless to say, for example, that a DNA molecule is a living
thing. At the symbolic level, a lack of precision in defining
nanotechnology does not matter; what matters is the deliberate and
surreptitious attempt to blur a fundamental distinction that until now
has enabled human beings to steer a course through the world that was
given to them. In the darkness of dreams, there is no difference
between a living cat and a dead cat. Once again, we find that
science oscillates between two opposed attitudes: on the one hand,
vainglory, an excessive and often indecent pride; and on the other,
when it becomes necessary to silence critics, a false humility that
consists in denying that one has done anything out of the ordinary,
anything that departs from the usual business of normal science. As a
philosopher, I am more troubled by the false humility, for in truth it
is this, and not the vainglory, that constitutes the height of pride. I
am less disturbed by a science that claims to be the equal of God than
by a science that drains one of the most essential distinctions known
to humanity since the moment it first came into existence of all
meaning: the distinction between that which lives and that which does
not; or, to speak more bluntly, between life and death. Let me
propose an analogy that is more profound, I believe, than one may at
first be inclined to suspect. With the rise of terrorism in recent
years, specifically in the form of suicide attacks, violence on a
global scale has taken a radically new turn. The first edition of this
book belongs to a bygone era, which ended on 11 September 2001. In that
world, even the most brutal persecutor expressed his attachment to
life, because he killed in order to affirm and assert the primacy of
his own way of living. But when the persecutor assumes the role of
victim, killing himself in order to maximize the number of people
killed around him, all distinctions are blurred, all possibility of
reasoned dissuasion is lost, all control of violence is doomed to
impotence. If science is allowed, in its turn, to continue along this
same path in denying the crucial difference that life introduces in the
world, it will, I predict, prove itself to be capable of a violence
that is no less horrifying. Among the most extreme promises of
nanotechnology, as we have seen, is immortality (or “indefinite life
extension,†as it is called). But if there is thought to be no
essential difference between the living and the non-living, then there
is nothing at all extraordinary about this promise. Yet again, Hannah
Arendt very profoundly intuited what such a pact with the devil would
involve: The
ETC Group’s premonitory observation—“For the first time, God has
competitionâ€â€”can only strengthen the advocates of the NBIC Convergence
in their belief that those who criticize them do so for religious
reasons. The same phrases are always used to sum up what is imagined to
be the heart of this objection: human beings do not have the right to
usurp powers reserved to God alone; playing God is forbidden. Often it is added that this taboo is specifically “Judeo-Christian.†Let
us put to one side the fact that this allegation wholly misconstrues
the teaching of the Talmud as well as that of Christian theology. In
conflating them with the ancient Greek conception of the sacred—the
gods, jealous of men who have committed the sin of pride, hubris,
send after them the goddess of vengeance, Nemesis—it forgets that the
Bible depicts man as co-creator of the world with God. As the French
biophysicist and Talmudic scholar Henri Atlan notes with regard to the
literature about the Golem: Within
the Christian tradition, authors such as G. K. Chesterton, René Girard,
and Ivan Illich see Christianity as the womb of Western modernity,
while arguing that modernity has betrayed and corrupted its message.
This analysis links up with the idea, due to Max Weber, of the
desacralization of the world—its famous “disenchantmentâ€â€”in regarding
Christianity, or at least what modernity made of it, as the main factor
in the progressive elimination of all taboos, sacred prohibitions, and
other forms of religious limitation. It fell to science itself to
extend and deepen this desacralization, inaugurated by the religions of
the Bible, by stripping nature of any prescriptive or normative value.
It is utterly futile, then, to accuse science of being at odds with the
Judeo-Christian tradition on this point. Kantianism, for its part,
conferred philosophical legitimacy on the devaluation of nature by
regarding it as devoid of intentions and reasons, inhabited only by
causes, and by severing the world of nature from the world of freedom,
where the reasons for human action fall under the jurisdiction of moral
law. Where, then, is the ethical problem located, if in fact
there is one here? It clearly does not lie in the transgression of this
or that taboo sanctioned by nature or the sacred, since the joint
evolution of religion and science has done away with any such
foundation for the very concept of a moral limitation, and hence of a
transgression. But that is precisely the problem. For there is no free
and autonomous human society that does not rest on some principle of
self-limitation. We will not find the limits we desperately need in the
religions of the Book, as though such limits are imposed on us by some
transcendental authority, for these religions do nothing more than
confront us with our own freedom and responsibility. The ethical
problem weighs more heavily than any specific question dealing, for
instance, with the enhancement of a particular cognitive ability by one
or another novel technology. But what makes it all the more intractable
is that, whereas our capacity to act into the world is increasing
without limit, with the consequence that we now find ouselves faced
with new and unprecedented responsibilities, the ethical resources at
our disposal are diminishing at the same pace. Why should this be?
Because the same technological ambition that gives mankind such power
to act upon the world also reduces mankind to the status of an object
that can be fashioned and shaped at will; the conception of the mind as
a machine—the very conception that allows us to imagine the possibility
of (re)fabricating ourselves—prevents us from fulfilling these new
responsibilities. Hence my profound pessimism. 6. Alcmena’s Paradox To
pay Heinz von Foerster a final homage, I would like to conclude by
recounting a very lovely and moving story he told me, one that has a
direct bearing on the arguments developed here. The story takes
place in Vienna toward the end of 1945, and it concerns another
Viennese Jew, the psychiatrist Viktor Frankl, whose celebrated book Man’s Search for Meaning was to be published the following year.
Frankl had just returned to Vienna, having miraculously survived the
Auschwitz-Birkenau camp; in the meantime he had learned that his wife,
his parents, his brother, and other members of his family had all been
exterminated. He decided to resume his practice. Here, then, is the
story as my friend Heinz told it: This,
at least, is the lesson that von Foerster drew from this story—in
typical cybernetic fashion. But I think that another lesson can be
drawn from it, one that extends the first. What was it that this man
suddenly saw, which he did not see before? The thought experiment that
Frankl invited his patient to perform echoes one of the most famous
Greek myths, that of Amphitryon. In order to seduce Amphitryon’s wife,
Alcmena, and to pass a night of love with her, Zeus assumes the form of
Amphytryon. When
we love somebody, we do not love a list of characteristics, even one
that is sufficiently exhaustive to distinguish the person in question
from anyone else. The most perfect simulation still fails to
capture something, and it is this something that is the essence of
love—this poor word that says everything and explains nothing. I very
much fear that the spontaneous ontology of those who wish to set
themselves up as the makers or re-creators of the world know nothing of
the beings who inhabit it, only lists of characteristics. If the
nanobiotechnological dream were ever to come true, what still today we
call love would become incomprehensible. Endnotes 1 N. Katherine Hayles, How we became posthuman. Virtual Bodies in Cybernetics, Literature, and Informatics, Chicago, Chicago University Press, 1999. 2 Jean-Pierre Dupuy, L'essor de la première cybernétique (1943-1953), Paris, Ecole Polytechnique, Cahiers du CREA, 7, 1985. 3 Jean-Pierre Dupuy, Aux origines des sciences cognitives, Paris, La Découverte, 1994. 4 Jean-Pierre Dupuy, The Mechanization of the Mind,
Princeton, Princeton University Press, 2000. A revised paperback
edition is about to be published by the MIT Press under the title On the Origins of Cognitive Science. The Mechanization of the Mind (2008). 5 This point is clearly established by Luc Ferry and Alain Renaut, French Philosophy of the Sixties: An Essay on Antihumanism, trans. Mary H. S. Cattani, Amherst, University of Massachusetts Press, 1990. 6 Martin Heidegger, “Letter on Humanism†in Basic Writings, ed. David Farrell Krell, New York, Harper and Row, 1977, p. 225. 7 This phrase is found in the review Sartre wrote in 1943 of Albert Camus’s The Stranger, “Explications de l’Etrangerâ€, reprinted in Critiques littéraires (Situations I), Paris, Gallimard, 1947; available in English in Literary and Philosophical Essays, trans. Annette Michelson, New York, Criterion Books, 1955. 8 “To render philosophy inhuman†– thus the task Jean-François Lyotard set himself in 1984. 9 This expression is borrowed from Steve Heims’s indispensable book, The Cybernetics Group, Cambridge, Mass., MIT Press, 1991. 10
Peter Sloterdijk, “On the Rules of the Human Fleetâ€, a paper delivered
at a conference on Heidegger at Elmau Castle, Upper Bavaria, on July
17, 1999, and presented as a reply to Heidegger’s “Letter on Humanism.†11 Louis Dumont, Essays on Individualism: Modern Ideology in Anthropological Perspective, Chicago, University of Chicago Press, 1986. 12 See Jean-Pierre Dupuy, “Some Pitfalls in the Philosophical Foundations of Nanoethics,†Journal of Medicine and Philosophy
32, no. 3 (2007): 237-261; Jean-Pierre Dupuy, “Complexity and
Uncertainty: A Prudential Approach to Nanotechnology,†in John Weckert
et al., eds., Nanoethics: Examining the Social Impact of Nanotechnology (Hoboken,
N.J.: John Wiley and Sons, 2007), 119-131; Jean-Pierre Dupuy, “The
double language of science, and why it is so difficult to have a proper
public debate about the nanotechnology program,†Foreword to Fritz
Allhoff and Patrick Lin, eds., Nanoethics: Emerging Debates
(Dordrecht: Springer, 2008); and Jean-Pierre Dupuy and Alexei Grinbaum,
“Living with Uncertainty: Toward a Normative Assessment of
Nanotechnology,†Techné (joint issue with Hyle) 8, no. 2 (2004): 4-25. 13 Mihail C. Roco and William Sims Bainbridge, Converging
Technologies for Improving Human Performance: Nanotechnology,
Biotechnology, Information Technology, and Cognitive Science (Washington, D.C.: National Science Foundation, 2002), 13. 14 Damien Broderick, The Spike: How Our Lives Are Being Transformed by Rapidly Advancing Technologies (New York: Forge, 2001), 118. 15 See Kevin Kelly, “Will Spiritual Robots Replace Humanity by 2100?â€, in The Technium, a work in progress, https://www.kk.org./thetechnium/. 16 Hannah Arendt, The Human Condition (Chicago: University of Chicago Press, 1958), 231. 17 Ibid., 2-3. 18 Günther Anders, Die Antiquiertheit des Menschen: über die Seele im Zeitalter der zweiten industriellen Revolution, vol. 1 (Munich: Beck, 1980), 21-97.
By Jean-Pierre Dupuy
It
suffices to clearly understand that the next long periods of history
will be periods of choice as far as the [human] species is concerned.
Then it will be seen if humanity, or at least its cultural elites, will
succeed in establishing effective procedures for self-domestication. It
will be necessary, in the future, to forthrightly address the issue and
formulate a code governing anthropological technologies. Such a code
would modify, a posteriori, the meaning of classical humanism, for it
would show that humanitas consists not only in the friendship
of man with man, but that it also implies . . . , in increasingly
obvious ways, that man represents the supreme power for man.10
the
model of modern artificialism in general, the systematic application of
an extrinsic, imposed value to the things of the world. Not a value
drawn from our belonging to the world, from its harmony and our harmony
with it, but a value rooted in our heterogeneity in relation to it: the
identification of our will with the will of God (Descartes: man makes
himself master and possessor of nature). The will thus applied to the
world, the end sought, the motive and the profound impulse of the will
are [all] foreign. In other words, they are extra-worldly.
Extra-worldliness is now concentrated in the individual will.11
If the Cognitive Scientists can think it,
The Nano people can build it,
The Bio people can implement it, and
The IT people can monitor and control it.13To what extent we have begun to act into nature,
in the literal sense of the word, is perhaps best illustrated by a
recent casual remark of a scientist [Wernher von Braun, December 1957]
who quite seriously suggested that “basic research is when I am doing what I don’t know what I am doing.â€
This
started harmlessly enough with the experiment in which men were no
longer content to observe, to register, and contemplate whatever nature
was willing to yield in her own appearance, but began to prescribe
conditions and to provoke natural processes. What then developed into
an ever-increasing skill in unchaining elemental processes,
which, without the interference of men, would have lain dormant and
perhaps never have come to pass, has finally ended in a veritable art
of “making†nature, that is, of creating “natural†processes
which without men would never exist and which earthly nature by herself
seems incapable of accomplishing....
[N]atural sciences have become exclusively sciences of process and, in their last stage, sciences of potentially irreversible, irremediable “processes of no returnâ€....16The
human artifice of the world separates human existence from all mere
animal environment, but life itself is outside this artificial world,
and through life man remains related to all other living organisms. For
some time now, a great many scientific endeavors have been directed
toward making life also “artificial,†toward cutting the last tie
through which even man belongs among the children of nature....
This future man, whom the scientists tell us they will produce in no more than a hundred years, seems to be possessed by a rebellion against human existence as it has been given, a free gift from nowhere (secularly speaking), which he wishes to exchange, as it were, for something he has made himself.17God
is supposed to have made man in His own image, and the propagation of
the race may also be interpreted as a function in which one living
being makes another in its own image. In our desire to glorify God with
respect to man and Man with respect to matter, it is thus natural to
assume that machines cannot make other machines in their own image;
that this is something associated with a sharp dichotomy of systems
into living and non-living; and that it is moreover associated with the
other dichotomy between creator and creature. Is this, however, so?20
The greatest and most appalling danger for
human thought is that what we once believed could be wiped out by the
discovery of some fact that had hitherto remained unknown; for example,
it could be that one day we succeed in making men immortal, and
everything we had ever thought concerning death and its profundity
would then become simply laughable. Some may think that this is too
high a price to pay for the suppression of death.26
One does not find [in it],
at least to begin with, the kind of negative judgment one finds in the
Faust legend concerning the knowledge and creative activity of men “in
God’s image.†Quite to the contrary, it is in creative activity that
man attains his full humanity, in a perspective of imitatio Dei that allows him to be associated with God, in a process of ongoing and perfectible creation.27
Concentration camps
were the setting for many horrific stories. Imagine then the
incredulous delight of a couple who returned to Vienna from two
different camps to find each other alive. They were together for about
six months, and then the wife died of an illness she had contracted in
the camp. At this her husband lost heart completely, and fell into the
deepest despair, from which none of his friends could rouse him, not
even with the appeal “Imagine if she had died earlier and you had not
been reunited!†Finally he was convinced to seek the help of Viktor
Frankl, known for his ability to help the victims of the catastrophe.
They
met several times, conversed for many hours, and eventually one day
Frankl said: “Let us assume God granted me the power to create a woman
just like your wife: she would remember all your conversations, she
would remember the jokes, she would remember every detail: you could
not distinguish this woman from the wife you lost. Would you like me to
do it?†The man kept silent for a while, then stood up and said, “No
thank you, doctor!†They shook hands; the man left and started a new
life.
When I asked him about this astonishing and simple change,
Frankl explained, “You see, Heinz, we see ourselves through the eyes of
the other. When she died, he became blind. But when he saw that he was blind, he could see!â€28All through the night, Alcmena loves a man
whose qualities are in every particular identical to those of her
husband. The self-same description would apply equally to both. All the
reasons that Alcmena has for loving Amphitryon are equally reasons for
loving Zeus, who has the appearance of Amphitryon, for Zeus and
Amphitryon can only be distinguished numerically: they are two rather
than one. Yet it is Amphitryon whom Alcmena loves and not the god who
has taken on his form. If one wishes to account for the emotion of love
by appeal to arguments meant to justify it or to the qualities that
lovers attribute to the objects of their love, what rational
explanation can be given for that “something†which Amphitryon
possesses, but that Zeus does not, and which explains why Alcmena loves
only Amphitryon, and not Zeus?29