Adapted from the Caltech commencement address given in 1974.
During the Middle Ages there were all kinds of crazy ideas, such as that a piece of rhinoceros horn would increase potency. Then amethod was discovered for separating the ideas--which was to tryone to see if it worked, and if it didn't work, to eliminate it.This method became organized, of course, into science. And itdeveloped very well, so that we are now in the scientific age. Itis such a scientific age, in fact that we have difficulty inunderstanding how witch doctors could ever have existed, whennothing that they proposed ever really worked--or very little ofit did.
But even today I meet lots of people who sooner or later get meinto a conversation about UFOS, or astrology, or some form ofmysticism, expanded consciousness, new types of awareness, ESP, andso forth. And I've concluded that it's not a scientific world.
Most people believe so many wonderful things that I decided toinvestigate why they did. And what has been referred to as mycuriosity for investigation has landed me in a difficulty where Ifound so much junk that I'm overwhelmed. First I started out byinvestigating various ideas of mysticism, and mystic experiences.I went into isolation tanks and got many hours of hallucinations,so I know something about that. Then I went to Esalen, which is ahotbed of this kind of thought (it's a wonderful place; you shouldgo visit there). Then I became overwhelmed. I didn't realize howmuch there was.
At Esalen there are some large baths fed by hot springs situatedon a ledge about thirty feet above the ocean. One of my mostpleasurable experiences has been to sit in one of those baths andwatch the waves crashing onto the rocky shore below, to gaze intothe clear blue sky above, and to study a beautiful nude as shequietly appears and settles into the bath with me.
One time I sat down in a bath where there was a beautiful girlsitting with a guy who didn't seem to know her. Right away I beganthinking, "Gee! How am I gonna get started talking to this beautiful nude babe?"
I'm trying to figure out what to say, when the guy says to her, "I'm, uh, studying massage. Could I practice on you?"
"Sure," she says. They get out of the bath and she lies down on amassage table nearby.
I think to myself, "What a nifty line! I can never think ofanything like that!" He starts to rub her big toe. "I think I feelit, "he says. "I feel a kind of dent--is that the pituitary?"
I blurt out, "You're a helluva long way from the pituitary, man!"
They looked at me, horrified--I had blown my cover--and said, "It'sreflexology!"
I quickly closed my eyes and appeared to be meditating.
That's just an example of the kind of things that overwhelm me. Ialso looked into extrasensory perception and PSI phenomena, and thelatest craze there was Uri Geller, a man who is supposed to be ableto bend keys by rubbing them with his finger. So I went to hishotel room, on his invitation, to see a demonstration of bothmindreading and bending keys. He didn't do any mindreading thatsucceeded; nobody can read my mind, I guess. And my boy held a keyand Geller rubbed it, and nothing happened. Then he told us itworks better under water, and so you can picture all of us standingin the bathroom with the water turned on and the key under it, andhim rubbing the key with his finger. Nothing happened. So I wasunable to investigate that phenomenon.
But then I began to think, what else is there that we believe? (AndI thought then about the witch doctors, and how easy it would havebeen to cheek on them by noticing that nothing really worked.) SoI found things that even more people believe, such as that we havesome knowledge of how to educate. There are big schools of readingmethods and mathematics methods, and so forth, but if you notice,you'll see the reading scores keep going down--or hardly going upin spite of the fact that we continually use these same people toimprove the methods. There's a witch doctor remedy that doesn'twork. It ought to be looked into; how do they know that theirmethod should work? Another example is how to treat criminals. Weobviously have made no progress--lots of theory, but no progress--in decreasing the amount of crime by the method that we use tohandle criminals.
Yet these things are said to be scientific. We study them. And Ithink ordinary people with commonsense ideas are intimidated bythis pseudoscience. A teacher who has some good idea of how toteach her children to read is forced by the school system to do itsome other way--or is even fooled by the school system intothinking that her method is not necessarily a good one. Or a parentof bad boys, after disciplining them in one way or another, feelsguilty for the rest of her life because she didn't do "the rightthing," according to the experts. So we really ought to look into theories that don't work, andscience that isn't science.
I think the educational and psychological studies I mentioned areexamples of what I would like to call cargo cult science. In theSouth Seas there is a cargo cult of people. During the war they sawairplanes land with lots of good materials, and they want the samething to happen now. So they've arranged to imitate things likerunways, to put fires along the sides of the runways, to make awooden hut for a man to sit in, with two wooden pieces on his headlike headphones and bars of bamboo sticking out like antennas--he'sthe controller--and they wait for the airplanes to land. They'redoing everything right. The form is perfect. It looks exactly theway it looked before. But it doesn't work. No airplanes land. SoI call these things cargo cult science, because they follow all theapparent precepts and forms of scientific investigation, butthey're missing something essential, because the planes don't land.
Now it behooves me, of course, to tell you what they're missing.But it would be just about as difficult to explain to the South SeaIslanders how they have to arrange things so that they get somewealth in their system. It is not something simple like tellingthem how to improve the shapes of the earphones. But there is onefeature I notice that is generally missing in cargo cult science.That is the idea that we all hope you have learned in studyingscience in school--we never explicitly say what this is, but justhope that you catch on by all the examples of scientificinvestigation. It is interesting, therefore, to bring it out nowand speak of it explicitly. It's a kind of scientific integrity,a principle of scientific thought that corresponds to a kind ofutter honesty--a kind of leaning over backwards. For example, ifyou're doing an experiment, you should report everything that youthink might make it invalid--not only what you think is right aboutit: other causes that could possibly explain your results; andthings you thought of that you've eliminated by some otherexperiment, and how they worked--to make sure the other fellow cantell they have been eliminated.
Details that could throw doubt on your interpretation must begiven, if you know them. You must do the best you can--if you knowanything at all wrong, or possibly wrong--to explain it. If youmake a theory, for example, and advertise it, or put it out, thenyou must also put down all the facts that disagree with it, as wellas those that agree with it. There is also a more subtle problem.When you have put a lot of ideas together to make an elaboratetheory, you want to make sure, when explaining what it fits, thatthose things it fits are not just the things that gave you the ideafor the theory; but that the finished theory makes something elsecome out right, in addition.
In summary, the idea is to try to give all of the information tohelp others to judge the value of your contribution; not just theinformation that leads to judgment in one particular direction oranother.
The easiest way to explain this idea is to contrast it, forexample, with advertising. Last night I heard that Wesson oildoesn't soak through food. Well, that's true. It's not dishonest;but the thing I'm talking about is not just a matter of not beingdishonest, it's a matter of scientific integrity, which is anotherlevel. The fact that should be added to that advertising statementis that no oils soak through food, if operated at a certaintemperature. If operated at another temperature, they all will--including Wesson oil. So it's the implication which has beenconveyed, not the fact, which is true, and the difference is whatwe have to deal with.
We've learned from experience that the truth will come out. Otherexperimenters will repeat your experiment and find out whether youwere wrong or right. Nature's phenomena will agree or they'lldisagree with your theory. And, although you may gain sometemporary fame and excitement, you will not gain a good reputationas a scientist if you haven't tried to be very careful in this kindof work. And it's this type of integrity, this kind of care not tofool yourself, that is missing to a large extent in much of theresearch in cargo cult science.
A great deal of their difficulty is, of course, the difficulty ofthe subject and the inapplicability of the scientific method to thesubject. Nevertheless it should be remarked that this is not theonly difficulty. That's why the planes didn't land--but they don'tland.
We have learned a lot from experience about how to handle some ofthe ways we fool ourselves. One example: Millikan measured thecharge on an electron by an experiment with falling oil drops, andgot an answer which we now know not to be quite right. It's alittle bit off, because he had the incorrect value for theviscosity of air. It's interesting to look at the history ofmeasurements of the charge of the electron, after Millikan. If youplot them as a function of time, you find that one is a littlebigger than Millikan's, and the next one's a little bit bigger thanthat, and the next one's a little bit bigger than that, untilfinally they settle down to a number which is higher. Why didn't they discover that the new number was higher right away?It's a thing that scientists are ashamed of--this history--becauseit's apparent that people did things like this: When they got anumber that was too high above Millikan's, they thought somethingmust be wrong--and they would look for and find a reason whysomething might be wrong. When they got a number closer toMillikan's value they didn't look so hard. And so they eliminatedthe numbers that were too far off, and did other things like that.We've learned those tricks nowadays, and now we don't have thatkind of a disease.
But this long history of learning how not to fool ourselves--ofhaving utter scientific integrity--is, I'm sorry to say, somethingthat we haven't specifically included in any particular course thatI know of. We just hope you've caught on by osmosis.
The first principle is that you must not fool yourself--and you arethe easiest person to fool. So you have to be very careful aboutthat. After you've not fooled yourself, it's easy not to fool otherscientists. You just have to be honest in a conventional way afterthat.
I would like to add something that's not essential to the science,but something I kind of believe, which is that you should not foolthe layman when you're talking as a scientist. I am not trying totell you what to do about cheating on your wife, or fooling yourgirlfriend, or something like that, when you're not trying to bea scientist, but just trying to be an ordinary human being. We'llleave those problems up to you and your rabbi. I'm talking abouta specific, extra type of integrity that is not lying, but bendingover backwards to show how you are maybe wrong, that you ought tohave when acting as a scientist. And this is our responsibility asscientists, certainly to other scientists, and I think to laymen.
For example, I was a little surprised when I was talking to afriend who was going to go on the radio. He does work on cosmologyand astronomy, and he wondered how he would explain what theapplications of this work were. "Well," I said, "there aren't any."He said, "Yes, but then we won't get support for more research ofthis kind." I think that's kind of dishonest. If you'rerepresenting yourself as a scientist, then you should explain tothe layman what you're doing--and if they don't want to support youunder those circumstances, then that's their decision. One example of the principle is this: If you've made up your mindto test a theory, or you want to explain some idea, you shouldalways decide to publish it whichever way it comes out. If we onlypublish results of a certain kind, we can make the argument lookgood. We must publish both kinds of results. I say that's also important in giving certain types of governmentadvice. Supposing a senator asked you for advice about whetherdrilling a hole should be done in his state; and you decide itwould be better in some other state. If you don't publish such aresult, it seems to me you're not giving scientific advice. You'rebeing used. If your answer happens to come out in the direction thegovernment or the politicians like, they can use it as an argumentin their favor; if it comes out the other way, they don't publishit at all. That's not giving scientific advice.
Other kinds of errors are more characteristic of poor science. WhenI was at Cornell, I often talked to the people in the psychologydepartment. One of the students told me she wanted to do anexperiment that went something like this--it had been found byothers that under certain circumstances, X, rats did something, A.She was curious as to whether, if she changed the circumstances toY, they would still do A. So her proposal was to do the experimentunder circumstances Y and see if they still did A. I explained to her that it was necessary first to repeat in herlaboratory the experiment of the other person--to do it undercondition X to see if she could also get result A, and then changeto Y and see if A changed. Then she would know that the realdifference was the thing she thought she had under control.
She was very delighted with this new idea, and went to herprofessor. And his reply was, no, you cannot do that, because theexperiment has already been done and you would be wasting time.This was in about 1947 or so, and it seems to have been the generalpolicy then to not try to repeat psychological experiments, butonly to change the conditions and see what happens. Nowadays there's a certain danger of the same thing happening, evenin the famous (?) field of physics. I was shocked to hear of anexperiment done at the big accelerator at the National AcceleratorLaboratory, where a person used deuterium. In order to compare hisheavy hydrogen results to what might happen with light hydrogen"he had to use data from someone else's experiment on lighthydrogen, which was done on different apparatus. When asked why,he said it was because he couldn't get time on the program (becausethere's so little time and it's such expensive apparatus) to do theexperiment with light hydrogen on this apparatus because therewouldn't be any new result. And so the men in charge of programsat NAL are so anxious for new results, in order to get more moneyto keep the thing going for public relations purposes, they aredestroying--possibly--the value of the experiments themselves,which is the whole purpose of the thing. It is often hard for theexperimenters there to complete their work as their scientificintegrity demands.
All experiments in psychology are not of this type, however. Forexample, there have been many experiments running rats through allkinds of mazes, and so on--with little clear result. But in 1937a man named Young did a very interesting one. He had a longcorridor with doors all along one side where the rats came in, anddoors along the other side where the food was. He wanted to see ifhe could train the rats to go in at the third door down fromwherever he started them off. No. The rats went immediately to thedoor where the food had been the time before.
The question was, how did the rats know, because the corridor wasso beautifully built and so uniform, that this was the same dooras before? Obviously there was something about the door that wasdifferent from the other doors. So he painted the doors verycarefully, arranging the textures on the faces of the doors exactlythe same. Still the rats could tell. Then he thought maybe the ratswere smelling the food, so he used chemicals to change the smellafter each run. Still the rats could tell. Then he realized therats might be able to tell by seeing the lights and the arrangementin the laboratory like any commonsense person. So he covered thecorridor, and still the rats could tell. He finally found that they could tell by the way the floor soundedwhen they ran over it. And he could only fix that by putting hiscorridor in sand. So he covered one after another of all possibleclues and finally was able to fool the rats so that they had tolearn to go in the third door. If he relaxed any of his conditions,the rats could tell. Now, from a scientific standpoint, that is an A-number-oneexperiment. That is the experiment that makes rat-runningexperiments sensible, because it uncovers the clues that the ratis really using--not what you think it's using. And that is theexperiment that tells exactly what conditions you have to use inorder to be careful and control everything in an experiment withrat-running.
I looked into the subsequent history of this research. The nextexperiment, and the one after that, never referred to Mr. Young.They never used any of his criteria of putting the corridor onsand, or being very careful. They just went right on running ratsin the same old way, and paid no attention to the great discoveriesof Mr. Young, and his papers are not referred to, because he didn'tdiscover anything about the rats. In fact, he discovered all thethings you have to do to discover something about rats. But notpaying attention to experiments like that is a characteristic ofcargo cult science. Another example is the ESP experiments of Mr. Rhine, and otherpeople. As various people have made criticisms--and they themselveshave made criticisms of their own experiments--they improve thetechniques so that the effects are smaller, and smaller, andsmaller until they gradually disappear. All the parapsychologistsare looking for some experiment that can be repeated--that you cando again and get the same effect--statistically, even. They run amillion rats no, it's people this time they do a lot of things andget a certain statistical effect. Next time they try it they don'tget it any more. And now you find a man saying that it is anirrelevant demand to expect a repeatable experiment. This isscience? This man also speaks about a new institution, in a talk in whichhe was resigning as Director of the Institute of Parapsychology.And, in telling people what to do next, he says that one of thethings they have to do is be sure they only train students who haveshown their ability to get PSI results to an acceptable extent--not to waste their time on those ambitious and interested studentswho get only chance results. It is very dangerous to have such apolicy in teaching--to teach students only how to get certainresults, rather than how to do an experiment with scientificintegrity.
So I have just one wish for you--the good luck to be somewherewhere you are free to maintain the kind of integrity I havedescribed, and where you do not feel forced by a need to maintainyour position in the organization, or financial support, or so on,to lose your integrity. May you have that freedom.
(No Ratings Yet)
Loading...