Saturday, December 17, 2005

Science and Replicability

In a discussion of natural and unnatural methods of childbirth, it was asserted that “Scientific data is intended to be replicated—indeed, must be replicable—and objective”. I don’t have anything to say about childbirth, but I would like to point out that depending on what is meant by “replicated,” the above formulation has the potential to give an overly narrow or even misleading notion of science.

Being able to set up and perform an artificial experiment at will is less important to the definition of science than that data be publicly available and that there be sufficient objective control on hypotheses. It is great when a repeatable artificial experiment is possible, but such historical disciplines as geology, paleontology, in some cases astronomy and astrophysics, and so on depend in part on events (and sometimes even observations) that are not repeatable: we will not observe a real-time magnetic field reversal in our lifetime, dinosaurs will not go extinct again, SN1987A will not explode again, and so forth. Nevertheless the public shareability of recorded observations of these events and adequate cross-examinations to control interpretations justify calling their study ‘scientific.’

Statements like ‘many different people can analyze relics of the event and come to the same conclusion’ or ‘more dinosaur bones could be found’ are arguably instances of ‘replication’ in a different sense than repeatable, artificially controlled experiments; but I resist considering these as alternative ways of satisfying some putative overarching scientific ‘replicability criterion,’ for the first may simply collapse to the more useful notion of public transferability, and the second is obviously too restrictive: the study of dinosaur bones would remain scientific even if no new ones were ever found.

Finally, there is a danger in overemphasizing repeatability without sufficient attention to public shareability and objective controls (in this connection I appreciate the use of the word “objective” in the above formulation, I just think it deserves more emphasis—hence my present ascent to the soapbox). For example, it might be observed that instances of Mormon testimony are eminently repeatable (like the McDonalds’ slogan, Millions and Millions served), while the generation of Earth’s biodiversity by evolution cannot be replicated in the size or time scales of human laboratories and lifetimes. With an overemphasis on replicability some might therefore be tempted to call Mormon testimony ‘scientific’ and the study of evolution ‘unscientific.’ (I hasten to say that the commenter I quoted above didn’t say anything remotely like this; perhaps there are very few Mormons who would consider testimony a form of scientific knowledge, though Alma 32 might tempt some to say so. I am simply using her statement as an occasion to take off on an unrelated tangent that may be of little interest to anyone but myself.)

The problem here is that the more important consideration than replicability to the question of whether testimony or evolution are scientific is whether the experiences motivating belief in the respective principles are publicly transferable and controlled by objective data against plausible alternative explanations. In the case of testimony the answer is ‘no,’ and it must therefore be considered unscientific, notwithstanding its wide (if far from universal) replicability. To the extent the answer is ‘yes’ to various aspects of evolution, those aspects are scientific, in spite of the fact that we have only a single precious example of the historical unfolding of a great pageant of life.


we will not observe a real-time magnetic field reversal in our lifetime 

Ahem . 

Comment by MT | 12/17/2005 03:02:00 PM  

Objectively "applicable" might be better than "replicable." 

Comment by MT | 12/17/2005 03:06:00 PM  

This really cuts to the core of quite a few somewhat naive religious claims. One of the active evangelists for young earth creationism trains his audiences to simply ask "were you there?" when the issue of evolution comes up. No, the scientists weren't there, but neither was this man nor was Moses. His argument does far more damage against his own position than against the evolutionists he is fighting against.

More to your point is the issue of "replicability" in religious experiences. The error most commonly commited here is the same one which prevents most forms of evolutionary epistemology from getting off the ground. Just because we all experience the same thing, or better said, describe certain experiences in the same way doesn't mean that we should trust them all that much.

Compounding this problem is the fact that the few instances of "replicable" (with serious scare quotes) religious experiences tend to cast doubt on them rather than dispel it. For instance, when people say that they see the same vision as somebody else, the details are never exactly the same upon independent attestation. Take for instance the experience of the 3 witness. Each person experience a similar "vision" but that is to be expected given the context. What is more important is the differences which exist as they do in every similar accounts in other religious traditions.

For these reasons and more I am very suspicious of any accounts of "scientific" spiritual knowledge. 

Comment by Jeffrey Giliam | 12/17/2005 05:21:00 PM  

Jeffrey, when three people say they see the same anything  the details are never exactly the same. The points you raise aren't unique to religious experience but experience in general. Psychological studies of witness accounts deem them always highly interpreted and therefore not always 'accurate.' This has been written a lot on, especially in the context of legal testimony.

With regards to the three witnesses, I'm certainly open to alternative ways of viewing them and fully agree they aren't the slam dunk many Mormons take them to be. But I confess to being curious about why you say "that is to be expected given the context." I'm not sure what you mean.

Getting to the more fundamental issue that Christian raises, regarding science and past processes. I think it important to point out that replicability is but one of the aspects important to science. The bigger one is predictability and "falsification" (in scare quotes to avoid falling into Popper's errors). Further it seems there that the issue of replicability that Christian raises actually does occur. We need not have repeatability of the process to infer that it took place. What needs repeatability are the elements out of which our inferences are made. Thus we expect future dinosaur bone discoveries to line up with past measurements and calculations. If suddenly we found a T Rex bone dating from 10,000 years ago then I think it would be a big fly in the ointment. But that doesn't happen. And there are hundreds of small, repeated measurements. The same thing is true with arguments regarding cosmological evolution.

Now certainly one can criticize scientists for how often in practice they do attempt to falsify data and how often they rigorously attempt replication. I think for theories considered uncontroversial it doesn't take place as often as perhaps it ought. But then science is a fallible social practice.  

Comment by clark | 12/17/2005 05:36:00 PM  


While I do realize that the descriptions will always differ from person to person, I am claiming something far more radical, namely that the actual experiences themselves differed from person to person not just in perspecitve but in actual content. Take the reception of sec. 76. I think that I am on rather firm ground in saying that Rigdon did not see everything in that vision that Joseph did. Not because it was a difference in perspective but rather due to it being an entirely separate vision altogether. Accounts of the reception of the vision colaborate this interpretation where Joseph asks them if they see such and such and Rigdon responds yes or no.

This is actually the position that Geoff has been trying to advocate for ALL revelation and my argument against such was that that can hardly be considered the fool proof guide to truth that Mormons always believe it to be. And this is exactly for the reasons which Christian mentions.

When I say "that is to be expected given the context" I was referring to the fact that in that particular instance, as with others, the men were united in desiring the very thing which they in turn saw. In other words, their common expectation colored their experience making it more unified than it surely would have been otherwise. Additionally, the fact that there appears to have been communication between the men during and immediately after the experience this would in turn guide and coloer both their experiences as well as their memory of such in a way which would make the general outline of the experience VERY similar to one another.

The problem I was trying to briefly articulate is that when such people are asked about specifics which were not directly communicated between individuals or strongly inferred by such, the details lose their unified appearance. It becomes clear that the vision which each person saw was not the same as that seen by their companions at all. Atran mentions this in his book. 

Comment by Jeffrey Giliam | 12/17/2005 06:40:00 PM  

MT,  thanks for the interesting link. While magnetic field variation is observable, I think complete reversal---where magnetic north is near the south pole---occurs on timescales of thousands, maybe tens of thousands of years---and is a relatively sudden transition.

I'm not sure exactly what you're getting at in your second comment. 

Comment by Christian Y. Cardall | 12/17/2005 08:10:00 PM  

Jeffrey  and Clark, thanks for the interesting discussion.

Clark, unfortunately I'm not intimately familiar with Popper or his errors re falsification. Is it an easily stateable issue? (The only thing I think I know about Popper is that I absorbed the idea somewhere that he pointed out that typically there's no such thing as scientific proof, but only increasing confidence born of passing more and more falsifiable tests.)

Perhaps I was a little too dismissive of the importance of repeated examination of the relics of non- (or rarely-) repeating past events, but in some cases even relics are not available---the neutrinos from SN1987A, for example. But we still have the data they left behind, and there are good grounds for considering their use scientific. Of course even here I hope for at least one Galactic supernova in my lifetime that, with today's detectors, will give staggeringly better data... 

Comment by Christian Y. Cardall | 12/17/2005 08:37:00 PM  

Jeffrey, you make some great points about the difficulties of getting to the bottom of religious experiences. In-depth study of the witnesses of the founding Mormon generation is something I'm interested in exploring as time goes by here. Not long ago I bought a copy of Opening the Heavens  which supposedly collects all the firsthand accounts, but I haven't dug into it yet.

Also I agree that in the case of spiritual knowledge replicability can actually cast doubt rather than instill confidence, maybe in a slightly different sense than the example you give of differing witness accounts. The sorts of experiences we take as a basis for testimony occur in other contexts in our own lives (sometimes even in situations contrary to Church standards). They also occur to people of other faiths in such a way as to confirm them in their own beliefs rather than bring them to the One True Church. These observations strongly suggest that the danger of attribution error is pretty severe. 

Comment by Christian Y. Cardall | 12/17/2005 08:49:00 PM  

One last quick point about something Clark mentioned, that I need to do more justice to another time: predictability. Like replicability, predictability is highly desirable and should be sought whenever possible. Confirmation of counterintuitive predictions are particularly persuasive. But it is not always possible (for many reasons, including chaos, complexity, quantum measurement, the fact that some theories cannot be formulated until after data is in hand, ...) and I think it might be a mistake to also claim that predictability is a sine qua non  of science.

If no new orbits of heavenly bodies could ever have been detected beyond those known to Kepler, would we today say his laws of planetary motion were not scientific because there were no new instances beyond those he used to work out his laws? I think not. Hence in some cases post hoc hypotheses may be sufficiently compelling to be considered scientific. 

Comment by Christian Y. Cardall | 12/17/2005 09:07:00 PM  

Christian, Wikipedia has a reasonable albeit not great summary . The brief idea is that science progressing by showing theories wrong but never shows them right. It is based on the idea that we can prove individual claims false. The Quine-Duhem thesis argues that it's impossible to falsify individual elements. You can only falsify collected theories. Popper was very influential in science and I still meet scientists who buy into his approach. It's brought up in ID contexts as well - typically incorrectly. I think few take Popper seriously now, not because falsification isn't important, just that they recognize that we can't get at individual entities the way Popper wanted - further it seems like science is more of a complex and perhaps inconsistent mix of falsification, testability (predictions), simplicity, and other such things.

Jeffrey, since I think any individual experience is colored by our knowledge and expectations, I think it impossible that two people can ever experience the same thing. Consider me meeting my wife at the store. The experience for me is radically different from say someone else meeting my wife. This is contrary to say the positivists who'd argue that the experience is the same and that then we draw secondary conclusions from it. I tend to invert that relationship putting the experience as a whole first.

I recognize you are claiming more than this. I raise it though to simply suggest that determining whether or not your claim is true is problematic precisely because of the way experience is given to us. 

Comment by clark | 12/17/2005 11:26:00 PM  

I agree a field flip within the lives of our grandchildren even is unlikely, but until that news story I'd figured it was unimaginably far off and I assumed you'd thought so too. I was really startled by that story. Your point that we suppose the flip will be sudden only makes it seem more imaginable. I imagine this is super non-linear, and the geodynamo simulations are rough approximations, even if they do them on super computers.

With my "applicable" versus "replicable" comment my point was just that if you want a better definition of science than "replicable," which I took as partly your aim in your post, then you might reach no further than "applicable." A mixture of serious point and wordplay. 

Comment by MT | 12/18/2005 12:33:00 AM  

By "applicable" I mean of a theory when you apply it to phenomena other than that from which you deduced it or by which you initially confirmed it. Science is objectively "useful" I mean. 

Comment by MT | 12/18/2005 12:38:00 AM  

Actually, regarding the magnetic field reversal. I am not as learned as you all, and actually haven't quite read through everything yet, although I've skimmed.

I also know a documentary isn't conclusive proof, but I have done SOME reading, anyway.

The field is weakening, well it usally always has anomalies, but those anomalies are growing (they have a record that helps them see this going back to the last 300 years or so, of seagoing vessels and compass usage and their very meticulous keeping of logs). So. They have access to alot of data over a period of 300 years.

They also, can tell about shifts and weakening in it, geologically, from how things align in lava as it cools on the surface. It will show which way is magnetic north, etc. As the lava cools, the "things" ok, very technical term there, whatever material, crystals, or whatnot is in it, align with the magnetic field.

There is documented evidence of a lava field, of lava flows that were very very very close in age, basically one after another, where the field flipped, then flipped back again, and, I THINK, might have flipped one more time, or shifted greatly.

I THINK there may be a SINGLE FLOW where the first parts to cool were one way, and then the rest of it shows the shifting. I think. It's been a year or so seen I saw this.

So it can happen faster than one may think. I thought the whole thing was rather interesting. 

Comment by sarebear | 12/18/2005 03:27:00 AM  

So did Einstein. I think he called it the greatest unsolved problem in physics. Of course, he'd already knocked a couple of the harder ones out of the way. 

Comment by MT | 12/18/2005 04:19:00 PM  

Clark,  thanks for the summary on Popper.

MT and sarebear, thanks for the additional comments on magnetic field flips.

The magnetic striping in lava at spreading crustal plates that sarebear referred to is what I meant to be an example of a relic of past events that will not be replicated in our lifetime but can nevertheless be studied scientifically.

Again, the fact that many different people can study the magnetic striping is replicability in a sense, but I think in thinking about science philosophically it is more useful to consider this a manifestation of the public nature of scientific study, and reserve the term "replicable" for phenomena that can be generated and proactively experimented and tinkered with on demand. This is also related to a distinction between 'observation' and 'experimentation'; sometimes only the former is possible. 

Comment by Christian Y. Cardall | 12/19/2005 09:07:00 AM  



<< Home