http://geocities.yahoo.com.br/criticandokardec/criticizingskepticism.htm
Doubting Hyman et al...
Comments about Ray Hyman's article "Testing Natasha".
| It is highly recommended
that the reader see this page,
where I provide a summary on this issue and a regular update on it. |
___________________________________________________
Abstract: At long last, in May 2005 (after two postponings...), Ray Hyman published their official report, in the Skeptical Inquirer, about CSICOP's sloppy tests with russian "psychic" Natasha Demkina. Anyone seeking trustworthy information about the event will be fooled one more time by these so called "skeptics". One wonders: Will this ever end?
Note: Previous articles on this issue, and a follow-up
article:
My first critique of the CSICOP's test, at this link.
My reply to their "answer to critics", at this link.
Also, in November 2005, my critique on Hyman's article in the Skeptical
Inquirer Sep/Oct 2005, at this link.
____________________________________________________________________________
It is always surprising. And at the same time disappointing. Once again, CSICOP fellows have material to make a good, truly skeptical, rational, and scientific work. But instead they end up spoiling it all. Below, I will make some comments about Ray Hyman's repport in his article "Testing Natasha", which is available at the link below:
http://www.csicop.org/specialarticles/natasha.html
Also, I will make some comments about the article by Andrew Skolnick, "The Girl with Normal Eyes", available at the link below:
http://www.csicop.org/specialarticles/demkina.html
First, Hyman's article. I have kept only the points that I want to comment on. Hyman's and colleagues' comments appear in black type. My comments appear in dark bold blue.
Extracts from Hyman's Article (In Hyman's text, any bold type passage, or underlined passage, was highlighted by me):
We had to conduct the test of Natasha's claim to fit the constraints of a television program. We had only a month to devise a protocol that would be acceptable to all parties. After everyone agreed to the procedure, we had less than a week to locate a testing site in New York City and to find seven willing and suitable test subjects.
This extract above is important to highlight the intrinsically flawed nature of the test from the strictly scientific point of view. It does not invalidate the endeavor, which even has some "para-scientific" merits (especially in helping to inform the lay people in a rational way about the issue). But it is clear that, through its "lax" nature, either a misleading rejection of "Natasha's power" (that is, a false negative result) or a misleading endorsement to it (a false positive) is to be expected.
----------------
Monica Garnsey, director and producer of the program, told us how Natasha
operates and what she claimed to do.
This comment carefully hides the problem number one of the test: the "researchers" simply did not bother to have any direct contact with the claimant (that is, with Natasha - Note that Hyman himself acknowleges that they had an entire month to do it. So, in this point it was not lack of time, but lack of care), neither physically directly nor in any other way. With nowadays' technology, the least that one would expect would be a telephone contact (with interpreters online-realtime direct talk). As a result, they relied only on reports in the media (!) and on what the producer of the program told them. They ended up not actually testing the claim.
----------------
Garnsey e-mailed us the following information from Russia, where she was taping material for the television program: I double-checked a few things with her last night. Since the age of ten, a few days after having a religious dream, and also having had an operation to have her appendix removed that went wrong, swabs were left in her and she had to have another operation, Natasha has claimed to be able to see into people. . . . Natasha can see through clothing, but not see what someone is holding behind their back. She cannot see inside people if she shuts her eyes. Daylight is better. She does not need to talk to them to diagnose. She can also diagnose from a photograph. She usually scans people all over first, by making them stand up fully clothed and looking them up and down; delivers a general diagnosis; and then goes into more detail when the patients have discussed their concerns with her. She says she can certainly see ribs, heart, lungs, initially in general "like in an anatomy book," but can see right down to the cell level if she concentrates. She says that she can examine the whole body, but it can give her a bad headache if she does too much. The idea of restricting the test to the chest area appeals [to her], though her claims extend further than that.
Ray Hyman has a very good article entitled "Proper Criticism" where he highly recommends that you should "try to get the specifics of the claim directly from the claimant." (this link). It is curious why he decided to discard his own recommendations. Especially in light of the fact that this report above from Monica Garnsey (producer of the Discovery Channel program) can be used as a starting point to understanding the claim, but certainly not as the ultimate source for designing an acceptable test. For example, we can't even know, from the report above, if Natasha can diagnose problems in ribs (she certainly sees them; but does she see problems in them, and what kinds of?). This problem went so far that Andrew Skolnick (co-designer of the test) ended up believing (and telling us repeatedly through email) that Natasha could not only see cells in minute details but that she could diagnose problems in isolated molecules too, and that she herself had said that on the Discovery Channel Documentary (I pressured him on that, and he never showed me where he got that information from, and ended up stopping our email exchanges when I found out that he had been lying to me in this regard and in many other items as well - Caught in the act...).
----------------
Natasha's story is like thousands of other accounts. Alleged psychics
and their supporters make claims that, if true, defy the physical limitations
and laws of modern science.
Hyman, just as his colleague Richard Wiseman (also a co-designer of the test), likes to stigmatize unknown phenomena. He probably said, in 1988, that John Cairns' finding (bacterial "directed mutations", or "adaptive mutations"; published in the journal Nature) could not be true for it would defy the physical limitations and laws of modern science... A "new phenomenon" may bring lots of novelties with it, and yet accomodate perfectly nicely within the constraints of known laws of nature. Hyman's and Wiseman's insistance on this kind of stigmatization reveals, rather, an agenda to discredit especially the parapsychological field of investigation.
--------------
The proponents support the reality of these claims with testimonials of outstanding successes. They argue for the reality of the claim passionately and unreservedly. Although some proponents have had scientific training, none of the supporting evidence comes from well-controlled scientific studies. In the long history of psychical research, not one of these claims has produced convincing scientific evidence for the existence of paranormal ability (see Joe Nickell's column in this issue, p. 18). A few researchers have claimed that they did have scientific proof for a paranormal claim. Scrutiny by other scientists, however, showed that the "scientific proof" had serious flaws. Furthermore, none of these claims could be independently replicated.
Hyman knows that what he is saying above is not true. He is deliberately lying here. The best example (and, as a matter of fact, the only one that I consider a thorough scientific validation of a parapsychological claim) is the evidence for "telepathy" in tests using the Ganzfeld Protocol. The last time Hyman uttered some criticism to these group of tests was in an article in 1996 (in the Skeptical Inquirer). From then on, he has remained silent about it. The evidence is getting stronger and stronger, and Hyman more and more silent... But then (now), he comes to us and writes this paragraph above! Another interesting example of partially well proved "paranormal" phenomenon is the so called micro-psychokines (micro-PK), with human intention influencing random subatomic events. It has recently been replicated (in 2003, Jounal of Scientific Exploration) by a skeptic researcher that James Alcock often talks about: Stanley Jeffers.
---------------
From photo #1: Her friend, Svetlana Skarbo, who acted as her translator, holds a cell phone over which they had sent and received text messages from unknown parties during the test (in violation of test protocols.)
Hyman conveniently "forgot to say" that this violation
of test protocol above was authorized by himself!
[Important Comment added in January 29, 2006: I was mistaken in this line above. The violation was not actually authorized. The full passage of an email from Natasha's agent, Will Stewart, to Richard Wiseman where he talks about it, in May 2004, is: Now the mobile phone allegations. I understand Svetlana was asked by you to turn off her phone at the start of the test. This was done. It was used during the break, as I understand it. Had I been allowed in the small room, I would have stopped this at once. Yet there is nothing at all about not using mobiles in the rules. Svetlana wanted to calm Natasha's stress and so let her use her mobile. The messages concerned were to and from Natasha's boyfriend in Saransk, Russia. If you are concerned about them, we can provide documentary proof of the phone numbers and all the intimate love messages. But I am intrigued: just who do you suppose Natasha could be texting to help her cheat in this test? Given the elaborate security, who on earth would this provincial Russian girl know who could help her? If there was such concern, why didn't Andrew or Ray confiscate the phone or object to its use? My comments on this previous passage now: first, it was a break of protocol indeed. Second, it might have been unintentional from the part of Svetlana and Natasha (Skolnick confirmed, in the forum mentioned below, that the incident happened during the breaktime). Third, it is strange that the test proctor, Ray Hyman (helped by so many people, including Joe Nickell, Richard Wiseman, etc), did not notice it right away and corrected it. Recently, in the James Randi Educational Foundation Forum, it has been pointed out that it is the researchers' duty to garantee that the protocols are followed, and Andrew Skolnick just replied that cell phones are now so minute that they can be accidently inhaled...(see this link) and that they (i.e. he) got to know of this breach of protocol only afterwards. Obviously it only further attests to the sloppiness of the test... Fourth, it would have been very hard indeed for Natasha to get any help from these phone calls in her "diagnosing" of the seven subjects. Who could possibly help her, and how?]
----------------
The protocol stated how we would conduct the test and how we would interpret
the results. We wanted all parties to be clear about what would and would
not be considered a "successful" outcome. What makes a scientific experiment
or a test meaningful is just such an explicit commitment to the interpretation
of the outcome before we observe the data. This is a critical distinction
between the post hoc interpretation of testimonial evidence and the prior
commitment to specified outcomes of a meaningful test. Natasha's defenders
apparently fail to grasp this essential point. The written protocol protects
the interests of all parties. Natasha and her supporters had the opportunity
to study the document, to suggest modifications, and finally to agree or
disagree with its provisions. The protocol also protects the investigators
against a variety of false accusations about how we conducted the test.
Hyman is partially right above. The problem that he, again, conveniently "forgets to remember" is that CSICOP violated the protocols in many crucial ways. As a result, the test ended up being almost (almost!) devoid of any value.
----------------
We made sure to include in the protocol the statement that the "test is not in any way a definitive test. Deciding the truth of Natasha's claims with comfortable certainty is too simple and brief. It can only help to decide whether further studies of Natasha's claimed abilities are warranted." This statement is worth elaborating.
I call the readers' attention to this last phrase of Hyman (he is cunning with words...): "This statement is worth elaborating". He is hiding something here. And what he is hiding is precisely the fact that the statement was not elaborated in the way that it should have been, in the protocols. If Hyman himself acknowledges that the statement is worth elaborating, then what he is saying is that it should have been elaborated (that is, explained) in the written protocol itself (and it was not!) so as to give Natasha (and her legal representatives) the means to understand the implications of it. It is also important to mention here that despite this limitation above stated in the protocols, Hyman appeared in the Discovery Channel documentary saying that Natasha was living an illusion regarding her alleged powers. When Professor Brian Josephson stressed the unethical element in that kind of statements from the "researchers", Andrew Skolnick considered that an unacceptable criticism. Yet, it is a one-hundred-percent precise definition of what they have done.
---------------
Understanding what the test can and cannot do is essential. Even under ideal circumstances this test could not clearly decide if Natasha does or does not have X-ray vision. Any scientific hypothesis-especially a paranormal one-cannot be confirmed or disconfirmed by one test or one experiment. ... We knew that our test could not distinguish between two possibilities: (1) she can make correct matches using external clues; or (2) she can make correct matches using paranormal X-ray vision. The alternatives we could control or reduce were that she gets correct matches just by luck or that her correct matches are due to those factors that make vague statements seem like hits. We were also aware that our test could only detect a large effect. Natasha's claim can be considered in several contexts. The testimonials imply that she is highly accurate. This has practical consequences. If clients are depending upon her for medical diagnoses, Natasha's readings should be reliable. Otherwise, she can do much harm. Of course, Natasha could possess paranormal powers, but they could be weak and erratic. Such unreliable and weak ability would be useless for medical diagnosis, but would still be of theoretical interest. We lacked the resources and time to try to detect such a weak effect.
This paragraph above is interesting and informative (in the good sense). But still, Hymans makes some confusions. The test, as stated above, is reducing the "Luck Factor" and the "Cold Reading" factor. And also, aimed at detecting a "Large Effect". Perfect. But then come the confusions. Hyman says "the testimonials imply that"... . That is, he is doing away with "getting the specifics of the claim directly from the claimant" (and also getting them from Monica Garnsey, for Monica did not say that Natasha is highly accurate), and he is relying on tabloids to design, to conduct, and to interpret his "scientific" test. Instead of being highly accurate, she could be very accurate (or moderately accurate) with special instances of inaccuracy (her own report of her limitations, to Monica Garnsey, actually imply that). Hyman also says that "Such unreliable and weak ability would be useless for medical diagnosis". I tend to agree with him that a weak ability might be of little use (and even dangerous). But even that requires careful thought, for depending on the way that the therapist uses it, it could prove valuable, even though weak and erratic (it could be used as an additional/complementary element to the diagnosing procedure).
----------------
The outcome of the test could be from zero to seven correct matches. We set the criterion for success at five correct matches. We clearly stated this criterion in the test protocol and all parties agreed to this in advance.
Yes. And everybody agreed also (according to the protocols) that the subjects would show medical proof (documents) of their alleged conditions. They never did that. Not even Hyman knows for sure if they really had the conditions that they claimed to have. Also, the medical conditions cited as examples in the protocols would not lead anyone to expect the two alien conditions that were introduced: the removed appendix and the minute circular scar in the esophagus.
---------------
Although Natasha's mother says that her daughter never makes a mistake, we did not want to demand that Natasha perform perfectly. We wanted to give her some margin for error.
They gave her the least possible margin for error. Of course if they had demanded seven hits, everybody would say that it was absurd. What was the next possible value then? Six hits was an impossible outcome (for if Natasha got six conditions correctly, automatically she would get the remaining seventh condition right, by exclusion). So, five was the minimal non-perfect value for Natasha's performance. CSICOP demanded from Natasha the most they could.
---------------
Dr. Barrie Casselith, Chief of Integrative Medicine Service at Memorial Sloan-Kettering Cancer Center, helped us with the daunting task of assembling seven appropriate and willing subjects.
Hyman seems to be using the following strategy here: "Let us present to the readers the idea that the recruiting task was very challenging, daunting indeed (frightening and perhaps almost impossible), and if we convince the readers of that, then they will forget about the serious mistakes that we committed in this recruiting process". I think it is a gross exaggeration to call this recruiting task something "daunting". It may not be a piece of cake, but it is very far from daunting. And above all, in a research process, no matter how really daunting a task is, if it has serious flaws then the only scientific attitude is to consider the results inconclusive, and to stress the need for further studies.
----------------
On the morning of the day of the test we learned that two of the subjects had withdrawn. Again, Andrew and Austin saved the day by finding two replacements at the last moment.
"Andrew saved the day"... Most likely these two hasty replacements were the appendix and the scar in the esophagus (i.e. the two violations of the protocol).
----------------
Each volunteer had an internal condition that should be easy to detect if Natasha's claim is correct.
Again, Hyman lies. The removed appendix and the minute scar in the esophagus (I believe the scar would be invisible even to a tomography) were well beyond her claims (as far as I know, and as far as Hyman, Wiseman, and Skolnick know as well). And they were well beyond what was stated in the protocols too. They were violations of the protocol. Natasha felt visibly depressed by these two conditions at the beginning of the test. And Wiseman then "saved the day", by swindling her into accepting these two conditions using technically incorrect and misleading arguments. She ended up getting these two conditions wrong...
----------------
The target conditions were as follows: One patient had metal surgical staples in his chest from open heart surgery; one had a section of her esophagus surgically removed; one had a large section of one lung removed; one had an artificial hip replacement; one had a missing appendix (we discovered afterwards that another subject also had a missing appendix, which he didn't mention when we recruited him. Natasha chose neither of these two as the one with the missing appendix); one had a large brain tumor removed and now has a large hole in his skull covered by a metal plate; and the final subject had none of these target conditions.
Comments: the protocol stated that these subjects would show medical documents attesting their conditions. They never did. No one knows if they really had the conditions they claim to have. This is a violation of the protocols. The missing appendix was a violation of the protocols too (that conditions was not to be expected by the examples cited in the protocols). The additional missing appendix (if true...) was not only a violation of protocol. It also shows how sloppy was the interviewing procedure for recruting the subjects (the so called "daunting task"...). It is clear that the researchers did not ask this guy: "Do you, besides having this skull condition, also have any of these other conditions?". How come it was so?
----------------
One interpreter was Natasha's friend Sveta Skarbo. We allowed her in the test room to make Natasha feel comfortable.
Not exactly. Rather, they "allowed her in the test room" because the protocols made provision for that.
----------------
The other interpreter was supplied by the Discovery Channel. Ideally, only I, as the head proctor, Richard Wiseman as my co-investigator, Natasha and the two interpreters, and the seven subjects should have been present during the test. The realities of television production and the requests of Natasha's companions forced us to compromise here, and in some matters of protocol.
Again Hyman seems to be trying to mislead the readers into believing that the protocols were relaxed during the conduction of the test by his allowing the presence of all this people. But instead, the presence of all this "crowd" was clearly stated in the protocols beforehand...
----------------
The test room also included a television crew of three persons from the production company (Shine, Ltd.); Austin Dacey, who was videotaping the proceedings for CSICOP; Joe Nickell as an observer;
This "Joe Nickell", an "observer", is the one that, at the very beginning of the test, demanded that Natasha's mother be out of the test room, even though the protocols clearly stated that she could be in. Another violation of protocols from the part of the CSICOP fellows! Nickell did not read the protocols (or read it poorly), nor did Richard Wiseman, who also helped in worsening this initial confusion. Wiseman acknowledged in email to Will Stewart (Natasha's agent) that he had committed a mistake in this incident. Also, Monica Garnsey clearly states, in an email that she sent to Andrew Skolnick, that Nickell felt strongly and expressed himself strongly that Natasha's mother must be out. The CSICOPers, including Hyman, keep hiding this little incident...
----------------
Andrew and I met with Natasha in this room before the test to review the procedure and to remind her about the details of the protocol. She had agreed to this protocol, which Monica had shown her five days previously.
What Monica showed her was something from which Natasha could not expect to meet a removed appendix or a minute scar in the esophagus. That is, the protocols, prepared by CSICOP, listed a number of medical conditions as examples of physical problems that could be presented to Natasha. If you see the examples used, it is clear that these two conditions (removed appendix and scar in esophagus) are beyond what was stated in the protocols (as I said before: The test rule number 2 lists examples of clinical conditions, or ones similar to these, that could be used: The Subject Recruiter will choose and "shepherd" the test subjects to the testing facility and will provide us with each subject's "target" medical condition, such as implanted pace maker, resected lung, bone plate and screws, that are clearly documented by medical records and/or x-rays. The target conditions will all be different, no two subjects will have the same condition - see this link and this link. Those two conditions, removed appendix and scar in the esophagus, differ drastically from the examples used in protocols). Natasha protested about those two conditions right at the beginning of the test (she did not complain of any other condition). And she was taken in into accepting it through technically incorrect reasoning from wise Wiseman.
-----------------
We reviewed each condition that we would ask her to detect. She expressed concerns about the removed appendix and the resected esophagus. She was worried that if the appendix had been removed long enough ago it might have grown back. Andrew assured her that appendices do not grow back. Her concern about the resected esophagus was that individuals might normally differ in the length of their esophagus and this could mislead her. Andrew told her that instead of the length she should look for the scar that completely encircled the place where the two ends of the resected esophagus had been surgically joined.
When we see the documentary, it seems that her concerns were not only these ones. She was not really satisfied even knowing that the appendix would not be there (that is, that they do not grow back) and that she was supposed to look for a scar. She only accepted it, reluctantly, after Wiseman's wise (i.e. cunning) and misleading reasoning that "even if she got those two conditions wrong, she would still get five and pass" (Note added in November 2005: this is my free rephrasing of Wiseman's actual words, while preserving their original meaning). That is, she was not being tested for seven conditions, but only for five. As to the other conditions, no complaint at all from Natasha. It is sure that there was something problematic (i.e. beyond her claim) with the tiny mucosa scar and the elusive missing appendix sign.
----------------
Midway through the proceedings, Natasha told us she would feel better if her mother could be in the briefing room. I immediately agreed to her request. [4].
Hidden in note #4: At the start of the test some initial confusion existed as to who would be allowed into the test and briefing rooms. This was quickly corrected and Natasha's mother and Will Stewart were given the option of staying in one of these rooms.
Here Hyman is downplaying the relevance of this initial conflict, and hiding facts in bottom line notes.
----------------
Natasha succeeded in correctly matching four target conditions out of a possible seven. Our protocol required that Natasha get five or more correct matches to "pass" our test. ... Natasha has complained that if she had gotten five correct she would have been a success. Isn't four close enough? Our answer is that five was the minimum score that everyone agreed upon.
That is, Hyman is saying that there is no problem in protocol violations, as long as they favor CSICOP. There is no problem in the facts that: the subjects did not have medical proof of their alleged conditions; the appendix and the minute scar were beyond what could be expected from the written protocols; Natasha was convinced into taking the test through deceiving arguments from Wiseman; they attempted to cast out Natasha's mother at the beginning of the test; there were two guys with removed appendixes (which would have confused someone with true x-ray vision); the guy with the metal plate in his skull (who also missed an appendix) seemed to be improperly blinded (he reacted with an eyebrow movement when the camera faced him directly.
----------------
It was also the minimum score that would convince us of a possible ability to diagnose subjects with sufficient reliability to be useful. We designed our test to detect a large effect. We were looking for something that would distinguish Natasha's claims from many similar ones. We wanted a good reason to justify using the additional time and resources to investigate her ability further.
Here, for the first time ever, Hyman says what the score five really meant: a number that would convince CSICOP that it might be important for them to proceed with their investigations. Natasha's task was to convince Hyman! Not even Superman would conceivably achieve that...
----------------
Both inherent and unforeseen limitations of our test provided possible clues to the target conditions for some subjects. ... These subjects differed sufficiently in outward appearance to provide possible clues about their conditions. Another problem occurred through two violations of the test protocol. Together these problems created the possibility for identifying the target conditions-by external, normal means-for the following four subjects:
This thing above really deserves a careful look, and may indeed have helped Natasha through non-paranormal ways.
----------------
1. The "control" subject, the one who had no internal medical condition, was obviously the youngest of the group. He also looked in good physical condition and appeared much healthier. He was a good candidate for the person with no defects.
Yes. But he could also be a decoy. But Hyman's (and colleagues') warning is possible, and important.
----------------
2. The subject with the staples in his chest (because of major heart surgery) was male, the oldest of the group and looked the least healthy. He was an obvious choice for the person with the staples in his chest.
Again the same as above.
-----------------
3. A breach of protocol occurred on the first trial. Natasha posed a question and her interpreter translated it aloud in front of the subjects. The question, contrary to our protocol, allowed the subjects to know that Natasha was looking for the subject with part of her lung removed. Here it was possible that, knowing which condition Natasha was looking for, the subject with the missing lung might have given herself away through bodily reaction.
This is indeed interesting, and a little problematic at least. Anyway I wonder why Hyman does not tell us the time that elapsed from this moment (the aloud remark) until Natasha came with her answer for that condition. This was the first condition to be identified by her, and the one that, really weirdly (for someone with a good X-Ray-like vision), she took one full hour to identify. If, say, forty minutes elapsed between the "breach" in protocol and Natasha's verdict, then it would be very different from, say, a five minutes delay. I tend to think that here again Hyman is hiding information that speaks against his suspicious hypotheses...
----------------
4. After the test was over, I learned that Natasha and her companions, because of an apparent misunderstanding, had arrived at the test site before we had expected them. They waited outside the test building where they reportedly observed at least two of the test subjects climb the long flight of stairs and enter the test building. This breach of protocol may have provided them clues about which subjects did or did not have the artificial hip.
Natasha's agent (Will Stewart) said that they stayed were they were instructed to stay, and that they saw no one. Anyway, it really might have (or may have) provided the clues Hyman talks about above. But what we have is a doubt. And in this case, the most ethical thing to do would be to declare the test invalid.
----------------
We do not know if Natasha took advantage of the clues I've described in the previous four paragraphs.
But he is treating the issue as if she did take advantage of them. This is a problem.
----------------
However, it is suggestive that these were just the four subjects for whom Natasha achieved her correct matches.
Yes, it is suggestive. Yes, it cannot be discarded by Natasha's believers. That is, each and every feedback that any given test gives must be carefully thought about by those interested in understanding a phenomenon (or a claim). But just as Natasha probably has many blind believers who will throw away these suggestive feedbacks above, Hyman seems to be blindly disbelieving her and throwing away other suggestive feedbacks from the test too. He is discarding the fact that she failed precisely in two conditions that she complained about before the test began: the removed appendix and the esophagus scar. (also she failed a third condition that seemed to be a hundred percent within her claim: the metal plate on the skull - this, again, must not be downplayed by Natasha's believers).
----------------
The probability that she was relying upon nonparanomal clues increases when we consider her misses. She wrongly picked the subject who was wearing a baseball cap as the one who had the metal plate in his head. Conceivably, she picked this subject because one might assume (falsely in this case) that the subject was trying to cover a scar on his head. We should also emphasize that her failure to correctly match the subject with the metal plate in his head further argues against any fledgling paranormal powers. If she truly can see into bodies, she should have easily detected the large area of missing skull along with the metal plate covering the hole.
We could also hypothise that she could not see any appendix at all (because it was an item beyond her claim), and also no scar in the esophagus at all (another item beyond her claim); and also that the metal plate guy, through his ultra spkeptic attitude (he is reported to have behaved in a rather inapproppriate way towards Natasha), shielded himself from Natasha's eyes. So she could see nothing in three subjects, and therefore she exchanged blind guesses between these three, getting all these incorrectly.
----------------
Our test included five subjects for whom external clues were available concerning their internal condition.
Here Hyman is performing his private miracle (mimicking Jesus turning water into wine): he is changing a possibility into a certainty. He is right about the old man with staples in his chest, and about the young healthy subject. Also he is right about the man wearing a baseball cap. But when it comes to the artificial joint hip woman, it is not certain if Natasha saw her walking. I don't even know if everybody that has this kind of "problem" would perform bad (that is, in a give-away manner) in climbing stairs. And we are not told if this woman in particular walked and climbed stairs in a manner that would give away her condition (nor did the "researchers" attempt to find it out afterwards, even though they are quick to use this possibility as a certainty - Also, some of the other subjects might walk or climb stairs in a way that would mislead people into thinking they have some sort of hip problem. This was also not checked afterwards). As to the last example, the woman with lung problem, no one knows if this woman gave away any kind of clue during the test after hearing that it was her condition that was being sought at that time. At least, her eyes and pupils were away from Natasha's scanning. It seems unlikely that she would give herself away in any manner.
----------------
The clues correctly pointed to the true target condition for four subjects. The external clue for the fifth subject falsely pointed to the hole in the skull. In each of these five cases Natasha made her choice consistent with how the external clue was pointing.
Again: it is important not to discredit these observations that Hyman is making. They may indeed reveal something problematic about Natasha's actual performance. But these are suggestions. Indications. Not definetly proved possibilities. They aren't even highly likely possibilities. The true scientific and ethical stand in this case would be to consider the results inconclusive due to the breaches in the test.
----------------
Because a single test, even one done under ideal conditions, cannot settle a paranormal claim, we conceived our test as the first stage of a potential series. The first stage would not necessarily rule out nonparanormal alternatives. If Natasha could pass the first stage, this would justify continuing onto the next stage. If she passed that stage, then we would continue studying her claim. On the other hand, if she failed at any of the early stages, this would end our interest in her claim.
So, getting four hits out of seven ended CSICOP's interest in the matter. Five out of seven would thus give Natasha that: CSICOP's further interest. Nothing really worth gaining, IMHO...
----------------
Keep in mind that the burden of proof belongs to the parties making an extraordinary claim.
Now Hyman appeals to their mantra. Actually, the burden of proof belongs to anyone that makes any claim. If Hyman says that Natasha is living an illusion (like he did say in the documentary, in clear violation of the protocols), then he has to prove it. Keep in mind, Mr. Hyman, that it was not Natasha that asked to be tested. She did not apply to any "Randi-like" test. The ones that really initiated the whole process, I think, were the Discovery Channel people. Both you (Hyman et al) and Natasha were invited. Therefore, things can and should be put in a more, say, polite and respectful way.
----------------
Extraordinary claims require extraordinary proof.
Yes. And acceptance of extraordinary assignments require extraordinary performances. This is what Hyman does not have the sensibility to understand. And surprisingly, even Andrew Skolnick, who worked for many years as editor at the scientific journal JAMA, does not understand either. They fail to grasp the true impact of the "Natasha Phenomenon". The "Natasha Phenomenon" is not one of a girl with X ray eyes. It is one of a girl that, regardless of having some paranormality or not, acts as an informal diagnosing therapist in a poverty stricken country. And one of a girl that, despite honestly believing to have these odd powers (even Richard Wiseman said he trusted that Natasha was sincere in her belief, and the Discovery Channel producer, Monica Garnsey, attested to the unanimous agreement on that from the part of all of the program crew), is nevertheless willing to put it to test, even under skeptic and stringent conditions. She is amenable to feedbacks from scientists. She is engaged in an activity that may save or compromise lives. The CSICOP fellows had the opportunity to help her in that. Even to help her to discover and to understand the weaknesses of her "powers". They failed to do that. They failed in their social duty. It is a shame.
----------------
Our test had its limitations. None of these limitations, however, worked against Natasha's claim. If anything, they may have artificially enhanced her score.
Again, Hyman lies. Contrary to the protocols, the subjects never proved their alleged conditions. Contrary to the protocols, removed appendixes and minute scars were presented. Contrary to the protocols, Natasha was taken in into accepting these two allien conditions by the deceiving (and technically wrong) arguments from Wiseman. Contrary to the protocols, the man with the metal plate on his skull seemed to be unblind (it seemed that he could see the camera, when it pointed in his direction). Contrary to the protocols, Nickell and Wiseman demanded that Natasha's mother not be present with her during the test (though this demand was subsequently reversed). Why is it that Hyman cannot see the obvious? How can he find the "paranormal" (if it really exists) if he can't even see the "normal"?...
----------------
Our task was not to prove that Natasha does not have X-ray vision. Rather, Natasha and her supporters had the responsibility to show us that she could perform well enough to deserve further scientific investigation. This they failed to do.
And she is still performing as an informal diagnosing therapist... Because you failed to do a decent job. She has all the right in the world to believe that she was treated unfairly in this test. She was treated so indeed. As to her powers, they may be true. How can we tell? At least from this CSICOP test, the only conclusion that true scientists can gather is that the test was inconclusive...
Now, some comments about the article from Andrew Skolnick. His article is entitled: "The Girl with Normal Eyes". After reading this article, and thinking about its author, the only title that I can give to this review is:
The Man With Abnormal Eyes...
Natasha claims she can see everything inside of people's bodies down to the cellular level,
The one who claims this is Skolnick, not Natasha. Skolnick is deliberately lying here, and in many other instances too. I don't know what he thinks that he or anyone is going to gain from it. Skolnick, who has never looked cells through a microscope, does not have the faintest idea of what it means to be (that is, what is implied in being) able to see at the cellular level. If true, this power of vision, rather than being a diagnosing gift, would actually be a curse for the untrained eye, like Natasha's surely is (it is not something you can easily train yourself on your own).
----------------
and her mother says her readings are 100 percent accurate.
Skolnick gives such a tremendous importance to what Natasha's mother said once (yes, she only said it at one occasion, that Skolnick knows about) that Freud would surely have lots to say about it.
----------------
So the test-which required her to match at least five of the target medical conditions to the correct subjects-should have been very easy compared with her normal readings.
So it is sure that, in Skolnick's mind, the test was meant to investigate Natasha's mother's claims. Freud might explain that...
----------------
She was told exactly what to look for and exactly where to look.
Oh, yes. Told her exactly what to look for. She asked Skolnick what was the size of the extracted part of the esophagus, and he replied exactly the size of it: "You don't know it, Natasha, and neither do I" (Note added in November 2005: this is my free rephrasing of what Skolnick actually said, while keeping the original meaning). Couldn't be any more exact than that...
----------------
Yet it took her more than four hours to complete the test and, inexplicably, she took an hour to examine the seven subjects before deciding which one was missing a large part of her left lung!
This was indeed, to me, the most serious problem in Natasha's performance at the test. It is really important to point it out.
----------------
She guessed that one correctly, but why would anyone who claims to be able to see "every cell" inside a person take an hour to decide which person was missing a large portion of one lung?
Well, just because that was indeed a very serious problem (and an indication of lack of "paranormality", especially lack of a strong and trustworthy "paranormality"), we don't need to make lies about Natasha's claim, like Skolnick is doing above. No souce that Skolnick knows of has ever passed to him the information that Natasha can see every cell in the body. That is why I say that CSICOP gathered material for a good analysis of the case (though a preliminary one) but ended up spoiling it all in many ways.
----------------
Natasha matched only four of the conditions correctly-a score that everyone prior to the test had agreed would not justify further testing.
Again, CSICOPers forget to remember the very many violations of the protocol that they themselves committed. What a shame.
----------------
There was an error in our test protocol, which could have been much more embarrassing for us, had Natasha had the power to see it. Although our test design required no more than one subject to have any of the target medical conditions, we discovered after the test was over that the man who had a metal plate in his head also had undergone an appendectomy. He had forgotten to mention this when he was recruited but brought it up after the test because a missing appendix had been one of the test target conditions. This mistake gave Natasha twice the chance of correctly guessing which subject was missing his or her appendix. Despite having twice the odds of being correct, she still chose a subject with an intact appendix.
Well, this error was embarrasing enough, especially because it reveals that the "daunting task" of recruiting the subjects was done in a careless way. If the recruiter had asked that specific subject about his having or not any of the other conditions that the other subjects had, no doubt he would have remembered that he had no appendix. What a sloppy test...
----------------
Natasha protested during the test that appendixes can grow back after an appendectomy. When told this isn't possible, she insisted that they do grow back in Russia.
When we watch the documentary, we see that the situation was not exactly this one described by Skolnick. He concluded that Natasha was insisting that appendixes grow back, but it seems that he reached this conclusion through his hasty reasoning, because he didn't even let the translator say all that Natasha was objecting to. The man is a army tank...
----------------
After the test, Natasha asked if she could give me a medical reading in the hope of convincing me that her powers are real. With CSICOP's Senior Research Fellow Joe Nickell taking notes, I agreed.
Now, the man that could not read (Nickell, who failed to read the protocol and so demanded that Natasha's mother not be in the test room) is asked to write... What can possibly come out of it? Note that these neandertal skeptics don't carry video cameras or even tape recorders. So scientific and technologically well equipped...
----------------
Her reading, in which she scanned me from head to toes while describing the abnormalities she saw, took about ten minutes. Few of my organs passed the inspection: My neck vertebrae were too tight, too close. The bronchial tubes of my lungs had phlegm causing me to cough. The muscle on the left side of my heart is a bit weak and the valve closes late. The mucosa of my stomach is abnormal. A segment of my liver was enlarged and I was suffering poor bile circulation. The head of my pancreas is increased and abnormally dark (although not seriously). My duodenum has a little scar. My prostate gland has a nodule and is inflamed. My right kidney has "sand," while my left kidney's urethra is enlarged. In other words, I should forget about ever again signing an organ donor card. Neither my physician nor I are aware of any of these problems. Nevertheless, Natasha and her supporters claim she sees what doctors and their tests often miss. The only way I could prove her wrong would be to submit to an autopsy-which I'm not quite ready to do.
Oh, what a pitty... Anyway, we will gladly wait for the autopsy... :-). Come on, Skolnick, don't be so drastic. For the vertebrae, some innocent X-raying would do (not altogether harmless, but no "Hulk" side effects too); also for your bronchial tubes. As to the heart, it is high time it got checked anyway; leave the stomach mucosa be, and in that I agree; for the liver, an ultrasonography might do a harmless work; and for the prostate, some routine exams would penetrate the issue smoothly... ;-).
----------------
I believe Natasha may have been making some bad guesses based on non-paranormal observations. While the test was ongoing, I swallowed some water down my wind pipe, which triggered a loud fit of coughing.
So the autopsy does not seem to be that far away after all...
----------------
It was so loud that someone had to close the briefing room door. Natasha apparently misjudged the cause of my coughing. My shoulders and neck were sore and tired from lugging around my luggage and a heavy camera bag. She apparently misread my drooping posture as a sign of a spinal problem. And earlier, I had complained to the Russian translator that I had a headache (from stress and lack of sleep). I suspect that information may have been passed on to Natasha, who wrongly attributed the problem to narrowed blood vessels in my neck. Even more telling was what Natasha didn't "see." She failed to point out any of my confirmed health problems, which include hypertension, hypercholesterolemia, recent colon surgery, nasal polyps, a very narrow pharynx, and sleep apnea. She also didn't see the calcified plaque in two of my coronary arteries that was documented by spiral CT scan.
These comments above from Skolnick really deserve serious consideration. They are indeed indicative of Natasha's getting clues through non-paranormal means. I think he exaggerates when he demands that she identify his cholesterol (he is still being haunted by the legend of the russian psychic that could spot isolated molecules from afar). But I would, just like him, be a little, let's say, "doubtful" about her missing items like the calcified plaque and the very narrow pharynx.
----------------
Natasha Demkina supporters widely site this "diagnostic" drawing of a "sarcoidosis granuloma" as evidence of her remarkable X-ray vision. Yale Rosen, M.D., a leading expert on sarcoidosis granuloma, however, disagrees.
Mr. Yale Rosen does not disagree on anything. He, too, was taken in by CSICOPers, and ended up issuing a comment about a claim that neither Natasha nor anyone directly involved with the case ever made (as far as I can tell, and as far as Skolnick can tell too).
----------------
Unable to describe what she saw in the man's lungs, she drew a picture. With the help of Natasha's drawing, the man got a referral to a doctor in Moscow, who claimed to see the same thing through her microscope Skolnick needs to watch the documentary again. Things did not happen that way. and that the drawing is indicative of an inflammatory disease lesion called sarcoidosis granuloma. That's a remarkable confirmation of the young psychic's powers-at least for viewers who don't have a clue what sarcoidosis granulomas look like.
It is important to watch carefully this section in the documentary (Skolnick should go see it again!). It may be (by what I saw in the documentary) that the doctor wasn't really saying that what she was seeing through the microscope looked like Natasha's drawing. She might have only said that the drawing looked like a macroscopic lung with sarcoidosis. I myself managed to find a photo of a lung with sarcoidosis in Yale Rosen's site that looks very much like Natasha's drawing.
----------------
It didn't look like any pathological finding I've seen,
Then, Skolnick, go take a look at Yale Rosen's site! The link below:
http://www.granuloma.homestead.com/sarcoidosis_gross.html
Honeycomb Lung With Emphysema In The Upper Part Of Specimen.
----------------
so I sent an image of the drawing to Yale Rosen, M.D., Professor of Pathology at SUNY Downstate Medical Center, a leading expert on sarcoidosis granuloma, whose Atlas of Granulomatous Diseases is widely used in the training of pathologists. Here is Dr. Rosen's opinion of Demkina's drawing:
Try as I might I can see no resemblance whatsoever of this drawing to a granuloma or to any other microscopic pathologic finding that I know of. If I were presented with this drawing and no background history I would guess that it was made by a 4-6 year old child who was trying to depict a human-like head with four appendages (? arms and legs) attached. If Ms. Demkina is claiming that a physician made a diagnosis of sarcoidosis based upon this drawing I would say that that's simply unbelievable.
Skolnick told Rosen that the drawing was of microscopic cells that Natasha saw in the man's lung, and that the man showed that drawing to the pathologist before the pathologist saw the cells through the microscope. All wrong. No one knows (and Skolnick never bothered trying to find...) if the drawing is of cells or of the macroscopic lung. No one asked Natasha that. No one even asked the man with sarcoidosis that. And no one said that the diagnosis of sarcoidosis was made based upon that. This last point is a hundred percent clear in the documentary. First the pathologist in russia looked through the microscope and issued her diagnosis. Only after that did the man show to the doctor the drawing. Skolnick got it all wrong. Sloppy work. How did he manage to stay so long in JAMA is a mystery.
----------------
We've asked the producer/director several times when the documentary may be broadcast in the United States. We have not received an answer. I am afraid that Discovery Channel may consider the program too skeptical for the American audience. I hope I will be proven wrong.
Certainly the Discovery Channel people have already realized the obvious: that although blind beliefs may be a serious problem in the United States, a far worse problem is this pseudo-skepticism, as practiced by Hyman, Wiseman, and Skolnick.
Well, one good thing in this article from Skolnick is the fact that he, at last, decided to throw away his private (and wrong) statistics regarding the sum of Natasha's hits and misses (he had wrongly inflated the statistical significance of her false negatives, leaving aside her true negatives altogether). Instead, he focused correctly on the qualitative aspect of this issue of false negatives and false positives, and by doing this he really, and properly, addressed the concerns of public health in this issue.
It may be that not everything is lost after all...
Julio Siqueira - juliocbsiqueira@terra.com.br
Microbiologist (non-practicing)
May 15, 2005.