The Not So Seductive Allure of Colorful Brain Images

by The Neurocritic in The Neurocritic

We all know that the mere presence of a brain scan image or a neuro-prefix adds instant credibility to any news story, right? And that the public (i.e., undergraduates) is easily swayed into believing in bogus psychological findings if accompanied by pretty colorful brains? Well count me in! But wait...Neuroscience Fiction Fiction?The day after the high-profile Neuroscience Fiction article by Dr. Gary Marcus appeared in The New Yorker, a stealthy blog post in Brain Myths summarized an unpublished paper (Farah & Hook, in press, PDF) that  refutes this notion.1Are Brain Scans Really So Persuasive? New evidence suggests the allure of brain scans is a myth Published on December 3, 2012 by Christian Jarrett, Ph.DA pair of psychologists at The University of Pennsylvania have highlighted a delicious irony. Sceptical neuroscientists and journalists frequently warn about the seductive allure of brain scan images. Yet the idea that these images are so alluring and persuasive may in fact be a myth. Martha Farah and Cayce Hook refer to this as the “seductive allure of ‘seductive allure’” (PDF via author website).Most of their evidence against the "seductive allure" is from unpublished data described in their in press article (which we can't evaluate yet):Two series of as yet unpublished experiments have failed to find evidence for the seductive allure of brain images. Michael, Newman, Vuorre, Cumming, and Garry (2012, under review) reported a series of replication attempts using McCabe & Castel’s Experiment 3 materials. Across nearly 2000 subjects, a meta‐analysis of these studies and McCabe & Castel’s original data produced a miniscule estimated effect size whose plausible range includes a value of zero. Our own work (Hook & Farah, in preparation) has also failed to find evidence that brain images enhance readers’ evaluation of research in three experiments comprising a total of 988 subjects.However, one published paper did fail to find an effect of fMRI images on how participants judged the scientific reasoning and credibility of a fake news story titled, “Scientists Can Reconstruct Our Dreams” (Gruber & Dickerson, 2012).2  The study was designed to replicate the previous study of McCabe and Castell (2008) with some notable exceptions. Rather than using a bar graph or an ugly and cluttered EEG topographic map as the comparison images in separate groups, Gruber and Dickerson used:...a fantastical, artistic image of a human head and a cyberspace-esque background with swirly lines. The final group was given an image from the popular science fiction film Minority Report in which three children’s dreams of the future are projected on a screen and used to prevent crime.Very io9... But both studies did have a no-image condition.The Gruber and Dickerson study also added additional questions to explicitly assess credibility and authoritativeness, in addition to whether the scientific reasoning made sense. Results showed that all cases, ratings did not differ statistically across the conditions, including the fMRI vs. no-image comparison.Hmm... Farah and Hook also debunked the study of Weisberg et al., (2008), which didn't use images at all but added neuroscience-y explanations to 18 actual psychological phenomenon. The problem was that the neuroscience-y paragraphs were longer than the no-neuroscience paragraphs. The author of the excellent but now-defunct Brain In A Vat blog had a similar objection, as explained in I Was a Subject in Deena Weisberg's Study:So how does it feel being held up to the scientific community as an exemplar idiot? Well, it’s a bit embarrassing. One of my coping mechanisms has been to criticize the experimental design. For instance, I think its problematic that the with neuroscience explanations were longer than the without neuroscience explantions. If subjects merely skimmed some of the questions (not that I would ever do such a thing), they might be more likely to endorse lengthier explanations.Neuroskeptic also raised this point in his otherwise [mostly] positive evaluation of the study, Critiquing a Classic: "The Seductive Allure of Neuroscience Explanations":Perhaps the authors should have used three conditions - psychology, "double psychology" (with additional psychological explanations or technical terminology), and neuroscience (with additional neuroscience). As it stands, the authors have strictly shown is that longer, more jargon-filled explanations are rated as better - which is an interesting finding, but is not necessarily specific to neuroscience.He noted that the authors acknowledged this objection, but also that the conclusions we can draw from the study are fairly modest.What does this mean for Neuro Doubt and Neuroscience Fiction and Neurobollocks? The takedowns of overreaching interpretations, misleading press releases, and boutique neuro-fields are still valid, of course, but the critics themselves shouldn't succumb to the seductive allure of seductive allure. But we must also remember that the most thorough critiques of seductive allure still await peer review.3 Footnotes1 And makes me feel a little silly.2 The experiment must have been designed before these actual 2012 headlines: Scientists read dreams (Nature) and Scientists decode contents of dreams (Telegraph).3 I wrote to two of the authors of the original studies (Weisberg and Castel) to get their reactions, but haven't heard back. Very, very tragically, we... Read more »