A Carrot and a Stick to Motivate Data Fusion: Two Stories About EEG, MEG and fMRI Analysis

Sergey Plis
The Mind Research Network


Abstract: Surprisingly, data fusion analysis is not a universally accepted way to study brain function. This is true, despite the common understanding that any brain-imaging modality alone provides only a limited view into brain function; despite growing availability of multimodal datasets; and despite general acceptance of the complementary nature of information hidden in various modalities. Among the multitude of reasons preventing researchers from practicing multimodal analysis, arguably, two reasons contribute the most: a doubt that modalities that are very different contain any common information to reinforce their signal, and a comfortable feeling that what is learned from one modality cannot be wrong, but, in the worst case, incomplete. This talk will mitigate the doubt of impossibility and will demonstrate that unimodal analysis can be misleading without any way for one to know it. I will demonstrate differing analyses that each in turn rely on unique strengths of EEG (temporal dynamics) and fMRI (spatial resolution) and yet find overlapping intrinsic brain activity. And show how identical but separate analysis of MEG and fMRI collected from the same subjects in identical experiments leads to strictly opposite conclusions about effective connectivity and thus brain function. With these examples, I will demonstrate importance for analysis frameworks to correctly account for the properties of each of the data sources and argue for taking multiple data sources into account.

Conference Sponsors