Scientists have used a chapter of Harry Potter and the Sorcerer’s Stone to identify what different regions of the brain are doing while people read.
Researchers from the machine learning department at Carnegie Mellon University performed functional magnetic resonance imaging (fMRI) scans on eight people as they read a chapter of the Harry Potter book.
They then analyzed the scans, cubic millimeter by cubic millimeter, for every four-word segment of that chapter. The result was the first integrated computational model of reading, which identifies which parts of the brain are responsible for sub-processes like parsing sentences, determining the meaning of words, and understanding relationships between characters.
As Leila Wehbe, a PhD student in the machine learning department, and Tom Mitchell, the department head, report in PLOS ONE, the model was able to predict fMRI activity for novel text passages with sufficient accuracy to tell which of two different passages a person was reading with 74 percent accuracy.
“At first, we were skeptical of whether this would work at all,” says Mitchell, who notes that analyzing multiple sub-processes of the brain at the same time is unprecedented in cognitive neuroscience. “But it turned out amazingly well and now we have these wonderful brain maps that describe where in the brain you’re thinking about a wide variety of things.”
Dyslexia, stroke, or learning trouble
Wehbe and Mitchell say the model is still inexact, but might someday be useful in studying and diagnosing reading disorders, such as dyslexia, or to track the recovery of patients whose speech was affected by a stroke. It could also be useful to educators for identifying what might be giving a student trouble when learning a foreign language.
“If I’m having trouble learning a new language, I may have a hard time figuring out exactly what I don’t get,” Mitchell says. “When I can’t understand a sentence, I can’t articulate what it is I don’t understand. But a brain scan might show that the region of my brain responsible for grammar isn’t activating properly, or perhaps instead I’m not understanding the individual words.”
Researchers at Carnegie Mellon and elsewhere have used fMRI scans to identify activation patterns associated with particular words or phrases or even emotions. But these have always been tightly controlled experiments, with only one variable analyzed at a time. The experiments were unnatural, usually involving only single words or phrases, but the slow pace of fMRI—one scan every two seconds—made other approaches seem unfeasible.
Wehbe was still convinced that multiple cognitive sub-processes could be studied simultaneously while people read a compelling story in a near-normal manner. She believed that using a real text passage as an experimental stimulus would provide a rich sample of the different word properties, which could help to reveal which brain regions are associated with these different properties.
“No one falls asleep in the scanner during Leila’s experiments,” Mitchell says.
They devised a technique in which people see one word of a passage every half second—or four words for every two-second fMRI scan. For each word, they identified 195 detailed features—everything from the number of letters in the word to its part of speech. They then used a machine learning algorithm to analyze the activation of each cubic centimeter of the brain for each four-word segment.
Bit by bit, the algorithm was able to associate certain features with certain regions of the brain, Wehbe says.
When Harry Potter flies
“The test subjects read Chapter 9 of Sorcerer’s Stone, which is about Harry’s first flying lesson,” she notes. “It turns out that movement of the characters—such as when they are flying their brooms—is associated with activation in the same brain region that we use to perceive other people’s motion.
“Similarly, the characters in the story are associated with activation in the same brain region we use to process other people’s intentions.”
Exactly how the brain creates these neural encodings is still a mystery, they say, but it is the beginning of understanding what the brain is doing when a person reads.
“It’s sort of like a DNA fingerprint—you may not understand all aspects of DNA’s function, but it guides you in understanding cell function or development,” Mitchell says. “This model of reading initially is that kind of a fingerprint.”
A complementary study by Wehbe and Mitchell, presented earlier this fall at the Conference on Empirical Methods in Natural Language Processing, used magnetoencephalography (MEG) to record brain activity in subjects reading Harry Potter.
MEG can record activity every millisecond, rather than every two seconds as in fMRI scanning, but can’t localize activity with the precision of fMRI. Those findings suggest how words are integrated into memory—how the brain first visually perceives a word and then begins accessing the properties of the word, and fitting it into the story context.
The National Science Foundation, the National Institute of Child Health and Human Development, and the Rothberg Brain Imaging Award supported this work.
Source: Carnegie Mellon University