¡Descarga Neuroimaging: Emotional Processing in Reading with Amygdala Activation - Prof. Mateos y más Apuntes en PDF de Psicología solo en Docsity!
Amygdala and heart rate variability responses from listening to emotionally intense
parts of a story
Mikkel Wallentin
a,b,
⁎, Andreas Højlund Nielsen
a,c
, Peter Vuust
a,d
, Anders Dohn
a,d
Andreas Roepstorff
a,c
, Torben Ellegaard Lund
a
a (^) Center of Functionally Integrative Neuroscience, Aarhus University Hospital, Nørrebrogade, 8000 Aarhus C, Denmark b (^) Center for Semiotics, Aarhus University, Denmark c (^) Department of Anthropology, Archaeology and Linguistics, Aarhus University, Denmark d (^) Royal Academy of Music, Aarhus/Aalborg, Denmark
a r t i c l e i n f o a b s t r a c t
Article history: Received 16 December 2010 Revised 14 June 2011 Accepted 24 June 2011 Available online 2 July 2011
Emotions are often understood in relation to conditioned responses. Narrative emotions, however, cannot be reduced to a simple associative relationship between emotion words and their experienced counterparts. Intensity in stories may arise without any overt emotion depicting words and vice versa. In this fMRI study we investigated BOLD responses to naturally fluctuating emotions evoked by listening to a story. The emotional intensity profile of the text was found through a rating study. The validity of this profile was supported by heart rate variability (HRV) data showing a significant correspondence across participants between intensity ratings and HRV measurements obtained during fMRI. With this ecologically valid stimulus we found that narrative intensity was accompanied by activation in temporal cortices, medial geniculate nuclei in the thalamus and amygdala, brain regions that are all part of the system for processing conditioned emotional responses to auditory stimuli. These findings suggest that this system also underpins narrative emotions in spite of their complex nature. Traditional language regions and premotor cortices were also activated during intense parts of the story whereas orbitofrontal cortex was found linked to emotion with positive valence, regardless of level of intensity. © 2011 Elsevier Inc. All rights reserved.
Introduction
Stories are universally used for giving readers or listeners emotional experiences. Any coherent sequence of events could be said to constitute a narrative, but the stories that engage us and which we share are stories that move us, somehow (Hogan, 2003). Whole departments are dedicated to the study of narratives at almost every university, but the brain processes underlying the experience of narrative emotions have received surprisingly little attention. There is a long tradition for using story comprehension as a means for studying other cognitive and clinical phenomena, such as theory of mind and autism (e.g. Gallagher et al., 2000; Happé, 1994; Saxe and Kanwisher, 2003; Vogeley et al., 2001). Recently, researchers have also used stories to investigate the structure of complex linguistic representations (Crinion et al., 2003; Wilson et al., 2007; Xu et al., 2005; Yarkoni et al., 2008), such as syntax (Brennan et al., in press), event boundaries (Speer et al., 2007) and motor/perceptual repre- sentations (Deen and McCarthy, 2010; Speer et al., 2009; Wallentin et
al., in press). None of these studies, however, take into consideration the emotional experiences, central to story processing. Emotions in the brain have often been linked to the limbic system, coined by MacLean (1949). However, no strict criteria exists for inclusion in this system, and it has been suggested to abandon the concept while maintaining the notion that emotions involve relatively primitive circuits found throughout mammalian evolution (LeDoux, 2000). An alternative avenue for the study of emotions has been in terms of stimulus–reinforcement association, i.e. conditioning (Pavlov, 1927). Emotions are in this context considered as states elicited by reinforcing stimuli, i.e. rewards and punishers and the signs associated with them (e.g. Rolls, 2000, 2005). Using fear as a model this approach has led to the discovery of a well-characterized network of brain regions involved in emotion evoked by sound. Conditioned fear responses to auditory stimuli are processed primarily through a nucleus in the lateral amygdala (LeDoux, 1995, 2000). Auditory inputs to the lateral amygdala come both directly from the auditory thalamus (the medial geniculate nucleus) and via the temporal cortices. Cortical inputs to the amygdala are thought to arise from association areas of the temporal cortices, rather than directly from the primary auditory cortex. These secondary auditory regions supposedly provide the amygdala with a more dense representation of the input than can be achieved with thalamic
NeuroImage 58 (2011) 963– 973
⁎ Corresponding author at: Center of Functionally Integrative Neuroscience, Aarhus University Hospital, Nørrebrogade, 8000 Aarhus C, Denmark. Fax: +45 89494400. E-mail address: [email protected] (M. Wallentin).
1053-8119/$ – see front matter © 2011 Elsevier Inc. All rights reserved. doi:10.1016/j.neuroimage.2011.06.
Contents lists available at ScienceDirect
NeuroImage
j o u r n a l h o m e p a g e : w w w. e l s ev i e r. c o m / l o c a t e / y n i m g
input alone (LeDoux, 2007). This suggests that there may be two routes to emotion, a fast “automatic” route, linked to direct associa- tions between reinforcers and experience, and a more “cognitive” route with a broader contextual outlook. Output projections from the lateral nucleus of the amygdala go to, among other regions, the central nucleus of the amygdala which via the brain stem and hypothalamus mediates arousal in terms of behavioral (fight-or-flight) and sympathetic autonomic (e.g. in- creased heart rate) responses (LeDoux, 2007). Recent developments within the study of emotional processing, however, have suggested that rather than being a fear “module”, these pathways may constitute a more general system for relevance detection (Herbert et al., 2009; Paton et al., 2006; Sander et al., 2003; Sergerie et al., 2008) monitoring both positive and negative emotions. Amygdala also has dense projections to orbitofrontal parts of the brain, known to be involved in the experience of emotions, in particular hedonic experiences (Kringelbach, 2005; Kringelbach and Rolls, 2004). Furthermore, the notion of “conflict”, central to narratives, has been described as the situation where an environ- mental stimulus is linked to both positive and negative reinforcers (Rolls, 2005, 2008) This suggests that experienced emotions in narratives might initially be studied with a two-dimensional model of emotions, including level of arousal and valence (Russell, 1979), i.e. using a general emotional intensity measure and a measure that treats positive and negative feelings separately (i.e. valence). Linguistic emotional content has primarily been studied at the single word level. A number of studies have studied the phenomenon with electroencephalography (EEG) using event-related potentials (ERP). Kissler et al. (2007) studied reading of words and found that both positive and negative words lead to an enhanced negativity in the ERP peaking around 200–400 ms after stimulus onset and that pleasant and unpleasant words did not differ. This result has since been replicated a number of times (Herbert et al., 2008; Kissler et al., 2009; Schacht and Sommer, 2009; Scott et al., 2009) and the authors suggest that this enhancement may be driven by cortico-amygdaloid connections (Kissler et al., 2007). Faster effects have also been reported in the N1 time domain (135–180 ms after word presenta- tion) before normal lexical processing is thought to occur (Scott et al., 2009). This effect was enhanced for high frequency words, indicating that multiple exposures may turn the processing of linguistic emotional content into an automatic response. The neuroimaging literature also finds a role for amygdala in the processing of emotional words. Isenberg et al. (1999) in a PET study reported increased blood flow in the amygdala in response to threat words (such as “torture” or “beat”) relative to neutral words (such as “wash” or “instruct”) even though these words were embedded in a Stroop task where the participants were instructed to report the color of the letters rather than the content of the words. Amygdala activation has also been found with positive words, e.g. “successful” or “happy” (Hamann and Mao, 2002; Herbert et al., 2009). Furthermore, direct evidence exists for linguistic stimuli being able to alter neurotransmitter release in emotion related brain regions. In their study, Badgaiyan et al. (2009) contrasted a condition where healthy participants read either emotionally neutral words (such as “park” or “pencil”) with a condition where they read words with emotional connotations (such as “fire” or “blood”) while undergoing a PET scan with a dopamine receptor ligand ( 18 F-Fallypride). Dopamine release was found to increase both in left amygdala and in left inferior frontal cortex as a function of increased intensity. Lastly, intracranial recordings in epilepsy patients have also demonstrated amygdala activation related to subliminally presented emotional words (Naccache et al., 2005). Together these data suggest that words may form direct long-term associations with structures relevant for processing emotions, to the extent that conscious awareness of having processed the words may no longer be necessary. However, within narratives a one-to-one mapping between words denoting emotions and actual experienced emotion is rarely found.
Skinner famously tried to describe language processing in behavioristic terms: “If a verbal stimulus frequently accompanies some state of affairs which is the unconditioned or previously conditioned stimulus for an emotional reaction, the verbal stimulus eventually evokes this reaction” (Skinner, 1957, p. 154). In his critique, Chomsky (1959), pointed to the difficulty of a direct probabilistic mapping between linguistic stimuli and non-linguistic experiences, saying that according to this logic a person would not be able to respond appropriately to the utterance Your money or your life “unless he has a past history of being killed” (ibid. p. 46). One speculation could be that one function of narratives is to give people the opportunity to simulate exactly these experiences, but even if we allow for a transfer of emotional associations via other means than direct experience (e.g. through narratives) a Skinnerian approach to narrative emotions is still facing the problem that emotions in stories often are evoked without any mentioning of words directly describing emotional states or situations, e.g. a descriptive sentence like “the roulette landed on the number 25” may or may not be highly arousing depending on the context. Analogous to the fear conditioning circuit we therefore tentatively suggest that two routes to emotion also exist in emotional responses to narratives, a more “automatic” route, based on direct associations between reinforcers and descriptors (e.g. words and known scenes) and a more “cognitive” route, with dense contextual input. Evidence for the latter system comes from reading studies showing that readers do build elaborate representations of the emotions of characters in stories (De Vega et al., 1997; Graesser et al., 1994; Gygax et al., 2003). We may thus form the hypothesis that both kinds of processes exist in parallel and that experienced emotion resulting from listening to a story will tap into the same basic network of brain regions found to process emotionally relevant processes even in simple conditioning. Alternatively, the complexity of narratives might cause the emotions they evoke to rely on a completely different set of brain regions. This question, however, can only be addressed using real stories as experimental material. Among the few studies that do study emotions in a greater narrative setting, both results and methods are highly mixed. Some use video clips (Decety and Chaminade, 2003; Hasson et al., 2004), others auditory input (Ferstl et al., 2005; Sabatinelli et al., 2006) and some use reading paradigms (Jabbi et al., 2008). One influential study investigated inter-participant synchronization of BOLD activity during viewing of a motion picture (Hasson et al., 2004). Widespread parts of the brain were found to synchronize across participants and cortical areas (around 30% of the total brain) in a non- selective manner. Using a reverse correlation approach the authors found that peaks in the BOLD responses for this non-selective component was primarily found during perception of “emotionally arousing scenes”. How these findings translate to verbally mediated narratives is unclear. More problematic, however, is the fact that the authors' claim that these scenes are in fact arousing builds entirely on a heuristic analysis of the film clips that come out of the reverse correlation analysis. No attempts at qualifying what constitutes an arousing scene were made. Decety and Chaminade(2003) also used video clips in a PET study on sympathy. These video clips showed actors telling short (60–80 s) either sad or neutral stories with either appropriate or non- appropriate motor expression. Sad stories were found to yield increased blood flow in frontal cortices, including traditional language regions (left inferior frontal gyrus), motor and premotor regions as well as temporal poles and amygdala, bilaterally. Sabatinelli et al. (2006) studied narrative imagery using very short (12 s), verbally presented pleasant, unpleasant or neutral scenes. They found an effect of both pleasant and unpleasant scenes relative to neutrals in secondary auditory cortices and left middle frontal gyrus during listening to the stories. No effects of emotional valence were found. Left inferior frontal cortex and supplementary motor regions were found to respond more to emotional than neutral content during subsequent imagery of the same scenes.
median age: 25 years, range: 20–47 years). Each received a 100 DKK payment per hour for participation. Gender was not considered important (Wallentin, 2009). All participants reported having Danish as their primary language. All participants were right-handed as measured by the Edinburgh Inventory (Oldfield, 1971; score range 80 – 100). One participant was excluded due to not performing above chance level in the post scanning memory test (see results in Memory test). fMRI data from one participant was lost due to a technical problem. Another technical problem caused physiological data to be discarded from three participants. Thus, heart rate data analyses (see Heart rate variability (HRV)) only include 21 out of the 24 analyzed fMRI participants.
Stimuli and material
Narrative. The same recording of The Ugly Duckling as that used in the rating session was used for the scanning experiment (see Narrative and procedure).
Memory test. Before and after the story, participants answered a multiple-choice test probing their detailed knowledge of The Ugly Duckling. Participants read 20 sentences from the narrative and chose for each between five possible completions of the sentence.
Procedure After initial instructions and screening, participants were given the pre-scan test for knowledge about the story. They were then placed in the MR-scanner and scanned while listening to the story. The story started 12 s after the onset of fMRI acquisition and ended 12 s before the end of the scanning session. Stimuli were delivered through pneumatic headphones from Avotec (Stuart, FL USA). Participants were instructed to listen carefully to the story during which they
would have to press a button four times whenever a voice, different from the one telling the story, uttered the Danish word “nu” (English: “now”). Immediately after the scanning session, participants were taken out of the scanner and given the memory test again. All participants confirmed being able to hear the story during scanning. Structural scans were obtained at a subsequent scanning session. A 3 T General Electrics Medical Systems (Milwaukee, WI USA) MR system with a standard head coil was used to acquire both T 2- weighted gradient echo, echo-planar images (EPI) with Blood Oxygenation Level-Dependent (BOLD) contrast and T 1 -weighted structural images. Five hundred seventy EPI volumes were acquired per participant, not including the first 5 volumes that were discarded to allow for effects of T1 equilibrium. Whole brain coverage was achieved using 35 axial slices of 3 mm thickness with an in-plane resolution of 3 × 3 mm in a 64 × 64 voxel matrix (FOV 192 mm). Images were obtained with a TR of 2200 ms, a 30 ms TE and a 90° flip angle. A high-resolution 3D GR T 1 anatomical scan was acquired for spatial processing of the fMRI data. It consisted of 256 × 256 × 134 voxels with a 0.94 × 0.94 × 1.2 mm 3 voxel size, obtained with a TR of 6.552 ms, a 2.824 ms TE and a 14° flip angle. Cardiac responses and respiration data were sampled continuously during scanning at 1000 Hz using the scanner equipment. Time stamps for each heartbeat were automatically generated.
Analysis The two regressors of interest (emotional intensity and valence) were used as independent variables in the analysis of both heart rate variability and fMRI data. We also created an additional regressor, based on the valence ratings, which modeled valence as a bivariate measure, i.e. either positive or negative, not including degree of arousal (see Fig. 2).
0 500 1000 1500 2000 2500 3000 0
1
2
3
4
5
6
7
8
9
10
Word number
intensity +SD
0 500 1000 1500 2000 2500 3000
0
5
Word number
valence +SD
1651
1289 1651
1289 3097
3097
Fig. 1. Top) solid: average intensity estimate (with standard deviations) across the story The Ugly Duckling by H.C. Andersen, obtained from 27 independent raters. Bottom) valence estimates obtained from 26 raters. Both estimates were used as independent variables in the fMRI experiment in a search for the neural underpinnings of narrative emotion. Using reverse correlation analyses we were able to study the content of the most intense parts of the story (e.g. at word numbers 1289, 1651 and 3097).
Heart rate variability (HRV). The high frequency (HF) components of the HRV signal (in the 0.15–0.4 Hz range) are thought to reflect parasympathetic modulation of the heart rate. Low frequency (LF) components of the HRV signal (in the 0.05–0.15 Hz range), on the other hand, are thought to reflect sympathetic nervous system, although this is more controversial since LF components have also been argued to contain a parasympathetic component, to some extent. Due to this concern, it has been argued that the ratio of LF to HF power constitutes the most appropriate index of shifts in biases toward sympathetic or parasympathetic control over cardiac function (Appelhans and Luecken, 2006; Berntson et al., 1997; Malik et al., 1996). The LF/HF ratio regressors were constructed in the following way: — Differences between time stamps for each heartbeat were used to generate a heart rate time series for each participant. This was first up-sampled to 10 Hz (using a linear interpolation method— interp1.m in MATLAB) in order to achieve common samples points across participants. — Next a spectrogram (power-density as a function of time and frequency) was created on the basis of the up-sampled heart rate time course using the MATLAB function spectrogram.m. In order to achieve a smoothly varying signal we used segments of 200 samples (20 s) with an overlap of 199 samples, each segment was windowed with a hamming window of 20 s. This relatively short time window allows for a control over when a shift in HRV takes place while still allowing for a determination of frequency components with a 0.05 Hz resolution. The estimated frequency components from each window were considered as belonging to a time point in the middle of the time window (i.e. 10 s after the beginning of each time frame). This meant that there were no HRV measures for the first and last 10 s of the experiment. Since the scanning session started 12 s before the start of the story and ended 12 s after the end of the story, this procedure still enabled us to study the impact of the story on HRV across the whole story. — The power profiles for the first 2 non-DC frequency bins were then summed to create a LF time series and similarly the power profiles of the next 5 frequency bins were summed to create a HF time series. From these time series a LF/HF ratio time series were created. The LF/HF ratio time series was used as dependent variable in a multiple regression analysis using the high-pass filtered intensity and valence regressors (see above) as independent variables. Analyses were performed using a two-level general linear model approach similar to that used in the analysis of fMRI data (Penny and
Holmes, 2007; Worsley and Friston, 1995). At the single subject level regression coefficients were estimated for each of the three predictors (using the glmfit.m function in MATLAB), and these beta-estimates were then each submitted to a group level random effects one-sample t-test.
Pre-processing of fMRI data. All fMRI image pre-processing and data analyses were performed using Statistical Parametric Mapping software (SPM8; Wellcome Department of Imaging Neuroscience, www.fil.ion.ucl.ac.uk/spm), implemented in MATLAB. Functional images were motion-corrected and registered to the first EPI image. The mean of the motion-corrected images was then co-registered to the individual's structural MRI using a 9-parameter affine transfor- mation. The anatomical image was spatially normalized to the standard MNI space using the unified segmentation–normalization algorithm within SPM (Ashburner and Friston, 2005) with the resulting deformation field subsequently applied to the fMRI data. Finally, data were spatially smoothed with an isotropic 8 mm full width at half maximum (FWHM) Gaussian kernel to account for differences between participants.
Statistical analyses of fMRI data. Statistical analyses of fMRI data were performed using a two-level general linear model approach (Penny and Holmes, 2007; Worsley and Friston, 1995). Single subject analyses. The three regressors of interest (intensity, valence and valence bivariate) were convolved with the standard HRF, resampled to the sampling frequency of the scanner and high-pass filtered (128 s cut off) before being used in a multiple regression SPM analysis. The HRV LF/HF regressor (see above) was also resampled and included. Due to the assumed relationship between emotional responses and physiological variables that are known to cause artefacts in the fMRI signal, such as cardiac pulsation and respiration, studies like the present are under a great risk of reporting false positives directly caused by these physiological effects (and not by their neurological causes). It is therefore important to filter these effects out prior to analysis of emotional effects. In this experiment nuisance effects were factored out using a nuisance variable regression approach (Lund et al., 2006). This approach includes regressors based on cardiac and respiratory oscillations (Glover et al., 2000—see also Figure S2 for an illustrative example) and six motion parameters (Friston et al., 1996; Lund et al., 2005) in addition to the SPM8 standard discrete cosine set high-pass filter (128 s cut off). Sound intensity was modeled by taking
−4 0 200 400 600 800 1000 1200
−
−
−
0
1
2
3
4
5
Story time (Seconds)
Ratings
Fig. 2. An additional valence regressor modeling only positivity or negativity was constructed, based on the original valence ratings (dashed line). Both were convolved with the hemodynamic response function before entering into SPM analysis.
were used to obtain continuous measures of experienced emotional intensity and valence while listening to The Ugly Duckling by H.C. Andersen. These measures were used to model both heart rate variability responses and the BOLD signal from the fMRI study using the same story as stimulus. We found that intense parts of a story yielded comparable shifts in the low frequency/high frequency power ratio in the HRV signal. These shifts are thought to reflect changes in the sympathetic and parasympathetic control of heart rate (Appelhans and Luecken, 2006; Berntson et al., 1997; Malik et al., 1996). This indicates
that intense parts of the narrative are accompanied by an increased sympathetic response. The negative correlation between our HRV measure and the valence regressor (see Fig. 3) seems to suggest that this primarily reflects the parts with negative emotions, but this may also be due to the fact that these were overall more intense than the positive parts. The stronger correlation for the intensity measure supports an interpretation where it is intensity, regardless of valence, which causes arousal, as measured by changes in HRV. We also found that intensity was accompanied by BOLD responses in the network of brain regions known to process conditioned emotional responses from auditory stimuli (LeDoux, 2000, 2007). These regions include temporal cortices, thalamus (medial geniculate nuclei) and amygdala. Our results therefore suggest that these structures may also be involved in processing the emotional experiences evoked by stories. These findings are consistent with single word studies (see Introduc- tion) and also corroborate findings from studies of narrative emotion of which some found amygdala activations (Decety and Chaminade, 2003; Ferstl et al., 2005) and others found temporal activation (Decety and Chaminade, 2003; Jabbi et al., 2008; Sabatinelli et al., 2006). We believe, however, that we are the first to show the whole network linked to emotional processing of sound stimuli, including the thalamic nuclei, known to project directly to the amygdala. This indicates that even though narratives form very complex relationships between verbal content and emotional impact, the underlying basic emotional circuitry responsible for processing the emotion is similar to that found in simple conditioning experiments. This network of activations was coupled with increased activation in other cortical regions, including language areas, such as left inferior gyrus and left posterior temporal cortex, and motor cortices (Fig. 4). This more extended network of activity is consistent with a dual route hypothesis, involving also more contextualized emotions (see Introduction). Primary language regions, such as left inferior frontal cortex have been found to increase their BOLD response as a function of increased linguistic processing cost or integration error (Christensen and Wallentin, 2011; Fiebach et al., 2005; Hagoort et al., 2004; Wallentin et al., 2006; Xu et al., 2005). In ERP research, increased emotional
Fig. 3. Heart rate variability. Left-top: normalized median low frequency/high frequency power ratio across participants reflecting changes in sympathetic/parasympathetic neural bias (line) showing a clear correlation with intensity ratings across the story (dashed line). Left-bottom: an anti-correlation was observed for the valence regressor, suggesting that it was the negative parts of the story that were primarily associated with the physiological response. Right: intensity was also found to be correlated with LF/HF HRV ratio when analyzing each participant individually and conducting a random effects analysis across participants. Both valence measures revealed non-significant trends. **: P b 0.01.
Table 2 Intensity effects.
Putative anatomical region Peak MNI Z-score Positive effects of intensity Inf. frontal L—BA 45 −50, 24, 0 4. Inf. frontal L—BA 44 −54,16, 18 4. Inf. frontal R—BA 45 56, 24, 12 3. Premotor L—BA 6 −46, −2, 54 4. Premotor R—BA 6 30, −16, 68 3. Premotor R—BA 6 52, 0, 48 3. Postcentral R—BA 3 44,−20, 52 3. Mid. temporal L—BA 21 −54, 0, − 20 4. Mid. temporal L—BA 21 −60, −10, − 2 4. Mid. temporal L—BA 21 −50, −54, 6 4. Sup. temporal L (auditory cortex)—BA 41/42 −62, −26, 6 5. Mid. temporal L—BA 22 −54, −38, 4 4. Mid. temporal R—BA 21 48, −28, − 10 5. Mid. temporal R—BA 21 46, −64, 8 3. Mid. temporal R—BA 22 66, −18, 2 5. Sup. temporal R—BA 21/22 46, −20, − 6 5. Precuneus L—BA 7 −6, −52, 44 3. Precuneus L—BA 7 −6, −50, 68 3. Amygdala R 24, −8, − 12 3. Thalamus L (medial geniculate body) −4, −34, − 4 3. Thalamus R (medial geniculate body) 14, −30, − 2 4.
Negative effects of intensity Inf. parietal lobule R—BA 40 58, −40, 46 4.
P-FDR b 0.05, cluster N 25 voxels.
intensity has also been found to be correlated with increased processing costs (e.g. Kissler et al., 2007) as measured by the increased deflection of the potentials. This raises the question of causality. Do the primary linguistic responses cause the amygdala responses or vice versa? The fact that high frequency emotional words may be accompanied by increased processing already in the pre- lexical time domain (Scott et al., 2009) points to the fact that fast mappings may exist that surpass cortical processing, such as those observed in conditioning, and that these may account for some of the effect, i.e. suggesting that the causal arrow goes from amygdala to language cortex. According to this line of thinking the emotional response may then be followed by enhanced attention (Dolan, 2002; Lewis, 2005) which then subsequently causes enhanced linguistic processing. This interpretation is consistent with two of the studies on narrative emotions reviewed in the introduction (Decety and Chaminade, 2003; Jabbi et al., 2008) that both find increased activity in left inferior frontal gyrus as a function of emotional content. This interpretation is also consistent with findings showing that amygdala also modulates condition-specific activity in other cognitive modal- ities, such as visual cortex, e.g. during processing of emotional faces (Rotshtein et al., 2001). One could, on the other hand, argue that it might rather be something in the linguistic structure of the narrative that imposes an increased cost which is then followed by an emotional response, i.e.
increased linguistic processing cost may in itself, regardless of content, also be enhancing arousal, suggesting that the causal arrow could go from language cortex to amygdala. Increased syntactic complexity, for example, has been found to affect pupil size, indexing sympathetic arousal (Just and Carpenter, 1993; Schluroff et al., 1986). Another avenue for investigating effects of structural complexity on arousal is music. Processing of increased rhythmic complexity, such as poly- rhythms, has been found to involve language cortex and is followed by the experience of increased tension, both during comprehension and production (Vuust et al., 2006; Vuust et al., 2011). The present study was not designed to determine whether amygdala causes language cortex to activate or the other way around, but we suggest that the two options may not be mutually exclusive. The fact that we find motor regions activated for the emotional scenes is likely to reflect that some of these scenes contain descriptions of physical action and danger (e.g. see examples above and in Supplementary Table S1). A number of studies have documented the involvement of premotor cortices in the processing of language related to action (Boulenger et al., 2006; Hauk et al., 2004; Jeannerod, 2008; Kemmerer et al., 2008; Pulvermüller et al., 2005; Tettamanti et al., 2005), also during processing of narratives (Speer et al., 2009). In the data analysis we factored out signal arising from action verbs (Wallentin et al., 2005; Wallentin et al., in press). The observed signal, therefore, is unlikely to arise due to a confound between intensity and story
amygdala
A
B
C
D
t (23) 8
6
4
2
0
thalamus
Fig. 4. A) Positive whole brain effects (P-FDR b 0.05) of emotional intensity rendered on a standard brain; B) same effects displayed on a glass brain; C) sagittal slice (x: 24) displaying right amygdala activation; D) axial slice (z: −2) displaying bilateral medial thalamus activation.
A B^ t^ (23)
Fig. 5. Positive effects of the bivariate valence regressor (P-FDR b 0.05); A) coronal slice (y: −54) displaying precuneus and bilateral temporoparietal junction activation; B) sagittal slice (x = −12) displaying medial prefrontal and precuneus activations. This pattern is similar to the “default” network.
References
Appelhans, B.M., Luecken, L.J., 2006. Heart rate variability as an index of regulated emotional responding. Rev. Gen. Psychol. 10, 229–240. Ashburner, J., Friston, K.J., 2005. Unified segmentation. NeuroImage 26, 839–851. Badgaiyan, R.D., Fischman, A.J., Alpert, N.M., 2009. Dopamine release during human emotional processing. NeuroImage 47, 2041–2045. Bechara, A., Damasio, H., Damasio, A.R., 2000. Emotion, decision making and the orbitofrontal cortex. Cerebral Cortex 10, 295–307. Berntson, G.G., Bigger, J.T., Eckberg, D.L., Grossman, P., Kaufmann, P.G., Malik, M., Nagaraja, H.N., Porges, S.W., Saul, J.P., Stone, P.H., van der Molen, M.W., 1997. Heart rate variability: origins, methods, and interpretive caveats. Psychophysiology 34, 623 – 648. Boulenger, V., Roy, A.C., Paulignan, Y., Deprez, V., Jeannerod, M., Nazir, T.A., 2006. Cross- talk between language processes and overt motor behavior in the first 200 msec of processing. J. Cogn. Neurosci. 18, 1607–1615. Brennan, J., Nir, Y., Hasson, U., Malach, R., Heeger, D.J., Pylkkänen, L., in press. Syntactic structure building in the anterior temporal lobe during natural story listening. Brain and Lang. doi:10.1016/j.bandl.2010.04.002. Chomsky, N., 1959. Reviews: verbal behavior. By B.F. Skinner. Language 35, 26–58. Christensen, K.R., Wallentin, M., 2011. The locative alternation: distinguishing linguistic processing cost from error signals in Broca's region. Neuroimage 56, 1622–1631. Crinion, J.T., Lambon-Ralph, M.A., Warburton, E.A., Howard, D., Wise, R.J.S., 2003. Temporal lobe regions engaged during normal speech comprehension. Brain 126, 1193 – 1201. Critchley, H.D., Mathias, C.J., Josephs, O., O'Doherty, J., Zanini, S., Dewar, B.-K., Cipolotti, L., Shallice, T., Dolan, R.J., 2003. Human cingulate cortex and autonomic control: converging neuroimaging and clinical evidence. Brain 126, 2139–2152. Dalgleish, T., 2004. The emotional brain. Nat. Rev. Neurosci. 5, 583–589. De Vega, M., Diaz, J.M., León, I., 1997. To know or not to know: comprehending protagonists' beliefs and their emotional consequences. Discourse Processes 23, 169 – 192. Decety, J., Chaminade, T., 2003. Neural correlates of feeling sympathy. Neuropsycho- logia 41, 127–138. Deen, B., McCarthy, G., 2010. Reading about the actions of others: biological motion imagery and action congruency influence brain activity. Neuropsychologia 48, 1607 – 1615. Dien, J., O'Hare, A.J., 2008. Evidence for automatic sentence priming in the fusiform semantic area: convergent ERP and fMRI findings. Brain Res. 1243, 134–145. Dolan, R.J., 2002. Emotion, cognition, and behavior. Science 298, 1191–1194. Ferstl, E.C., Rinck, M., von Cramon, D.Y., 2005. Emotional and temporal aspects of situation model processing during text comprehension: an event-related fMRI study. J. Cogn. Neurosci. 17, 724–739. Fiebach, C.J., Schlesewsky, M., Lohmann, G., Von Cramon, D.Y., Friederici, A.D., 2005. Revisiting the role of Broca's area in sentence processing: syntactic integration versus syntactic working memory. Hum. Brain Mapp. 24, 79–91. Friston, K.J., Williams, S., Howard, R., Frackowiak, R.S., Turner, R., 1996. Movement- related effects in fMRI time-series. Magn. Reson. Med. 35, 346–355. Gallagher, H.L., Happé, F., Brunswick, N., Fletcher, P.C., Frith, U., Frith, C.D., 2000. Reading the mind in cartoons and stories: an fMRI study of ‘theory of mind’ in verbal and nonverbal tasks. Neuropsychologia 38, 11–21. Glover, G.H., Li, T.Q., Ress, D., 2000. Image-based method for retrospective correction of physiological motion effects in fMRI: RETROICOR. Magn. Reson. Med. 44, 162–167. Graesser, A.C., Singer, M., Trabasso, T., 1994. Constructing inferences during narrative text comprehension. Psychol. Rev. 101, 371–395. Gygax, P., Oakhill, J., Garnham, A., 2003. The representation of characters' emotional responses: do readers infer specific emotions? Cogn. Emotion 17, 413–428. Hagoort, P., Hald, L., Bastiaansen, M., Petersson, K., 2004. Integration of word meaning and world knowledge in language comprehension. Science 304, 438–441. Hamann, S., Mao, H., 2002. Positive and negative emotional verbal stimuli elicit activity in the left amygdala. Neuroreport 13, 15–19. Happé, F.G., 1994. An advanced test of theory of mind: understanding of story characters' thoughts and feelings by able autistic, mentally handicapped, and normal children and adults. J. Autism Dev. Disord. 24, 129–154. Hargreaves, D., 1984. The effects of repetition on liking for music. J. Res. Music. Educ. 32, 35 – 47. Hasson, U., Nir, Y., Levy, I., Fuhrmann, G., Malach, R., 2004. Intersubject synchronization of cortical activity during natural vision. Science 303, 1634–1640. Hauk, O., Johnsrude, I., Pulvermuller, F., 2004. Somatotopic representation of action words in human motor and premotor cortex. Neuron 41, 301–307. Herbert, C., Junghofer, M., Kissler, J., 2008. Event related potentials to emotional adjectives during reading. Psychophysiology 45, 487–498. Herbert, C., Ethofer, T., Anders, S., Junghofer, M., Wildgruber, D., Grodd, W., Kissler, J.,
- Amygdala activation during reading of emotional adjectives—an advantage for pleasant content. Soc. Cogn. Affect. Neurosci. 4, 35–49. Hogan, P.C., 2003. The Mind and Its Stories—Narrative Universals and Human Emotion. Cambridge University Press, Cambridge. Isenberg, N., Silbersweig, D., Engelien, A., Emmerich, S., Malavade, K., Beattie, B., Leon, A.C., Stern, E., 1999. Linguistic threat activates the human amygdala. Proc. Natl. Acad. Sci. U.S.A 96, 10456–10459. Jabbi, M., Bastiaansen, J., Keysers, C., 2008. A common anterior insula representation of disgust observation, experience and imagination shows divergent functional connectivity pathways. PLoS ONE 3, e2939. Jeannerod, M., 2008. Language, perception and action. How words are grounded in the brain. Eur. Rev. 16, 389–398.
Just, M.A., Carpenter, P.A., 1993. The intensity dimension of thought: pupillometric indices of sentence processing. Can. J. Exp. Psychol. 47, 310–339. Kemmerer, D., Castillo, J.G., Talavage, T., Patterson, S., Wiley, C., 2008. Neuroanatomical distribution of five semantic components of verbs: evidence from fMRI. Brain Lang. 107, 16–43. Kissler, J., Herbert, C., Peyk, P., Junghofer, M., 2007. Buzzwords: early cortical responses to emotional words during reading: research report. Psychol. Sci. 18, 475–480. Kissler, J., Herbert, C., Winkler, I., Junghofer, M., 2009. Emotion and attention in visual word processing—an ERP study. Biol. Psychol. 80, 75–83. Kringelbach, M.L., 2005. The human orbitofrontal cortex: linking reward to hedonic experience. Nat. Rev. Neurosci. 6, 691–702. Kringelbach, M.L., Rolls, E.T., 2004. The functional neuroanatomy of the human orbitofrontal cortex: evidence from neuroimaging and neuropsychology. Prog. Neurobiol. 72, 341–372. Kringelbach, M.L., Vuust, P., Geake, J., 2008. Sci. Rev. 33, 321–335. Kutas, M., Hillyard, S.A., 1984. Brain potentials during reading reflect word expectancy and semantic association. Nature 307, 161–163. Lane, R.D., McRae, K., Reiman, E.M., Chen, K., Ahern, G.L., Thayer, J.F., 2009. Neural correlates of heart rate variability during emotion. NeuroImage 44, 213–222. LeDoux, J.E., 1995. Emotion: clues from the brain. Annu. Rev. Psychol. 46, 209–235. LeDoux, J.E., 2000. Emotion circuits in the brain. Annu. Rev. Neurosci. 23, 155–184. LeDoux, J.E., 2007. The amygdala. Curr. Biol. 17, R868–R874. Lewis, M.D., 2005. Bridging emotion theory and neurobiology through dynamic systems modeling. Behav. Brain Sci. 28, 169–194 discussion 194–245. Lund, T.E., Nørgaard, M.D., Rostrup, E., Rowe, J.B., Paulson, O.B., 2005. Motion or activity: their role in intra- and inter-subject variation in fMRI. NeuroImage 26, 960–964. Lund, T.E., Madsen, K.H., Sidaros, K., Luo, W.-L., Nichols, T.E., 2006. Non-white noise in fMRI: does modelling have an impact? NeuroImage 29, 54–66. MacLEAN, P.D., 1949. Psychosomatic disease and the visceral brain; recent developments bearing on the Papez theory of emotion. Psychosom. Med. 11, 338–353. Maldjian, J.A., Laurienti, P.J., Kraft, R.A., Burdette, J.H., 2003. An automated method for neuroanatomic and cytoarchitectonic atlas-based interrogation of fMRI data sets. NeuroImage 19, 1233–1239. Maldjian, J.A., Laurienti, P.J., Burdette, J.H., 2004. Precentral gyrus discrepancy in electronic versions of the Talairach atlas. NeuroImage 21, 450–455. Malik, M., Bigger, J.T., Camm, A.J., Kleiger, R.E., Malliani, A., Moss, A.J., Schwartz, P.J., 1996. Heart rate variability. Standards of measurement, physiological interpretation, and clinical use. Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology. Eurean Heart J. 17, 354–381. Naccache, L., Gaillard, R., Adam, C., Hasboun, D., Clémenceau, S., Baulac, M., Dehaene, S., Cohen, L., 2005. A direct intracranial record of emotions evoked by subliminal words. Proc. Natl. Acad. Sci. U.S.A 102, 7713–7717. Napadow, V., Dhond, R., Conti, G., Makris, N., Brown, E.N., Barbieri, R., 2008. Brain correlates of autonomic modulation: combining heart rate variability with fMRI. NeuroImage 42, 169–177. Obleser, J., Kotz, S.A., 2010. Expectancy constraints in degraded speech modulate the language comprehension network. Cereb Cortex 20, 633. Oldfield, R.C., 1971. The assessment and analysis of handedness: the Edinburgh inventory. Neuropsychologia 9, 97–113. Paton, J.J., Belova, M.A., Morrison, S.E., Salzman, C.D., 2006. The primate amygdala represents the positive and negative value of visual stimuli during learning. Nature 439, 865–870. Pavlov, I.P., 1927. Conditioned Reflexes: An Investigation of the Physiological Activity of the Cerebral Cortex. Oxford University Press, London. Penny, W., Holmes, A.P., 2007. Random effects analysis. In: Friston, K.J., Ashburner, J., Kiebel, S., Nichols, T., Penny, W. (Eds.), Statistical Paramtric Mapping: The Analysis of Functional Brain Images. Academic Press, London, pp. 156–165. Pulvermüller, F., Shtyrov, Y., Ilmoniemi, R., 2005. Brain signatures of meaning access in action word recognition. J. Cogn. Neurosci. 17, 884–892. Raichle, M.E., MacLeod, A.M., Snyder, A.Z., Powers, W.J., Gusnard, D.A., Shulman, G.L.,
- A default mode of brain function. Proc. Natl. Acad. Sci. U.S.A. 98, 676–682. Rolls, E.T., 2000. Précis of the brain and emotion. Behav. Brain Sci. 23, 177– 191 discussion 192–233. Rolls, E.T., 2005. Emotion Explained. Oxford University Press, Oxford. Rolls, E.T., 2008. Emotion, higher-order syntactic thoughts, and consciousness. In: Weiskrantz, L., Davies, M. (Eds.), Frontiers of Consciousness. Oxford University Press, Oxford. Rotshtein, P., Malach, R., Hadar, U., Graif, M., Hendler, T., 2001. Feeling or features: different sensitivity to emotion in high-order visual cortex and amygdala. Neuron 32, 747–757. Russell, J.A., 1979. Affective space is bipolar. J. Personal. Soc. Psychol. 37, 345–356. Sabatinelli, D., Lang, P.J., Bradley, M.M., Flaisch, T., 2006. The neural basis of narrative imagery: emotion and action. Prog. Brain Res. 156, 93–103. Sander, D., Grafman, J., Zalla, T., 2003. The human amygdala: an evolved system for relevance detection. Rev. Neurosci. 14, 303–316. Saxe, R., Kanwisher, N., 2003. People thinking about thinking people. The role of the temporo-parietal junction in “theory of mind”. NeuroImage 19, 1835–1842. Schacht, A., Sommer, W., 2009. Time course and task dependence of emotion effects in word processing. Cogn. Affect. Behav. Neurosci. 9, 28–43. Schluroff, M., Zimmermann, T.E., Freeman, R.B., Hofmeister, K., Lorscheid, T., Weber, A.,
- Pupillary responses to syntactic ambiguity of sentences. Brain Lang. 27, 322–344. Scott, G.G., O'Donnell, P.J., Leuthold, H., Sereno, S.C., 2009. Early emotion word processing: evidence from event-related potentials. Biol. Psychol. 80, 95–104. Sergerie, K., Chochol, C., Armony, J.L., 2008. The role of the amygdala in emotional processing: a quantitative meta-analysis of functional neuroimaging studies. Neurosci. Biobehav. Rev. 32, 811–830.
Shulman, G.L., Fiez, J.A., Corbetta, M., Buckner, R.L., Miezin, F.M., Raichle, M.E., Petersen, Steven E., 1997. Common blood flow changes across visual tasks: II. Decreases in cerebral cortex. J. Cogn. Neurosci. 9, 648–663. Skinner, B.F., 1957. Verbal Behavior. Copley Publishing Group, Acton, MA. Speer, N.K., Zacks, J.M., Reynolds, J.R., 2007. Human brain activity time-locked to narrative event boundaries. Psychol. Sci. 18, 449–455. Speer, N.K., Reynolds, J.R., Swallow, K.M., Zacks, J., 2009. Reading stories activates neural representations of visual and motor experiences. Psychol. Sci.: J. Am. Psychol. Soc./ APS 20, 989–999. Talmy, L., 2000. Toward a Cognitive Semantics Cambridge. MIT Press, Mass. Tettamanti, M., Buccino, G., Saccuman, M.C., Gallese, V., Danna, M., Scifo, P., Fazio, F., Rizzolatti, G., Cappa, S.F., Perani, D., 2005. Listening to action-related sentences activates fronto-parietal motor circuits. J. Cogn. Neurosci. 17, 273–281. Tzourio-Mazoyer, N., Landeau, B., Papathanassiou, D., Crivello, F., Etard, O., Delcroix, N., Mazoyer, B., Joliot, M., 2002. Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain. NeuroImage 15, 273–289. Vogeley, K., Bussfeld, P., Newen, A., Herrmann, S., Happé, F., Falkai, P., Maier, W., Shah, N.J., Fink, G.R., Zilles, K., 2001. Mind reading: neural mechanisms of theory of mind and self-perspective. NeuroImage 14, 170–181. Vuust, P., Roepstorff, A., Wallentin, M., Mouridsen, K., Østergaard, L., 2006. It don't mean a thing … keeping the rhythm during polyrhythmic tension, activates language areas (BA47). NeuroImage 31, 832–841. Vuust, P., Wallentin, M., Mouridsen, K., Ostergaard, L., Roepstorff, A., 2011. Tapping polyrhythms in music activates language areas. Neurosci. Lett. 494, 211–216. Wallentin, M., 2009. Putative sex differences in verbal abilities and language cortex: a critical review. Brain Lang. 108, 175–183.
Wallentin, M., Lund, T., Østergaard, S., Østergaard, L., Roepstorff, A., 2005. Motion verb sentences activate left posterior middle temporal cortex despite static context. Neuroreport 16, 649. Wallentin, M., Roepstorff, A., Glover, R., Burgess, N., 2006. Parallel memory systems for talking about location and age in precuneus, caudate and Broca's region. NeuroImage 32, 1850–1864. Wallentin, M., Vuust, P., Mouridsen, K., Dohn, A., Nielsen, A.H., Roepstorff, A., Lund, T.E., 2009a. Word co-occurrence effects driving language cortex during listening to a narrative. NeuroImage 47, S165-165. Wallentin, M., Vuust, P., Mouridsen, K., Roepstorff, A., Lund, T.E., 2009b. Covariance structures in narratives studied with fMRI—proof of concept. CNS 2009—The 16th Annual Meeting of the Cognitive Neuroscience Society, San Francisco. Wallentin, M., Nielsen, A.H., Vuust, P., Dohn, A., Roepstorff, A., Lund, T.E., in press. BOLD response to motion verbs in left posterior middle temporal gyrus during story comprehension. Brain and Lang., doi:10.1016/j.bandl.2011.04.006. Wilson, S.M., Molnar-Szakacs, I., Iacoboni, M., 2007. Beyond superior temporal cortex: intersubject correlations in narrative speech comprehension. Cereb. Cortex 18, 230 – 242. Worsley, K.J., Friston, K.J., 1995. Analysis of fMRI time-series revisited—again. Neuro- Image 2, 173–181. Xu, J., Kemeny, S., Park, G., Frattali, C., Braun, A., 2005. Language in context: emergent features of word, sentence, and narrative comprehension. NeuroImage 25, 1002 – 1015. Yarkoni, T., Speer, N.K., Zacks, J.M., 2008. Neural substrates of narrative comprehension and memory. NeuroImage 41, 1408–1425. Zacks, J.M., Speer, N.K., Reynolds, J.R., 2009. Segmentation in reading and film comprehension. J Exp. Psychol. Gen. 138, 307–327.
