Abstract
Responding to multiple stimuli of different modalities has been shown to reduce reaction time (RT), yet many different processes can potentially contribute to multisensory response enhancement. To investigate the neural circuits involved in voluntary response initiation, an acoustic stimulus of varying intensities (80, 105, or 120 dB) was presented during a visual RT task to a patient with profound bilateral cortical deafness and an intact auditory brainstem response. Despite being unable to consciously perceive sound, RT was reliably shortened (~100 ms) on trials where the unperceived acoustic stimulus was presented, confirming the presence of multisensory response enhancement. Although the exact locus of this enhancement is unclear, these results cannot be attributed to involvement of the auditory cortex. Thus, these data provide new and compelling evidence that activation from subcortical auditory processing circuits can contribute to other cortical or subcortical areas responsible for the initiation of a response, without the need for conscious perception.
Similar content being viewed by others
Introduction
Responding to a stimulus with a voluntary action is typically thought to involve several processes that are mediated by cortical structures. For example, a sprinter will “get set” to launch their body out of the starting blocks, and once the starting pistol fires the sprinter must recognize that the signal has occurred and initiate that planned response. The processes involved in planning a response (preparation) and holding it ready until the go-signal occurs (storage) have been ascribed principally to cortical motor and premotor areas1,2,3. Similarly, processes involved in stimulus recognition (detection/discrimination) and response initiation are thought to occur in cortical sensory and association areas as well as frontal cortical regions4.
However, data from a paradigm involving the presentation of an unpredictable high-intensity auditory stimulus that is capable of eliciting a startle reflex has suggested that lower subcortical structures may play a role in response initiation processes5. Typical mean reaction time (RT) to the onset of an auditory stimulus is approximately 140–150 ms, with reaction to a visual stimulus being slightly longer at 180–200 ms6. Yet, when a loud startling acoustic stimulus (SAS) is presented, RT is dramatically shortened to approximately 70–80 ms on average, while the characteristic features of the response remain largely unchanged7,8. The startle reflex is primarily mediated by brainstem nuclei9 and elicits a pattern of generalized flexion in various muscles with latencies of 50–100 ms. Given the short latency of both the startle reflex and prepared response, early explanations of RT speeding by startle were that the “voluntary” reaction was too short to have involved the normal cortically mediated stimulus-response processing pathways and was somehow driven at the speed of the startle response. In order for this to occur, it was argued that storage and/or initiation of the response must have involved subcortical structures common to both the startle and voluntary response channels7. This notion of subcortical involvement has also been tested in patients with hereditary spastic paraplegia, whose corticospinal tracts have been severely damaged resulting in substantial RT delays. Presenting a SAS led to RT shortening in these patients similar to that seen in healthy controls, suggesting that the startle-elicited response output somehow bypassed the damaged corticospinal pathways10.
While the presentation of a startling stimulus is thought to decrease response latencies through the involvement of subcortical pathways, the presentation of a non-startling auditory stimulus in conjunction with a visual stimulus can also result in shorter RTs. This so-called “redundant signal effect” occurs when stimuli of different modalities are presented, and a variety of models have been proposed to explain the RT reduction including statistical facilitation, multisensory integration, and crossmodal attention; see reviews11,12,13. Each viewpoint outlines different mechanisms responsible for shortened response latency, which also depend on the relative stimulus locations and timing of presentation14,15. The integration of auditory and visual information is thought to involve a variety of cortical and subcortical structures including unisensory cortices and superior temporal cortex16,17, as well as reticular formation and superior colliculus18,19. However, the relative contributions of these structures during multisensory integration is still a matter of debate20, as is the degree of cortical versus subcortical involvement in response initiation processes.
In order to investigate the involvement of subcortical circuits in response initiation, we carried out a simple RT experiment in a patient with bilateral cortical deafness, in which an auditory stimulus of varying intensities was presented in conjunction with a visual go-signal. Complete bilateral cortical deafness is an exceedingly rare condition, and the patient reports no conscious perception of sound, a result of cortical auditory damage following a bilateral stroke21 (Fig. 1a). However, he exhibits a normal auditory brainstem response (ABR, see Fig. 1b), indicating that secondary pathways are intact. In our experiment, the patient was required to make a wrist extension movement as quickly as possible in response to the onset of a visual go-signal. On some trials the visual stimulus was accompanied by an acoustic stimulus of variable intensity: Quiet (80 dB), loud (105 dB), very loud (120 dB). The stimulus was presented either concurrent with, or 200 ms prior to the visual signal. Our primary research question was whether the presence of an unperceived auditory stimulus would result in RT facilitation. If the patient exhibited a startle reflex and accompanying reduction in RT when presented with the very loud (120 dB) acoustic stimulus, this result would be attributed to involvement of startle reflex circuitry in the accelerated initiation of a prepared response, due to the intact subcortical auditory circuits. Alternatively, if no startle reflex was observed, yet a reduction in RT was nevertheless found when presented with an acoustic stimulus (irrespective of intensity), this result would be indicative of multisensory response enhancement occurring in the absence of awareness of auditory input.
Results
Reaction time
Even though the patient reported a complete lack of awareness of any acoustic stimulus, results show that regardless of stimulus intensity, an unperceived acoustic stimulus presented concurrent with the visual go-signal led to a reliable and substantial (~100 ms) reduction in RT (Fig. 2). Although single-participant designs are often descriptive in nature, the present results were statistically verified using non-parametric randomization tests (see Methods). First, randomization tests compared RTs obtained when the three intensities of acoustic stimuli (80, 105, and 120 dB) were presented simultaneous with the visual go-signal. These tests showed no statistically reliable difference between stimulus intensities (all p-values > 0.442). Similar results were obtained for trials where the acoustic stimulus was presented 200 ms prior to the visual go-signal (all p-values > 0.701). This is exemplified in Fig. 2 where individual trial premotor RT for all conditions is plotted alongside grouped descriptive statistics. Thus, for randomization tests comparing acoustic stimulus trial RTs to visual-only trials, RTs were collapsed for the acoustic stimulus conditions presented simultaneous with the visual go-signal, as well as for the acoustic stimulus presented 200 ms prior to the visual go-signal. This analysis confirmed that when an auditory stimulus was presented simultaneous with the visual stimulus, mean RT was reliably shorter (p = 0.012) even though no differences existed between intensities. A Bayesian t-test also corroborated this finding, providing moderate evidence that the data were ~4.6 times more likely under the alternative hypothesis that the stimulus led to shorter RTs (BF+0 = 4.567). When the acoustic stimulus was presented 200 ms prior to the visual go, RT was also shortened by ~100 ms on average; however, this result did not reach statistical significance (p = 0.077), and would be considered inconclusive with respect to the null hypothesis using Bayesian inference (BF+0 = 1.312).
Percentage of non-overlapping data (PND) is presented in Fig. 3a as a surrogate for effect size22 for the visual stimulus only (unfilled circles) as compared when the auditory stimulus was presented simultaneous with the visual stimulus (filled circles). When all obtained RT values in these conditions are considered, only 4 of 32 (12.5%) values are non-overlapping (see values outside of horizontal bars, Fig. 3a). However, one limitation of PND is that it may erroneously represent effects when outliers are present in the data23. When the top and bottom values of each distribution are trimmed, 9 of 30 (30.0%) of values are non-overlapping (see values shown outside shaded area, Fig. 3a), which may be considered as a small effect24.
Trial-by-trial RT is presented in Fig. 3b to accurately depict RT trends as they evolved over time for the visual stimulus only condition (unfilled circles) as compared to all of the auditory stimulus conditions (different filled symbols), and as a function of whether the auditory stimulus occurred simultaneous with the visual stimulus (filled circles), or 200 ms prior to it (filled squares). While the mean RTs (horizontal lines) clearly indicate that reactions were faster when an acoustic stimulus was presented (dashed lines) as compared to when the visual stimulus was presented alone (solid line), this is also supported by the individual trial data.
Startle reflex indicators
Although our primary finding is that RT facilitation due to an acoustic stimulus can occur without conscious perception of the stimulus, a secondary result was that a startle reflex was not observed following presentation of the auditory stimuli, irrespective of intensity. Specifically, no activity was observed in the muscles typically associated with a startle reflex in the 200 ms following the onset of the 120 dB auditory stimulus25, including orbicularis oculi (OOc), sternocleidomastoid (SCM), and C4 paraspinal (C4P) (Fig. 4). Although we expected to observe a startle and/or blink reflex due to the intact auditory brainstem response, some previous studies have reported the startle reflex was also abolished in patients who suffered lesions to the temporal lobes and auditory cortices26,27. The absence of startle reflex activity in this patient, even following first presentation of the loudest stimulus, may be directly related to the loss of input from auditory cortical areas. It is possible that higher structures may act in a top-down manner to supress the startle reflex even though the brainstem circuits are intact. As such, the lack of startle reflex observed here seems to represent top-down chronic sensory gating of these brainstem mediated responses and may represent a cortically-mediated neuroplastic adaptation to prevent unwanted reactions to unperceived stimuli.
Discussion
The primary finding from the present experiment is that RT was significantly (~100 ms) shorter in the presence of a concurrent auditory stimulus (Fig. 2), despite the participant being unable to consciously process sound21. This novel result indicates that secondary auditory pathways can contribute to the processes involved in response initiation, even when the auditory stimulus is not perceived. These pathways appear to be activated at all auditory intensities presented in the current study (80, 105, & 120 dB) and the data provide compelling evidence that activation solely from subcortical auditory processing circuits can contribute to other cortical or subcortical areas responsible for the initiation of a response. The reduction in RT associated with the presentation of both an auditory and visual stimulus has previously been ascribed to many different mechanisms, including statistical facilitation, multisensory integration, and crossmodal attention11,12,13. Although our results cannot definitively support one particular explanation, they strongly implicate a subcortically mediated pathway that does not involve primary auditory cortex in the production of the faster responses, and indicate that multisensory response enhancement can occur without overt perception of the stimulus.
The initiation of a response requires a number of processes including preparation of the motor commands, identification of the go-signal, and an increase in activation in the initiation-related circuits in order to release the prepared motor commands. Although concurrent presentation of a tone and visual target has been shown to enhance response preparation in a choice RT task28, the current study employed a simple RT paradigm, in which preparation of the response would be expected to occur in advance of the go-signal, and thus faster response preparation is an unlikely candidate for the observed RT facilitation. In terms of identification of the go-signal, perceptual processing of multiple stimuli has been shown to be affected by both multisensory integration and crossmodal spatial attention, although the relative contribution of these effects depends on the relative timing between stimuli13. Crossmodal attention is considered largely a bottom-up/automatic process29,30 and thus could occur even without conscious perception of the auditory stimulus. However, the benefits of crossmodal attention shifts are typically maximized when the auditory stimulus precedes the visual stimulus by 50–300 ms31,32, yet in the current study the RT facilitation only reached statistical significance when the two stimuli were presented concurrently. Instead, concurrent stimuli are thought to promote multisensory integration, whereby sensory information is processed more efficiently33,34, which may have contributed to the faster RT observed when the auditory stimulus was presented.
While the observed response latency may have been affected by a faster identification of the go-signal, the data indicate it is unlikely that RT facilitation is solely due to these perceptual/sensory processes. It is well documented that RT is shorter with increasing stimulus intensity, an effect attributed to faster sensory processing35, yet in the present experiment there was no differential effect of stimulus intensity (80 dB, 105 dB, or 120 dB) on the RT effect observed (Fig. 2). Once the go-signal is identified, neural activation in initiation-related circuitry still needs to be increased to above an “ignition point” or threshold that results in the release of the prepared motor commands4,36. This initiation process has shown to be sped up by multiple stimuli, due to additive contributions associated with each individual stimulus37,38. In this co-activation model, each stimulus provides additive activation to the thalamus, which receives input from both visual and auditory stimuli39 and has been previously implicated in response initiation40. Thus, in co-activation, as well as in drift diffusion models41, redundant signals provide a faster rate of activation accumulation, resulting in a shorter time required to reach initiation threshold. Interestingly, in a previous investigation of this additive initiation model, when an 80 dB auditory cue was presented concurrently with a visual go-signal, the RT facilitation was 93 ms37, similar in magnitude to that seen in the current experiment. Furthermore, this multisensory RT reduction could not be explained by a statistical facilitation model, whereby the faster reaction is due to a race between processing of the independent stimuli42,43. In the current study, we also believe statistical facilitation to be an unlikely explanation, as the racehorse model requires a response to each stimulus, which independently race to reach a response threshold. Although it is conceivable that an unperceived stimulus could cause a response via subcortical circuitry, the lack of response to the auditory stimulus when presented 200 ms prior to the visual go-signal suggests this did not occur in the present experiment. On these trials, the patient did not respond to the auditory stimulus, as this would likely have resulted in a RT reduction of ~200 ms compared to the condition when the auditory cue was presented simultaneously with the visual stimulus; yet RTs for both auditory conditions were similar to each other (and both ~100 ms faster than visual stimulus-only RTs). Similarly, it does not appear that involuntary response triggering by the startling stimulus is a plausible explanation for the results, the patient did not exhibit a startle reflex (Fig. 4) and showed a similar RT reduction for all stimulus intensities.
Thus, the current results may be attributable to a variety of processes involved in multisensory response enhancement, including both faster identification of the go-signal and less time to reach the threshold for the initiation of the motor commands. While the exact locus of the RT facilitation effect is unknown, the lack of awareness of the auditory stimulus by the patient clearly implicates unconscious stimulus processing in subcortical and secondary auditory pathways, which contribute activation to voluntary response initiation circuits and thus shorter RTs. When considering other models of hierarchical multisensory processing, it is typically posited that information from different sensory modalities is transmitted to their respective primary sensory cortices via separate channels before being combined in various sensory integration centres44,45,46. However, some studies have shown that multi-modality processing can occur in cortical regions thought to be unisensory in nature16,47. This multi-modality activation is thought to result from cortico-cortical activation, but in the present patient this is unlikely to be the case due to extensive damage to the auditory cortices. This is not to say that the primary auditory cortex does not contribute to multisensory integration in non-patient populations. Indeed, studies involving animal models have shown that cryogenic deactivation of the auditory cortex modulates multisensory integration in the superior colliculus48,49, and other findings have demonstrated multisensory contributions in low level cortical structures that were previously thought to be unisensory in nature; see review16. However, it does appear that many brain structures are involved in perceptual processing and activation from subcortical auditory processing circuits can contribute to other cortical or subcortical areas responsible for sensory integration and response initiation. We next turn our attention to the possible structures and pathways that may contribute to the observed decrease in response latency.
The subcortical structures involved in providing this additional activation may either be part of the primary auditory circuit or secondary auditory pathways (Fig. 5). The primary auditory pathway leads from the cochlear nuclei to the superior olivary nuclei, which are concerned with sound localization. From there, axons join others from the cochlear nuclei in an ascending pathway, where some axons synapse in the nuclei of the lateral lemniscus, at the level of the pons. All ascending fibres passing through the lateral lemniscus synapse at the inferior colliculus, at the level of the midbrain. Postsynaptic cells of the inferior colliculus project to the medial geniculate nucleus of the thalamus with neurons from the geniculate body terminating in primary auditory cortex50. Structures in the primary auditory pathway appear to primarily integrate auditory and somatosensory information51,52, whereas secondary auditory pathways (through reticular formation and superior colliculus), provide information that is integrated with vision18,19. Indeed, it has been demonstrated that activity in superior colliculus is increased when visual and auditory stimuli are combined, and that many of the neurons that exhibited changes in firing following multisensory input had descending projections to motor and premotor areas in brainstem and spinal cord18. Furthermore, increased activity in superior colliculus has been associated with shorter latency onset of saccades19, as well as covert visual processing in hemianopic patients53. Thus, the present data point to the possibility that an acoustic stimulus could act to increase premotor activation in subcortical structures including thalamus, brainstem, and spinal cord, thereby “priming” the motor output system for upcoming descending efferent drive. Alternatively, higher structures implicated in secondary auditory pathways also project directly to motor thalamus and primary visual cortex. Because thalamus has been strongly implicated in providing activation leading to the initiation of motor responses40, inputs from subcortical auditory circuits may directly impact response initiation processing. For the current patient, the intact auditory brainstem response suggests that input from the cochlea is transferred at least to the level of the inferior colliculus and/or medial geniculate nucleus54,55. Thus, lower level structures involved in secondary auditory circuits likely contributed to the faster RTs observed in this patient following the presentation of an unperceived acoustic stimulus, via increases in response-related activity in cortical and subcortical structures. These data show that secondary auditory pathways and centres associated with stimulus integration are a potential mechanism by which the response speeding occurred. Greater knowledge of the neural circuitry associated with sensory processing and response initiation may ultimately lead to a better understanding of how human movements are generated and significant advances in rehabilitation of patients with disorders involving sensory perception56, movement initiation, and movement execution.
Methods
Patient
One male participant (Age = 40 years) with complete bilateral cortical deafness participated in this case study. His condition has been thoroughly described previously21, but in short, as the result of a 2007 rupture of the right internal carotid-posterior communicating artery and a subarachnoidal hemorrhage, the patient suffered extensive infarction of both left and right primary auditory cortices and auditory radiations (Fig. 1a). While other localized damage occurred resulting in the loss of some motor function, this was recovered completely; however, complete loss of hearing and some somatosensory and vestibular function did not recover at all. Pure tone audiometry showed thresholds that were “off the scale” (i.e., no perception even at the highest tested intensity) and speech audiometry showed 0% speech discrimination21. The patient reports no sensation or perception of sound whatsoever; however, audiometry showed a normal ABR with thresholds of 20 dB in both ears (Fig. 1b). Prior to testing he completed a written assessment of hearing, reporting 0% sensation in all sound categories including perception of sound, noise, sound of breathing, heartbeat, sneezing, growling stomach, or vibration sounds. Otherwise, the patient has shown normal reading, writing, and intellectual abilities, with only a minor visual memory impairment. The patient provided written informed consent prior to beginning the experiment, and testing occurred at the National Tokyo Medical Centre: Otorhinolaryngology Outpatient Clinic, Tokyo, Japan. Following the experiment, the patient was debriefed and reported being unaware that auditory stimuli were presented during the testing protocol. The procedure was approved by members of the National Tokyo Medical Centre ethics review board and followed all of its guidelines and regulations.
Apparatus and task
During the experiment the patient sat in a chair in front of a desk with his right arm resting comfortably on the desktop approximately parallel to the body midline with the ventral forearm and palm down. A laptop computer (Dell XPS13) was placed approximately 1 m in front of the patient. The task for the patient was to perform a simple right wrist extension movement as quickly as possible upon appearance of a green square “go” signal on the laptop screen. Although the movement was untargeted, the patient was instructed to use only a wrist movement to perform the reaction time (RT) task and to not lift the entire arm. Each trial began with a visual warning signal “準備します” (“prepare”) displayed on the computer screen for 1 s along with a black outline (2 mm wide) of a 3.4 cm square displayed against a grey background. The warning text then disappeared which was followed by a 2–2.5 s variable foreperiod before the square turned green. The patient then reacted as quickly as possible with a right wrist extension. RT feedback was then provided for 3.5 s before the sequence started again.
Procedure
The patient performed 20 practice trials where only the visual stimulus was presented. This was followed by 4 blocks of 21 testing trials. Each block was made up of 9 control (visual stimulus only) trials, 9 trials where a 25 ms duration broadband acoustic stimulus (equal power from 1 Hz to 22 kHz) was presented concurrent with the onset of the visual stimulus (3 @ 80 dB, 3 @ 105 dB, and 3 @ 120 dB), and 3 trials where the acoustic stimulus preceded the visual stimulus by 200 ms (1 @ 80 dB, 1 @ 105 dB, and 1 @ 120 dB). Different stimulus intensities were chosen as they would be expected to result in a different proportion of trials in which a reflexive startle response would be observed. Previous research has shown that presentation of a 80 dB, 105 dB and 120 dB result in elicitation of a startle reflex on approximately 0%, 50% and >80% of trials, respectively57,58. Trials in which the acoustic stimulus preceded the visual stimulus were included such that if the response was triggered involuntarily, latencies would clearly show that response initiation was due to the acoustic rather than the visual stimulus. In addition, a presentation of the acoustic stimulus 200 ms prior to the visual stimulus was chosen because at this time point, the participant's preparation level would be expected to be at a high and consistent level59. Trial conditions were randomized within each block and the entire testing procedure comprised 84 RT trials. The acoustic stimulus was generated using a custom LabView program (National Instruments Inc.) and output as an analog signal, amplified (Lepai LP-2020TI), and presented via a loudspeaker (MG Electronics M58-H) placed 30 cm directly behind the head of the patient. Sound level intensity was verified using a precision sound level meter (Casella CEL-254, A-weighting, impulse setting).
Recording equipment
Surface electromyographic (EMG) data were collected from the muscle bellies of the right extensor carpi radialis longus (ECR; agonist), right flexor carpi radialis (FCR; antagonist), as well as traditional startle indicator muscles including the left orbicularis oculi (OOc), left sternocleidomastoid (SCM), and left C4 paraspinal (C4P) using bipolar preamplified (gain = 10) surface electrodes (DE-2.1, Delsys Inc.), attached to an external amplifier (Delsys Bagnoli-8). The electrode sites were prepared by lightly scrubbing and wiping with alcohol swabs to decrease electrical impedance. A reference electrode (4 ×4 cm, tens electrode, Lemonbest Inc.) was attached to the right lateral epicondyle. To capture displacement onset, a single axis piezoelectric accelerometer (Type 4375, Brüel & Kjær) was attached to the middle of the back of the patient's hand (proximal to the third metacarpophalangeal joint) using tape. The accelerometer was connected through an inline charge amplifier (Type 422, PCB Electronics) to a power supply and coupler (Model 5114, Kistler Inc.). Raw bandpassed (20–450 Hz) EMG and raw accelerometer data were digitally sampled at 1000 Hz for 3 s using a USB data acquisition unit (USB-6001, National Instruments Inc.). For each trial, data acquisition began 1 s prior to appearance of the visual go-signal, and was stored for later analysis.
Data analysis
Practice trials were not included in analysis. Five testing trials were removed from analysis because the patient reacted prior to the visual or acoustic stimulus, and were considered to be anticipation. Three trials were removed for slow responses (RT > 1 s). This resulted in an inclusion rate of 90.5%. EMG burst onsets were identified for all muscles on each trial using a thresholding algorithm that identified the point at which EMG activity began a sustained increase above baseline. The temporal location of his point was determined as the first point where the baseline corrected, rectified and filtered (25 Hz low-pass elliptic filter) EMG signal increased more than 2 standard deviations above the level of activity observed in baseline (mean of the first 500 ms of data collection and prior to any stimulus onset), and remained elevated for at least 25 ms. This point was visually confirmed and corrected if necessary60. Reaction time (RT) was defined as the time of EMG onset in the ECR muscle with respect to the visual go-signal. The occurrence of a startle reflex was defined as an onset of EMG occurring in either the OOc, SCM, or C4P within 200 ms of presentation of the auditory stimulus. A comparatively large startle reflex onset window25 was used because a delayed startle reflex has been previously observed in some patient populations61.
RT data were assessed using visual analysis techniques, and trial-by-trial descriptive statistics are presented in Fig. 3 including percentage of non-overlapping data, a measure of effect size for single case experiments, whereby a smaller amount of overlap (higher amount non-overlapping data) indicates a larger effect of the intervention or treatment24. RT data were then subjected to permutation (randomization) analyses, as this type of test is valid for single case experiments because there are no underlying distribution assumptions23. The randomization procedure was carried out using R software62 with the single case data analysis (SCDA) plugin63 using a 1000 iteration Monte Carlo randomization. In brief, the observed RT values are randomly reordered while the order of the independent variable (trial type/condition) is held constant. The difference between the obtained condition means based on the new randomization is then determined. This procedure is carried out N times (1000 in the present case), and the number of randomly re-ordered data sets where the mean difference exceeds the actual observed mean difference is determined. The p value of the randomization test is the proportion of randomizations that exceed the observed difference23. That is, if this number is less than 5%, it can be concluded that the observed difference is unlikely to have occurred due to chance63. Initial tests were carried out comparing the different intensity acoustic stimulus trials to test if RT differences existed between intensities (80, 105, 120 dB). This procedure was completed for trials where the acoustic stimulus was presented simultaneous with, and 200 ms prior to the visual go-signal. No differences were detected between intensities (see Results), therefore, the acoustic stimulus trial data were collapsed across intensities and randomization tests were carried out comparing trials where an acoustic stimulus was presented versus those with only the visual stimulus. This was done separately for trials where the acoustic stimulus was presented concurrent with the visual go, or presented 200 ms prior to it. In order to further examine any differences in RT seen between conditions, Bayesian statistical analysis was used to test the likelihood that these results fit the alternative hypothesis that presentation of an acoustic stimulus led to shorter RTs. As such, separate Bayesian t-tests were performed comparing using JASP software (version 0.9) to estimate Bayes factors for comparisons between trials with the visual stimulus alone and trials where the acoustic stimulus was presented concurrent with the visual go, or presented 200 ms prior to it. All acoustic stimulus intensities were grouped for each analysis.
Data availability
All data generated or analysed during this study are included in this published article.
References
Deecke, L., Scheid, P. & Kornhuber, H. H. Distribution of readiness potential, pre-motion positivity, and motor potential of the human cerebral cortex preceding voluntary finger movements. Exp. Brain Res. 7, 158–168 (1969).
Wickens, J., Hyland, B. & Anson, G. Cortical cell assemblies: a possible mechanism for motor programs. J. Motor Behav. 26, 66–82, https://doi.org/10.1080/00222895.1994.9941663 (1994).
Churchland, M. M., Yu, B. M., Ryu, S. I., Santhanam, G. & Shenoy, K. V. Neural variability in premotor cortex provides a signature of motor preparation. J. Neurosci. 26, 3697–3712, https://doi.org/10.1523/JNEUROSCI.3762-05.2006 (2006).
Hanes, D. P. & Schall, J. D. Neural control of voluntary movement initiation. Science 274, 427–430 (1996).
Carlsen, A. N. & Maslovat, D. Startle and the StartReact Effect: Physiological Mechanisms. J. Clin. Neurophysiol. 36, 452–459, https://doi.org/10.1097/WNP.0000000000000582 (2019).
Brebner, J. M. T. & Welford, A. T. In Reaction times (ed. Welford, A. T.) 1–23 (Academic Press, 1980).
Valls-Solé, J., Rothwell, J. C., Goulart, F. R. & Cossu, G. Patterned ballistic movements triggered by a startle in healthy humans. J. Physiol. 516, 931–938 (1999).
Carlsen, A. N., Chua, R., Inglis, J. T., Sanderson, D. J. & Franks, I. M. Prepared movements are elicited early by startle. J. Motor Behav. 36, 253–264 (2004).
Yeomans, J. S. & Frankland, P. W. The acoustic startle reflex: neurons and connections. Brain Res. Rev. 21, 301–314, https://doi.org/10.1016/0165-0173(96)00004-5 (1996).
Nonnekes, J. et al. StartReact restores reaction time in HSP: evidence for subcortical release of a motor program. J. Neurosci. 34, 275–281, https://doi.org/10.1523/JNEUROSCI.2948-13.2014 (2014).
Miller, J. Statistical facilitation and the redundant signals effect: What are race and coactivation models? Atten. Percept. Psychophys. 78, 516–519, https://doi.org/10.3758/s13414-015-1017-z (2016).
Stevenson, R. A. et al. Identifying and quantifying multisensory integration: a tutorial review. Brain Topogr. 27, 707–730, https://doi.org/10.1007/s10548-014-0365-7 (2014).
Van der Stoep, N., Spence, C., Nijboer, T. C. & Van der Stigchel, S. On the relative contributions of multisensory integration and crossmodal exogenous spatial attention to multisensory response enhancement. Acta Psychol. (Amst.) 162, 20–28, https://doi.org/10.1016/j.actpsy.2015.09.010 (2015).
Cecere, R., Romei, V., Bertini, C. & Ladavas, E. Crossmodal enhancement of visual orientation discrimination by looming sounds requires functional activation of primary visual areas: a case study. Neuropsychologia 56, 350–358, https://doi.org/10.1016/j.neuropsychologia.2014.02.008 (2014).
Spence, C., Lee, J. & Van der Stoep, N. Responding to sounds from unseen locations: crossmodal attentional orienting in response to sounds presented from the rear. Eur. J. Neurosci., https://doi.org/10.1111/ejn.13733 (2017).
Schroeder, C. E. & Foxe, J. Multisensory contributions to low-level, ‘unisensory’ processing. Curr. Opin. Neurobiol. 15, 454–458, https://doi.org/10.1016/j.conb.2005.06.008 (2005).
Calvert, G. A. & Thesen, T. Multisensory integration: methodological approaches and emerging principles in the human brain. J. Physiol. Paris 98, 191–205, https://doi.org/10.1016/j.jphysparis.2004.03.018 (2004).
Meredith, M. A. & Stein, B. E. Visual, auditory, and somatosensory convergence on cells in superior colliculus results in multisensory integration. J. Neurophysiol. 56, 640–662, https://doi.org/10.1152/jn.1986.56.3.640 (1986).
Bell, A. H., Meredith, M. A., Van Opstal, A. J. & Munoz, D. P. Crossmodal integration in the primate superior colliculus underlying the preparation and initiation of saccadic eye movements. J. Neurophysiol. 93, 3659–3673, https://doi.org/10.1152/jn.01214.2004 (2005).
Deroy, O. et al. The Complex Interplay Between Multisensory Integration and Perceptual Awareness. Multisensory research 29, 585–606, https://doi.org/10.1163/22134808-00002529 (2016).
Kaga, K., Shinjo, Y., Enomoto, C. & Shindo, M. A case of cortical deafness and loss of vestibular and somatosensory sensations caused by cerebrovascular lesions in bilateral primary auditory cortices, auditory radiations, and postcentral gyruses - complete loss of hearing despite normal DPOAE and ABR. Acta Otolaryngol 135, 389–394, https://doi.org/10.3109/00016489.2014.980914 (2015).
Scruggs, T. E., Mastropieri, M. A. & Casto, G. The Quantitative Synthesis of Single-Subject Research - Methodology and Validation. Rem. Spec. Educ. 8, 24–33 (1987).
Heyvaert, M. & Onghena, P. Analysis of single-case data: randomisation tests for measures of effect size. Neuropsychol. Rehabil. 24, 507–527, https://doi.org/10.1080/09602011.2013.818564 (2014).
Scruggs, T. E. & Mastropieri, M. A. Summarizing single-subject research. Issues and applications. Behav. Modif. 22, 221–242, https://doi.org/10.1177/01454455980223001 (1998).
Carlsen, A. N., Maslovat, D., Lam, M. Y., Chua, R. & Franks, I. M. Considerations for the use of a startling acoustic stimulus in studies of motor preparation in humans. Neurosci. Biobehav. Rev. 35, 366–376, https://doi.org/10.1016/j.neubiorev.2010.04.009 (2011).
Woods, D. L., Knight, R. T. & Neville, H. J. Bitemporal lesions dissociate auditory evoked potentials and perception. Electroencephalogr. Clin. Neurophysiol. 57, 208–220 (1984).
Ho, K. J., Kileny, P., Paccioretti, D. & McLean, D. R. Neurologic, audiologic, and electrophysiologic sequelae of bilateral temporal lobe lesions. Arch. Neurol. 44, 982–987 (1987).
Los, S. A. & Van der Burg, E. Sound speeds vision through preparation, not integration. J. Exp. Psychol. Hum. Percept. Perform. 39, 1612–1624, https://doi.org/10.1037/a0032183 (2013).
Mazza, V., Turatto, M., Rossi, M. & Umilta, C. How automatic are audiovisual links in exogenous spatial attention? Neuropsychologia 45, 514–522, https://doi.org/10.1016/j.neuropsychologia.2006.02.010 (2007).
Santangelo, V. & Spence, C. Is the exogenous orienting of spatial attention truly automatic? Evidence from unimodal and multisensory studies. Conscious. Cogn. 17, 989–1015, https://doi.org/10.1016/j.concog.2008.02.006 (2008).
Berger, A., Henik, A. & Rafal, R. Competition between endogenous and exogenous orienting of visual attention. J. Exp. Psychol. Gen. 134, 207–221, https://doi.org/10.1037/0096-3445.134.2.207 (2005).
Spence, C. & McDonald, J. In The handbook of multisensory processes. 3–25 (MIT Press, 2004).
Leone, L. M. & McCourt, M. E. The roles of physical and physiological simultaneity in audiovisual multisensory facilitation. Iperception 4, 213–228, https://doi.org/10.1068/i0532 (2013).
Stevenson, R. A., Fister, J. K., Barnett, Z. P., Nidiffer, A. R. & Wallace, M. T. Interactions between the spatial and temporal stimulus factors that influence multisensory integration in human performance. Exp. Brain Res. 219, 121–137, https://doi.org/10.1007/s00221-012-3072-1 (2012).
Nissen, M. J. Stimulus-Intensity and Information-Processing. Percept. Psychophys. 22, 338–352 (1977).
Summers, J. J. & Anson, J. G. Current status of the motor program: revisited. Hum. Mov. Sci. 28, 566–577, https://doi.org/10.1016/j.humov.2009.01.002 (2009).
Maslovat, D., Hajj, J. & Carlsen, A. N. Coactivation of response initiation processes with redundant signals. Neurosci. Lett. 675, 7–11, https://doi.org/10.1016/j.neulet.2018.03.029 (2018).
Maslovat, D., Carter, M. J., Kennefick, M. & Carlsen, A. N. Startle neural activity is additive with normal cortical initiation-related activation. Neurosci. Lett. 558, 164–168, https://doi.org/10.1016/j.neulet.2013.11.009 (2014).
Herkenham, M. Laminar organization of thalamic projections to the rat neocortex. Science 207, 532–535, https://doi.org/10.1126/science.7352263 (1980).
Haider, M., Ganglberger, J. A. & Groll-Knapp, E. Thalamo-cortical components of reaction time. Acta Psychol. (Amst.) 30, 378–381 (1969).
Drugowitsch, J., DeAngelis, G. C., Klier, E. M., Angelaki, D. E. & Pouget, A. Optimal multisensory decision-making in a reaction-time task. Elife 3, https://doi.org/10.7554/eLife.03005 (2014).
Raab, D. H. Statistical facilitation of simple reaction times. Trans. N. Y. Acad. Sci. 24, 574–590 (1962).
Ulrich, R., Miller, J. & Schroter, H. Testing the race model inequality: an algorithm and computer programs. Behav. Res. Methods 39, 291–302 (2007).
Jones, E. G. & Powell, T. P. An anatomical study of converging sensory pathways within the cerebral cortex of the monkey. Brain 93, 793–820 (1970).
Wang, Y., Celebrini, S., Trotter, Y. & Barone, P. Visuo-auditory interactions in the primary visual cortex of the behaving monkey: electrophysiological evidence. BMC Neurosci. 9, 79, https://doi.org/10.1186/1471-2202-9-79 (2008).
Masterton, R. B. & Berkley, M. A. Brain function: changing ideas on the role of sensory, motor, and association cortex in behavior. Annu. Rev. Psychol. 25, 277–312, https://doi.org/10.1146/annurev.ps.25.020174.001425 (1974).
Allman, B. L., Keniston, L. P. & Meredith, M. A. Subthreshold auditory inputs to extrastriate visual neurons are responsive to parametric changes in stimulus quality: sensory-specific versus non-specific coding. Brain Res. 1242, 95–101, https://doi.org/10.1016/j.brainres.2008.03.086 (2008).
Alvarado, J. C., Stanford, T. R., Vaughan, J. W. & Stein, B. E. Cortex mediates multisensory but not unisensory integration in superior colliculus. J. Neurosci. 27, 12775–12786, https://doi.org/10.1523/JNEUROSCI.3524-07.2007 (2007).
Jiang, W., Wallace, M. T., Jiang, H., Vaughan, J. W. & Stein, B. E. Two cortical areas mediate multisensory integration in superior colliculus neurons. J. Neurophysiol. 85, 506–522, https://doi.org/10.1152/jn.2001.85.2.506 (2001).
Kelly, J. P. In Principles of neural science (eds. Kandel, E. R., Schwartz, J. H. & Jessell, T. M.) 481–499 (Elsevier, 1991).
Shneiderman, A. & Henkel, C. K. Banding of lateral superior olivary nucleus afferents in the inferior colliculus: a possible substrate for sensory integration. J. Comp. Neurol. 266, 519–534, https://doi.org/10.1002/cne.902660406 (1987).
Jain, R. & Shore, S. External inferior colliculus integrates trigeminal and acoustic information: unit responses to trigeminal nucleus and acoustic stimulation in the guinea pig. Neurosci. Lett. 395, 71–75, https://doi.org/10.1016/j.neulet.2005.10.077 (2006).
Leo, F., Bolognini, N., Passamonti, C., Stein, B. E. & Ladavas, E. Cross-modal localization in hemianopia: new insights on multisensory integration. Brain 131, 855–865, https://doi.org/10.1093/brain/awn003 (2008).
Eggermont, J. J. In The Auditory Brain and Age-Related Hearing Impairment (ed. Jos J. Eggermont) 27–48 (Academic Press, 2019).
Rowe, M. J. III. Normal variability of the brain-stem auditory evoked response in young and old adult subjects. Electroencephalogr. Clin. Neurophysiol. 44, 459–470, https://doi.org/10.1016/0013-4694(78)90030-5 (1978).
Ladavas, E. Multisensory-based approach to the recovery of unisensory deficit. Ann. N. Y. Acad. Sci. 1124, 98–110, https://doi.org/10.1196/annals.1440.008 (2008).
Carlsen, A. N., Dakin, C. J., Chua, R. & Franks, I. M. Startle produces early response latencies that are distinct from stimulus intensity effects. Exp. Brain Res. 176, 199–205, https://doi.org/10.1007/s00221-006-0610-8 (2007).
Carlsen, A. N. A broadband acoustic stimulus is more likely than a pure tone to elicit a startle reflex and prepared movements. Physiol Rep 3, e12509, https://doi.org/10.14814/phy2.12509 (2015).
Carlsen, A. N. & MacKinnon, C. D. Motor preparation is modulated by the resolution of the response timing information. Brain Res. 1322, 38–49, https://doi.org/10.1016/j.brainres.2010.01.076 (2010).
Hodges, P. W. & Bui, B. H. A comparison of computer-based methods for the determination of onset of muscle contraction using electromyography. Electroencephalogr. Clin. Neurophysiol. 101, 511–519, https://doi.org/10.1016/S0921-884x(96)95190-5 (1996).
Vidailhet, M., Rothwell, J. C., Thompson, P. D., Lees, A. J. & Marsden, C. D. The Auditory Startle Response in the Steele-Richardson-Olszewski Syndrome and Parkinsons-Disease. Brain 115, 1181–1192 (1992).
R: A language and environment for statistical computing v. 3.6.1 (R Foundation for Statistical Computing, 2019).
Bulte, I. & Onghena, P. An R package for single-case randomization tests. Behav. Res. Methods 40, 467–478 (2008).
Acknowledgements
This research was supported by the Natural Sciences and Engineering Research Council of Canada (RGPIN 418361–2012) and the Ontario Ministry of Research and Innovation (ER14-10-133). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Author information
Authors and Affiliations
Contributions
A.N.C. and D.M. designed the experiments, A.N.C. and K.K. performed the experiments. A.N.C. and D.M. analyzed the data. A.N.C., D.M. and K.K. interpreted the results. A.N.C. and D.M. drafted the manuscript. All authors approved the final version of the manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
About this article
Cite this article
Carlsen, A.N., Maslovat, D. & Kaga, K. An unperceived acoustic stimulus decreases reaction time to visual information in a patient with cortical deafness. Sci Rep 10, 5825 (2020). https://doi.org/10.1038/s41598-020-62450-9
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598-020-62450-9
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.