Abstract
Reading is a rapid, distributed process that engages multiple components of the ventral visual stream. To understand the neural constituents and their interactions that allow us to identify written words, we performed direct intra-cranial recordings in a large cohort of humans. This allowed us to isolate the spatiotemporal dynamics of visual word recognition across the entire left ventral occipitotemporal cortex. We found that mid-fusiform cortex is the first brain region sensitive to lexicality, preceding the traditional visual word form area. The magnitude and duration of its activation are driven by the statistics of natural language. Information regarding lexicality and word frequency propagates posteriorly from this region to visual word form regions and to earlier visual cortex, which, while active earlier, show sensitivity to words later. Further, direct electrical stimulation of this region results in reading arrest, further illustrating its crucial role in reading. This unique sensitivity of mid-fusiform cortex to sub-lexical and lexical characteristics points to its central role as the orthographic lexicon—the long-term memory representations of visual word forms.
This is a preview of subscription content, access via your institution
Access options
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 / 30 days
cancel any time
Subscribe to this journal
Receive 12 digital issues and online access to articles
$119.00 per year
only $9.92 per issue
Buy this article
- Purchase on Springer Link
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
Data availability
The datasets generated from this research are not publicly available due to their containing information non-compliant with HIPAA, and the human participants from whom the data were collected have not consented to their public release. However, they are available on request from the corresponding author.
Code availability
The custom code that supports the findings of this study is available from the corresponding author on request.
References
Dehaene, S., Le Clec’H, G., Poline, J.-B., LeBihan, D. & Cohen, L. The visual word form area: a prelexical representation of visual words in the fusiform gyrus. Neuroreport 13, 321–325 (2002).
Dehaene, S. & Cohen, L. The unique role of the visual word form area in reading. Trends Cogn. Sci. 15, 254–262 (2011).
Dehaene, S., Cohen, L., Sigman, M. & Vinckier, F. The neural code for written words: a proposal. Trends Cogn. Sci. 9, 335–341 (2005).
Grainger, J. & Van Heuven, W. J. B. in The Mental Lexicon: Some Words to Talk about Words (ed. Bonin, P.) 1–23 (Nova Science, 2003).
McClelland, J. L. & Rumelhart, D. E. An interactive activation model of context effects in letter perception: I. An account of basic findings. Psychol. Rev. 88, 375–407 (1981).
Davis, C. J. The spatial coding model of visual word identification. Psychol. Rev. 117, 713–758 (2010).
Whitney, C. How the brain encodes the order of letters in a printed word: the SERIOL model and selective literature review. Psychon. Bull. Rev. 8, 221–243 (2001).
Grainger, J., Dufau, S. & Ziegler, J. C. A vision of reading. Trends Cogn. Sci. 20, 171–179 (2016).
Vinckier, F. et al. Hierarchical coding of letter strings in the ventral stream: dissecting the inner organization of the visual word-form system. Neuron 55, 143–156 (2007).
Binder, J. R., Medler, D. A., Westbury, C. F., Liebenthal, E. & Buchanan, L. Tuning of the human left fusiform gyrus to sublexical orthographic structure. Neuroimage 33, 739–748 (2006).
Whaley, M. L., Kadipasaoglu, C. M., Cox, S. J. & Tandon, N. Modulation of orthographic decoding by frontal cortex. J. Neurosci. 36, 1173–1184 (2016).
Heilbron, M., Richter, D., Ekman, M., Hagoort, P. & de Lange, F. P. Word contexts enhance the neural representation of individual letters in early visual cortex. Nat. Commun. 11, 321 (2020).
Kronbichler, M. et al. The visual word form area and the frequency with which words are encountered: evidence from a parametric fMRI study. Neuroimage 21, 946–953 (2004).
Price, C. J. & Devlin, J. T. The myth of the visual word form area. Neuroimage 19, 473–481 (2003).
Price, C. J. & Devlin, J. T. The interactive account of ventral occipitotemporal contributions to reading. Trends Cogn. Sci. 15, 246–253 (2011).
Kay, K. N. & Yeatman, J. D. Bottom-up and top-down computations in word- and face-selective cortex. Elife 6, e22341 (2017).
Song, Y., Hu, S., Li, X., Li, W. & Liu, J. The role of top-down task context in learning to perceive objects. J. Neurosci. 30, 9869–9876 (2010).
Starrfelt, R. & Gerlach, C. The visual what for area: words and pictures in the left fusiform gyrus. Neuroimage 35, 334–342 (2007).
White, A. L., Palmer, J., Boynton, G. M. & Yeatman, J. D. Parallel spatial channels converge at a bottleneck in anterior word-selective cortex. Proc. Natl Acad. Sci. USA 116, 10087–10096 (2019).
Pammer, K. et al. Visual word recognition: the first half second. Neuroimage 22, 1819–1825 (2004).
Woodhead, Z. V. J. et al. Reading front to back: MEG evidence for early feedback effects during word recognition. Cereb. Cortex 24, 817–825 (2014).
Schuster, S., Hawelka, S., Hutzler, F., Kronbichler, M. & Richlan, F. Words in context: the effects of length, frequency, and predictability on brain responses during natural reading. Cereb. Cortex 26, 3889–3904 (2016).
Graves, W. W., Desai, R., Humphries, C., Seidenberg, M. S. & Binder, J. R. Neural systems for reading aloud: a multiparametric approach. Cereb. Cortex 20, 1799–1815 (2010).
Kadipasaoglu, C. M., Conner, C. R., Whaley, M. L., Baboyan, V. G. & Tandon, N. Category-selectivity in human visual cortex follows cortical topology: a grouped icEEG study. PLoS ONE 11, e0157109 (2016).
Forseth, K. J. et al. A lexical semantic hub for heteromodal naming in middle fusiform gyrus. Brain 141, 2112–2126 (2018).
Lerma-Usabiaga, G., Carreiras, M. & Paz-Alonso, P. M. Converging evidence for functional and structural segregation within the left ventral occipitotemporal cortex in reading. Proc. Natl Acad. Sci. USA 115, 9981–9990 (2018).
Brysbaert, M. & New, B. Moving beyond Kučera and Francis: a critical evaluation of current word frequency norms and the introduction of a new and improved word frequency measure for American English. Behav. Res. Methods 41, 977–990 (2009).
Fischl, B., Sereno, M. I., Tootell, R. B. H. & Dale, A. High-resolution inter-subject averaging and a surface-based coordinate system. Hum. Brain Mapp. 8, 272–284 (1999).
Argall, B. D., Saad, Z. S. & Beauchamp, M. S. Simplified intersubject averaging on the cortical surface using SUMA. Hum. Brain Mapp. 27, 14–27 (2006).
Saad, Z. S. & Reynolds, R. C. SUMA. Neuroimage 62, 768–773 (2012).
Miller, K. J. et al. Spectral changes in cortical surface potentials during motor movement. J. Neurosci. 27, 2424–2432 (2007).
Esposito, F. et al. Cortex-based inter-subject analysis of iEEG and fMRI data sets: application to sustained task-related BOLD and gamma responses. Neuroimage 66, 457–468 (2013).
Conner, C. R., Chen, G., Pieters, T. A. & Tandon, N. Category specific spatial dissociations of parallel processes underlying visual naming. Cereb. Cortex 24, 2741–2750 (2014).
Woolnough, O., Forseth, K. J., Rollo, P. S. & Tandon, N. Uncovering the functional anatomy of the human insula during speech. Elife 8, e53086 (2019).
Pflugshaupt, T. et al. About the role of visual field defects in pure alexia. Brain 132, 1907–1917 (2009).
Rodríguez-López, C., Guerrero Molina, M. P. & Martínez Salio, A. Pure alexia: two cases and a new neuroanatomical classification. J. Neurol. 265, 436–438 (2018).
Tsapkini, K. & Rapp, B. The orthography-specific functions of the left fusiform gyrus: evidence of modality and category specificity. Cortex 46, 185–205 (2010).
Hirshorn, E. A. et al. Decoding and disrupting left midfusiform gyrus activity during word reading. Proc. Natl Acad. Sci. USA 113, 8162–8167 (2016).
Mani, J. et al. Evidence for a basal temporal visual language center: cortical stimulation producing pure alexia. Neurology 71, 1621–1627 (2008).
Bouhali, F., Bézagu, Z., Dehaene, S. & Cohen, L. A mesial-to-lateral dissociation for orthographic processing in the visual cortex. Proc. Natl Acad. Sci. USA 116, 21936–21946 (2019).
Coltheart, M. Are there lexicons? Q. J. Exp. Psychol. Sect. A 57, 1153–1171 (2004).
Coltheart, M., Rastle, K., Perry, C., Langdon, R. & Ziegler, J. DRC: a dual route cascaded model of visual word recognition and reading aloud. Psychol. Rev. 108, 204–256 (2001).
Glezer, L. S., Kim, J., Rule, J., Jiang, X. & Riesenhuber, M. Adding words to the brain’s visual dictionary: novel word learning selectively sharpens orthographic representations in the VWFA. J. Neurosci. 35, 4965–4972 (2015).
Taylor, J. S. H., Davis, M. H. & Rastle, K. Mapping visual symbols onto spoken language along the ventral visual stream. Proc. Natl Acad. Sci. USA 116, 17723–17728 (2019).
Norris, D. The Bayesian reader: explaining word recognition as an optimal Bayesian decision process. Psychol. Rev. 113, 327–357 (2006).
Gold, J. I. & Shadlen, M. N. Banburismus and the brain: decoding the relationship between sensory stimuli, decisions, and reward. Neuron 36, 299–308 (2004).
Rayner, K. & Duffy, S. A. Lexical complexity and fixation times in reading: effects of word frequency, verb complexity, and lexical ambiguity. Mem. Cogn. 14, 191–201 (1986).
Rayner, K. Visual attention in reading: eye movements reflect cognitive processes. Mem. Cogn. 5, 443–448 (1977).
Carreiras, M., Perea, M. & Grainger, J. Effects of orthographic neighborhood in visual word recognition: cross-task comparisons. J. Exp. Psychol. Learn. Mem. Cogn. 23, 857–871 (1997).
Grainger, J., Dufau, S., Montant, M., Ziegler, J. C. & Fagot, J. Orthographic processing in baboons (Papio papio). Science 336, 245–249 (2012).
Rice, G. A. & Robinson, D. O. The role of bigram frequency in the perception of words and nonwords. Mem. Cogn. 3, 513–518 (1975).
Meade, G., Grainger, J. & Holcomb, P. J. Task modulates ERP effects of orthographic neighborhood for pseudowords but not words. Neuropsychologia 129, 385–396 (2019).
Balota, D. A., Cortese, M. J., Sergent-Marshall, S. D., Spieler, D. H. & Yap, M. J. Visual word recognition of single-syllable words. J. Exp. Psychol. Gen. 133, 283–316 (2004).
Perry, C., Ziegler, J. C. & Zorzi, M. Nested incremental modeling in the development of computational theories: the CDP+ model of reading aloud. Psychol. Rev. 114, 273–315 (2007).
Bar, M. et al. Top-down facilitation of visual recognition. Proc. Natl Acad. Sci. USA 103, 449–454 (2006).
Lochy, A. et al. Selective visual representation of letters and words in the left ventral occipito-temporal cortex with intracerebral recordings. Proc. Natl Acad. Sci. USA 115, E7595–E7604 (2018).
Thesen, T. et al. Sequential then interactive processing of letters and words in the left fusiform gyrus. Nat. Commun. 3, 1284–1288 (2012).
Chang, C. H. C. et al. Adaptation of the human visual system to the statistics of letters and line configurations. Neuroimage 120, 428–440 (2015).
Dehaene, S., Cohen, L., Morais, J. & Kolinsky, R. Illiterate to literate: behavioural and cerebral changes induced by reading acquisition. Nat. Rev. Neurosci. 16, 234–244 (2015).
Szwed, M., Qiao, E., Jobert, A., Dehaene, S. & Cohen, L. Effects of literacy in early visual and occipitotemporal areas of Chinese and French readers. J. Cogn. Neurosci. 26, 459–475 (2014).
Agrawal, A., Hari, K. V. S. & Arun, S. P. Reading increases the compositionality of visual word representations. Psychol. Sci. 30, 1707–1723 (2019).
Lochy, A., Van Reybroeck, M. & Rossion, B. Left cortical specialization for visual letter strings predicts rudimentary knowledge of letter-sound association in preschoolers. Proc. Natl Acad. Sci. USA 113, 8544–8549 (2016).
Thomas Yeo, B. T. et al. The organization of the human cerebral cortex estimated by intrinsic functional connectivity. J. Neurophysiol. 106, 1125–1165 (2011).
Devlin, J. T. et al. Susceptibility-induced loss of signal: comparing PET and fMRI on a semantic task. Neuroimage 11, 589–600 (2000).
Borchers, S., Himmelbach, M., Logothetis, N. & Karnath, H. O. Direct electrical stimulation of human cortex–the gold standard for mapping brain functions? Nat. Rev. Neurosci. 13, 63–70 (2012).
Tandon, N. in Clinical Brain Mapping (eds. Yoshor, D. & Mizrahi, E.) 203–218 (McGraw Hill Education, 2012).
Conner, C. R., Ellmore, T. M., Pieters, T. A., Disano, M. A. & Tandon, N. Variability of the relationship between electrophysiology and BOLD-fMRI across cortical regions in humans. J. Neurosci. 31, 12855–12865 (2011).
Pieters, T. A., Conner, C. R. & Tandon, N. Recursive grid partitioning on a cortical surface model: an optimized technique for the localization of implanted subdural electrodes. J. Neurosurg. 118, 1086–1097 (2013).
Tandon, N. et al. Analysis of morbidity and outcomes associated with use of subdural grids vs stereoelectroencephalography in patients with intractable epilepsy. JAMA Neurol. 76, 672–681 (2019).
Rollo, P. S., Rollo, M. J., Zhu, P., Woolnough, O. & Tandon, N. Oblique trajectory angles in robotic stereoelectroencephalography. J. Neurosurg. https://doi.org/10.3171/2020.5.JNS20975 (2020).
Cox, R. W. AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. Comput. Biomed. Res. 29, 162–173 (1996).
Dale, A. M., Fischl, B. & Sereno, M. I. Cortical surface-based analysis: I. segmentation and surface reconstruction. Neuroimage 9, 179–194 (1999).
Kleiner, M., Brainard, D. & Pelli, D. What’s new in Psychtoolbox-3? Perception 36, ECVP ‘07 Abstracts (2007).
Balota, D. A. et al. The English lexicon project. Behav. Res. Methods 39, 445–459 (2007).
Fedorenko, E. et al. Neural correlate of the construction of sentence meaning. Proc. Natl Acad. Sci. USA 113, E6256–E6262 (2016).
Glasser, M. F. et al. A multi-modal parcellation of human cerebral cortex. Nature 536, 171–178 (2016).
Yarkoni, T., Balota, D. & Yap, M. Moving beyond Coltheart’s N: a new measure of orthographic similarity. Psychon. Bull. Rev. 15, 971–979 (2008).
Berry, M. W., Browne, M., Langville, A. N., Pauca, V. P. & Plemmons, R. J. Algorithms and applications for approximate nonnegative matrix factorization. Comput. Stat. Data Anal. 52, 155–173 (2007).
Kadipasaoglu, C. M. et al. Development of grouped icEEG for the study of cognitive processing. Front. Psychol. 6, 1008 (2015).
Kadipasaoglu, C. M. et al. Surface-based mixed effects multilevel analysis of grouped human electrocorticography. Neuroimage 101, 215–224 (2014).
Acknowledgements
The authors thank Y. Wang for assistance coordinating participant data transfers and E. Klier for comments on previous versions of this manuscript. We thank all the individuals who participated in this study, the neurologists at the Texas Comprehensive Epilepsy Program who participated in the care of these people and all the nurses and technicians in the Epilepsy Monitoring Unit at Memorial Hermann Hospital who helped make this research possible. This work was supported by the National Institute of Neurological Disorders and Stroke and the National Institute on Deafness and Communicable Disorders via the BRAIN initiative ‘Research on Humans’ grant NS098981. The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.
Author information
Authors and Affiliations
Contributions
Conceptualization: O.W., N.T. and S.D.; Methodology: O.W., C.D., N.T. and S.D.; Data curation: O.W., C.D., P.S.R. and N.E.C.; Software: O.W., K.J.F. and C.D.; Formal analysis: O.W.; Writing – original draft: O.W.; Writing – review and editing: O.W.,. N.T., S.F.B., Y.L. and S.D.; Visualization: O.W.; Supervision: N.T.; Project administration: N.T.; Funding acquisition: N.T.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests
Additional information
Peer review information Primary Handling Editor: Marike Schiffer.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Extended data
Extended Data Fig. 1 Lateralization of word-responsive electrodes in ventral cortex.
Map of word responsive (yellow; activation >20% above baseline) and unresponsive (red) electrodes in the passive viewing (a; 27 patients) and sentence (b; 28 patients) tasks. In the non-dominant right hemisphere (14 patients), word responses were confined to occipital cortex.
Extended Data Fig. 2 Spatiotemporal mapping of selectivity to hierarchical orthographic stimuli.
Word-amplitude normalized selectivity profiles grouped in 20 mm intervals along the y (antero-posterior) axis in Talairach space for three consecutive time windows (20 patients). Within each time window, electrodes with >20% activation above baseline in response to words were utilized. Averaged within patient. Standard errors represent between patient variability. Individual data points are overlaid. Horizontal dashed lines represent word response.
Extended Data Fig. 3 Lexical and Sub-Lexical Frequency Effects in Mid-Fusiform Cortex.
a, Mid-fusiform responses to real words from the word list condition separated by word frequency and length (49 electrodes, 15 patients). b, Pseudoword responses in mid-fusiform cortex from the Jabberwocky condition separated by bigram frequency (BGF) and word length (49 electrodes, 15 patients).
Extended Data Fig. 4 Timing of the selectivity to hierarchical orthographic stimuli in the passive viewing task.
a, Temporal representations of the two archetypal components generated from NNMF of the z-scores of words against each non-word condition. b, Spatial map of the NNMF decompositions of the z-score word selectivity (207 electrodes, 20 patients). c, Spatiotemporal representation of word vs non-word selectivity (non-word normalized to word activity) for each of the letter-form conditions. Electrode selectivity profiles were grouped every 20 mm along the antero-posterior axis in Talairach space. Each condition shows an anterior-to-posterior spread of word selectivity (red). FF: False Font, IL: Infrequent Letters, FL: Frequent Letters, BG: Frequent Bigrams, QG: Frequent Quadrigrams.
Supplementary information
Supplementary Video 1
Spatiotemporal map of lexical sensitivity in ventral visual cortex. MEMA activation video showing the regions of significant activation to the real word (W; left) stimuli and infrequent letter (IL; middle) stimuli (27 patients). The word normalized amplitude map (right) shows regions with preferential activation to words (red) or infrequent letters (blue)
Supplementary Video 2
Spatiotemporal map of sensitivity to sub-lexical structure in ventral visual cortex. MEMA video showing word-normalized activation amplitudes for each of the non-word conditions from the sub-lexical task, demonstrating regions with preferential activation to words (red) or non-words (blue) (27 patients). FF, false font; IL, infrequent letters; FL, frequent letters; BG, frequent bigrams; QG, frequent quadrigrams
Supplementary Video 3
Cortical stimulation mapping (CSM) of mid-fusiform cortex and lateral occipitotemporal gyrus. CSM session for TA774B showing stimulation of either site results in selective deficits in word reading with no associated naming or speech production deficits. Transcriptions are of the presented reading stimuli. Recording provided with patient’s consent
Rights and permissions
About this article
Cite this article
Woolnough, O., Donos, C., Rollo, P.S. et al. Spatiotemporal dynamics of orthographic and lexical processing in the ventral visual pathway. Nat Hum Behav 5, 389–398 (2021). https://doi.org/10.1038/s41562-020-00982-w
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s41562-020-00982-w
This article is cited by
-
The spatiotemporal dynamics of semantic integration in the human brain
Nature Communications (2023)
-
Seeking the neural representation of statistical properties in print during implicit processing of visual words
npj Science of Learning (2023)
-
Dataset of human intracranial recordings during famous landmark identification
Scientific Data (2022)
-
Brains and algorithms partially converge in natural language processing
Communications Biology (2022)
-
Neurophysiological considerations for visual implants
Brain Structure and Function (2022)