Abstract
Cognitive neuroscience seeks generalizable theories explaining the relationship between behavioral, physiological and mental states. In pursuit of such theories, we propose a theoretical and empirical framework that centers on understanding task demands and the mutual constraints they impose on behavior and neural activity. Task demands emerge from the interaction between an agent’s sensory impressions, goals and behavior, which jointly shape the activity and structure of the nervous system on multiple spatiotemporal scales. Understanding this interaction requires multitask studies that vary more than one experimental component (for example, stimuli and instructions) combined with dense behavioral and neural sampling and explicit testing for generalization across tasks and data modalities. By centering task demands rather than mental processes that tasks are assumed to engage, this framework paves the way for the discovery of new generalizable concepts unconstrained by existing taxonomies, and moves cognitive neuroscience toward an action-oriented, dynamic and integrated view of the brain.
This is a preview of subscription content, access via your institution
Access options
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 / 30 days
cancel any time
Subscribe to this journal
Receive 12 print issues and online access
$209.00 per year
only $17.42 per issue
Buy this article
- Purchase on SpringerLink
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1038%2Fs41593-024-01711-6/MediaObjects/41593_2024_1711_Fig1_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1038%2Fs41593-024-01711-6/MediaObjects/41593_2024_1711_Fig2_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1038%2Fs41593-024-01711-6/MediaObjects/41593_2024_1711_Fig3_HTML.png)
Similar content being viewed by others
References
Gibson, J. J. The ecological approach to visual perception. J. Aesthet. Art. Crit. 39, 203 (1979).
Ledergerber, D. et al. Task-dependent mixed selectivity in the subiculum. Cell Rep. 35, 109175 (2021).
Kay, K., Bonnen, K., Denison, R. N., Arcaro, M. J. & Barack, D. L. Tasks and their role in visual neuroscience. Neuron https://doi.org/10.1016/j.neuron.2023.03.022 (2023).
Burlingham, C. S. et al. Task-related hemodynamic responses in human early visual cortex are modulated by task difficulty and behavioral performance. eLife 11, e73018 (2022).
Ito, T. & Murray, J. D. Multitask representations in the human cortex transform along a sensory-to-motor hierarchy. Nat. Neurosci. 26, 306–315 (2023).
Koida, K. & Komatsu, H. Effects of task demands on the responses of color-selective neurons in the inferior temporal cortex. Nat. Neurosci. 10, 108–116 (2007).
Lee, J. J., Krumin, M., Harris, K. D. & Carandini, M. Task specificity in mouse parietal cortex. Neuron 110, 2961–2969 (2022).
Cole, M. W. et al. Multi-task connectivity reveals flexible hubs for adaptive task control. Nat. Neurosci. 16, 1348–1355 (2013).
Cox, P. H., Kravitz, D. J. & Mitroff, S. R. Great expectations: minor differences in initial instructions have a major impact on visual search in the absence of feedback. Cogn. Res. Princ. Implic. 6, 19 (2021).
Nastase, S. A., Goldstein, A. & Hasson, U. Keep it real: rethinking the primacy of experimental control in cognitive neuroscience. NeuroImage 222, 117254 (2020).
Topalovic, U. et al. A wearable platform for closed-loop stimulation and recording of single-neuron and local field potential activity in freely moving humans. Nat. Neurosci. https://doi.org/10.1038/s41593-023-01260-4 (2023).
Schmid, A. C., Barla, P. & Doerschner, K. Material category of visual objects computed from specular image structure. Nat. Hum. Behav. 7, 1152–1169 (2023).
Nau, M., Navarro Schröder, T., Frey, M. & Doeller, C. F. Behavior-dependent directional tuning in the human visual-navigation network. Nat. Commun. 11, 3247 (2020).
Chiu, C. Q., Barberis, A. & Higley, M. J. Preserving the balance: diverse forms of long-term GABAergic synaptic plasticity. Nat. Rev. Neurosci. 20, 272–281 (2019).
Lindenberger, U. & Lövdén, M. Brain plasticity in human lifespan development: the exploration–selection–refinement model. Annu. Rev. Dev. Psychol. 1, 197–222 (2019).
Cisek, P. Resynthesizing behavior through phylogenetic refinement. Atten. Percept. Psychophys. 81, 2265–2287 (2019).
Hedrick, N. G. et al. Learning binds new inputs into functional synaptic clusters via spinogenesis. Nat. Neurosci. 25, 726–737 (2022).
Musall, S., Kaufman, M. T., Juavinett, A. L., Gluf, S. & Churchland, A. K. Single-trial neural dynamics are dominated by richly varied movements. Nat. Neurosci. 22, 1677–1686 (2019).
Stringer, C. et al. Spontaneous behaviors drive multidimensional, brainwide activity. Science 364, eaav7893 (2019).
Sommer, M. A. & Wurtz, R. H. Brain circuits for the internal monitoring of movements. Annu. Rev. Neurosci. 31, 317–338 (2008).
Rolfs, M. & Schweitzer, R. Coupling perception to action through incidental sensory consequences of motor behaviour. Nat. Rev. Psychol. 1, 112–123 (2022).
Wynn, J. S., Shen, K. & Ryan, J. D. Eye movements actively reinstate spatiotemporal mnemonic content. Vision 3, 21 (2019).
Zhao, W. et al. Task fMRI paradigms may capture more behaviorally relevant information than resting-state functional connectivity. NeuroImage 270, 119946 (2023).
Bradley, C., Nydam, A. S., Dux, P. E. & Mattingley, J. B. State-dependent effects of neural stimulation on brain function and cognition. Nat. Rev. Neurosci. 23, 459–475 (2022).
Gardner, R. J. et al. Toroidal topology of population activity in grid cells. Nature 602, 123–128 (2022).
Driscoll, L. N., Duncker, L. & Harvey, C. D. Representational drift: emerging theories for continual learning and experimental future directions. Curr. Opin. Neurobiol. 76, 102609 (2022).
Schoonover, C. E., Ohashi, S. N., Axel, R. & Fink, A. J. P. Representational drift in primary olfactory cortex. Nature 594, 541–546 (2021).
Kramer, M. R., Cox, P. H., Mitroff, S. R. & Kravitz, D. J. A precise quantification of how prior experience informs current behavior. J. Exp. Psychol. Gen. 151, 1854–1865 (2022).
Sadeh, S. & Clopath, C. Contribution of behavioural variability to representational drift. eLife 11, e77907 (2022).
Jangraw, D. C. et al. A highly replicable decline in mood during rest and simple tasks. Nat. Hum. Behav. https://doi.org/10.1038/s41562-023-01519-7 (2023).
Busch, E. L. et al. Multi-view manifold learning of human brain-state trajectories. Nat. Comput. Sci. https://doi.org/10.1038/s43588-023-00419-0 (2023).
Vaidya, A. R., Pujara, M. S., Petrides, M., Murray, E. A. & Fellows, L. K. Lesion studies in contemporary neuroscience. Trends Cogn. Sci. 23, 653–671 (2019).
Gidon, A. et al. Dendritic action potentials and computation in human layer 2/3 cortical neurons. Science 367, 83–87 (2020).
Pessoa, L. The entangled brain. J. Cogn. Neurosci. 35, 349–360 (2023).
Bechtel, W. & Bich, L. Grounding cognition: heterarchical control mechanisms in biology. Philos. Trans. R. Soc. B Biol. Sci. 376, 20190751 (2021).
Long, M. A. & Fee, M. S. Using temperature to analyse temporal dynamics in the songbird motor pathway. Nature 456, 189–194 (2008).
Folloni, D. et al. Manipulation of subcortical and deep cortical activity in the primate brain using transcranial focused ultrasound stimulation. Neuron 101, 1109–1116 (2019).
Siddiqi, S. H., Kording, K. P., Parvizi, J. & Fox, M. D. Causal mapping of human brain function. Nat. Rev. Neurosci. 23, 361–375 (2022).
Jun, S., Lee, S. A., Kim, J. S., Jeong, W. & Chung, C. K. Task-dependent effects of intracranial hippocampal stimulation on human memory and hippocampal theta power. Brain Stimul. 13, 603–613 (2020).
Basile, B. M., Templer, V. L., Gazes, R. P. & Hampton, R. R. Preserved visual memory and relational cognition performance in monkeys with selective hippocampal lesions. Sci. Adv. 6, eaaz0484 (2020).
Ausra, J. et al. Wireless, battery-free, subdermally implantable platforms for transcranial and long-range optogenetics in freely moving animals. Proc. Natl Acad. Sci. USA 118, e2025775118 (2021).
Uselman, T. W., Medina, C. S., Gray, H. B., Jacobs, R. E. & Bearer, E. L. Longitudinal manganese‐enhanced magnetic resonance imaging of neural projections and activity. NMR Biomed. 35, e4675 (2022).
Rust, N. C. & LeDoux, J. E. The tricky business of defining brain functions. Trends Neurosci. 46, 3–4 (2023).
Burnston, D. C. A contextualist approach to functional localization in the brain. Biol. Philos. 31, 527–550 (2016).
Huk, A. C. & Shadlen, M. N. Neural activity in macaque parietal cortex reflects temporal integration of visual motion signals during perceptual decision making. J. Neurosci. 25, 10420–10436 (2005).
Levi, A. J., Zhao, Y., Park, I. M. & Huk, A. C. Sensory and choice responses in MT distinct from motion encoding. J. Neurosci. 43, 2090–2103 (2023).
Fusi, S., Miller, E. K. & Rigotti, M. Why neurons mix: high dimensionality for higher cognition. Curr. Opin. Neurobiol. 37, 66–74 (2016).
Bedny, M., Pascual-Leone, A., Dodell-Feder, D., Fedorenko, E. & Saxe, R. Language processing in the occipital cortex of congenitally blind adults. Proc. Natl Acad. Sci. USA 108, 4429–4434 (2011).
Liu, Y., Vannuscorps, G., Caramazza, A. & Striem-Amit, E. Evidence for an effector-independent action system from people born without hands. Proc. Natl Acad. Sci. USA 117, 28433–28441 (2020).
Levenstein, D. et al. On the role of theory and modeling in neuroscience. J. Neurosci. 43, 1074–1088 (2023).
Haji-Abolhassani, A. & Clark, J. J. An inverse Yarbus process: predicting observers’ task from eye movement patterns. Vis. Res. 103, 127–142 (2014).
Kay, L., Keogh, R., Andrillon, T. & Pearson, J. The pupillary light response as a physiological index of aphantasia, sensory and phenomenological imagery strength. eLife 11, e72484 (2022).
Krakauer, J. W., Ghazanfar, A. A., Gomez-Marin, A., MacIver, M. A. & Poeppel, D. Neuroscience needs behavior: correcting a reductionist bias. Neuron 93, 480–490 (2017).
Bimbard, C. et al. Behavioral origin of sound-evoked activity in mouse visual cortex. Nat. Neurosci. 26, 251–258 (2023).
Miller, C. T. et al. Natural behavior is the language of the brain. Curr. Biol. 32, R482–R493 (2022).
DiCarlo, J. J., Zoccolan, D. & Rust, N. C. How does the brain solve visual object recognition? Neuron 73, 415–434 (2012).
Datta, S. R., Anderson, D. J., Branson, K., Perona, P. & Leifer, A. Computational neuroethology: a call to action. Neuron 104, 11–24 (2019).
Frey, M., Nau, M. & Doeller, C. F. Magnetic resonance-based eye tracking using deep neural networks. Nat. Neurosci. 24, 1772–1779 (2021).
Schneider, S., Lee, J. H. & Mathis, M. W. Learnable latent embeddings for joint behavioural and neural analysis. Nature https://doi.org/10.1038/s41586-023-06031-6 (2023).
Hazeltine, E., Dykstra, T. & Schumacher, E. In Experimental Psychology (eds. Gozli, D. & Valsiner, J.) 75–95 https://doi.org/10.1007/978-3-031-17053-9_6 (Springer International Publishing, 2022).
Khona, M. & Fiete, I. R. Attractor and integrator networks in the brain. Nat. Rev. Neurosci. 23, 744–766 (2022).
Ascoli, G. A., Maraver, P., Nanda, S., Polavaram, S. & Armañanzas, R. Win–win data sharing in neuroscience. Nat. Methods 14, 112–116 (2017).
Buzsáki, G. The Brain From Inside Out (Oxford University Press, 2019).
Beam, E., Potts, C., Poldrack, R. A. & Etkin, A. A data-driven framework for mapping domains of human neurobiology. Nat. Neurosci. 24, 1733–1744 (2021).
Poeppel, D. & Adolfi, F. Against the epistemological primacy of the hardware: the brain from inside out, turned upside down. eNeuro 7, ENEURO.0215-20.2020 (2020).
Fiebelkorn, I. C. & Kastner, S. Functional specialization in the attention network. Annu. Rev. Psychol. 71, 221–249 (2020).
Zhang, X. et al. Active information maintenance in working memory by a sensory cortex. eLife 8, e43191 (2019).
Rademaker, R. L., Chunharas, C. & Serences, J. T. Coexisting representations of sensory and mnemonic information in human visual cortex. Nat. Neurosci. https://doi.org/10.1038/s41593-019-0428-x (2019).
Teng, C. & Kravitz, D. J. Visual working memory directly alters perception. Nat. Hum. Behav. 3, 827–836 (2019).
Lee, S.-H., Kravitz, D. J. & Baker, C. I. Goal-dependent dissociation of visual and prefrontal cortices during working memory. Nat. Neurosci. 16, 997–999 (2013).
Favila, S. E., Kuhl, B. A. & Winawer, J. Perception and memory have distinct spatial tuning properties in human visual cortex. Nat. Commun. 13, 5864 (2022).
Dijkstra, N. & Fleming, S. M. Subjective signal strength distinguishes reality from imagination. Nat. Commun. 14, 1627 (2023).
Albers, A. M., Kok, P., Toni, I., Dijkerman, H. C. & de Lange, F. P. Shared representations for working memory and mental imagery in early visual cortex. Curr. Biol. 23, 1427–1431 (2013).
Kragel, P. A., Reddan, M. C., LaBar, K. S. & Wager, T. D. Emotion schemas are embedded in the human visual system. Sci. Adv. 5, eaaw4358 (2019).
Yarkoni, T., Poldrack, R. A., Nichols, T. E., Van Essen, D. C. & Wager, T. D. Large-scale automated synthesis of human functional neuroimaging data. Nat. Methods 8, 665–670 (2011).
Shine, J. M. et al. Human cognition involves the dynamic integration of neural activity and neuromodulatory systems. Nat. Neurosci. 22, 289–296 (2019).
Nakai, T. & Nishimoto, S. Quantitative models reveal the organization of diverse cognitive functions in the brain. Nat. Commun. 11, 1142 (2020).
Gitelman, D. R., Nobre, A. C., Sonty, S., Parrish, T. B. & Mesulam, M.-M. Language network specializations: an analysis with parallel task designs and functional magnetic resonance imaging. NeuroImage 26, 975–985 (2005).
Niv, Y. The primacy of behavioral research for understanding the brain. Behav. Neurosci. 135, 601–609 (2021).
Hebart, M. N., Zheng, C. Y., Pereira, F. & Baker, C. I. Revealing the multidimensional mental representations of natural objects underlying human similarity judgements. Nat. Hum. Behav. 4, 1173–1185 (2020).
Marek, S. et al. Reproducible brain-wide association studies require thousands of individuals. Nature 603, 654–660 (2022).
Moncrieff, J. et al. The serotonin theory of depression: a systematic umbrella review of the evidence. Mol. Psychiatry https://doi.org/10.1038/s41380-022-01661-0 (2022).
Cuthbert, B. N. & Insel, T. R. Toward the future of psychiatric diagnosis: the seven pillars of RDoC. BMC Med. 11, 126 (2013).
Malik-Moraleda, S. et al. An investigation across 45 languages and 12 language families reveals a universal language network. Nat. Neurosci. 25, 1014–1019 (2022).
Jaffe, P. I., Poldrack, R. A., Schafer, R. J. & Bissett, P. G. Modelling human behaviour in cognitive tasks with latent dynamical systems. Nat. Hum. Behav. https://doi.org/10.1038/s41562-022-01510-8 (2023).
Gschwind, T. et al. Hidden behavioral fingerprints in epilepsy. Neuron https://doi.org/10.1016/j.neuron.2023.02.003 (2023).
Porsteinsson, A. P., Isaacson, R. S., Knox, S., Sabbagh, M. N. & Rubino, I. Diagnosis of early Alzheimer’s disease: clinical practice in 2021. J. Prev. Alzheimers Dis. https://doi.org/10.14283/jpad.2021.23 (2021).
Montague, P. R., Dolan, R. J., Friston, K. J. & Dayan, P. Computational psychiatry. Trends Cogn. Sci. 16, 72–80 (2012).
Zador, A. et al. Catalyzing next-generation artificial intelligence through NeuroAI. Nat. Commun. 14, 1597 (2023).
Körding, K. P. & Wolpert, D. M. Bayesian integration in sensorimotor learning. Nature 427, 244–247 (2004).
Friston, K., FitzGerald, T., Rigoli, F., Schwartenbeck, P. & Pezzulo, G. Active inference: a process theory. Neural Comput. 29, 1–49 (2017).
Dayan, P. & Daw, N. D. Decision theory, reinforcement learning, and the brain. Cogn. Affect. Behav. Neurosci. 8, 429–453 (2008).
de Cothi, W. et al. Predictive maps in rats and humans for spatial navigation. Curr. Biol. 32, 3676–3689 (2022).
Momennejad, I. Learning structures: predictive representations, replay, and generalization. Curr. Opin. Behav. Sci. 32, 155–166 (2020).
Tomov, M. S., Schulz, E. & Gershman, S. J. Multi-task reinforcement learning in humans. Nat. Hum. Behav. 5, 764–773 (2021).
Yang, G. R., Joglekar, M. R., Song, H. F., Newsome, W. T. & Wang, X.-J. Task representations in neural networks trained to perform many cognitive tasks. Nat. Neurosci. 22, 297–306 (2019).
Flesch, T., Juechems, K., Dumbalska, T., Saxe, A. & Summerfield, C. Orthogonal representations for robust context-dependent task performance in brains and neural networks. Neuron 110, 1258–1270 (2022).
Johnston, W. J. & Fusi, S. Abstract representations emerge naturally in neural networks trained to perform multiple tasks. Nat. Commun. 14, 1040 (2023).
Scholte, H. S., Losch, M. M., Ramakrishnan, K., de Haan, E. H. F. & Bohte, S. M. Visual pathways from the perspective of cost functions and multi-task deep neural networks. Cortex 98, 249–261 (2018).
Maloney, L. T. & Mamassian, P. Bayesian decision theory as a model of human visual perception: testing Bayesian transfer. Vis. Neurosci. 26, 147–155 (2009).
Acknowledgements
We thank B. Averbeck, B. Conway, K. Kay, J. B. Ritchie, and M. Rolfs for helpful comments on an earlier version of this paper. M.N. was supported by a Feodor Lynen Research Fellowship funded by the Alexander von Humboldt Foundation. A.C.S. was supported by a Walter Benjamin Fellowship funded by the German Research Foundation (DFG). M.N., A.C.S. and C.I.B. were further supported by C.I.B.’s funding provided by the Intramural Research Program of the NIMH (ZIAMH002909). S.M.K. and D.J.K. were supported by D.J.K.’s funding provided by the National Science Foundation (grant no. BCS2022572).
Author information
Authors and Affiliations
Contributions
The framework presented in this article was developed through the collaborative efforts of all authors. C.I.B. and D.J.K. initiated the project. M.N., C.I.B., and D.J.K. conceptualized the key points of the article. M.N. wrote the manuscript, created the figures, and incorporated feedback from A.C.S., S.K., C.I.B., and D.J.K. All authors contributed to the ideas and writing of the manuscript over the course of weekly meetings, with M.N. and A.C.S. drafting the final version of the manuscript together, with edits from C.I.B. and D.J.K.
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing interests.
Peer review
Peer review information
Nature Neuroscience thanks David Barack, Rachel Denison and Luiz Pessoa for their contribution to the peer review of this work.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Nau, M., Schmid, A.C., Kaplan, S.M. et al. Centering cognitive neuroscience on task demands and generalization. Nat Neurosci (2024). https://doi.org/10.1038/s41593-024-01711-6
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41593-024-01711-6