Abstract
Most behavioral learning in biology is trial and error, but how these learning processes are influenced by individual brain systems is poorly understood. Here we show that ventral-to-dorsal hippocampal subdivisions have specific and sequential functions in trial-and-error maze navigation, with ventral hippocampus (vH) mediating early task-specific goal-oriented searching. Although performance and strategy deployment progressed continuously at the population level, individual mice showed discrete learning phases, each characterized by particular search habits. Transitions in learning phases reflected feedforward inhibitory connectivity (FFI) growth occurring sequentially in ventral, then intermediate, then dorsal hippocampal subdivisions. FFI growth at vH occurred abruptly upon behavioral learning of goal-task relationships. vH lesions or the absence of vH FFI growth delayed early learning and disrupted performance consistency. Intermediate hippocampus lesions impaired intermediate place learning, whereas dorsal hippocampus lesions specifically disrupted late spatial learning. Trial-and-error navigational learning processes in naive mice thus involve a stereotype sequence of increasingly precise subtasks learned through distinct hippocampal subdivisions. Because of its unique connectivity, vH may relate specific goals to internal states in learning under healthy and pathological conditions.
This is a preview of subscription content, access via your institution
Access options
Subscribe to this journal
Receive 12 print issues and online access
$209.00 per year
only $17.42 per issue
Buy this article
- Purchase on Springer Link
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
References
Sutton, R.S. & Barto, A.G. Reinforcement Learning: An Introduction (MIT Press, 1998).
Dayan, P. & Balleine, B.W. Reward, motivation and reinforcement learning. Neuron 36, 285–298 (2002).
Graybiel, A.M. Habits, rituals, and the evaluative brain. Annu. Rev. Neurosci. 31, 359–387 (2008).
Barnes, T.D., Kubota, Y., Hu, D., Jin, D.Z. & Graybiel, A.M. Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories. Nature 437, 1158–1161 (2005).
Desrochers, T.M., Jin, D.Z., Goodman, N.D. & Graybiel, A.M. Optimal habits can develop spontaneously through sensitivity to local cost. Proc. Natl. Acad. Sci. USA 107, 20512–20517 (2010).
Thorn, C.A., Atallah, H., Howe, M. & Graybiel, A.M. Differential dynamics of activity changes in dorsolateral and dorsomedial striatal loops during learning. Neuron 66, 781–795 (2010).
Amemori, K., Gibb, L.G. & Graybiel, A.M. Shifting responsibility: the importance of striatal modularity to reinforcement learning in uncertain environments. Front. Hum. Neurosci. 5, 47 (2011).
Botvinick, M.M., Niv, Y. & Barto, A.C. Hierarchically organized behavior and its neural foundations: a reinforcement-learning perspective. Cognition 113, 262–280 (2009).
O'Keefe, J. & Nadel, L. The Hippocampus as a Cognitive Map (Oxford Univ. Press, 1978).
Moser, E.I., Kropff, E. & Moser, M.B. Place cells, grid cells, and the brain's spatial representation system. Annu. Rev. Neurosci. 31, 69–89 (2008).
Bird, C.M. & Burgess, N. The hippocampus and memory: insights from spatial processing. Nat. Rev. Neurosci. 9, 182–194 (2008).
Morris, R.G.M., Garrud, P., Rawlins, J.N.P. & O'Keefe, J. Place navigation impaired in rats with hippocampal lesions. Nature 297, 681–683 (1982).
Moser, M.B. & Moser, E.I. Functional differentiation in the hippocampus. Hippocampus 8, 608–619 (1998).
Dong, H.W., Swanson, L.W., Chen, L., Fanselow, M.S. & Toga, A.W. Genomic-anatomic evidence for distinct functional domains in hippocampal field CA1. Proc. Natl. Acad. Sci. USA 106, 11794–11799 (2009).
Fanselow, M.S. & Dong, H.-W. Are the dorsal and ventral hippocampus functionally distinct structures? Neuron 65, 7–19 (2010).
Moser, M.B., Moser, E.I., Forrest, E., Andersen, P. & Morris, R.G.M. Spatial learning with a minislab in the dorsal hippocampus. Proc. Natl. Acad. Sci. USA 92, 9697–9701 (1995).
Thompson, C.L. et al. Genomic anatomy of the hippocampus. Neuron 60, 1010–1021 (2008).
Viard, A., Doeller, C.F., Hartley, T., Bird, C.M. & Burgess, N. Anterior hippocampus and goal-directed spatial decision making. J. Neurosci. 31, 4613–4621 (2011).
Royer, S., Sirota, A., Patel, J. & Buzsaki, G. Distinct representations and theta dynamics in dorsal and ventral hippocampus. J. Neurosci. 30, 1777–1787 (2010).
Young, J.J. & Shapiro, M.L. Dynamic coding of goal-directed paths by orbital prefrontal cortex. J. Neurosci. 31, 5989–6000 (2011).
Pennartz, C.M., Ito, R., Verschure, P.F., Battaglia, F.P. & Robbins, T.W. The hippocampal-striatal axis in learning, prediction and goal-directed behavior. Trends Neurosci. 34, 548–559 (2011).
Ruediger, S. et al. Learning-related feedforward inhibitory connectivity growth required for memory precision. Nature 473, 514–518 (2011).
Bednarek, E. & Caroni, P. β-adducin is required for stable assembly of new synapses and improved memory upon environmental enrichment. Neuron 69, 1132–1146 (2011).
Morris, R. Development of a water-maze procedure for studying spatial learning in the rat. J. Neurosci. Methods 11, 47–60 (1984).
Wolfer, D.P. & Lipp, H.P. Dissecting the behaviour of transgenic mice: is it the mutation, the genetic background, or the environment? Exp. Physiol. 85, 627–634 (2000).
Janus, C. Search strategies used by APP transgenic mice during navigation in the Morris water maze. Learn. Mem. 11, 337–346 (2004).
Garthe, A., Behr, J. & Kempermann, G. Adult-generated hippocampal neurons allow the flexible use of spatially precise learning strategies. PLoS ONE 4, e5464 (2009).
Luo, A.H., Tahsili-Fahadan, P., Wise, R.A., Lupica, C.R. & Aston-Jones, G. Linking context with reward: a functional circuit from hippocampal CA3 to ventral tegmental area. Science 333, 353–357 (2011).
Wiltgen, B.J. et al. The hippocampus plays a selective role in the retrieval of detailed contextual memories. Curr. Biol. 20, 1336–1344 (2010).
Schultz, W., Dayan, P. & Montague, P.R. A neural substrate of prediction and reward. Science 275, 1593–1599 (1997).
Bethus, I., Tse, D. & Morris, R.G.M. Dopamine and memory: modulation of the persistence of memory for novel hippocampal NMDA receptor-dependent paired associates. J. Neurosci. 30, 1610–1618 (2010).
Rabenstein, R.L. et al. Impaired synaptic plasticity and learning in mice lacking β-adducin, an actin-regulating protein. J. Neurosci. 25, 2138–2145 (2005).
Bast, T., Wilson, I.A., Witter, M.P. & Morris, R.G.M. From rapid place learning to behavioral performance: a key role for intermediate hippocampus. PLoS Biol. 7, e1000089 (2009).
Bannerman, D.M., Good, M.A., Butcher, S.P., Ramsay, M. & Morris, R.G.M. Distinct components of spatial learning revealed by prior training and NMDA receptor blockade. Nature 378, 182–186 (1995).
Hoh, T., Beiko, J., Boon, S., Weiss, S. & Cain, D.P. Complex behavioral strategy and reversal learning in the water maze without NMDA receptor-dependent long-term potentiation. J. Neurosci. 19, RC2 (1999).
Gallistel, C.R., Fairhust, S. & Balsam, P. The learning curve: implications of a quantitative analysis. Proc. Natl. Acad. Sci. USA 101, 13124–13131 (2004).
Kjelstrup, K.G. et al. Reduced fear expression after lesions of the ventral hippocampus. Proc. Natl. Acad. Sci. USA 99, 10825–10830 (2002).
De Paola, V., Arber, S. & Caroni, P. AMPA receptors regulate dynamic equilibrium of presynaptic terminals in mature hippocampal networks. Nat. Neurosci. 6, 491–500 (2003).
Acknowledgements
We thank B. Sacchetti (University of Torino), C. Sandi (École Polytechnique Fédérale de Lausanne) and S. Arber (Friedrich Miescher Institut) for comments on the manuscript. The Friedrich Miescher Institut is part of the Novartis Research Foundation.
Author information
Authors and Affiliations
Contributions
S.R. devised and carried out the analysis of hippocampal behavior, connectivity, vH lesions and β-adducin rescue; D.S. carried out the analysis of FFI growth and c-Fos immunoreactivity; F.D. devised and carried out behavioral and lesion studies relating vH, iH and dH FFI growth to subdivision function in learning. P.C. helped devise the experiments and wrote the manuscript. All authors discussed the results and commented the manuscript.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing financial interests.
Supplementary information
Supplementary Text and Figures
Supplementary Figures 1–8 (PDF 0 kb)
Rights and permissions
About this article
Cite this article
Ruediger, S., Spirig, D., Donato, F. et al. Goal-oriented searching mediated by ventral hippocampus early in trial-and-error learning. Nat Neurosci 15, 1563–1571 (2012). https://doi.org/10.1038/nn.3224
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/nn.3224
This article is cited by
-
Specific patterns of neural activity in the hippocampus after massed or distributed spatial training
Scientific Reports (2023)
-
Visual instrumental learning in blindsight monkeys
Scientific Reports (2021)
-
Characteristics of the Neuronal Support for Operative Behavior Formed by Mono- and Multistep Methods
Neuroscience and Behavioral Physiology (2020)
-
A sample efficient model-based deep reinforcement learning algorithm with experience replay for robot manipulation
International Journal of Intelligent Robotics and Applications (2020)
-
Learning, memory and the expression of cholinergic components in mice are modulated by the pesticide chlorpyrifos depending upon age at exposure and apolipoprotein E (APOE) genotype
Archives of Toxicology (2019)