Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Matters Arising
  • Published:

Overfitting to ‘predict’ suicidal ideation

Nature Human Behaviour volume 7, pages 680–681 (2023)Cite this article

Subjects

The Original Article was published on 30 October 2017

arising from: M. A. Just et al. Nature Human Behaviour https://doi.org/10.1038/s41562-017-0234-y (2017).

Unlike many areas of medicine, the fields of psychiatry and clinical psychology suffer from a critical lack of ability to directly measure the internal processes that are the root of most psychiatric disorders1. Instead, these fields rely on indirect assessments, via verbal report or behavioural analyses, that can often be unreliable indicators of internal thoughts and experiences. Over the past few years, machine learning methods applied to functional neuroimaging data have presented a promising avenue for the field of computational psychiatry to potentially measure preverbal internal processes2, offering hope for the development of neural biomarkers of psychiatric diseases.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Data availability

The data used in these analyses can be found at https://doi.org/10.1184/R1/22086995.v1 and http://www.ccbi.cmu.edu/Suicidal-ideation-NATHUMBEH2017/Just-NatHumBeh2017-data-and-code.html.

Code availability

The code used in these analyses can be found at https://doi.org/10.1184/R1/22086995.v1 and the original code shared by ref. 3 can be found at http://www.ccbi.cmu.edu/Suicidal-ideation-NATHUMBEH2017/Just-NatHumBeh2017-data-and-code.html.

References

  1. Fuchs, T. Subjectivity and intersubjectivity in psychiatric diagnosis. Psychopathology 43, 268–274 (2010).

    Article  PubMed  Google Scholar 

  2. Bzdok, D. & Meyer-Lindenberg, A. Machine learning for precision psychiatry: opportunities and challenges. Biol. Psychiatry Cogn. Neurosci. Neuroimaging 3, 223–230 (2018).

    PubMed  Google Scholar 

  3. Just, M. A. et al. Machine learning of neural representations of suicide and emotion concepts identifies suicidal youth. Nat. Hum. Behav. 1, 911–919 (2017).

    Article  PubMed  PubMed Central  Google Scholar 

  4. Flint, C. et al. Systematic misestimation of machine learning performance in neuroimaging studies of depression. Neuropsychopharmacology 46, 1510–1517 (2021).

    Article  PubMed  PubMed Central  Google Scholar 

  5. Cawley, G. C. & Talbot, N. L. C. On over-fitting in model selection and subsequent selection bias in performance evaluation. J. Mach. Learn. Res. 11, 2079–2107 (2010).

    Google Scholar 

  6. Kriegeskorte, N., Simmons, W. K., Bellgowan, P. S. F. & Baker, C. I. Circular analysis in systems neuroscience: the dangers of double dipping. Nat. Neurosci. 12, 535–540 (2009).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Departments of Psychology, Carnegie Mellon Neuroscience Institute, and Biomedical Engineering, Carnegie Mellon University, Pittsburgh, PA, USA

    Timothy Verstynen

  2. Departments of Bioengineering and Neuroscience, University of Pennsylvania, Philadelphia, PA, USA

    Konrad Paul Kording

Authors
  1. Timothy Verstynen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Konrad Paul Kording
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

T.V. and K.P.K. contributed to writing and conceptualization. T.V. wrote the analysis.

Corresponding author

Correspondence to Timothy Verstynen.

Ethics declarations

Competing interests

T.V. works at the same institution as the senior and first author of ref. 3. The authors have no other competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Verstynen, T., Kording, K.P. Overfitting to ‘predict’ suicidal ideation. Nat Hum Behav 7, 680–681 (2023). https://doi.org/10.1038/s41562-023-01560-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41562-023-01560-6

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing