Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Comment
  • Published:

Towards foundation models of biological image segmentation

Nature Methods volume 20, pages 953–955 (2023)Cite this article

Subjects

In the ever-evolving landscape of biological imaging technology, it is crucial to develop foundation models capable of adapting to various imaging modalities and tackling complex segmentation tasks.

This is a preview of subscription content, access via your institution

Relevant articles

Open Access articles citing this article.

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Three fundamental biological image segmentation tasks.

References

  1. de Teresa-Trueba, I. et al. Nat. Methods 20, 284–294 (2023).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  2. Stringer, C., Wang, T., Michaelos, M. & Pachitariu, M. Nat. Methods 18, 100–106 (2021).

    Article  CAS  PubMed  Google Scholar 

  3. Greenwald, N. F. et al. Nat. Biotechnol. 40, 555–565 (2022).

    Article  CAS  PubMed  Google Scholar 

  4. Heinrich, L. et al. Nature 599, 141–146 (2021).

    Article  CAS  PubMed  Google Scholar 

  5. Berg, S. et al. Nat. Methods 16, 1226–1232 (2019).

    Article  CAS  PubMed  Google Scholar 

  6. Krizhevsky, A., Sutskever, I. & Hinton, G. E. Adv. Neural Inf. Process. Syst. 25, 1097–1105 (2012).

    Google Scholar 

  7. Ronneberger, O., Fischer, P. & Brox, T. In Intl C. Medical Image Computing and Computer-Assisted Intervention (eds Navab, N. et al.) 234–241 (Springer, 2015).

  8. Pachitariu, M. & Stringer, C. Nat. Methods 19, 1634–1641 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  9. Cutler, K. J. et al. Nat. Methods 19, 1438–1448 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  10. Sheridan, A. et al. Nat. Methods 20, 295–303 (2023).

    Article  CAS  PubMed  Google Scholar 

  11. Isensee, F., Jaeger, P. F., Kohl, S. A. A., Petersen, J. & Maier-Hein, K. H. Nat. Methods 18, 203–211 (2021).

    Article  CAS  PubMed  Google Scholar 

  12. Antonelli, M. et al. Nat. Commun. 13, 4128 (2022).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  13. Graham, S. et al. Preprint at arXiv https://doi.org/10.48550/arXiv.2303.06274 (2023).

  14. Brown, T. B. et al. Adv. Neural Inf. Process. Syst. 33, 1877–1901 (2020).

    Google Scholar 

  15. Rombach, R., Blattmann, A., Lorenz, D., Esser, P. & Ommer, B. In Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition 10674–10685 (IEEE, 2022).

  16. Jain, J. et al. Preprint at arXiv https://doi.org/10.48550/arXiv.2211.06220 (2023).

  17. Zou, X. et al. In Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition (in the press).

  18. Zhou, B. et al. Int. J. Comput. Vis. 127, 302–321 (2019).

    Article  Google Scholar 

  19. Caesar, H., Uijlings, J & Ferrari, V. In Computer Vision and Pattern Recognition 1209–1218 (CVPR, 2018).

  20. Vaswani, A. et al. Adv. Neural Inf. Process. Syst. (2017); https://proceedings.neurips.cc/paper_files/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html

  21. Edlund, C. et al. Nat. Methods 18, 1038–1045 (2021).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  22. Lin, J.-R. et al. Cell 186, 363–381.e19 (2023).

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  23. He, K. et al. In Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition 15979–15988 (IEEE, 2022).

  24. Ouyang, L. et al. Adv. Neural Inf. Process. Syst. 35, 27730–27744 (2022).

    Google Scholar 

  25. Hu, E. J. et al. In Intl Conf. Learning Representations (2022); https://openreview.net/forum?id=nZeVKeeFYf9

Download references

Acknowledgements

We thank R. Xie and K. Mckeen for insightful discussions. This work was supported by the Natural Sciences and Engineering Research Council of Canada (NSERC, RGPIN-2020-06189 and DGECR-2020-00294), Canadian Institute for Advanced Research (CIFAR) AI Catalyst Grants, and CIFAR AI Chair programs.

Author information

Authors and Affiliations

  1. Peter Munk Cardiac Centre, University Health Network, Toronto, Ontario, Canada

    Jun Ma & Bo Wang

  2. Department of Laboratory Medicine and Pathobiology, University of Toronto, Toronto, Ontario, Canada

    Jun Ma & Bo Wang

  3. Vector Institute for Artificial Intelligence, Toronto, Ontario, Canada

    Jun Ma & Bo Wang

  4. AI Hub, University Health Network, Toronto, Ontario, Canada

    Bo Wang

  5. Department of Computer Science, University of Toronto, Toronto, Ontario, Canada

    Bo Wang

Authors
  1. Jun Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bo Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

J.M. wrote the manuscript and B.W. edited the original draft and provided funding support. All authors wrote, edited and gave final approval to the manuscript.

Corresponding author

Correspondence to Bo Wang.

Ethics declarations

Competing interests

The authors declare no competing interests.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ma, J., Wang, B. Towards foundation models of biological image segmentation. Nat Methods 20, 953–955 (2023). https://doi.org/10.1038/s41592-023-01885-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41592-023-01885-0

This article is cited by

Search

Advanced search

Quick links

Nature Briefing

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing