Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Correspondence
  • Published:

EfficientBioAI: making bioimaging AI models efficient in energy and latency

Nature Methods volume 21, pages 368–369 (2024)Cite this article

Subjects

In recent years, artificial intelligence (AI)-based microscopy image analysis tools (for example, DeepImageJ1, ZeroCostDL4Mic2, Cellpose3 and mmv_im2im4) have developed rapidly. As AI methods set records in various benchmarks and permit quantitative biological studies not feasible before, we want to raise the awareness of another aspect of bioimaging AI models’ performance: efficiency, particularly latency and energy efficiency. The complexity of AI models has been growing considerably to solve intricate problems, potentially escalating network latency (that is, inference time), especially on very large images or on CPU or edge devices (for example, running Smart Microscope on an embedded system5). This increasing computational overhead contributes substantially to energy consumption, with energy utilization for one inference pass rising exponentially over the last decade6.

One way to address these issues is to compress the models to enhance efficiency, using strategies such as neural network quantization (lowering numerical precision to, for example, int8)7, pruning (removing unimportant neurons) and knowledge distillation (distilling smaller models from the original model)8. These techniques can reduce memory consumption, speed model inference and save energy. Despite the success of model compression in computer vision fields such as object detection9, it remains scarcely adopted in bioimaging AI.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

Fig. 1: Overview of the toolbox.

Code availability

The whole project including the source code can be accessed from https://github.com/MMV-Lab/EfficientBioAI.

References

  1. Gómez-de-Mariscal, E. et al. Nat. Methods 18, 1192–1195 (2021).

    Article  PubMed  Google Scholar 

  2. von Chamier, L. et al. Nat. Commun. 12, 2276 (2021).

    Article  ADS  Google Scholar 

  3. Stringer, C., Wang, T., Michaelos, M. & Pachitariu, M. Nat. Methods 18, 100–106 (2021).

    Article  CAS  PubMed  Google Scholar 

  4. Sonneck, J. & Chen, J. Gigascience https://doi.org/10.1093/gigascience/giad120 (2023).

  5. Mahecic, D. et al. Nat. Methods 19, 1262–1267 (2022).

    Article  CAS  PubMed  Google Scholar 

  6. Desislavov, R., Martínez-Plumed, F. & Hernández-Orallo, J. Sustain. Comput. Inform. Syst. 38, 100857 (2023).

    Google Scholar 

  7. Gholami, A. et al. Preprint at arXiv https://doi.org/10.48550/arXiv.2103.13630 (2021).

  8. Gou, J., Yu, B., Maybank, S. J. & Tao, D. Int. J. Comput. Vis. 129, 1789–1819 (2021).

    Article  Google Scholar 

  9. Ding, C. et al. in Proc. 2019 ACM/SIGDA Int. Sympos. Field-Programmable Gate Arrays 33–42 (Association for Computing Machinery, 2019).

  10. Cai, Y. et al. in Proc. IEEE/CVF Conf. Computer Vision and Pattern Recognition 13169–13178 (IEEE, 2020).

Download references

Acknowledgements

Y.Z., J.S., S.B. and J. Chen are funded by the Federal Ministry of Education and Research (Bundesministerium für Bildung und Forschung, BMBF) in Germany under the funding reference 161L0272. Y.Z., J.S., S.B., S.D., A.G., K.L. and J. Chen are also supported by the Ministry of Culture and Science of the State of North Rhine-Westphalia (Ministerium für Kultur und Wissenschaft des Landes Nordrhein-Westfalen, MKW NRW). J. Cao and S.Z. are supported by the National Key Research and Development Project of China (No. 2022ZD0117801).

Author information

Authors and Affiliations

  1. Leibniz-Institut für Analytische Wissenschaften – ISAS – e.V., Dortmund, Germany

    Yu Zhou, Justin Sonneck, Sweta Banerjee, Stefanie Dörr, Anika Grüneboom, Kristina Lorenz & Jianxu Chen

  2. National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University, Beijing, China

    Jiajun Cao & Shanghang Zhang

  3. Faculty of Computer Science, Ruhr-University Bochum, Bochum, Germany

    Justin Sonneck

  4. Institute of Pharmacology and Toxicology, University of Würzburg, Würzburg, Germany

    Kristina Lorenz

Authors
  1. Yu Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiajun Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Justin Sonneck
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sweta Banerjee
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Stefanie Dörr
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Anika Grüneboom
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Kristina Lorenz
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Shanghang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Jianxu Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Y. Z. initiated the research and wrote the source code. J. Cao and S.Z. enhanced the extensibility of the code. J. S. contributed the pretrained model in the 2D instance segmentation experiment. S.B. performed the data annotation and model pretraining in the 3D semantic segmentation experiment. S.D. and K.L. contributed the annotated data in the 2D instance segmentation experiment. A.G. contributed the data in the 3D semantic segmentation experiment. J. Chen supervised the project. Y.Z. and J. Chen wrote the manuscript with input from all coauthors.

Corresponding author

Correspondence to Jianxu Chen.

Ethics declarations

Competing interests

The authors declare no competing interests.

Peer review

Peer review information

Nature Methods thanks Péter Horváth, Weisong Zhao, and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Supplementary information

Supplementary Information

Supplementary Figures 1–7, Tables 1–4, Notes and Discussion

Reporting Summary

Supplementary Code

Reviewed version of EfficientBioAI toolbox

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, Y., Cao, J., Sonneck, J. et al. EfficientBioAI: making bioimaging AI models efficient in energy and latency. Nat Methods 21, 368–369 (2024). https://doi.org/10.1038/s41592-024-02167-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41592-024-02167-z

Search

Advanced search

Quick links

Nature Briefing AI and Robotics

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing: AI and Robotics