Abstract
Optical microscopy methods such as calcium and voltage imaging enable fast activity readout of large neuronal populations using light. However, the lack of corresponding advances in online algorithms has slowed progress in retrieving information about neural activity during or shortly after an experiment. This gap not only prevents the execution of real-time closed-loop experiments, but also hampers fast experiment–analysis–theory turnover for high-throughput imaging modalities. Reliable extraction of neural activity from fluorescence imaging frames at speeds compatible with indicator dynamics and imaging modalities poses a challenge. We therefore developed FIOLA, a framework for fluorescence imaging online analysis that extracts neuronal activity from calcium and voltage imaging movies at speeds one order of magnitude faster than state-of-the-art methods. FIOLA exploits algorithms optimized for parallel processing on GPUs and CPUs. We demonstrate reliable and scalable performance of FIOLA on both simulated and real calcium and voltage imaging datasets. Finally, we present an online experimental scenario to provide guidance in setting FIOLA parameters and to highlight the trade-offs of our approach.
This is a preview of subscription content, access via your institution
Access options
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 / 30 days
cancel any time
Subscribe to this journal
Receive 12 print issues and online access
$259.00 per year
only $21.58 per issue
Buy this article
- Purchase on Springer Link
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
Data availability
Voltage data with simultaneous electrophysiology can be found at https://janelia.figshare.com/collections/Simultaneous_Voltron_1_0_imaging_and_whole-cell_patch-clamp_recordings_of_somatosensory_cortex_layer_1_interneurons_in_vivo/5325254/1 and https://zenodo.org/record/4515768/export/hx#.ZEK_sXaZO5d. Calcium imaging datasets can be found at https://zenodo.org/record/1659149#.ZELAEHaZO5c and https://zenodo.org/record/7779164#.ZELAJ3aZO5c. The HPR dataset can be found at https://dandiarchive.org/dandiset/000054/draft. Source data are provided with this paper.
Code availability
Code for FIOLA can be found in the FIOLA GitHub repository: https://github.com/nel-lab/FIOLA. FIOLA is under GNU General Public License v2.0. A google colab demo that allows users to quickly try the FIOLA pipeline can be found at https://colab.research.google.com/drive/1y98SHqjAqalJ0LXvVF2drjtVdH8tzMa2?usp=sharing. Voltage imaging simulation code can be found at https://github.com/KasparP/PositronSimulations. License information: Creative Commons Attribution 4.0 International.
References
Grienberger, C., Giovannucci, A., Zeiger, W. & Portera-Cailliau, C. Two-photon calcium imaging of neuronal activity. Nature Reviews Methods Primers 2, 67 (2022).
Peterka, D. S., Takahashi, H. & Yuste, R. Imaging voltage in neurons. Neuron 69, 9–21 (2011).
Sofroniew, N. J., Flickinger, D., King, J. & Svoboda, K. A large field of view two-photon mesoscope with subcellular resolution for in vivo imaging. Elife 5, e14472 (2016).
Voleti, V. et al. Real-time volumetric microscopy of in vivo dynamics and large-scale samples with SCAPE 2.0. Nature Methods 16, 1054–1062 (2019).
Demas, J. et al. High-speed, cortex-wide volumetric recording of neuroactivity at cellular resolution using light beads microscopy. Nature Methods 18, 1103–1111 (2021).
Zhang, Y. et al. Fast and sensitive GCaMP calcium indicators for imaging neural populations. Nature 615, 884–891 (2023).
Abdelfattah, A. S. et al. Bright and photostable chemigenetic indicators for extended in vivo voltage imaging. Science 365, 699–704 (2019).
Villette, V. et al. Ultrafast two-photon imaging of a high-gain voltage indicator in awake behaving mice. Cell 179, 1590–1608 (2019).
Adam, Y. et al. Voltage imaging and optogenetics reveal behaviour-dependent changes in hippocampal dynamics. Nature 569, 413–417 (2019).
Carrillo-Reid, L., Han, S., Yang, W., Akrouh, A. & Yuste, R. Controlling visually guided behavior by holographic recalling of cortical ensembles. Cell 178, 447–457 (2019).
Dalgleish, H. W. et al. How many neurons are sufficient for perception of cortical activity? Elife 9, e58889 (2020).
Robinson, N. T. et al. Targeted activation of hippocampal place cells drives memory-guided spatial behavior. Cell 183, 1586–1599 (2020).
Shemesh, O. A. et al. Temporally precise single-cell-resolution optogenetics. Nat. Neurosci. 20, 1796–1806 (2017).
Packer, A. M., Russell, L. E., Dalgleish, H. W. & Häusser, M. Simultaneous all-optical manipulation and recording of neural circuit activity with cellular resolution in vivo. Nat. Methods 12, 140–146 (2015).
Dal Maschio, M., Donovan, J. C., Helmbrecht, T. O. & Baier, H. Linking neurons to network function and behavior by two-photon holographic optogenetics and volumetric imaging. Neuron 94, 774–789 (2017).
Athalye, V. R., Carmena, J. M. & Costa, R. M. Neural reinforcement: re-entering and refining neural dynamics leading to desirable outcomes. Curr. Opin. Neurobiol. 60, 145–154 (2020).
Marshel, J. H. et al. Cortical layer-specific critical dynamics triggering perception. Science 365, eaaw5202 (2019).
Grosenick, L., Marshel, J. H. & Deisseroth, K. Closed-loop and activity-guided optogenetic control. Neuron 86, 106–139 (2015).
Zhang, Z., Russell, L. E., Packer, A. M., Gauld, O. M. & Häusser, M. Closed-loop all-optical interrogation of neural circuits in vivo. Nat. Methods 15, 1037–1040 (2018).
Pnevmatikakis, E. A. Analysis pipelines for calcium imaging data. Curr. Opin. Neurobiol. 55, 15–21 (2019).
Cai, C. et al. VolPy: automated and scalable analysis pipelines for voltage imaging datasets. PLoS Comput. Biol. 17, e1008806 (2021).
Pnevmatikakis, E. A. & Giovannucci, A. NoRMCorre: an online algorithm for piecewise rigid motion correction of calcium imaging data. J. Neurosci. Methods 291, 83–94 (2017).
Giovannucci, A. et al. OnACID: online analysis of calcium imaging data in real time. In Guyon, I. et al. (eds) Advances in Neural Information Processing Systems 30, 2381–2391 (Curran Associates, 2017). http://papers.nips.cc/paper/6832-onacid-online-analysis-of-calcium-imaging-data-in-real-time.pdf
Friedrich, J., Zhou, P. & Paninski, L. Fast online deconvolution of calcium imaging data. PLoS Comput. Biol. 13, e1005423 (2017).
Bao, Y., Soltanian-Zadeh, S., Farsiu, S. & Gong, Y. Segmentation of neurons from fluorescence calcium recordings beyond real time. Nat. Mach. Intell. 3, 590–600 (2021).
Friedrich, J., Giovannucci, A. & Pnevmatikakis, E. A. Online analysis of microendoscopic 1-photon calcium imaging data streams. PLoS Comput. Biol. 17, e1008565 (2021).
Giovannucci, A. et al. CaImAn: an open source tool for scalable calcium imaging data analysis. eLife 8, e38173 (2019).
Mitani, A. & Komiyama, T. Real-time processing of two-photon calcium imaging data including lateral motion artifact correction. Front. Neuroinform. 12, 98 (2018).
Chen, Z., Blair, G. J., Blair, H. T. & Cong, J. BLINK: bit-sparse LSTM inference kernel enabling efficient calcium trace extraction for neurofeedback devices. In Proceedings of the ACM/IEEE International Symposium on Low Power Electronics and Design, 217–222 (2020).
Yang, W., Carrillo-Reid, L., Bando, Y., Peterka, D. S. & Yuste, R. Simultaneous two-photon imaging and two-photon optogenetics of cortical circuits in three dimensions. Elife 7, e32671 (2018).
Pnevmatikakis, E. A. et al. Simultaneous denoising, deconvolution, and demixing of calcium imaging data. Neuron 89, 285–299 (2016).
Guizar-Sicairos, M., Thurman, S. T. & Fienup, J. R. Efficient subpixel image registration algorithms. Opt. Lett. 33, 156–158 (2008).
Abdou, I. E. Practical approach to the registration of multiple frames of video images. In Visual Communications and Image Processing’99, vol. 3653, 371–382 (International Society for Optics and Photonics, 1998).
Abadi, M. et al. TensorFlow: a system for large-scale machine learning. In 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), 265–283 (2016).
Lawson, C. L. & Hanson, R. J. Solving Least Squares Problems, vol. 15 (Siam, 1995).
Tseng, P. Approximation accuracy, gradient methods, and error bound for structured convex optimization. Math. Program. 125, 263–295 (2010).
Pachitariu, M. et al. Suite2p: beyond 10,000 neurons with standard two-photon microscopy. Preprint at bioRxiv https://doi.org/10.1101/061507 (2017).
Xie, M. E. et al. High-fidelity estimates of spikes and subthreshold waveforms from 1-photon voltage imaging in vivo. Cell Rep. 35, 108954 (2021).
Plitt, M. H. & Giocomo, L. M. Experience-dependent contextual codes in the hippocampus. Nat. Neurosci. 24, 705–714 (2021).
Zhu, F. et al. A deep learning framework for inference of single-trial neural population dynamics from calcium imaging with subframe temporal resolution. Nat. Neurosci. 25, 1724–1734 (2022).
Vladimirov, N. et al. Brain-wide circuit interrogation at the cellular level guided by online analysis of neuronal function. Nat. Methods 15, 1117–1125 (2018).
Che, S. et al. A performance study of general-purpose applications on graphics processors using CUDA. J. Parallel Distrib. Computing 68, 1370–1380 (2008).
Buchanan, E. K. et al. Penalized matrix decomposition for denoising, compression, and improved demixing of functional imaging data. Preprint at bioRxiv https://doi.org/10.1101/334706 (2019).
Deneux, T. et al. Accurate spike estimation from noisy calcium signals for ultrafast three-dimensional imaging of large neuronal populations in vivo. Nat. Commun. 7, 12190 (2016).
Liu, Z. et al. Sustained deep-tissue voltage recording using a fast indicator evolved for two-photon microscopy. Cell 185, 3408–3425 (2022).
Smith, J. O. Introduction to Digital Filters: With Audio Applications, vol. 2 (Julius Smith, 2007).
Rozsa, M., Singh, A. & Svoboda, K. Simultaneous Voltron (1.0) Imaging and Whole-cell Patch-clamp Recordings of Somatosensory Cortex Layer 1 Interneurons In Vivo (Janelia Research Campus, 2021). https://janelia.figshare.com/collections/Simultaneous_Voltron_1_0_imaging_and_whole-cell_patch-clamp_recordings_of_somatosensory_cortex_layer_1_interneurons_in_vivo/5325254/1
Kuhn, H. W. The Hungarian method for the assignment problem. Naval Res. Logistics Q 2, 83–97 (1955).
Koay, S. A., Charles, A. S., Thiberge, S. Y., Brody, C. D. & Tank, D. W. Sequential and efficient neural-population coding of complex task information. Neuron 110, 328–349 (2022).
Najafi, F., Giovannucci, A., Wang, S. S.-H. & Medina, J. F. Sensory-driven enhancement of calcium signals in individual Purkinje cell dendrites of awake mice. Cell Rep. 6, 792–798 (2014).
Walker, E. Y. et al. Inception loops discover what excites neurons most using deep predictive models. Nat. Neurosci. 22, 2060–2065 (2019).
Acknowledgements
The authors thank A. M. S. Ang from University of Mons for useful discussions and for the formalism of Supplementary Note 3. The authors also thank P. Gunn from the Flatiron Institute, Simons Foundation for valuable suggestions and help with the GPU experiments, K. Podgorski, K. Svoboda and A. Singh from Janelia Research Campus for the ground truth datasets, K. Podgorski for useful discussions, M. Xie and A. Cohen from Harvard University for useful discussions, and J. Tabet and W. Heffley from UNC for assistance with the manuscript. A.G. is supported by the Beckman Young Investigator award.
Author information
Authors and Affiliations
Contributions
C.C. and A.G. designed the study with input from C.D. and E.A.P. Data acquisition was done by M.R. for simultaneous voltage imaging and electrophysiology. C.C., C.D., J.F. and A.G. wrote the code and performed the data analysis. C.C., C.D. and A.G. wrote the manuscript, with feedback from J.F., E.A.P. and M.R.
Corresponding author
Ethics declarations
Competing interests
The authors declare no competing interests.
Peer review
Peer review information
Nature Methods thanks Philipp Berens and the other, anonymous, reviewer(s) for their contribution to the peer review of this work. Peer reviewer reports are available. Primary Handling Editor: Nina Vogt, in collaboration with the Nature Methods team.
Additional information
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Extended data
Supplementary information
Supplementary Information
Supplementary Notes 1–7, Supplementary Figs. 1–17 and Supplementary Tables 1–14.
Source data
Source Data Fig. 2
Result of motion correction.
Source Data Fig. 3
Result of source separation.
Source Data Fig. 4
Result of spike extraction for voltage imaging.
Source Data Fig. 5
Result of timing performance.
Source Data Fig. 6
Result of simulated experiments.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Cai, C., Dong, C., Friedrich, J. et al. FIOLA: an accelerated pipeline for fluorescence imaging online analysis. Nat Methods 20, 1417–1425 (2023). https://doi.org/10.1038/s41592-023-01964-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1038/s41592-023-01964-2