Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Volume 5 Issue 8, August 2023

Feedback states for robot motor learning

As deep reinforcement learning gains prominence in robot learning, understanding the importance of sensory feedback becomes crucial. Yu et al. quantitatively identify essential sensory feedback for effective learning of locomotion skills, enabling robust performance with minimal sensing dependencies and providing insights into the relationship between state observations and motor skills.

See Yu et al.

Image: Zhibin Li, University College London; Christopher McGreavy, CERN. Cover design: Thomas Phillips

Editorial

  • Machine learning and quantum computing approaches are converging, fuelling considerable excitement over quantum devices and their capabilities. However, given the current hardware limitations, it is important to push the technology forward while being realistic about what quantum computers can do, now and in the near future.

    Editorial

    Advertisement

Top of page ⤴

Correspondence

Top of page ⤴

Comment & Opinion

  • To protect the integrity of knowledge production, the training procedures of foundation models such as GPT-4 need to be made accessible to regulators and researchers. Foundation models must become open and public, and those are not the same thing.

    • Fabian Ferrari
    • José van Dijck
    • Antal van den Bosch
    Comment
  • Medical artificial intelligence needs governance to ensure safety and effectiveness, not just centrally (for example, by the US Food and Drug Administration) but also locally to account for differences in care, patients and system performance. Practical collaborative governance will enable health systems to carry out these challenging governance tasks, supported by central regulators.

    • W. Nicholson Price II
    • Mark Sendak
    • Karandeep Singh
    Comment
Top of page ⤴

Reviews

  • Limited interpretability and understanding of machine learning methods in healthcare hinder their clinical impact. Imrie et al. discuss five types of machine learning interpretability. They examine medical stakeholders, highlight how interpretability meets their needs and emphasize the role of tailored interpretability in linking machine learning advancements to clinical impact.

    • Fergus Imrie
    • Robert Davis
    • Mihaela van der Schaar
    Perspective
Top of page ⤴

Research

  • Algorithmic super-resolution in the context of fluorescence microscopy is challenging due to the difficulty to reliably represent biological nanostructures in synthetically generated images. Bouchard and colleagues propose a deep learning model for live-cell imaging that can leverage auxiliary microscopy imaging tasks to guide and enhance reconstruction, while preserving the biological features of interest.

    • Catherine Bouchard
    • Theresa Wiesner
    • Flavie Lavoie-Cardinal
    Article Open Access
  • There are currently promising developments in deep learning for protein design, with applications in drug discovery and synthetic biology. For more efficient exploration of the design space, Wang et al. demonstrate a reinforcement learning method, EvoZero, for directed evolution in protein engineering towards desired functional or structure-related properties.

    • Yi Wang
    • Hui Tang
    • Meng Yang
    Article
  • To ensure that a machine learning model has learned the intended features, it can be useful to have an explanation of why a specific output was given. Slack et al. have created a conversational environment, based on language models and feature importance, which can interactively explore explanations with questions asked in natural language.

    • Dylan Slack
    • Satyapriya Krishna
    • Sameer Singh
    Article Open Access
  • The tendency of machine learning algorithms to learn biases from training data calls for methods to mitigate unfairness before deployment to healthcare and other applications. Yang et al. propose a reinforcement-learning-based method for algorithmic bias mitigation and demonstrate it on COVID-19 screening and patient discharge prediction tasks.

    • Jenny Yang
    • Andrew A. S. Soltan
    • David A. Clifton
    Article Open Access
  • Microscopic imaging and holography aim to decrease reliance on labelled experimental training data, which can introduce biases, be time-consuming and costly to prepare, and may lack real-world diversity. Huang et al. develop a physics-driven self-supervised model that eliminates the need for labelled or experimental training data, demonstrating superior generalization on the reconstruction of experimental holograms of various samples.

    • Luzhe Huang
    • Hanlong Chen
    • Aydogan Ozcan
    Article Open Access
  • A challenging problem in deep learning consists in developing theoretical frameworks suitable to study generalization. Feng and colleagues uncover a duality relation between neuron activities and weights in deep learning neural networks, and use it to show that sharpness of the loss landscape and norm of the solution act together in determining its generalization performance.

    • Yu Feng
    • Wei Zhang
    • Yuhai Tu
    Article
  • Traditional feedback-state selection in robot learning is empirical and requires substantial engineering efforts. Yu et al. develop a quantitative and systematic state-importance analysis, revealing crucial feedback signals for learning locomotion skills.

    • Wanming Yu
    • Chuanyu Yang
    • Zhibin Li
    Article Open Access
  • Deep learning applied to live-cell images of patient-derived neurons aids predicting underlying mechanisms and gains insights into neurodegenerative diseases, facilitating the understanding of mechanistic heterogeneity. D’Sa and colleagues use patient-derived stem cell models, high-throughput imaging and machine learning algorithms to investigate Parkinson’s disease subtyping.

    • Karishma D’Sa
    • James R. Evans
    • Sonia Gandhi
    Article Open Access
Top of page ⤴

Amendments & Corrections

Top of page ⤴

Search

Quick links