Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

Volume 5 Issue 3, March 2023

Pathways for small changes in large language models

Large language models have, as their name implies, a large number of parameters: over 175 million for example for GPT-3. An analysis by Ding et al. in this issue explores how changing only a few parameters can bring a model onto a new path (as conceptually visualized in the cover image) to fine-tune them for new tasks.

See Ding et al.

Image: Ruiqi Shao, Beijing ModelBest Technology Co., Ltd. Cover design: Thomas Phillips

Editorial

  • In the next phase of space exploration, human crews will be sent on missions beyond the low Earth orbit. Artificial intelligence (AI) is expected to play a main role in autonomous biomonitoring, research and Earth-independent healthcare.

    Editorial

    Advertisement

Top of page ⤴

Comment & Opinion

  • We explore the intersection between algorithms and the State from the perspectives of legislative action, public perception and the use of AI in public administration. Taking India as a case study, we discuss the potential fallout from the absence of rigorous scholarship on such questions for countries in the Global South.

    • Nandana Sengupta
    • Vidya Subramanian
    • Arul George Scaria
    Comment
Top of page ⤴

Reviews

  • An increasing number of regulations demand transparency in automated decision-making processes such as in automated online recruitment. To provide meaningful transparency, Sloane et al. propose the use of ‘nutritional’ labels that display specific information about an automated decision system, depending on the context.

    • Mona Sloane
    • Ian René Solano-Kamaiko
    • Julia Stoyanovich
    Perspective
  • Deep-space exploration missions require new technologies that can support astronaut health systems as well as biological monitoring and research systems that can function independently from Earth-based mission control centres. A NASA workshop explored how artificial intelligence advances could help address these challenges and, in this first of two Review articles based on the findings from the workshop, a vision for autonomous biomonitoring and precision space health is discussed.

    • Ryan T. Scott
    • Lauren M. Sanders
    • Sylvain V. Costes
    Review Article
  • Deep space exploration missions will require new technologies that can support astronaut health systems, as well as biological monitoring and research systems that can function independently from Earth-based mission control centres. A NASA workshop explored how artificial intelligence advances could help address these challenges and, in this second of two Review articles based on the findings from the workshop, the intersection between artificial intelligence and space biology is discussed.

    • Lauren M. Sanders
    • Ryan T. Scott
    • Sylvain V. Costes
    Review Article
Top of page ⤴

Research

  • Training a deep neural network can be costly but training time is reduced when a pre-trained network can be adapted to different use cases. Ideally, only a small number of parameters needs to be changed in this process of fine-tuning, which can then be more easily distributed. In this Analysis, different methods of fine-tuning with only a small number of parameters are compared on a large set of natural language processing tasks.

    • Ning Ding
    • Yujia Qin
    • Maosong Sun
    Analysis Open Access
  • Machine learning methods can predict and recognize binding patterns between T-cell receptors and human antigens, but they struggle with antigens for which no or little data exist regarding interactions with the immune system. A new method called PanPep based on meta-learning can learn quickly on new binding prediction tasks and accurately predicts pairing between T-cell receptors and new antigens.

    • Yicheng Gao
    • Yuli Gao
    • Qi Liu
    Article
  • Various post-hoc interpretability methods exist to evaluate the results of machine learning classification and prediction tasks. To better understand the performance and reliability of such methods, which is particularly necessary in high-risk applications, Turbe et al. have developed a framework for quantitative comparison of post-hoc interpretability approaches in time-series classification.

    • Hugues Turbé
    • Mina Bjelogrlic
    • Gianmarco Mengaldo
    Article Open Access
  • Developing proprioception systems for flexible structures such as soft robots is a challenge. Hu et al. report a stretchable e-skin for soft robot proprioception. Combined with deep learning, the e-skin enables high-resolution 3D geometry reconstruction of the soft robot and can be applied in many scenarios, such as human–robot interaction.

    • Delin Hu
    • Francesco Giorgio-Serchi
    • Yunjie Yang
    Article
  • High-quality annotation of datasets is critical for machine-learning-based biomedical image analysis. However, a detailed examination of recent image competitions reveals a gap between annotators’ needs and quality of labelling instructions. It is also found that annotator performance can be substantially improved by providing exemplary images.

    • Tim Rädsch
    • Annika Reinke
    • Lena Maier-Hein
    Article Open Access
  • Computational models can help predict metabolic profiles of microbial communities such as human gut microbiomes or environmental microbiomes, but they lack generalizability and interpretability. To address this challenge, Wang et al. report a deep learning approach for metabolic profile prediction called mNODE that incorporates a neural network module with hidden layers described by ordinary differential equations.

    • Tong Wang
    • Xu-Wen Wang
    • Yang-Yu Liu
    Article
  • Simulated data is an alternative to real data for medical applications where interventional data are needed to train AI-based systems. Gao and colleagues develop a model transfer paradigm to train deep networks on synthetic X-ray data and corresponding labels generated using simulation techniques from CT scans. The approach establishes synthetic data as a viable resource for developing machine learning models that apply to real clinical data.

    • Cong Gao
    • Benjamin D. Killeen
    • Mathias Unberath
    Article
  • Metal–organic frameworks are of high interest for a range of energy and environmental applications due to their stable gas storage properties. A new machine learning approach based on a pre-trained multi-modal transformer can be fine-tuned with small datasets to predict structure-property relationships and design new metal-organic frameworks for a range of specific tasks.

    • Yeonghun Kang
    • Hyunsoo Park
    • Jihan Kim
    Article
  • Explanatory interactive machine learning methods have been developed to facilitate the learning process between the machine and the user. Friedrich et al. provide a unification of various explanatory interactive machine learning methods into a single typology, and present benchmarks for evaluating such methods.

    • Felix Friedrich
    • Wolfgang Stammer
    • Kristian Kersting
    Article
Top of page ⤴

Search

Quick links