AI & robotics briefing: How ill-informed AI use is fuelling a reproducibility crisis

Hello Nature readers, would you like to get this Briefing in your inbox free every week? Sign up here.

Critics question ‘self-driving lab’

Last month, scientists demonstrated the A-lab, an autonomous chemistry system that can synthesize new materials without human intervention. Critics have now raised concerns about its ability to analyse the results and have questioned whether it really made 41 new compounds. “To me, this doesn’t invalidate the A-Lab concept, but it shows that there is considerable work still to do on the autonomy aspects,” says chemist Andy Cooper. Two critics’ teams are now collaborating to check the paper’s claims in more detail.

Nature | 6 min read

Google’s AI outperforms GPT-4 — barely

Google has released an AI model called Gemini that it says outperforms GPT-4, the OpenAI large language model that powers ChatGPT. Gemini beat GPT-4 in 30 of 32 benchmarks tests — including reading comprehension, scientific knowledge, problem solving and code writing. Yet without insight into Gemini’s training data, it’s difficult to interpret these findings, says computer scientist Percy Liang. “It’s not obvious to me that Gemini is actually substantially more capable than GPT-4,” says AI researcher Melanie Mitchell. A viral promotional video that shows the AI responding in real time to video and audio prompts seems to partially misrepresent what the system can do.

MIT Technology Review | 8 min read & Ars Technica | 6 min read

Chatbots can jailbreak each other

AI systems can trick each other into giving up dangerous information, such as how to make methamphetamine, build a bomb or launder money. Built-in rules usually prevent chatbots from answering these questions. Researchers instructed several chatbots to take on the persona of a research assistant and help to develop prompts that could get other chatbots to break the rules. These ‘jailbreak’ attacks were successful 43% of the time against GPT-4, 61% of the time against Anthropic’s Claude 2 and 36% of the time against the open-source bot Vicuna.

Scientific American | 4 min read

Reference: arXiv preprint (not peer reviewed)

EU reaches landmark deal on AI laws

The European Union has agreed on one of the first comprehensive attempts to regulate AI use. Under the plan, AI applications would be categorized on the basis of their potential to harm people and society. Some applications would be banned, such as using images from the Internet to create a facial-recognition database. The deal is provisional and legislation will not take effect until 2025 at the earliest — a long time for AI development.

The New York Times | 5 min read

Algorithm sniffs out fake wine

A machine-learning tool can trace the origin of wine by analysing the drink’s complex chemical profile. The algorithm was trained and tested on the ‘symphony’ of chemical compounds in wines from France’s Bordeaux region, and it can pinpoint the estate on which each bottle was produced with 99% accuracy. The approach could help the wine industry to authenticate products. “There’s a lot of wine fraud around with people making up some crap in their garage, printing off labels, and selling it for thousands of dollars,” says neuroscientist and study co-author Alexandre Pouget.

The Guardian | 4 min read

Reference: Communications Chemistry paper

Features & opinion

Ill-informed AI use fuels irreproducibility

The naive use of AI is driving a deluge of unreliable, useless or wrong research. This happens, for example, when researchers report that algorithms can reliably classify images or even diagnose diseases, but fail to realize that their systems are really only regurgitating artefacts in the training data. “AI provides a tool that allows researchers to ‘play’ with the data and parameters until the results are aligned with the expectations,” says computer scientist Lior Shamir. There are checklists that can help scientists to avoid common problems, such as insufficient separation between training and test data. Many researchers argue that the way forward is to make all code and data available for public scrutiny.

Nature | 14 min read

No women on list of AI pioneers

“I imagined millions of eye rolls happening at once,” says AI reporter Sharon Goldman about the fact that The New York Times’s list of 12 AI pioneers fails to include any women. For example, computer scientist Fei-Fei Li, who created a huge image data set that enabled advances in computer vision, is not on the list. This is “really just a glaring symptom of a larger ‘where’s the women’ problem”, Goldman writes. “Aren’t we all tired of it?”

VentureBeat | 3 min read

Cyborg cockroaches to the rescue

Cockroaches controlled by on-board computers could search for earthquake survivors; sensor-carrying jellyfish could monitor the ocean for the effects of climate change. Biohybrid robots such as these allow engineers to harness organisms’ natural capabilities — being able to fly or swim, for example. And animals can usually stay on the move for much longer than battery-powered machines. The biggest issue is scaling up production, because most biohybrids are essentially handmade. “It’s arts and crafts, it’s not engineering,” says biomedical engineer Kit Parker. “You’ve got to have design tools. Otherwise, these are just party tricks.”

Nature | 13 min read

This article is part of Nature Outlook: Robotics and artificial intelligence, an editorially independent supplement produced with financial support from FII Institute.