Artificial intelligence (AI) may have more to offer science than accurate predictions. The patterns it learns could point researchers towards new, testable explanations of how complex systems function.
A new Perspective co-authored by Assistant Professor Gianmarco Mengaldo from the Department of Mechanical Engineering under the College of Design and Engineering at the National University of Singapore (NUS CDE), sets out how explainable AI, or XAI, could bring those patterns to light. In particular, researchers would identify what drove an AI prediction, turn that clue into a hypothesis and test it through experiments, simulations or established scientific principles. Only then could it support a conclusion about the real world.
This process could unearth relationships buried in complex data, guide engineering design and help assess AI used in safety-critical fields such as healthcare, aviation, energy and infrastructure. The Perspective connects existing approaches into a proposed framework rather than reporting a newly tested system.
Published in Nature Communications on 6 August 2026, the work was carried out in collaboration with Associate Professor Ricardo Vinuesa from the University of Michigan and Professor Steve Brunton from the University of Washington.
A prediction is not an explanation
A deep-learning model, a type of AI, learns by analysing many examples. It can use the underlying patterns to make accurate predictions without presenting them as a readable explanation or scientific equation. Researchers may therefore know that a model works without knowing which information led to its answer.
XAI refers to methods for tracing that information. For example, in a model predicting aircraft drag, it might pinpoint the regions of airflow that most influenced the answer. It might also express a pattern learnt by AI as a simpler equation for scientists to examine.
“AI may tease out relationships that humans have overlooked, but an explanation of the model is only a lead,” said Asst Prof Mengaldo. “It tells us where to investigate. Experiments, simulations and established science must then go on to find out whether the relationship exists in the physical world.”
The model might rely on an airflow pattern because it reflects an important physical process or because that pattern frequently appeared in the training data. XAI can reveal this reliance, but further testing must distinguish between the two possibilities.
From an AI clue to a tested insight
A recent example from research in turbulence demonstrates how this approach could work. XAI was used to identify patterns in a turbulent airflow that had the greatest influence on an AI model’s predictions. Researchers then trained another AI system to modify those patterns, resulting in a strategy that reduced drag more effectively than one trained to reduce drag directly.
The example illustrates the potential value of looking inside a model. Instead of regarding drag as a single outcome to minimise, researchers found a more precise feature within the airflow to target. The explanation therefore helped turn a prediction into a more effective control strategy.
This sequence connects the Perspective’s three areas. Explainability supports discovery by revealing patterns that may point to how a system functions. Once tested, that knowledge can support optimisation by improving a design or control strategy. The record of what influenced the AI can also help assess whether it is reliable enough for use through certification.
Putting explanations to the test before trusting them
An earlier study published in Nature Machine Intelligence and co-authored by Asst Prof Mengaldo shows why AI explanations need testing. In the study, several tools were used to explain how the same AI model analysed an electrocardiogram, a recording of the heart’s electrical activity. The tools highlighted different parts of the signal despite examining the same prediction.
This earlier study supports a central argument of the new Perspective. Before using an AI explanation to form a hypothesis or make a safety decision, researchers must establish that it accurately reflects what influenced the model. The team developed numerical tests for doing so instead of judging whether an explanation merely looked plausible.
The Perspective extends this principle to certification, meaning the checks that may be needed before AI is approved for high-stakes application. Evaluators would consider its accuracy on past cases alongside evidence showing which information influenced its answers, whether it relied on different information under unfamiliar conditions and why it might fail. These explanations could strengthen existing safety and regulatory assessments, but importantly, would not replace them.
The need for such evidence is particularly pertinent in applications such as autonomous vehicles and AI-guided robots. If a system takes an unexpected action, investigators would need to reconstruct what information influenced that decision. This could help improve safety and inform questions of regulatory responsibility and legal liability.
“As AI systems increasingly match or outperform traditional methods in complex scientific and engineering tasks, can we extract new scientific understanding from what they have learnt?” said Asst Prof Mengaldo. “We believe so, but only if the explanations reflect what the model has learnt, and if they are treated as hypotheses to test rather than facts to accept.”
Asst Prof Mengaldo and his team are now exploring applications in healthcare, AI-guided robotics, weather and climate science. “A meaningful milestone would be to identify a previously unknown mechanism behind an important scientific problem, then demonstrate through independent testing that it represents a genuine causal process in the real world instead of merely a statistical pattern in the data,” he added.
