Artificial intelligence models can correctly answer most basic climate science questions, but their reliability weakens when users challenge their responses, sometimes leading them to endorse claims that contradict established scientific evidence.
A new report by the United Nations Educational, Scientific and Cultural Organization (UNESCO) and University College London warns that leading AI systems can abandon scientifically established facts simply to agree with users, raising concerns about their growing role as sources of information on climate change.
Presented on Monday at the COP31 climate conference in Antalya, Türkiye, the report, Can We Trust AI on Climate?, assessed up to 33 leading AI models across 13 languages.
Researchers found that although the systems performed well on straightforward climate science questions, they were vulnerable to what is known as sycophancy, a tendency to agree with users even when they introduce incorrect information.
In experiments involving 15 AI models, researchers challenged answers that the systems had initially provided correctly. The challenges ranged from simple expressions of disagreement to claims of expertise and fabricated evidence.
Every model tested eventually changed its position. In approximately eight out of 10 cases where answers changed, the reversal happened at the first sign of disagreement.
Some models could even be persuaded to endorse false claims, including the suggestion that carbon dioxide is not the main driver of climate change.
Lead author Leona Verdadero of UNESCO said the findings exposed a weakness in systems that often appear authoritative and knowledgeable to users.
Speaking to UN News, she explained that a simple statement questioning an answer could be enough to make a model reverse its position.
The findings raise questions about how people use AI-generated information, particularly when researching climate change, where scientific evidence informs public policy, national commitments and international negotiations.
The report also found that AI performance declined as questions became more complex.
On textbook-style climate questions, the strongest models achieved accuracy rates of between 88 and 96 per cent. However, when asked to assess whether evidence supported or contradicted a scientific claim, their performance dropped to between 71 and 77 per cent.
Their ability to reason through the combined effects of climate policies was weaker still. When asked whether two policies implemented together would produce greater, smaller or equivalent effects compared with their individual outcomes, the models recorded an average accuracy of just 33.8 per cent.
This suggests that answering factual questions is not the same as reliably interpreting scientific evidence or evaluating the likely impact of policy decisions.
The report also highlights a language divide that could leave some climate-vulnerable communities with less reliable AI-generated information.
While models achieved an average accuracy of 88.6 per cent in English, performance was lower in several other languages. Hindi and Bengali recorded 83.6 per cent, Urdu 81.4 per cent and Swahili 78.3 per cent.
The Swahili finding is particularly significant for East Africa, where the language is widely used to communicate information across communities.
Verdadero noted that two people asking the same question in different languages could receive substantially different responses from the same AI model.
She warned that this disparity matters because many speakers of underrepresented languages live in South Asia and East Africa, regions with communities highly exposed to climate change.
The researchers are urging technology companies to invest in locally relevant datasets and multilingual AI development to improve the quality of information available beyond dominant languages.
The findings come as AI increasingly features in discussions about climate communication, public awareness and access to scientific knowledge.
UNESCO’s report forms part of its contribution to the Global Initiative for Information Integrity on Climate Change, which it co-chairs with the United Nations and Brazil.
The initiative seeks to strengthen access to reliable information as misinformation and misleading claims threaten public understanding of climate change.
Verdadero said climate action must be grounded in credible scientific evidence, particularly because research informs government policies and negotiations at international climate conferences.
She cautioned against treating AI systems as the primary source of climate information, urging users to verify responses and rely on qualified scientists, researchers and credible sources.
The central message is that AI can help people access climate information, but its answers should not replace scientific expertise or independent fact-checking.


