Categories
Reading time
People rely on AI chatbots to answer questions about every aspect of their lives, from recipes and entertainment to current affairs and health advice. But when can they trust the responses? Are some chatbots more reliable than others? Or does it depend on the nature of the question? These are some of the issues we're addressing in our Public Interest AI Trust Benchmark.
As chatbots and the underlying large language models (LLMs) that drive them proliferate, there has also been a growth in the range of proposed methods to evaluate and compare them. This blog post is not a comprehensive review of such benchmarks but is a selective walk through some of the key features they have. We focus on evaluating the quality of the output, leaving aside important aspects such as training data, cost and model size.
One important aspect of any LLM benchmark is how the responses are analysed. While human annotations can produce high-quality trustworthy scores, the effort required is a significant bottleneck, limiting scale and repeatability. At the other extreme, asking a second LLM to judge responses risks penalising a test LLM because the judge has misinterpreted its answer. So let's look at these and other options.
Option 1: Limit to simple answers
If the benchmark questions are designed so that the correct answer is limited to a name, a date, a number or some similarly brief response, then evaluation becomes simple: just check if the target answer is included in the test chatbot output.
One example is AA-Omniscience [1], a benchmark based on 6000 challenging questions drawn from authoritative sources such as academic journals and industry-specific publications. Interestingly, this benchmark also tells the LLMs being tested "it is better to abstain from answering rather than getting the wrong answer." Incorrect answers are penalised but abstentions are not.
One drawback of this approach is of course that typical chatbot responses are hundreds of words long and sometimes thousands, so evaluating their quality based on the presence of just one or two words may be incomplete. Furthermore, it is relatively easy for this kind of dataset to leak out and end up in the training data of an LLM, meaning that the chatbot has already seen (and likely memorised) the answers to the test.
Option 2: Direct manual review
One approach to benchmarking is to draw up a list of questions and trusted answers and then manually review the chatbots' attempts at answering each question. While this inevitably limits the scale of the evaluation, it can produce high-quality insights.
For example, TruthfulQA [2] contains more than 800 questions that were deliberately chosen because people often get confused by related misinformation that is widespread online. The (human) evaluators follow a script to encourage systematic scoring and give each chatbot response a label such as "mostly true", "sometimes true" or "subjective".
NewsGuard takes a similar approach in its quarterly audit of popular AI chatbots. It selects 10 contemporary stories that are known to be controversial and associated with misinformation, then poses several related questions to each of 10 of the most popular chatbots. Researchers then manually review the resulting responses and identify trends and estimate how many of the models reproduce the misinformation and how many correct it.
Option 3: LLM-as-a-judge
As LLMs become increasingly capable, it's reasonable to ask if they can do the evaluation themselves. The "LLM-as-a-judge" method involves asking one LLM to compare the output of the chatbot being judged with a model answer, often written by a subject matter expert or at least carefully compiled from trusted sources. Further details and examples can be found in this recent survey [3].
This approach replaces human judgement with a machine, albeit a powerful model that can achieve high accuracy. However, it does risk evaluating one black box by using another and it's not always clear how reliable the evaluation metrics are. The LLM-as-a-judge is also provided with carefully curated reference answers and producing these requires considerable human effort and expertise, which is not always available or easy to reproduce.
Option 4: Extract and compare atomic facts
Rather than trying to mark a whole response as good or bad, some methods attempt a finer-grained analysis, such as FActScore [4]. Given any piece of text, such as the output of a chatbot, the method breaks it down into "atomic facts", each of which expresses a single piece of information. It then computes the percentage of these atomic facts that are supported by a trusted (even if imperfect) knowledge base, such as Wikipedia. To do this, it searches for information relevant to the atomic fact from the knowledge base and ask an "LLM Evaluator" to decide if the atomic fact is supported or not. So this is a form of LLM-as-a-judge, but one that makes the task more explicit by extracting all the facts directly. However, even the most trustworthy sources do contain factual errors and ambiguities, making evaluation less certain; and even if the correct information is provided by the trusted source, it may not be retrieved correctly by the search tool.
A similar approach from Google DeepMind and colleagues is called SAFE [5], which searches the whole depth and breadth of the internet (rather than just Wikipedia) for information that supports or contradicts each atomic fact to evaluate the model. SAFE is fully automatic, and can therefore scale up to many thousands of questions, but it does risk rewarding models that reproduce misleading information found online. And so because there is no manual review of the scoring process, models producing misleading output could still receive a high score without anyone knowing.
More recent work studied how ordinary people might search for information on government websites and how well chatbots can answer their queries [6]. The CitizenQuery project generated 22,000 queries whose answers could all be found on the official UK government website, gov.uk. This extended the FactScore approach described above to also capture how often models refused to answer questions and how verbose the answers were. By focussing on a single trusted site, this evaluation method can largely ignore questions of source quality.
Option 5: Atomic facts with manual review
As we started to develop our benchmarking process at Full Fact, we wanted to make the best use of our in-house fact checkers as well as our experienced AI team. We therefore developed a hybrid system, seeking the best of both worlds. We rely on automated API access to get the chatbot responses and automated tools to extract and group similar atomic facts. At the same time, we rely on expert human annotators to draft reference answers, to verify the grouping of atomic facts, and to assign appropriate marks to each group.
We thus combine the high scalability of an automated atomic facts approach with the high quality that comes with manual annotation. More details of our approach can be found in the companion methodology post.
Conclusion
Evaluation of chatbots remains an important problem with no clear answer. Purely manual approaches risk subjectivity and variance between individual annotators. Even when this is mitigated, they require substantial expert human effort and so cannot operate at a large scale. Purely automated approaches can scale up massively, limited only by the cost of computation. But they risk conflating errors from the models being tested with errors from the model judges.
The Full Fact AI Trust Benchmark attempts to balance these extremes by combining the automatic extraction, grouping and comparison of atomic facts with close human oversight provided by expert fact checkers.
References
[1] Jackson, D.I., Keating, W.E., Cameron, G. and Hill-Smith, M. (2025) "AA-Omniscience: Evaluating Cross-Domain Knowledge Reliability in Large Language Models." arXiv abs/2511.13029
[2] Lin, S., Hilton, J. and Evans, O. (2021) "TruthfulQA: Measuring How Models Mimic Human Falsehoods." arXiv abs/2109.07958
[3] Gu, J., Jiang, X., Shi, Z., Tan, H., Zhai, X., Xu, C., Li, W., Shen, Y., Ma, S., Liu, H. and Wang, S. (2026). "A survey on LLM-as-a-judge." The Innovation, 7(6).
[4] Min, S., Krishna, K., Lyu, X., Lewis, M., Yih, W., Koh, P. W., Iyyer, M., Zettlemoyer, L., and Hajishirzi, H. (2023). "FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation." arXiv abs/2305.14251
[5] Wei, J., Yang, C., Song, X., Lu, Y., Hu, N., Huang, J., Tran, D., Peng, D., Liu, R., Huang, D., Du, C., and Le, Q. V. (2024). "Long-form factuality in large language models." arXiv abs/2403.18802
[6] Majithia, N., Shinde, R., Chapman, Z., Trital, P., Decker, J., Maskey, M., Simperl, E., and Shadbolt, N. (2026). "The CitizenQuery Benchmark: A Novel Dataset and Evaluation Pipeline for Measuring LLM Performance in Citizen Query Tasks." arXiv abs/2602.04064