Is ChatGPT accurate: what OpenAI and the studies measured

There is no single accuracy figure

The figure a reader might look for, the share of ChatGPT answers that are right, is not on the pages we read. OpenAI’s Help Center page on whether ChatGPT tells the truth gives no percentage at all: it says ChatGPT can produce incorrect or misleading outputs and that it might sound confident even when it is wrong. OpenAI’s research page on hallucinations says accuracy will not reach 100%, and the reason it gives is that some real-world questions cannot be answered, because information is unavailable, a small model has limits or a question is ambiguous. The same page says that on simple tests such as SimpleQA some models come close to 100% accuracy, so a number on its own depends on the test.

What the pages do give are percentages for particular jobs, and the table below sets five of them side by side. A study of news answers counts the answers with a significant problem. OpenAI’s table counts how often two models, gpt-5-thinking-mini and OpenAI o4-mini, were right, wrong or declined to answer on questions with one correct answer. A review of clinical studies reports a range. Each is a figure for one task, not a rate for ChatGPT in general, and the pages we read do not combine them into one.

The EBU release does not give a figure for ChatGPT alone: its 45% covers ChatGPT, Copilot, Gemini and Perplexity together, so it cannot be read as a ChatGPT rate.

Versions matter as well. The clinical review tested ChatGPT 3.0, 3.5 and 4, and its authors write that different versions have shown different levels of accuracy. OpenAI’s research page, dated 5 September 2025, says GPT-5 has significantly fewer hallucinations, especially when reasoning, but that they still occur.

What OpenAI says about its own answers

OpenAI’s Help Center page, titled Does ChatGPT tell the truth?, opens by saying ChatGPT can be helpful but is not always right. It describes a language model that works from patterns in the data it was trained on and can produce incorrect or misleading outputs. It calls the phenomenon a hallucination and lists three examples: incorrect definitions, dates or facts; fabricated quotes, studies, citations or references to sources that do not exist; and overconfident answers to ambiguous or complex questions. OpenAI says that is why it encourages users to approach ChatGPT critically and to verify important information from reliable sources.

The page then lists four limits of its latest models:

• Knowledge cutoff. The models are trained on data up to a certain point, and responses do not include events beyond it unless tools are used.
• Confidence is not reliability. The model may express high confidence even in an incorrect answer.
• Lack of access. The model may not be able to get information from a website because of technical issues, paywalls or preferences set through robots.txt.
• Bias and over-simplification. In some cases it may present a single perspective as absolute truth, oversimplify complex issues, or misrepresent the weight of scientific consensus or social debate.

A short paragraph covers a related question, whether ChatGPT can tell if a text was written by AI, including by itself. OpenAI says it cannot reliably determine either, that it may make up answers to questions such as whether it wrote an essay, and that those answers are not reliable evidence of authorship. Our page on how accurate AI detectors are looks at that other kind of accuracy.

Why it sounds sure when it is wrong

OpenAI’s research page of 5 September 2025, Why language models hallucinate, defines hallucinations as plausible but false statements generated by language models, and says they can appear even for seemingly straightforward questions. Its example: the researchers asked a chatbot in common use for the title of the PhD dissertation of Adam Tauman Kalai, an author of the paper, and it confidently produced three different answers, and every one was wrong. Asked for his birthday, it gave three different dates, likewise all wrong. The page does not name the chatbot; separately, in its introduction, it says that ChatGPT also hallucinates.

The argument is about incentives. OpenAI compares a model to a student on a multiple-choice test: a wild guess might turn out right, while leaving the answer blank guarantees a zero. If a model is asked for someone’s birthday and guesses a date, it has a 1-in-365 chance of being right, while saying it does not know earns nothing. When models are graded only on accuracy, the percentage of questions answered exactly right, OpenAI argues they are encouraged to guess, and over thousands of test questions a guessing model ends up looking better on scoreboards than a careful one that admits uncertainty.

To show it, OpenAI reproduces a SimpleQA table from the GPT-5 system card for two models, drawn in the figure below. gpt-5-thinking-mini declined to answer 52% of the time, was right 22% of the time and wrong 26%. OpenAI o4-mini declined 1%, was right 24% and wrong 75%. OpenAI’s reading is that the older o4-mini is slightly better on accuracy but has a significantly higher error rate. It is a table for two models on questions with a single right answer, not a rate for ChatGPT as a whole.

Where the mistakes come from, and what OpenAI proposes

On the origin of the mistakes, OpenAI points to how models first learn: by predicting the next word in huge amounts of text, with no true or false labels attached to statements. Spelling mistakes and mismatched parentheses follow consistent patterns, it says, so errors of that kind disappear with scale. Arbitrary low-frequency facts, like a pet’s birthday, cannot be predicted from patterns alone and so lead to hallucinations.

The fix it proposes is to penalize confident errors more than uncertainty and to give partial credit for appropriate expressions of uncertainty. The page also answers two common ideas. To the idea that accuracy will reach 100%, it says some real-world questions are inherently unanswerable. To the idea that hallucinations are inevitable, it says they are not, because language models can abstain when uncertain. It adds that being calibrated takes much less computation than being accurate, and that its latest models have lower hallucination rates.

What the studies measured

The widest study on the pages we read is about news. It was coordinated by the European Broadcasting Union (EBU) and led by the BBC. According to the EBU release of 22 October 2025, 22 public service media organizations in 18 countries, working in 14 languages, took part, and professional journalists evaluated more than 3,000 responses from ChatGPT, Copilot, Gemini and Perplexity on accuracy, sourcing, separating opinion from fact, and context. Al Jazeera, reporting the same study, says more than 2,700 responses. The two pages differ on the count, and we use the EBU figure because it comes from the organization that coordinated the study.

The results as the EBU gives them:

• 45% of all answers had at least one significant issue.
• 31% showed serious sourcing problems: missing, misleading or incorrect attributions.
• 20% contained major accuracy issues, including hallucinated details and outdated information.
• Gemini did worst, with significant issues in 76% of responses, more than double the other assistants, largely because of poor sourcing.

Al Jazeera repeats the first three figures and adds that 14% of responses lacked appropriate context. These are percentages for four assistants taken together. The EBU release does not give a figure for ChatGPT alone, and neither does the report page we read. Al Jazeera’s article opens by saying that AI models such as ChatGPT routinely misrepresent news events, and it says all the models studied made basic factual errors, among them ChatGPT naming Pope Francis as the sitting pontiff months after his death. It also says OpenAI, Google, Microsoft and Perplexity did not immediately respond to requests for comment.

The EBU says a comparison with the BBC’s results from earlier in 2025 shows some improvements but still high levels of errors. Its report page says the problems appear no matter which language, territory or AI platform is tested. The release also cites the Reuters Institute Digital News Report 2025: 7% of online news consumers use AI assistants to get their news, rising to 15% of under-25s.

The other body of evidence on the pages we read is clinical. A literature review in Adv Health Inf Sci Pract, published on 18 April 2025, searched PubMed, found 122 unique references and included nine studies. It reports that clinical accuracy ranged from 20% to 95%, and it concludes that this variable accuracy makes ChatGPT unsuitable as a stand-alone point-of-care product. It also says ChatGPT can answer frequently asked questions, offer recommendations and categorize symptoms in less complicated scenarios.

The studies inside the review show how wide the range is. In a hip-replacement study, ChatGPT 3.5 answered 57 of 60 questions appropriately, or 95%, against 93.3% for arthroplasty-trained nurses; 79% of patients were uncertain whether they trusted AI, and 69.4% were more comfortable following the ChatGPT responses than the nurses’. In another hip study, 70% of responses were rated as needing moderate clarification and 20% as excellent. In a gynecologic oncology study of 64 questions, 34 answers (53.1%) were rated correct and comprehensive, while prevention and survivorship questions were answered at 91.7%; the review notes lower accuracy for treatment and diagnosis questions in the same study.

The review’s authors note that the studies used different methods, a narrow range of questions and subjective evaluation. The versions tested were 3.0, 3.5 and 4, while OpenAI’s research page of 2025 refers to GPT-5.

Checking an answer, as OpenAI describes it

OpenAI’s Help Center page gives the checks. It says to use ChatGPT as a first draft, not a final source, to check important information, and to always verify quotes, data, technical information or references to external documents. When accuracy matters, it says to use available tools like search or deep research, to check sources, and to visit the links directly.

The page names three tools that, depending on your plan, may help ChatGPT answer with more up-to-date or verifiable information: Search, Code interpreter / Data analysis and Deep research. The figure below puts OpenAI’s one-line descriptions side by side. The page does not say which plan has which tool.

Without search, OpenAI says, responses are based on what the model learned during training. With search or deep research, ChatGPT can access and cite real-time web sources. To check whether search is on, look at whether the search or deep research icons are enabled in the input bar, or whether the response cites sources from the web. The page adds, under a question about internet access and starting with “Generally, yes”, that these tools are enabled for all models by default; it does not reconcile this with its earlier “depending on your plan”.

Chatbase, a company that sells AI agents, reports two OpenAI comparisons that we did not check at an OpenAI page. One is that with web search on, GPT-5 answers were about 45% less likely to contain a factual error than GPT-4o answers. The other is that in OpenAI’s test of hard medical, legal and financial prompts, answers with at least one factual error were 68% less common with GPT-5.6 Sol than with GPT-5.5 Instant. Chatbase itself says OpenAI cautions that the second test measures difficult cases, and that benchmark scores are not a universal ChatGPT accuracy rate.

What we did not use

We tested nothing in ChatGPT ourselves; every statement here comes from the pages named at the foot of this page, read on 30 October 2026. We read the EBU release and the report’s web page, not the report’s PDF. We did not read community discussions, a Medical News Today article or a ScienceDirect review on the same question. We left out a Scribbr page from 2023 because it says ChatGPT was trained on data going up to 2021, and Chatbase is used only for what it reports from OpenAI. Figures that come from pages we did not open are not used.