ChatGPT hallucinations: three kinds, and a check for each

What OpenAI and a library mean by a hallucination

OpenAI’s research page Why language models hallucinate, dated 5 September 2025, defines hallucinations as plausible but false statements generated by language models, and says it means a model confidently producing an answer that is not true. It adds that ChatGPT also hallucinates: GPT-5 has significantly fewer, especially when reasoning, but they still occur. Its example is a question put to a chatbot in common use about the PhD dissertation of Adam Tauman Kalai, an author of the paper. The chatbot gave three different titles, none of them correct, and for the same author’s birthday it gave three different dates, all wrong.

The University of Arizona Libraries put it in similar words: a generative AI model produces inaccurate or invented information and presents it as if it were true. Its examples are incorrect facts, fabricated quotations, and citations to sources that do not exist.

OpenAI’s Help Center page Does ChatGPT tell the truth? goes a step further and sorts the problem into three examples. We use them as the three kinds this page follows:

• Wrong facts. Incorrect definitions, dates or facts.
• Invented sources. Fabricated quotes, studies, citations or references to non-existent sources.
• Overconfident answers. Answers to ambiguous or complex questions that sound surer than they should.

The Help Center page shows only “Updated: 14 days ago” when we read it, so we cannot give its date. It also says that ChatGPT can sound confident even when it is wrong, and that people should approach it critically and verify important information from reliable sources.

Why they happen: two explanations, kept apart

OpenAI’s research page gives two reasons. The first is about how models are scored. OpenAI says most evaluations measure performance in a way that encourages guessing, and that accuracy-only scoreboards dominate; accuracy is the share of questions answered exactly right, so a model that guesses looks better than one that says it does not know. OpenAI compares it to a multiple-choice test: a wild guess might be right, while a blank answer guarantees a zero. If a model is asked for someone’s birthday and guesses “September 10”, it has a 1-in-365 chance of being right.

To show the effect, OpenAI reproduces a SimpleQA table from the GPT-5 System Card, drawn in the figure below (our page Is ChatGPT accurate? reads the same table next to other accuracy figures; here it serves only to explain why guessing is rewarded). gpt-5-thinking-mini declined to answer 52% of the time, was right 22% and wrong 26%. OpenAI o4-mini declined 1%, was right 24% and wrong 75%. OpenAI’s reading is that o4-mini is slightly better on accuracy but has a much higher error rate, and that errors are worse than abstentions. The figures are for two models on one test, not for everyday use of ChatGPT.

The second reason is about how models first learn. Pretraining predicts the next word in huge amounts of text, with no true or false labels on each statement. OpenAI says spelling and parentheses follow consistent patterns, so errors there fade with scale, while arbitrary low-frequency facts, like a pet’s birthday, cannot be predicted from patterns alone. The page also says hallucinations are not inevitable, because a model can abstain when uncertain, and that accuracy will not reach 100%, since some real-world questions cannot be answered.

OpenAI’s announcement of ChatGPT in 2022 already said the system sometimes writes plausible-sounding but incorrect or nonsensical answers. It named three reasons the problem is hard: during reinforcement learning training there is, as OpenAI’s 2022 page puts it, currently no source of truth, training the model to be more cautious makes it decline questions it could answer, and supervised training misleads it because the ideal answer depends on what the model knows.

The SUNY Potsdam guide offers a different explanation. It says OpenAI copied the free parts of the internet to train ChatGPT but could not get behind paywalls, including many academic databases, and that the model invents articles because it lacks that material. The OpenAI pages we read do not give this explanation. The Help Center mentions paywalls only in its list of limits, as a reason the model may fail to get information from a website. The Arizona libraries add that models predict plausible sequences of words, and that their goal is a useful response, not retrieving verified facts.

Wrong facts: a right paragraph with a wrong link inside

The clearest case of a wrong fact in the sources we read is in a Cureus paper of February 2023 by Alkaissi and McFarlane, Artificial Hallucinations in ChatGPT: Implications in Scientific Writing. The authors asked ChatGPT for a short paragraph on how homocysteine causes osteoporosis. After reviewing the literature, they found the first two facts correct, on the balance between osteoblasts and osteoclasts. The third did not hold: the paragraph tied homocysteine to the vitamin K-dependent carboxylation of osteocalcin, and the authors write that homocysteine can reduce osteocalcin production but has nothing to do with that carboxylation.

The authors then tried a rare condition, late-onset Pompe disease. ChatGPT wrote an essay on liver involvement “with apparent confidence”, in the authors’ words, although such involvement had not been reported in the literature. The authors do not exclude that reports may exist in other languages. Their conclusion is that the data ChatGPT produces mixes true and completely invented material, which they say raises concerns about its use in academic writing.

This is the first of the kinds above, wrong facts, in a mild form: the paragraph reads well, and the error sits in one claim, the link to the carboxylation of osteocalcin, which the authors found by reviewing the literature on bone metabolism. It also matches the Kalai example on OpenAI’s page, where a specific fact about one person was asked for and wrong titles and dates came back.

On how often this happens, the pages we read are silent. They do not give a general rate of hallucination for ChatGPT today, and the OpenAI pages we read do not say how often numbers, dates or citations are wrong separately. The Potsdam guide says that ChatGPT tends to hallucinate on very specific things, such as a list of sources or detailed information about one person.

Invented sources: the identifier that points elsewhere

In the same Cureus paper the authors asked for references to support the claim. ChatGPT gave five, dating from the early 2000s. None of the titles existed, and every PubMed ID belonged to a different, unrelated paper. One of them was credited to “Kallajoki M, et al.” with the title Homocysteine and bone metabolism and a PubMed ID. Searching that ID leads to a 2002 paper on titanium staples in upper tract laparoscopic reconstructive surgery. When the authors asked for references from the last 10 years, ChatGPT returned the same list with different years and PubMed IDs of other papers.

The SUNY Potsdam guide tells a second story, which it takes from the Chronicle of Higher Education and which we did not read there. A researcher at the University of Southern California asked a librarian to find 35 articles. The librarian could not locate any of them, and the researcher said ChatGPT had supplied them. Each had a full citation with plausible titles, dates, page numbers and journal names, and none existed. The figure below puts these cases and the court case of the next section side by side.

The Arizona libraries list citations to sources that do not exist among their examples, and the OpenAI Help Center names fabricated quotes, studies, citations and references as one of its three examples. Potsdam adds that the model may take a real author and a real journal and invent an article that person never wrote.

The Cureus authors also report what they found useful. If the author does the literature review and gives ChatGPT short bullet points from each reference, it can turn them into coherent text. And when it could not find repeated PubMed IDs in a long discussion section on a first attempt, a request to write Python code to find them worked.

A court case, read in the ruling

One case that reached a federal court is Mata v. Avianca. We read the Opinion and Order on sanctions of 22 June 2023, written by Judge P. Kevin Castel in the Southern District of New York, as published on CourtListener. It begins by saying there is nothing inherently improper about using a reliable AI tool for assistance, but that the rules give lawyers a gatekeeping role for the accuracy of their filings. It then says the two lawyers and their firm abandoned their responsibilities when they submitted non-existent judicial opinions with fake quotes and citations created by ChatGPT, then continued to stand by them after judicial orders called their existence into question.

The ruling explains how it came about. The firm used a research service called Fastcase, whose account had limited access to federal cases. Attorney Steven Schwartz testified that he falsely assumed ChatGPT was a “super search engine”, and the ruling says the attorney had not used it before. The prompts in the record included “provide case law in support” of his argument, “show me specific holdings” and “show me more cases”. The court writes that the chatbot complied by making them up.

The ruling examines three of the fake decisions. For the one called Varghese, it records that the Eleventh Circuit’s clerk confirmed it was not an authentic ruling, and it calls the legal analysis gibberish. Its docket number belongs to another case, George Cornea v. U.S. Attorney General, and its citation belongs to J.D. v Azar, 925 F.3d 1291 (D.C. Cir. 2019). It cites other cases that do not exist: at the citation given for Holliday v. Atl. Capital Corp. the court found Gibbs v. Maxwell House.

The sanctions were letters to the plaintiff, Roberto Mata, and to each judge falsely named as the author of the fake opinions, plus a $5,000 penalty imposed jointly and severally on the respondents, to be paid within 14 days. The court did not require an apology. It ordered the sanctions under Rule 11 or, alternatively, its inherent authority. The Potsdam guide summarises the case as the court fining both lawyers $5,000; the ruling we read imposes one penalty of $5,000 on the respondents together, which include the law firm. The ruling does not give the version of ChatGPT used.

A way to check each kind

The pages we read do not give a step-by-step method for checking a citation against an outside archive, so the method below is our reading of the cases, not a procedure any source states. What the sources do say is in the figure that follows.

1. Wrong facts and dates. OpenAI says to verify data and technical information, and to check important information against reliable sources. In the Cureus case the authors found the wrong link by reviewing the literature on bone metabolism. A claim worth keeping needs a second page that says the same thing.
2. Invented sources. Search for the exact title together with the identifier it came with. In the Cureus paper the PubMed ID led to another paper; in the court case the docket number and the citation led to other cases. A title that finds nothing, or an identifier that finds something else, is the signal both cases show.
3. Overconfident answers. OpenAI says the model may express high confidence even in an incorrect answer, so the tone tells nothing. It names three tools that, depending on the plan, may help: Search, which gives cited answers from the web, Code interpreter or Data analysis for calculations, and Deep research for cited answers from several sources. The Arizona libraries warn that even search-grounded answers can misread sources, leave out context or cite sources that do not fully support the answer. Its advice is to open the cited sources and check them yourself. The Arizona libraries also say that reasoning models can work through complex problems but reasoning does not guarantee accuracy, which qualifies OpenAI’s remark that GPT-5 hallucinates less when reasoning. The OpenAI Help Center adds that without tools the answers do not include events after the training cut-off date, so a recent date or event is a case to check.

The sources differ on how far to trust the tool. OpenAI suggests using ChatGPT as a first draft and verifying what matters. The Potsdam guide tells students not to use it for research at all. The Help Center also says ChatGPT cannot reliably tell whether text was written by AI, and that it may make up an answer to a question such as whether it wrote an essay. That passage is about authorship. On re-checking, the court ruling quotes Schwartz’s declaration of 6 June: when he asked ChatGPT directly whether Varghese was a real case, it responded that the case “does indeed exist”.

What this page leaves out and why

We tested nothing in ChatGPT; every statement of fact here comes from the pages named at the foot of this page, read on 5 November 2026; the checking method in the last section is ours. We did not read the Chronicle of Higher Education or Associated Press articles that the Potsdam guide cites, so what they report is given as the guide tells it. We read an encyclopedia entry only to find our way and took no figure from it. The pages we read do not give a measured answer to whether ChatGPT hallucinates more than before.