Loading
Ask a chatbot a question and you get a fluent, confident answer. Sometimes it is wrong.
The confidence does not change when the answer does.
Predicting what text should come next, one piece at a time, from patterns in an enormous amount of writing.
That is the whole mechanism.
When a chatbot answers a factual question, it is:
A plausible-sounding wrong answer and a correct one look almost identical as patterns of text.
Nothing in the process distinguishes them, because the process is about fit, not truth.
It writes confidently because confident writing is what most of its training text looks like. Reference books do not hedge.
So the certainty is a style it copied, not a signal about the answer.
Specific, checkable details. Names, dates, numbers, citations, page references, quotations.
Those are exactly the things it can generate the shape of without having the content.
A fake citation is easy to produce: author, year, plausible journal, plausible title. All of that is pattern.
Whether the paper exists is not something the pattern knows.
Where is a chatbot more trustworthy?
Tap an item, then a bucket
Patterns seen many times are reproduced well. Patterns seen rarely are reconstructed from something adjacent.
So it is most confident and least reliable exactly where you are least able to check.
Push back and it will often change its answer, whether or not you were right.
Agreement is also a pattern in the text it learned from.
This is the practical point. Human confidence is a weak signal, but it is a signal. Here it is not one at all.
Certain and correct are unrelated.
Shaping language. Drafting, rephrasing, summarising something you supply, explaining a well-worn idea, or getting you started.
All cases where the pattern IS the job.
Ask yourself whether you could tell if this answer were wrong.
If you could not, that is precisely when to check it somewhere else.
It predicts likely text, not true text. It writes confidently because its sources did. And it is least reliable on the specific, checkable things it is most often asked for.