AI Answers: Authority Without Accountability

Recently, through a series of conversations, emails, staggeringly bad suggestions, and repeated attempts to correct misinformation, it has become more apparent that generative AI content not only sucks but can be potentially dangerous. When incorrect information is delivered confidently, repeated without challenge, and treated as authoritative, the problem quickly escalates from annoyance to risk.

If there is one lesson the security industry taught us the hard way, it is this:When a system is trusted, its errors are believed.

At MajorGeeks, we have lived this lesson for decades through the scourge of antivirus false positives. Legitimate tools flagged as malware. Clean utilities labeled as threats. Small developers are punished not for wrongdoing but for triggering automated heuristics that lack context brought on by companies more interested in scale than quality.

The technology is imperfect, but that was not the real problem.

The problem was authority. Stick with me here....

Trust turns guesses into verdicts.

Antivirus software does not gain influence because it is flawless. They gain influence because they are trusted, the branding is familiar, and the warnings are confident.

So when an antivirus engine declares something dangerous, users tend not investigate. They comply, delete, and blame the author - or MajorGeeks.

Even when the detection was wrong, the conclusion was accepted as fact. The user assumed the system knows more than they do, and in most cases, that assumption felt reasonable.

False positives did not persist because users are careless. They persist because people are trained to unquestioningly trust authority, which provides little incentive for the industry to change. Just check out the latest false positive test from Av Comparatives.

AI has claimed the same authority, without the same guardrails.

Today, AI-generated answers and summaries are being forced on consumers everywhere by every brand on the planet, occupying the same psychological space as antivirus warnings.

With a battle for AI dominance raging. We have AI responses appearing at the top of a search result, embedded in an operating system, and injected into a support interface; it no longer feels like an option or a suggestion. It feels like documentation.

And, Branding matters.

"Google AI Overview."
"Powered by OpenAI."
"Microsoft Copilot."

To most users, these names imply expertise, neutrality, and correctness. The answer is assumed to be vetted, official, and safe to rely on. But unlike traditional documentation, AI-generated answers are not verified and can be less accurate than a weather prediction.

False, but believed

AI systems are known to hallucinate. They can fabricate details, invent lists, and confidently state things that are simply not true. This is not a secret. It is a documented limitation of how these systems work.

The issue for me is not that errors exist. The issue is that uncertainty is invisible.

AI answers are delivered with the same tone and confidence, whether they are correct or not. There is no warning label. No friction. No pause that invites skepticism. Just misinformation presented as fact in a way that makes you feel good about yourself, popping off those sweet, sweet “I love me” endorphins.

So when an AI answer is wrong, it is not treated as a possibility. It is treated as a fact.

- Users repeat it.
- Publish it.
- Teach it.

And now the claim exists socially, even if it never existed technically.

As an example, you can see where ChatGPT quite authoritatively agreed with me that the 3rd founder of MajorGeeks was “Benjamin Dover” (Ok — yes, I have the humor of a 12-year-old, but my point stands.)

Why do people believe AI Answers?

This isn't an accident. AI works because it has been programmed and trained to tap directly into how people already think. It leans on authority bias, where answers feel trustworthy simply because they come from a big name, and then adds in the automation bias, where machine output is assumed to be objective. It then responds in a confident tone and polished explanations, making information feel correct because it is easy to read. It is programmed to agree with you to be seen and more likable. In a nutshell, Generative AI answers are programmed to program your brain to believe it.

AI does not need to persuade people; it only needs to agree with them confidently. It's like a very expensive company suck-up.

▶

The statistics no one likes to quote

Independent testing in the security world has long shown that antivirus false positives routinely occur at 5–15% in broad detection scenarios, especially for older, niche, or system-level software.

In AI research, controlled evaluations have demonstrated failure rates ranging from 20% to 50% on strictly graded, multi-step factual or reasoning tasks. Even for simpler factual recall, measurable hallucination rates persist. Generative AI false claims on news topics are up to 35%, doubling its failure rate in just over a year. It's not getting better. It is getting insanely worse!

These numbers do not mean AI is useless. They mean AI is probabilistic, not authoritative.

The danger is not the percentage. The danger is that users are never shown the probability and hence are convinced in 100% corectness. But the reality is, generative AI is now more realistic than your local weather forecast.

Authority flips the burden of proof.

Once a trusted system speaks, the burden shifts.

Instead of asking "Is this true?", users ask "Why does this not match what I am seeing?"

This is exactly why antivirus false positives survived for so long. The software was trusted more than the developer, the reviewer, or the user’s own experience, even though it was just an educated guess.

AI answers now create the same dynamic, but at a vastly greater scale.

Even a small error rate becomes corrosive when:

* The answer is forced rather than optional,
* The source is institutionally trusted,
* Skepticism is trained out of users.

Scale turns mistakes into infrastructure.

The danger, for me, is that a false antivirus detection affects a system, a program an author. In theory, you can report a false positive and correct a mistake- eventually. A false AI answer affects knowledge itself. AI-generated claims propagate instantly. They are repeated by other AI systems, summarized into overviews, indexed by search engines, and absorbed into future models with no real checks and balances to detect fact or truth.

Beyond citing a source, there is no practical way to independently verify accuracy, no meaningful oversight, no direct accountability, and no clear mechanism for reporting errors.

Realistically, there is limited incentive for companies to prioritize correcting individual mistakes, and some posit that the mistakes are actually a feature. Broad source citation, while useful in some contexts, would not necessarily address the underlying accuracy of a generative answer and may raise additional questions around attribution and copyright, which AI companies REALLY want to avoid.

Truth requires effort. Falsehood only requires plausibility. Once errors exist at scale, they stop being questioned. They become background assumptions.

Falsehood spreads faster when it wears a badge, and we aren't talking one of those fancy blue check marks either. We are talking about the companies that control the vast majority of the media that is consumed.

Authority does not eliminate error. It amplifies it.

We see this all the time in security, and it is one of the least talked-about problems in the industry.

We test every piece of software, and we see this a lot on VirusTotal. One antivirus engine flags a file with a vague "Generic" detection. Not a known malware family, not a confirmed threat, just "generic" behavior that may be something. No one actually tests the file, mind you. They just flag out of the algorithm. That first detection may carry weight because of the brand behind it. Other engines come along later, see that flag, and suddenly, more detections appear. Some now actually name it to be even scarier! Not because new evidence was found, but because nobody wants to be the one engine that missed something. Herd behavior kicks in.

Before long, a file that was never conclusively malicious is effectively blacklisted. To the average user, it now looks dangerous because "20 engines agree." In reality, many of those detections trace back to a single, weak assumption that got copied, echoed, and reinforced. Authority did not protect users from a mistake; it multiplied the mistake. One file that this happened to exactly is the ProduKey form of NirSoft. A perfectly fine piece of software that reads your installed keys, which you likely can't download anymore, because it is falsely accused of being a virus.

AI introduces the same problem, but at a scale and speed we have never dealt with before.

AI systems do not think. They correlate. They scan existing content, look for patterns, and reproduce what appears to be consensus. When the source material contains an error, especially one cloaked in authority, that error is not questioned. It gets normalized.

Now add volume. AI-generated content is being published at a rate no human editorial process can keep up with. A false idea appears once, then ten times, then ten thousand times. Other AI systems scan that content, see repetition, and treat it as validation. The mistake becomes "fact" simply because it is everywhere.

This creates a misinformation feedback loop. AI learns from content polluted by AI, reinforces the same false claims, and presents them back to users with the implied authority of a trusted platform. Most users will not question it. Why would they? It came from a big name. It sounds confident. It agrees with what they have already seen—Hell, there are even 3 sources!

The result is not just misinformation, but accelerated misinformation. Errors compound faster than corrections can be made.

At MajorGeeks, we have spent decades dealing with the fallout of this exact dynamic in the antivirus world. False positives, lazy classifications, and reputation-based assumptions cause real damage. Legitimate software gets buried. Developers get smeared. Users get misled.

AI did not invent this problem. It is just putting it on steroids.

Authority should be a starting point for scrutiny, not a substitute for it. Whether it is an antivirus engine or an AI model, trust without verification does not make information safer. It just makes mistakes louder.

What you should take away

AI is not going away. There is too much money invested, too much data to mine, and too much convenience for that. But how people use AI still matters, and that part is firmly in human hands.

The takeaway is simple. Trust, but verify. Use AI as a tool to speed things up, surface ideas, or point you in a direction, but never confuse it with a primary source. Branding is not validation. Confidence is not correctness.

Psychological authority does not eliminate error. It amplifies it. We have already seen what happens when automated systems are allowed to speak with unquestioned authority. Antivirus engines do it. Algorithms do it. AI is doing it now. The technology changes. The mistake do not.

In the end, the cleanup always falls to the same people, the ones willing to slow down, test assumptions, verify claims, and say when something is wrong, even when that answer is less convenient, less popular, or less profitable. That work is not glamorous, but it is the difference between information and noise.

At MajorGeeks — This is the way..


comments powered by Disqus