Welcome to DU! The truly grassroots left-of-center political community where regular people, not algorithms, drive the discussions and set the standards. Join the community: Create a free account Support DU (and get rid of ads!): Become a Star Member Latest Breaking News Editorials & Other Articles General Discussion The DU Lounge All Forums Issue Forums Culture Forums Alliance Forums Region Forums Support Forums Help & Search

moniss

(9,220 posts)
Thu Jul 2, 2026, 06:28 PM Thursday

When AI can't even get simple things straight

it should be a clear indicator that the rush for data centers should come to a grinding halt. This afternoon I casually inquired of Google, using my voice, whether the Post Office is open tomorrow July 3rd. Many of the Federal Agencies are recognizing the July 4th holiday on Friday the 3rd. Although the PO is a different animal I thought they might follow the Federal Agencies.

Google thought a moment and then came back on screen and showed in print that yes the Post Office is open for regular business on July 3rd, 2026. But almost simultaneously the AI voice came on and said "No the Post Office is not open on July 3rd, 2026 since they are recognizing the July 4th holiday on July 3rd. They will resume normal service and delivery on Monday July 6th, 2026."

So I stopped at my Post Office a short time ago and asked the clerk if they were open Friday July 3rd or if they were off that day. The clerk said it's business as usual tomorrow and they are closed on Saturday July 4th.

Amazingly there are people who want to let this garbage system do medical diagnosis and prescribe treatment. I think the f*cking Ouija Board or a trip to the local lady of mystery passing her hands over the crystal ball would do at least as well.

35 replies = new reply since forum marked as read
Highlight: NoneDon't highlight anything 5 newestHighlight 5 most recent replies
When AI can't even get simple things straight (Original Post) moniss Thursday OP
spending billions of dollars... RussBLib Thursday #1
Most of us could get all the lies we moniss Thursday #2
Call me simple minded, Faux pas Thursday #3
I think that's a bit short sighted... Happy Hoosier Friday #25
AI makes mistakes, but it has been life-changing for me in many ways. I feel lucky to Doodley Thursday #4
Did AI Write that? unweird Thursday #6
wonder how much here is AI text? RussBLib Thursday #11
I use ChatGPT every day. Have you ever tried it? Doodley Friday #16
I have tested it extensively, Ms. Toad Friday #18
No, but again I ask unweird Friday #20
Why? Mossfern Friday #29
I'm with your nephew! I love chatting with AI. anciano Friday #30
AI isn't perfect, but what is? anciano Thursday #5
The problem is that the information being given moniss Thursday #8
"often factually wrong".... anciano Thursday #9
How do you know that it wasn't wrong? Wiz Imp Thursday #12
Like any tool, understanding it has limits and weaknesses is important, but so is in seeing where its strengths are. Doodley Friday #15
Old info ThreeNoSeep Friday #26
🙄💤 Bye! Wiz Imp Friday #27
so have you rigorously researched the responses to be sure they are accurate? RussBLib Thursday #13
Have you done the research to back that up? Ms. Toad Friday #19
Had the same experience as Moniss with Copilot. I asked a question that I was allegorical oracle Friday #32
efficient? Wiz Imp Thursday #10
In my experience, it is more efficient in some areas than any human being. It has a depth of Doodley Friday #14
I believe that the people who knock it haven't discovered its potential. Doodley Friday #17
That is nonsense. Ms. Toad Friday #22
To be fair, the consumer available versions pale in comparison with those used in industry. harumph Thursday #7
Yeah... Hong Kong Cavalier Friday #21
It is downright dangerous for the general population Ms. Toad Friday #24
It's important to remember... Happy Hoosier Friday #23
"(which is what humans are supposedly good at)" Wiz Imp Friday #28
There are different types of LLMs and most do have extensive reasoning capabilities. They can 'think.' Appendix Friday #33
Hard disagree. Happy Hoosier Friday #34
Welcome to DU LetMyPeopleVote 12 hrs ago #35
The AI helping REAL Doctors with medical research and Diagnosis LostOne4Ever Friday #31

RussBLib

(10,863 posts)
1. spending billions of dollars...
Thu Jul 2, 2026, 06:31 PM
Thursday

...to be lied to? Shit, we get that for free from the Orange Menace and his goons.

https://russblib.blogspot.com

Happy Hoosier

(9,743 posts)
25. I think that's a bit short sighted...
Fri Jul 3, 2026, 12:10 PM
Friday

Human brains are essentially biological computers. They are physical systems assessing inputs. I think it’s just a matter of time that we are capable of replicating that function with technology.

Doodley

(12,166 posts)
4. AI makes mistakes, but it has been life-changing for me in many ways. I feel lucky to
Thu Jul 2, 2026, 06:45 PM
Thursday

experience AI and the impact it is had on my very soul.

unweird

(3,320 posts)
6. Did AI Write that?
Thu Jul 2, 2026, 07:04 PM
Thursday
I feel lucky to
experience AI and the impact it is had on my very soul.


?

RussBLib

(10,863 posts)
11. wonder how much here is AI text?
Thu Jul 2, 2026, 08:27 PM
Thursday

I think the tendency is to believe what AI produces for you, regardless of disclaimers.

HERE is your answer, and in small text at the bottom: "answers may be wrong." And if the answer confirms your bias, it is quickly accepted.

https://russblib.blogspot.com

Ms. Toad

(38,945 posts)
18. I have tested it extensively,
Fri Jul 3, 2026, 11:36 AM
Friday

As well as the AI summaries linked to various search engines.

Generative AI is designed to be a fluent, not factual, conversationalist. It is coded to keep the conversation flowing by making crap up to fill gaps in it's "knowledge" or to supplement skimpy sources. It doesn't "understand," for example, that it can't just combine Louisiana law (a system of law not used anywhere else in the US) with Federal law.

I have yet to find a single accurate answer. Parts arec correct, but much is gibberish - made up facts, facts taken out of context and stitched together like some kind of frankenstein answer - with no obvious clue as to what is made up garbage, or when multiple inconsistent sources are combined to create a dramatically inaccurate answer.

Most people didn't bother to fact check. They just copy-paste and spread the trash around. Many aren't even aware they are using AI. I have called people on it based on the copy-pasted answer (with the telltale, "ask me anything" tag line), and they have insisted what they posted is not AI. Those who do know, apparently don't care that they are spreading lies around (not to mention decimating local water supplies).

I find it terrifying that someone would believe that AI has touched their soul.

Mossfern

(4,927 posts)
29. Why?
Fri Jul 3, 2026, 01:09 PM
Friday

Honestly, maybe it's a generational thing, but I see no specific reason to use it at this point in my life.
My son and my nephew disagree with me, but when my nephew said that it's great to "talk" to Chat GPT I rolled my eyes.

anciano

(2,361 posts)
5. AI isn't perfect, but what is?
Thu Jul 2, 2026, 07:02 PM
Thursday

Personally, I still find it to be an efficient and useful tool to obtain information, evaluate ideas, and explore options.

moniss

(9,220 posts)
8. The problem is that the information being given
Thu Jul 2, 2026, 07:19 PM
Thursday

is often factually wrong and people don't know that and then go along believing false information.

anciano

(2,361 posts)
9. "often factually wrong"....
Thu Jul 2, 2026, 08:07 PM
Thursday

Personally, in my experience so far, I have not found that to be the case.



YMMV

Wiz Imp

(10,944 posts)
12. How do you know that it wasn't wrong?
Thu Jul 2, 2026, 08:27 PM
Thursday
https://futurism.com/the-byte/study-chatgpt-answers-wrong
Study Finds That 52 Percent of ChatGPT Answers to Programming Questions Are Wrong
In recent years, computer programmers have flocked to chatbots like OpenAI’s ChatGPT to help them code, dealing a blow to places like Stack Overflow, which had to lay off nearly 30 percent of its staff last year.

The only problem? A team of researchers from Purdue University presented research this month at the Computer-Human Interaction conference that shows that 52 percent of programming answers generated by ChatGPT are incorrect.

That’s a staggeringly large proportion for a program that people are relying on to be accurate and precise, underlining what other end users like writers and teachers are experiencing: AI platforms like ChatGPT often hallucinate totally incorrectly answers out of thin air.

For the study, the researchers looked over 517 questions in Stack Overflow and analyzed ChatGPT’s attempt to answer them.

“We found that 52 percent of ChatGPT answers contain misinformation, 77 percent of the answers are more verbose than human answers, and 78 percent of the answers suffer from different degrees of inconsistency to human answers,” they wrote.

https://joshbersin.com/2025/10/bbc-finds-that-45-of-ai-queries-produce-erroneous-answers/
BBC Finds That 45% of AI Queries Produce Erroneous Answers
This is mindblowing. Today the BBC and EBU (European Broadcasting Union) published a detailed study which shows that around 45% of AI news queries to ChatGPT, MS Copilot, Gemini, and Perplexity produce errors.

In other words, the “dangerously self-confident” AI systems we use are quite poor at giving us good analysis of news. While the study focused on news, this shows us that we have to be extremely careful when using and trusting these “open corpus” systems because they are answering questions based on faulty, exaggerated, outdated, or incorrect data.

Examples are quite astounding: the AI’s incorrectly answered “who is the Pope,” “who is the Chancellor of Germany,” and in response to the question “Should I be worried about the bird flu”, Copilot claimed “A vaccine trial is underway in Oxford”. The source for this was a BBC article from 2006, almost 20 years old.

“Some were potentially consequential errors on matters of law. Perplexity (CRo) claimed that surrogacy “is prohibited by law” in Czechia, when in fact it is not regulated by the law and is neither explicitly prohibited nor permitted. Gemini (BBC) incorrectly characterized a change to the law around disposable vapes, saying it would be illegal to buy them, when in fact it was the sale and supply of vapes which was to be made illegal.”

Doodley

(12,166 posts)
15. Like any tool, understanding it has limits and weaknesses is important, but so is in seeing where its strengths are.
Fri Jul 3, 2026, 09:31 AM
Friday

RussBLib

(10,863 posts)
13. so have you rigorously researched the responses to be sure they are accurate?
Thu Jul 2, 2026, 08:28 PM
Thursday

If not, how do you know the response is wrong? Or right?

https://russblib.blogspot.com

Ms. Toad

(38,945 posts)
19. Have you done the research to back that up?
Fri Jul 3, 2026, 11:48 AM
Friday

I have extensively tested ChatGPT and the summaries associated with a variety of search engines in areas of baking, art history, law, politics, medicine, and history of family members famous enough to have an internet presence that in theory was party of their training database. I have yet to find a single accurate response or summary - Even when I coach it with multiple redirects to the right answer after it initially made crap up.

Virtually all seem plausible, but when you trace back to the sources (if they happen to provide them), they aren't. They state stuff which is nowhere to be found in the sources, or they combine two (or more) inconsistent sources.

allegorical oracle

(6,737 posts)
32. Had the same experience as Moniss with Copilot. I asked a question that I was
Fri Jul 3, 2026, 02:17 PM
Friday

90 percent sure I knew the answer to. Two paragraphed answers popped up. The first one agreed with my understanding of the proper answer. The second paragraph provided an absolutely opposite answer. Both were unequivocal statements.

I double check everything when an AI reply pops up.

Wiz Imp

(10,944 posts)
10. efficient?
Thu Jul 2, 2026, 08:19 PM
Thursday

It wasn't very efficient in the situation described. Getting wrong information quickly is not efficient. It's dangerous.

Doodley

(12,166 posts)
14. In my experience, it is more efficient in some areas than any human being. It has a depth of
Fri Jul 3, 2026, 09:26 AM
Friday

understanding in some areas that exceed any genius-level human. Yes, if you ask for basic local information, it can often get things wrong, but its ability to comprehend and analyse has been life-changing, for me, my wife, and several of our friends.

Ms. Toad

(38,945 posts)
22. That is nonsense.
Fri Jul 3, 2026, 12:01 PM
Friday

If I was still teaching, I could have found multiple uses for AI. My nephew, an extremely experiences programmer, uses it for the initial draft of his programs - which he subsequently tests extensively.

But rather than being used to assist in areas where a predictive model might actually be useful, it is currently being used by the general population primarily as a substitute for research - for medical advice, for example - where relying on tool that is explicitly designed to make crap up (a predictive, gap filling model) can be life threatening.

It is also being used to create responses where truth is essential - in court documents, in medical records, in politics, etc.

My experience is that it is being used and promoted mainly by people who don't have even a basic understanding of how it functions - and who believe it without fact-checking.

harumph

(3,574 posts)
7. To be fair, the consumer available versions pale in comparison with those used in industry.
Thu Jul 2, 2026, 07:17 PM
Thursday

Those are not so much LLMs like Chat but rather special purpose types of AI. I think it has its uses, but so far its reach exceeds its grasp. I don't like the way many models are being trained on and hence infringing on people's creative works. The all consuming fascination that some have with it - is in my opinion flat out naive and creepy.

Hong Kong Cavalier

(4,609 posts)
21. Yeah...
Fri Jul 3, 2026, 11:56 AM
Friday

I've declared my contempt for AI when my sister had AI write our mother's obituary last year.

Rather than asking me, her brother.
Her brother, the writer.

For training to spot medical issues, yes, it's good. For other scientific research, it can be good.
For writing things like obituaries? Fuck. No.

Ms. Toad

(38,945 posts)
24. It is downright dangerous for the general population
Fri Jul 3, 2026, 12:10 PM
Friday

To use AI to spot medical issues.

https://pmc.ncbi.nlm.nih.gov/articles/PMC13043314/

It probably isn't dangerous for personal scientific research - but if you didn't already know what you are researching well enough to shoot the embedded crap, it probably won't be very helpful.

Happy Hoosier

(9,743 posts)
23. It's important to remember...
Fri Jul 3, 2026, 12:07 PM
Friday

That LLM’s do not reason. They determine what they calculate as the most likely answer based on their algorithms and training data. That’s it. They are not very good at comparing their conclusions to reality (which is what humans are supposedly good at).

 

Appendix

(1 post)
33. There are different types of LLMs and most do have extensive reasoning capabilities. They can 'think.'
Fri Jul 3, 2026, 02:24 PM
Friday

Your information is outdated. People making judgements about LLMs based on free tier versions of models are doing themselves a disservice.

Happy Hoosier

(9,743 posts)
34. Hard disagree.
Fri Jul 3, 2026, 08:25 PM
Friday

I deal with them every day. They are predictive models. Their “thinking” is a relatively somple staye machine.

LostOne4Ever

(9,772 posts)
31. The AI helping REAL Doctors with medical research and Diagnosis
Fri Jul 3, 2026, 01:48 PM
Friday

Is not the same as the LLMs (Large Language Models) that commercial AI companies are building data centers for.

Rather medical research AIs use tech like Convolutional Neural Networks that are vastly more specialized, accurate, and do NOT hallucinate. They are only used when absolutely needed and after meeting guidelines to prevent their over-use and only by a fraction of a fraction of the population {only medical professionals trained to use them} creating a microscopic footprint in comparison to LLMs while benefiting all of society rather than just answering stupid chat queries.

Criticize LLMs all you want but be aware medical AIs are different in how they work, environmental impact, and are a huge benefit to society.

AI bros can and will try to use ignorance of the difference between Medical AIs and shitty LLMs to score points. Don’t let them.



Latest Discussions»General Discussion»When AI can't even get si...