'Poisoned' AI: the ChatGPT shopping scams that lead to fake websites

[highplainsdem]

Source: Guardian

Ask Silver, a scam-checking service, says cloned sites have been showing up in search results on ChatGPT. The ones it has seen are rip-offs of Russell & Bromley and the furnishings retailer Dunelm.

Anna Jones of Ask Silver says it is possible that the large language model (LLM) that powers ChatGPT has been “poisoned”. This is when malicious content is inserted into the information an AI learns from – such as through cloned webpages put up by the fraudster.

-snip-

Louise Baxter, the head of the scams team at National Trading Standards, said people should not assume a website is genuine just because it is recommended by an AI tool.

“Consumers are increasingly turning to AI tools for advice and recommendations, but criminals are adapting just as quickly. The fact that scam websites can appear in AI-generated results is worrying, and is a stark reminder that fraudsters will exploit any new technology that helps them reach potential victims,” she said.

-snip-

Read more: https://www.theguardian.com/money/2026/jun/07/ai-chatgpt-shopping-scams-fake-websites

---

"Poisoned" is overly dramatic here - the fake websites are not a new scam in and of themselves.

What's new is AI companies trying to get their customers to stop using their own brains and doing their own searches, and instead just trust less-than-intelligent and often hallucinating chatbots to come up with the best recommendations.

ChatGPT can't be the only bot this is happening with, even though this is the first news story I've seen about it happening.

Generative AI does make fraudulent websites much easier to create, though. I've posted earlier threads about how easy genAI has made it to spread disinformation through fake news websites. It's a great tool for fraud, especially when AI users are dumbed down.

[nuxvomica]

1. These retail uses of AI should just be stopped

Promoters justify AI development by claiming that AI could discover new drugs and aid diagnosis, it could really help people, so maybe they should just focus on that. Even then, once AI gets deep into the business of creating things, even useful things, will humans even know how those things work? We are in the midst of a major accountability crisis, especially in politics. Do we really need to be subject to more entities that have no accountability?

[FakeNoose]

2. The AI creators are doing no fact-checking on these search results, obviously

They couldn't care less if users get fooled by a fraudulent website or some kind of internet scam. Anyone who uses AI-created search results needs to understand they're on their own, nobody has their back.

For myself, every time I do a Google search, I type "-AI" after the search term.

[highplainsdem]

3. You're right about them not caring. As for fact-checking AI results before the user sees them - they

not only can't do that without slowing down the results, but trying to do so would increase the operating cost, the cost of compute, and the AI companies are already losing money almost every time someone uses AI. (See the threads in GD about Ed Zitron for the best explanations.)

I saw an essay the other day (but forgot to bookmark it) about what would supposedly "solve" genAI's high failure rate. The recommendation was to give lots of AI models the same prompt and show the user only the answer that most of the chatbots offered. Which not only ignored the fact that they could still all be wrong, but would multiply the cost of getting the answer by the number of chatbots used.