The sorry state of search is an ominous sign for the AI era

On a recent Saturday night, a bored tech executive decided to play around with an artificial intelligence app he'd been hearing about. The executive, who we'll call Matthew, started the way most people look for things on the internet: He flipped open his laptop and typed the AI company's name, Midjourney, into Google. He clicked on the top result, and with a few more clicks he downloaded and installed the app.

Or that's what he thought he'd done: What he'd actually clicked on was an ad Google had unwittingly sold to a scammer disguised as Midjourney. Matthew, who requested anonymity because he's worried that whoever accessed his computer may still have his personal data, had inadvertently installed an "infostealer," a type of malware that combs through a victim's computer looking for usernames and passwords, then transmits those to hackers. This one, known as Aurora, had accessed his crypto wallet, social media accounts and who knows what else. The hackers transferred the contents of Matthew's Coinbase wallet—hundreds of thousands of dollars in crypto—to a bank that wasn't his. "It's your whole life," Matthew says. "You feel so naked and scared and vulnerable."

Matthew blames himself for not being more careful, but he also blames Google for allowing an obvious scam to reach the top of its results page. The failure was magnified by design choices that create the feeling that the Google search result links are perfectly objective and trustworthy while making it hard to tell which are there because someone paid to put them there. (Google labels ads as "sponsored," but it's easy to miss the label.)

Matthew's experience raises longstanding questions about the costs of the company's domination of the search market and its apparent inability to rein in scams. And it suggests that Google may have a more difficult-than-anticipated time keeping scams and misinformation off its experimental (and sometimes factually challenged) chatbot search engine. On Google Bard, as it's known, it's likely to be even harder to determine the provenance of the information the company provides.

By late July the ad Matthew had clicked on was gone, but there were still fake Midjourney websites showing up near the top of Google's search results. "The fact that it still shows up in the results is unconscionable," he says. "I trusted them, but it was a false trust." The fake Midjourney listings disappeared from Google shortly after a query from Bloomberg Businessweek. Google declined to comment on the fake listings, but says it uses a mix of human reviewers and software to remove such content, which it prohibits. It also notes that it removed 5.2 billion ads last year for violating company policies. "While there are bad actors who attempt to circumvent our protections, Google's systems are highly effective at surfacing high-quality information and fighting spam and malicious behavior," says Ned Adriance, a company spokesperson.

It shouldn't have taken a sophisticated anti-scam system to figure out that the ad Google showed to Matthew was malicious. The listing pointed to a website that purported to be Midjourney's, but it replaced the letter "o" in the company's name with a zero, a well-known scam tactic. There were threads on Reddit warning users of this exact thing before Matthew clicked, and Connecticut Democratic Senator Richard Blumenthal in a 2022 letter accused Google of having "routinely failed to address dangerous scams, impersonation, cybercrime." Moreover, Midjourney doesn't even have a downloadable app; you access it via the chat app Discord.

Senator Blumenthal speaks to reporters on May 16 before a Senate judiciary subcommittee hearing on artificial intelligence. Photographer: Eric Lee/Bloomberg

Similar examples have shown up periodically throughout Google's history. In late July, for example, Shmuli Evers, a digital designer, discovered that scammers had managed to trick Google into giving fake phone numbers for Delta Air Lines Inc. and other large carriers to sell fake tickets. Google corrected the error after Evers tweeted about it, but critics say the company has long known about similar phone number scams. "These hijackings have occurred every year over the last 15 years," says Mike Blumenthal, co-founder of Near Media, which advises businesses on digital and search marketing. "Google has long pushed the costs of these scams off onto society."

Scammers have switched phone numbers for florists, locksmiths and hotels, says Blumenthal, who's documented cases as far back as 2008. "We do not tolerate this misleading activity and are constantly monitoring and evolving our platforms to combat fraud," Adriance says, referring to the airline scams.

Google is also subject to a more subtle kind of manipulation. Search engine marketers—sometimes with the help of ChatGPT and similar services—have filled up results pages in certain categories with junky, self-promotional websites that are often devoid of anything useful. The problem, which the Atlantic described last year as "The Open Secret of Google Search," has caused users to resort to tricks to try to filter out bogus results. One of the most popular tactics involves adding the word "Reddit" to any search to try to get Google to focus on that social network, which is seen as having largely avoided getting overrun with marketing and spam. Another approach: replacing Google with TikTok, which produces results that can be a lot less spammy if you're looking for restaurants, recipes or travel tips. Finally there are chatbots, such as Bard and the one Microsoft Corp. developed for its Bing search engine, which some prognosticators say could eventually replace traditional web search.

Google’s Bard AI. Photographer: Gabby Jones/Bloomberg

Google says the vast majority of its customers are happy with the quality of its search results, which at least partly explains why its dominance in the category has gone more or less unchallenged for two decades. The company enjoys a 91% market share in search engines, according to Similarweb.

Congress could apply more pressure by reforming Section 230 of the Communications Decency Act, which shields websites from legal liability related to content their users have created. But such action would be heavily contested, and Congress hasn't shown the ability to take on such issues in recent years. A more likely path might be a lawsuit brought by the US Department of Justice and state attorneys general—which seeks to separate Google from two ad networks it purchased years ago. Google has said the lawsuit is an "attempt to pick winners and losers in the highly competitive advertising technology sector."

Those government actions are unlikely to be resolved for years. In the meantime, Matthew is warning friends and relatives to be careful. Even though he was eventually able to recover 90% of his money with the help of the FBI, he says he's putting his money in conventional banks instead of crypto from now on. But he worries that people are still falling for similar scams. "At this point Google is sophisticated enough that if I search for something simple like Midjourney, I expect the top link to be the Midjourney website," he says. "There are just practical problems they're not solving."

Disclaimer: This article first appeared on Bloomberg, and is published by special syndication arrangement