What Is Google AI Mode?

Google AI Mode is a new feature of the search engine that combines the breadth of a web search with the reasoning power of a large language model (LLM). It’s essentially an open-end AI chat (like Claude or ChatGPT) but tailored to information seeking.

Users can search the internet by asking questions, receive comprehensive answers, and follow up with clarifying questions or additional requests.

Anyone who’s used Perplexity might see echoes of it in Google’s AI Mode. Since its launch in 2022, Perplexity has been using AI to challenge the traditional approach to web search.

Conversational Format: Followup Prompts

Unlike Google’s traditional search pages, AI Mode uses a conversational format similar to  that used by many AI chatbots. This structure enables users to ask followup questions after the system’s initial response. AI Mode actively encourages conversations by offering clickable followup prompt suggestions (another feature shared with Perplexity).

This is a feature that differentiates AI Mode from Gemini and other AI chatbots — those tools sometimes fail to offer followup suggestions, or offer them as in-text suggestions. Compared to the clickable options in AI Mode, in-text suggestions require more work for users because they have to accept the suggestion by typing in the text field. 

Benefits of AI Mode: Supercharged Search

AI-powered search tools like Google AI Mode and Perplexity are essentially supercharging and automating the traditional search process.

Traditional Search Process

1. The user identifies information need.
2. They write a query containing keywords.
3. The query is executed by the search engine.
4. They explore results, gathering relevant information.
5. They revise the query and repeat as needed.

AI-Powered Search

1. The user communicates information need in as many words as needed.
2. LLM interprets the need based on user’s prompt.
3. LLM generates several sub-queries (which may include keywords the user didn’t know, effectively eliminating the keyword foraging problem of search).
4. The subqueries are executed through traditional search indexes.
5. LLM compiles and synthesizes many search results into a relevant answer for the user (thus performing the information-foraging step for the user).
6. In “deep research,” this process may be conducted iteratively to more deeply explore a domain or problem space.

For example, if a user searches best Mexican food Oakland in Google AI Mode, the system may submit many related queries, like Top taquerias in Oakland and Great Oakland Mexican restaurants. The LLM then synthesizes the results, summarizes them, and presents the answer back to the user.

Essentially, the AI-powered search is quickly conducting a broad, comprehensive series of searches that would take users hours to complete if done manually. This approach uses two techniques to expand the information-seeking value of LLMs: query fan-out and retrieval-augmented generation (RAG)

Query fan-out is an information-retrieval technique in which an information need is translated into multiple related subqueries, which are run concurrently, possibly across different data sources, before blending the results into a summarized answer.

Retrieval-augmented generation (RAG) is a technique that enhances LLMs by having them retrieve and reference external information before generating responses, instead of relying on training data alone.

Compared to pure LLM chatbots using only their training data, this approach is considered more reliable from an information-seeking perspective. Despite this, Google AI Mode did return false information and hallucinate during our study.

Google’s Big Advantage: Information Ecosystem

Google AI Mode tailors its response and UI based on the query. Google’s structured, indexed knowledge is vast, which makes AI Mode even more powerful. When the system sees relevance to the query, it can return data from:

• Google Maps: Locations and reviews
• Google Shopping: Carousels of available products with pricing
• Multimedia: Images and video, particularly from YouTube

Our Study

In a recent study on information seeking, 7 participants were encouraged to use AI Mode — if they didn’t naturally — when conducting online research on their own tasks.

Overall, participants were pleased with Google AI Mode. They appreciated the sources provided, suggested followups to continue the conversation, and information from products like Maps and Shopping embedded in the responses. However, we found several large usability issues that hindered some users from fully utilizing it.

Problems with Google AI Mode

Poor Discoverability

Currently, users can access Google AI Mode:

• from the Google homepage before searching
• through a new tab on Google search-results pages
• at the bottom of expanded AI Overviews

You might think these multiple access points would be enough for Google users to easily discover AI Mode… but you’d be wrong.

Like many AI features built into existing products, AI Mode has poor discoverability — meaning that people are unlikely to notice it on their own. While AI Mode can be accessed from several places, it’s still easy to miss. In our study, 4 out of 7 participants had not noticed or used AI Mode before we directed them to it.

We hypothesize that the familiarity of Google’s interface contributed to this problem. All our participants (like many users today) were extremely familiar and comfortable with Google search — they knew what they needed to perform a search (the search box) and that they could safely ignore whatever else Google put on their screen. Additionally, people tend to be extremely task-focused when looking for information. They just want to answer their question, find what they need, make decisions, and move on with their lives — they aren’t visiting Google just to look around and see what’s new.

Google is likely aware of this discoverability problem and experimenting with ways to fix it. Since our research sessions, Google has added a shimmery rainbow outline animation that appears around the AI Mode button, likely in the hopes of drawing user attention to it.

But we’re skeptical that the animation will make much of a difference. The larger problem seems to be the vague “AI Mode” name and the lack of differentiation from Google’s other AI products and features.

Multiple, Poorly Differentiated AI Products

A big hurdle in the use of AI Mode was the lack of clear differentiation from the rest of Google’s AI features. When participants encountered Google’s AI Mode, their initial reaction was often confusion. One participant confidently told us that he’d used AI Mode in the past, but it later became clear that he was thinking of past experiences with a different Google AI product.

With the addition of AI Mode, Google now offers three different paths for retrieving information with the help of AI.

When asked about AI Mode, participants often conflated it with these other AI features.

“And this doesn’t give me Gemini?”

“But isn’t Gemini Google as well? They’re not the same, though.”

“I’ve seen it, but I thought it was just how you search for a product or something. It gives you quick AI information. I thought it was just that. That’s why I never clicked on it, because I just assumed it was the same information that you get when you quickly search for something on here.”

The distinction between these three AI offerings is not immediately clear to users, and there’s a fair amount of overlap in their functionality. All three use Gemini (the LLM) models and can execute web searches.

This confusion is unlikely to improve in the short term and may even be a bit intentional from Google’s perspective. In Google’s announcement of AI Mode, Elizabeth Reid (head of Search) indicated that features that perform well in AI Mode will be integrated into core Google Search. If anything, the distinction between these three offerings will likely become blurrier and more confusing to Google users.

Missed Onboarding Microcopy

Even though the AI Mode’s welcome page contains useful information, our participants never saw it. The only way to view the page was to click the AI Mode button from the empty search box on Google’s homepage, and none of our participants accessed it in that way. Many Google users will also overlook the onboarding pages because modern browsers typically feature a default search feature that allows users to enter queries directly into the URL bar.

Help and explanatory text is only useful if people see it.

That’s a problem, because people are pretty ingrained in their longstanding information-seeking habits, and it will take them some time to adapt their queries to AI-based search.

For example, participants in our study relied on traditional keyword phrases like shop refrigerator smart instead of asking natural-language questions.

Or, at times, participants did not realize they could ask followup questions in the chat.  One participant repeatedly used Google search to reenter AI Mode, while another used the back button, expecting to return to a previous response, only to leave AI Mode and lose the entire conversation.

Wordy, Unformatted Responses

AI Mode suffers from a problem common across all LLMs: it provides thorough but overly long responses, which discourage and overload users.

At best, participants quickly skimmed or skipped to the bottom of the response for a summary. But in other cases, the amount of text caused participants to abandon AI Mode entirely. One participant told us they would go back to manual, traditional searching.

“So, at this point, to be honest, I get a little bit [lost]. Like this is just too much.”

Sometimes long responses can’t be avoided, but simple formatting (bold text, subheadings, bulleted lists) make them easier to read. Additionally, collapsible sections could allow users to control the amount of visible text and, thus, make browsing smoother.

Lost in Long Conversations

AI Mode’s ability to support information seeking through a conversation that can build over time was often an advantage over traditional, one-query-at-a-time search. It allowed participants to explore complex information needs in more depth. But after a few exchanges, the conversation thread became quite long (exacerbated by the AI Mode’s tendency to give wordy responses). As a result, particularly on mobile, revisiting the results of a previous query required a lot of scrolling.

One participant spent several minutes chatting with AI Mode in her search for a refrigerator She noticed that one option offered by LG included access to a “ThinkQ App.”  Unfamiliar with this oddly named app, she asked AI Mode about its purpose. Once she understood what it offered, she began scrolling back through her conversation, trying to find the LG fridge she had seen earlier. However, she struggled to locate it and eventually wondered if it would be easier to just ask AI Mode to provide the link again.

“I don’t know if it’s faster to just type it [again] or just look back up for it.”

Adding jump links or a table of contents pointing to various questions within a thread would save users a lot of effort and allow them to easily refer back to previous answers. This is critical on mobile devices, where moving back and forth within a conversation will require excessive scrolling.

Mismatch with Users’ Mental Model

Another issue related to navigation was that the tabs in the persistent top tab bar (containing AI Mode, All, Shopping, Images, etc.) lead to search results relevant to the first query in the AI-Mode conversation, instead of reflecting the most recent user question in the conversation.

For example, one participant started by researching Mexican restaurants in an area, but her conversation eventually evolved to best hikes. When she clicked on the Images tab in the top tab bar, she was shown pictures of restaurants, not hikes. This disconnect was jarring and broke her workflow.

“That was kind of confusing to have all your restaurants. And then I switched over to hikes, but this up here [the Images tab] was back to restaurants. So, I had to go back.”

This is an example where users’ mental model clashes with designers’ conceptual model. Conceptually, the AI Mode page is a tab within the search-results page and a followup question asked within the AI Mode conversation is not a new query; the tabs are thus relevant to the current query — that is to the query entered in the search engine. To the user, however, any question, regardless of where they asked it, counts as a query.

Hallucinations

Like all LLM-powered chats, AI Mode is not immune to hallucinations. It tries to mitigate this by including references and sources to build user trust. These were well received by participants but did not solve the hallucination problem. Like any LLM, its behavior seemed to be less reliable for niche topics, especially.

When a participant attempted to utilize Google AI Mode to look for scientific studies about a specific niche neuroscience topic, he was initially impressed by its ability to source relevant papers. But, upon closer inspection, it became clear that the paper and its author did not actually exist.

In this specific example, the expert user caught the error quickly. But this will not be the case for all users, particularly for those relying on the AI to summarize or compare information they are unfamiliar with.

Conclusion

AI Mode is a powerful tool if you know how to find and use it — which is a big “if,” considering the range of usability problems we observed.

Google’s wide reach as a major search engine means many of its users are less familiar with new technologies such as AI. To help these users benefit from AI features, Google should offer a clearer value proposition and improved onboarding.

One participant, previously unaware of AI Mode, highlighted the importance of addressing these usability issues:

“I’d have to play around with it a little bit more to not get lost or not but (…) I think I might try it, use this instead of just regular Google. I don’t know. We’ll see. Because I’m still like, set in my ways too.”

This hesitation also applies to users with more exposure to AI, as the conversational nature of AI Mode may conflict with their established mental model of a Google search.

However, once the usability issues of AI Mode are addressed, Google will be able to build on its existing dominance in the search space. The fact that people are “set in their ways” or have an existing mental model of Google search means they’ve already established a habit of using it. By demonstrating the value of AI mode, effectively onboarding users, and connecting AI Mode responses to the wider Google ecosystem (such as Shopping and YouTube) in the responses, AI mode will be well-positioned to compete with other AI chatbots and AI-based search engines as they become mainstream.