Watch out for low-quality sources when using Google Deep Research

A young man sits at a vintage computer in a traditional college office setting, typing with a focused and engaged expression. Behind him, a woman with curly hair, wearing a turquoise cardigan, leans in with a supportive and encouraging smile. The background features wooden bookshelves filled with books, a large window letting in natural light, and a desk covered with stacked books and papers, creating an academic atmosphere.

College interns aren’t perfect, and neither is Google Deep Research (Generated by Google Gemini Advanced).

Recently, my favorite newsletter linked an article about Sweden going back to paper textbooks after 15 years of using digital devices. This was the first I had heard about it.

The headline intrigued me because I was looking for the latest research on screen time and its impacts of using digital devices in the classroom. Unfortunately, the article was short and light on details. Ads flashed at me between every paragraph. Typos and poor grammar filled the comment section. These telltale signs of a content farm suggested to me I wasn’t getting the full story. What research was used to support this decision? What do teachers think of the policy change? These questions seemed like a good opportunity to try Google Deep Research.

What is Google Deep Research?

Deep Research is like web search on steroids. Tell it what you want to know, and Google’s new AI agent will search and synthesize a report based on dozens or even hundreds of web pages. After submitting your search request, Deep Research devises a web search plan. You can edit the plan if you feel it’s missing something or includes topics that don’t interest you.

Screenshot of a Google Deep Research session with labeled sections. The Prompt section in the top left contains a user query asking why Sweden decided to return to printed textbooks and what research supports this decision. Below, the Research Plan outlines steps, including finding articles and government reports. The Report Pane on the right displays a detailed analysis titled Sweden’s Return to Textbooks: A Shift in Educational Priorities, discussing Sweden’s move away from digital learning, historical context, and concerns about digital education. A Source from the Swedish government is cited as evidence. The layout includes headings, structured text, and a button to open the report in Google Docs.

Google Deep Research interface displaying the report for my first try at digging deeper into Sweden’s policy change.

Executing the plan takes several minutes. First, it lists all the websites it finds. Then it creates a report of its findings, complete with subheadings to help you quickly review the output. Nearly every paragraph has a “Learn more” button at the bottom that expands to display links to its sources. After Deep Research finishes, you can ask follow-up questions or save the report as a Google Doc.

This ability is amazing. It’s like having a college intern at your fingertips. While much faster than a human, is it better? Let’s see how it fared researching Sweden’s policy change on textbooks.

Deep Research Example

Here is the prompt I entered:

I want to know more about why Sweden decided to reverse course and buy textbooks for their classrooms. What research are they using to support this decision? What do teachers think of this decision?

The plan looked good, so I went ahead and started the research. The report’s first paragraph provided a decent summary. Moreover, it stated when the policy was announced, a detail lacking in the original article I read. Even better, the policy announcement from Sweden’s Ministry of Education and Research was the paragraph’s source. So far, so good, but the results degraded from there.

Screenshot of a research report titled Sweden’s Return to Textbooks: A Shift in Educational Priorities. The text discusses Sweden’s decision to reintroduce printed textbooks in classrooms after years of a digital-first approach, emphasizing a broader shift towards a knowledge-based education system focused on fundamental skills. A high-quality primary source from the Swedish government is cited, with a link to an article about government investment in more reading time and less screen use. The highlighted phrase “High-quality primary source” is in red text.

I wish the report used more high-quality sources like this one.

Statements in the report’s subsequent paragraphs sounded familiar. Drilling down into the sources revealed low-quality/content farm websites like bellesandgals.com, jasondeegan.com, glassalmanac.com. The original article from the newsletter was also cited. In fact, many of these sources were nearly identical to each other. They’re all likely AI generated rehashes of the same content.

Screenshot of a research report section titled “Challenges and Concerns: The Unforeseen Consequences of Digital Learning.” The text discusses drawbacks of Sweden’s digital-first education approach, highlighting concerns from parents and teachers about students’ declining fundamental skills in reading and writing. Teachers reported students struggling to focus during lessons and recall information from digital texts. Parents were also worried about children using computers for non-educational purposes, such as gaming and social media. Below the text, two cited sources are shown: one from bellesandgals.com and another from indiandefencereview.com, both discussing Sweden’s 2009 decision to replace books with computers.

Both of these sources are from content farms.

While none of these low-quality sites seemed to have errors, I became wary of Deep Research. If Deep Research can’t differentiate between high- and low-quality sources, can it reliably assess their accuracy? This is an important question because content farms don’t exist to inform. They exist to make money by firing as many ads as possible in front of your eyes. The possibility of fake content requires a default “trust but verify” attitude. You can review the entire report and sources for yourself in my first Deep Research conversation.

The Deep Research report used high-quality sources as well. Besides the Swedish government release, other sources included the European Union’s website, a Northern Michigan University Master’s thesis, and this Al-Estiklal article, which also linked to high-quality primary sources. Still, too much of Deep Research’s report was sourced from low-quality sites for my standards.

Never one to give up easily, I wondered if better prompting could help Deep Research improve the report.

Prompting for high-quality sources

What happens if I try to prompt it to avoid low quality websites? Can I ask it to use high page rank sites? Can I bias it toward pages containing high quality outbound links?

Here’s a second prompt I tried in a new conversation:

I want to know more about why Sweden decided to reverse course and buy textbooks for their classrooms. What research are they using to support this decision? What do teachers think of this decision? Please focus on high-quality sources like government press releases and primary news sources.

The output from this new prompt was marginally better. New sources appeared about declining reading test scores of Swedish fourth graders. However, the first paragraph used the content-farm site instead of the Swedish government announcement. Otherwise, this second report was about the same as the first.

After spending a couple hours vetting all the sources in these two documents, I still felt I was missing something. Evermore curious, I wondered what a good old-fashioned Google search would turn up.

Sometimes the old ways reign supreme

To compare with the Deep Research reports, I tried the following Google search: “why sweden go back textbooks”

Screenshot of a Google search results page for the query “why Sweden go back textbooks.” The featured snippet states that Sweden has announced a return to physical books, as “they believe digital education is not working,” with an estimated cost of €50 million for new books. Below the snippet, search results include a Newstalk article titled “Should schools go back to using textbooks?”, a Guardian article discussing Sweden’s shift to back-to-basics schooling, and a Reddit thread about Sweden’s past decision to replace books with computers. Some search results mention Sweden bringing back more books and handwriting practice.

The second link was better than two Google Deep Research reports.

The Guardian-branded AP article was much better than the Deep Research report. It went beyond the Deep Research output with much more detail and color. Here are several examples that were absent in the Deep Research reports:

More details on Swedish test scores (a major justification for bringing back paper textbooks).
Compared scores from other countries.
Noted that while Sweden’s test scores declined, they still tied for the seventh-highest score with Taiwan.
Cited education expert opinions on other possible explanations for the change in test scores (pandemic gap, more non-native Swedish speaking students).
Quotes from teachers, students, and other education experts.
Discussion of a UNESCO report issuing an “urgent call for appropriate use of technology in education.”
Critical comments from an Australian teacher wary of the government’s motives for the change.

So one article, quickly found by a traditional Google search, had much more (clears throat) depth than my Deep Research reports.

This disparity also begs the question, why didn’t the Deep Research reports include the AP article? I don’t know. Maybe the answer lies in understanding how OpenAI’s new deep research tool works.

A new deep research tool

OpenAI released their version of deep research last week, and it has already impressed AI experts like Ethan Mollick. He compares OpenAI’s tool to a beginning PhD student. Google’s Deep Research, however, performs more like an undergraduate student researcher. OpenAI powers deep research with their newest and most powerful reasoning model, o3. In contrast, Google built Deep Research on their older non-reasoning model, Gemini Pro 1.5. Reasoning models use a lot more compute resources to make better decisions. I suspect Google’s inability to filter low-quality content stems from using a less capable model.

Is Google’s Deep Research still worth using?

Yes. Despite its flaws, Google Deep Research is a good place to start and get leads on your topic, especially if the topic is complex. However, if you’re not yet paying for Google Gemini ($20/month), try a quick Google search first. As I showed above, sometimes a simple Google search surpasses Deep Research.

Also remember that it’s early, and AI is rapidly improving. Google’s Deep Research will improve in a matter of months, if not weeks. Using this imperfect tool still helps build your AI intuition. That “sixth sense” will pay dividends when you’re able to use Google’s next version more effectively.