6 Types of Duplicate Content and How Google Detects Them Online

Six Types Of Duplicated Content By Google Patent

Google Patents
November 17, 2024

Okay, so this paper called Detecting Query-Specific Duplicate Tokens is a patent by Google. And basically what this paper is all about is that it outlines the pattern of ways in which Google can use to detect duplicated documents. And then it knows how to find the unoriginal content or duplicated content and then use the best mechanics to display only relevant documents on the search engine result page.

How to Google Process Search Result Page to Filter Out Unoriginal Content or Duplicated Content

Below are the steps search engines like google can detect duplicate contents and only show users relevant results when they make a query on the search page.

Step One: User Enters a Query

So the first step, of course, is when the client goes into the search engine and puts in the query. Let’s say they put in a search term called “best coffee shops near me.”

Search Engine Analyzes the Query

And then what the search engine will do is it will take in this result, take in this term, and then try to check its own reposition. And then take this term into its repository and then try to see which tokens are most relevant to this term.

Grouping Relevant Documents

After, it tries to group the documents relevant to this term. Let’s say somebody searches about “best coffee shops near me” in the UK or in New York, it, it tends to group them based on the query.

Extraction of Relevant Snippets from Documents

The next thing for it to do is that it looks into each document and then looks at the most relevant part of the document that best serves the query to which the person or the user has re-positioned. This extraction of important pieces of text is called snippets.

Compression and Grouping Relevant Snippets

It then moves into the fifth stage, which is going to compress the document. So this relevant part, which it has drawn out from the stage 4, it tries to group them. And the reason why it’s grouping them, let’s say there is a document called “document A” and it has this “best coffee shops near me” information which is relevant to the query. Document B also has this “best coffee shops near me” information which is relevant to the query.

Filtering Out Duplicate Content

After it has gotten this snippet, which is the relevant permission it has drawn from each document, it then tries to see which one has the most similarity or which one leaves gaps. So based on these categorization of those relevant snippets, it tries to compare which are identical or similar. So whichever is identical and not authentic on its own, it tends to filter them out. This duplicate can be of different types (we will come to that).

Prevention of Duplicate Results for Improved User Experience

Then in the next step, after it has filtering out the duplicates, the search engine ranks the relevant documents in the search result page. And the reason why the system does all these (from stage one to 6) is because of user experience — to prevents the case where they need to go through ten links on Google which most of them seem alike.

Types of Duplicated Content According to Google Patent

So, this patent mentions six different types of duplicates, though it didn’t attribute a defined term used to explain each duplicate. It used diagrams, like from figure 1 to figure 6, so six different figures. So, I mentioned each figure, and then sought to explain each duplicate denoted by a figure. And then I try to give them a better name so you can easily understand what each duplicate means, even at first glance.

Figure 1 (An Expanded Duplicate)

For Fig. 1, I called name it “Expanded Duplicate.” This diagram contains two documents—the first document and the second document. So, as you can see (figure 1 in the image attached), the second document contains or encapsulates every bit and part of the first document. Moreso, you will notice that the writer of the second document went out of their way to include “a bit extra information” of what the first document does not.

Figure 2 (A Partial Overlapping Duplicates)

For the second figure, I call this a “Partial Overlap Duplicate.” When you check figure 2 in the image, you will see that both documents in that type of duplicate have similar content — but it doesn’t end there. Even as each has similar content, the writers for both documents went out of their way to research to provide information that is unique to both documents ensuring that each piece of info covers a particular aspect or topic that the other (writer) does not.

Figure 3 (A Minimal Common Duplicate)

Moving on to Fig. 3, I call this a “Minimal Common Duplicate.” In this case, the two different documents have some information in common. Document A and Document B each contain sections that are common to both, but it’s not necessarily the entirety of either document. This means a large part of each document is unique, with only a few differentiating common elements, which is why I call it a “Minimal Common Duplicate.”

Figure 4 (An Embedded Duplicate)

For Fig. 4, the first document contains all the content of the second document and a significant amount of additional information. So, for figure 4, we can see that document A entirely consumes document B. And document B, which is consumed by document A, is below 50% size of document A. So, to use a real life scenario to describe this, let’s say we have a big publisher like Yahoo News or BBC News, and then they publish a particular news story, and then a very small publisher website or a new publisher website just goes on that page, and they just copy half part of that document and they publish it. So, in essence, the first document or document A, which is from BBC, totally encapsulates the smaller version of information because the second or small publisher only picked few exact information from the document A here, which is the document from BBC. This is why I call it an “Embedded Duplicate.”

Figure 5 (A Contextual Duplicate)

For Fig. 5, we can call this a “Contextual Duplicate.” Here, document A and document B both have a little bit of information about Company X. In real-life context, we can see a news story from the BBC (Document A) about a release of product about Company X and the news story from the small publisher (Document b), in this case, the small publisher doesn’t entirely take a chunk about company X product release from the BBC’s article, but only a very little part. Let’s say the small publisher takes a small precise information about the product release from the news story from the BBC, and then they try to develop that with other information that they can get online or from the staffs of company X. So, in the very end, when this comes up in the search results, we will see that probably the title or even a small part of the body might be the same thing. For this can be in form of qoutes.

Figure 6 (An Embedded Contextual Duplicate)

Finally, for Fig. 6, which is the last kind of duplicate, I will call it an “Embedded Contextual Duplicate.” When we check the earlier diagram, we can see CompanyX was also used for illustration, but in a slightly different way. I also explain this using a real-life scenario that covers CompanyX (precisely its biography) and how its founder came to be. The Document B could be written by a BBC journalist who focused solely on the history of CompanyX.. As for Document A, it was written and published by a writer at Wikipedia detailing not only how CompanyX came about but also how its CEO grew from the grassroots to founding the company.

Complete Patent Document: Detecting Query-Specific Duplicate Tokens

AdminMuyi

Writer & Blogger

About SEO Expert

Started as a writer. Learnt journalism at university, although not my major. Later got an SEO content writing gig — my first exposure to Search Engine Optimization. I’ve been in love with it since then. And here we are today!

Menu

Social