Guide · Duplicate content

Canonical tags and duplicate content: how to set them up on a multilingual site so pages are not treated as duplicates

Updated 2026-11-07 · kagentx

Short answer

Give every page one self-referencing rel="canonical" (absolute URL, in the head). On a multilingual site, each language version's canonical points at itself, and the versions point back at each other with hreflang plus an x-default. MochiStamp's live pages do exactly this: each page's canonical is itself, hreflang lists only versions that exist, and x-default points to English. Bulk pages from one template with a few words swapped are a different matter: Google's spam policy has “scaled content abuse”, and pages that add no value for users are at risk. Whether AI search reads canonicals is unverified.

What is a canonical, and does AI search look at it?

A canonical is how you tell a search engine “these URLs have the same or very similar content; treat this one as the main one”. Google's documentation (checked 2026-11-07) says that without one it picks a version itself. Whether AI search reads canonicals, we found no explanation in OpenAI, Anthropic or Perplexity documentation, so it is unverified; but with the right canonical, at least the URL that traditional search indexes and shows is the one you want.

Which ways are there to set a canonical, and which is strongest?

Google lists three methods as signals of different strength, and they can be combined:

MethodStrength (Google documentation)When to use it
A permanent redirect such as 301StrongWhen retiring a duplicate page. Server-side redirects take effect fastest
A rel="canonical" link, or an HTTP headerStrongWhen the duplicate must keep existing. Use the HTTP header for non-HTML files such as PDFs
Listing in the sitemapWeakOnly a suggestion; do not rely on it as the main method

Do not point the same page at different URLs with different methods. Choose either the HTML link or the HTTP header; do not write both.

Where does a canonical go, and what makes it valid?

RuleHowConsequence of breaking it
LocationOnly in the head; scripts must not change itOne in the body is ignored (Google documentation: only accepted in the head)
URLUse an absolute URL with https and no fragment (#)A relative path or a fragment may not be used
Self-referencingThe canonical page also carries a canonical pointing at itselfWithout it, Google has to guess the canonical page's identity
Internal linksInternal links point to the canonical URL, not to a duplicateThe signals contradict each other
Do not use robots.txtDo not use robots.txt for canonicalizationA blocked URL can still be indexed, just without its content
Do not use noindex to choose a canonicalWithin one site use canonical, not noindexnoindex removes the page from search entirely
AttributeA canonical with hreflang, lang, media or type attributes has no effectWriting it is the same as not writing it
HTTP and HTTPSDo not put HTTP URLs in the sitemap or hreflangThe canonical URL gets split

For how to write internal links see Do internal links help AI search?; for the problems JavaScript rendering causes see Do AI crawlers render JavaScript? (the canonical must be in the HTML the server outputs).

How do canonical and hreflang work together on a multilingual site?

There are only three rules: each language version's canonical points at itself (same language); all language versions point back at each other with hreflang, including themselves; and add an x-default. Google's hreflang documentation says one-way annotations are ignored (checked 2026-11-07). Do not point every language version's canonical at the English version, which tells Google the other languages are duplicates.

Below is the head of mochistamp.com/en/guide/size on 2026-11-07 (only canonical and hreflang kept):

<!-- https://mochistamp.com/en/guide/size -->
<link rel="canonical" href="https://mochistamp.com/en/guide/size">
<link rel="alternate" hreflang="ja" href="https://mochistamp.com/guide/size">
<link rel="alternate" hreflang="en" href="https://mochistamp.com/en/guide/size">
<link rel="alternate" hreflang="zh-Hant" href="https://mochistamp.com/zh-hant/guide/size">
<link rel="alternate" hreflang="zh-Hans" href="https://mochistamp.com/zh-hans/guide/size">
<link rel="alternate" hreflang="x-default" href="https://mochistamp.com/en/guide/size">
Pagecanonicalhreflang
/guide/size (Japanese, no language prefix)Itselfja, en, zh-Hant, zh-Hans, x-default
/en/guide/size (English)ItselfSame as above, x-default points to English
/ar/phrases/kimochi (Arabic)Itself (starting with /ar/)Seven languages (Japanese, English, Traditional and Simplified Chinese, Arabic, Persian, Russian) plus x-default

You can see that each page's canonical is itself; hreflang lists only versions that exist (the size guide has only four languages, so it lists four); and x-default points to English. kagentx's own pages do the same: one self-referencing canonical per page, three languages pointing at each other, and x-default pointing to the English version. For other points on Arabic and Persian pages see How do you write hreflang on a multilingual site? How does AI search treat it?.

What are the most common mistakes on multilingual sites?

MistakeWhy it is wrongHow to fix
Every language page's canonical points at the English versionFor sites using hreflang, Google asks for a canonical pointing at a page in the same languagePoint each language version at itself
hreflang in one direction onlyTwo pages do not point back at each other, so the annotations are ignoredEach version lists all versions, and the list is identical on every version
The page is missing from its own listThe documentation requires each version to include itselfAdd an alternate that points at itself
A region alone, or an invalid region codeA region code cannot be used alone; the United Kingdom is GB, not UKLanguage code first, such as en-GB or zh-Hant
Relative pathshreflang URLs must be complete, including httpsWrite all of them as absolute URLs
hreflang pointing at pages that do not existOnly some pages were translated but all languages are listedList only the versions that exist
Different canonical URLs for the same page from different methodsThe signals contradict each otherKeep the HTML link, HTTP header and sitemap on the same URL

Write the sitemap lastmod separately for each language version; see How do you write a trustworthy sitemap lastmod?.

What is the risk of pages generated from one template with a few words swapped?

Google's spam policies include “scaled content abuse” (checked 2026-11-07): generating many pages that give users little value in order to manipulate rankings. The policy is about purpose, not how the content is made (the text says: no matter how it's created). Its examples include:

  • Using generative AI to produce many pages without adding value for users
  • Scraping content from elsewhere to generate many pages
  • Automated transformations such as synonymizing or translating that add little value for users
  • Stitching together content from different pages without adding value
  • Creating multiple sites to hide the scaled nature of the content
  • Many pages that make little sense to a reader and are stuffed with search keywords

The document ends by saying that if you host such content, you should exclude it from search. It does not define “duplicate content”, and it does not say that language versions are a violation in themselves; translation appears in the examples only when it adds “little value”. What multilingual versions should do is the canonical and hreflang setup in the previous section, not deleting pages.

Our own rule (the content operations handbook, 2026-11) is not to publish pages generated from one template with a single word swapped. How to write "X for Y" use-case pages that AI will cite shows how to give a use-case page something nobody else has.

How can you tell template pages from duplicate pages yourself?

MochiStamp's phrase pages are a set built from one template: 10 pages under /phrases/, each with 24 phrases and drawing ideas (read 2026-11-07). We took the main area of the page and compared three Japanese pages line by line after removing repeats:

ComparisonLines after removing repeatsIdentical linesTakeaway
kimochi (feelings) and kihon (basic greetings)156 vs 11341 (31% of kimochi)The identical lines are mostly navigation, buttons and descriptions
kimochi and koibito (lovers)156 vs 11341 (31% of kimochi)The phrases, drawing ideas and intro text all differ

This is a rough comparison of ours, not the way Google judges duplicates, and it cannot prove these pages will not be treated as duplicates. What really differs: each page has its own 24 phrases, a drawing idea for each phrase, an intro paragraph about that scenario, and a title in that language. The Arabic version of such a page (/ar/phrases/kimochi) had 86 impressions in the last 28 days (2026-11-05 data), which shows it has at least been indexed and shown.

A question to ask yourself: put two bulk pages side by side, remove the navigation and footer, and see how much of the remaining body exists only on that page. If only one word differs, merge them into one page or add something nobody else has.

How do you check your own site, step by step?

  1. 1. Look at the canonical in each page's head

    View the page source (not the element tree in developer tools, which may include scripted changes) and confirm there is exactly one, in the head, as an absolute URL, pointing at itself.

  2. 2. Check that multilingual pages point back with hreflang

    Open two language versions; their lists should be identical and include themselves and x-default.

  3. 3. Check the canonical of every language version

    Each version points at itself; none should all point at English.

  4. 4. Check the sitemap and internal links

    Put only canonical URLs in the sitemap, over HTTPS; internal links also point at canonical URLs.

  5. 5. Find bulk pages from one template

    Put pages from one template side by side, remove the shared parts and look at what remains; where little remains, add content or merge.

  6. 6. Review in Search Console

    In the page indexing report, look for reasons of the “duplicate page” kind and spot-check a few URLs. The exact wording in that report is whatever your account shows; we have not checked each item.

When trailing-slash forms are mixed, whether to merge them with 301 or 307: see Is a 307 on trailing slashes bad for SEO? How to choose between 301, 302, 307 and 308.

Whether a page with nothing unique should be noindexed, and what noindex and robots.txt Disallow each do for Google and for AI crawlers: see Should thin pages be noindexed? What it means for Google and AI search.

The actual hreflang and canonical pattern on live Arabic and Persian pages is in Arabic and Persian pages in search and AI answers: what one small site saw.

How a single-query page and its parent page divide the work when their content differs (a job for page roles, not canonical) is covered in Is one page per query worth it? Data from single-query pages on a small site.

FAQ

Does every page need a canonical?

Recommended. Google's documentation advises that the canonical page also carry a rel="canonical" pointing at itself, as an absolute URL, in the head.

On a multilingual site, can every language page's canonical point at English?

No. For sites using hreflang, Google asks for a canonical pointing at a page in the same language; pointing everything at English says the other language pages are duplicates of the English one.

What happens if hreflang is missing return links?

Google's documentation says that if two pages do not point at each other, the annotations are ignored.

Is translating into several languages duplicate content?

Google's policy page does not list language versions as a violation and does not define “duplicate content”; its examples are bulk pages made by automated transformations that add little value for users. The approach is a self-referencing canonical and mutually pointing hreflang for each language, with the title and body written in that language.

Does AI search read canonicals?

We found no explanation in the official OpenAI, Anthropic or Perplexity documentation, so it is unverified. Getting the canonical right at least ensures that the URL cited from traditional search is the one you want.

Sources

kagentx

Which canonical does a crawler read from your page?
Let the evaluation tell you

The evaluation checks whether crawlers can get in and whether the body is server-rendered, then sends buyer questions to ChatGPT, Perplexity and Claude (with web search on) and checks whether the URL they cite is the one you want. The Basic plan is $1, and results usually arrive in 2–3 minutes.