Magento Open Source Adobe Commerce Storefront design & front-end Independent · not affiliated with Adobe
Magento Themes Store Storefront Lab
Published 13 Nov 2026 13 min read

Magento SEO: the structural problems, and how to find them

A practical audit of Magento search indexing: crawl waste, duplicate URLs, redirects, pagination, multi-store signals and the order in which to repair them.

Start with the URLs search engines can reach

Magento’s layered navigation is useful for shoppers and dangerous for crawlers when every filter combination is exposed as a crawlable URL. A category might be reachable as a bare category URL, with a colour filter, with a size filter, with both filters, and with different sort orders. A catalogue with several attributes can create many more combinations than the number of visible categories suggests.

These are not automatically thin pages. Some filtered views have demand and deserve to be indexed. The problem is not the existence of filter pages; it is treating all of them as equally important destinations without controlling which ones search engines should crawl, follow or consolidate.

Magento can generate URLs for filterable attributes and category combinations through its URL rewrite system. The important question is whether those URLs are being discovered accidentally, rather than being part of a deliberate URL and indexation strategy.

How filter combinations multiply crawlable URLs

Start with one category and list the filters that can be applied. Then list the values for each filter and calculate the possible combinations. The calculation is only an estimate because Magento may generate URLs in different orders, include parameters such as sort order, and expose combinations through links, sitemaps or internal links.

For example, a category with three colours and four sizes can produce twelve colour-and-size combinations before pagination and sort variants are counted. Across dozens of categories, the number can become large enough to make crawling inefficient and to dilute internal link signals.

Check whether filterable options are set to appear in category URLs. Also check the configured search-engine optimisation settings for category URLs and product URLs. A common mistake is to add canonical tags while leaving the crawler free to discover every filtered variant. A canonical tag is an instruction about which URL should represent a page, not a mechanism for reducing the number of requests required to discover that page.

What robots.txt does and does not solve

Robots.txt controls crawling. It does not remove URLs from the index, and a blocked URL can still be indexed if search engines have learned about it from internal or external links. It is therefore a blunt instrument rather than a substitute for sensible URL generation.

Blocking parameter patterns in robots.txt can stop a crawler exploring a family of filter URLs, but it also prevents that crawler from seeing the canonical tag or any noindex directive on those pages. If a filtered URL needs to pass signals to a category, the crawler must be allowed to fetch it.

Noindex is a different instruction. It asks search engines not to keep a particular URL in the index, although the page may still be crawled to read that instruction. It can be useful for low-value filter combinations, but it is not a reason to leave every possible combination linked from the category template.

Canonicalisation is usually the cleaner approach when several URLs represent the same content. It tells search engines which URL should be the preferred version. It does not guarantee that the preferred URL will be crawled, selected or indexed, especially when the alternatives compete with many other pages.

Measure the scale in crawl data and log files

Search-engine crawl statistics show how many requests are spent on Magento URLs and which directories consume the most crawl activity. Look for repeated visits to category paths, query-string URLs, filter parameters and redirect chains. Crawl reports are not a complete list of every request, so they should be read alongside server logs.

Log files show what happened at the server. Group requests by path, query string and response code. The useful patterns are repeated requests for filter combinations, pages returning unexpected redirect responses, URLs requested by more than one search-engine bot, and requests for URLs that no longer appear in the catalogue.

Check the dates and user agents carefully. A bot name in a log is not proof that the request came from a search engine, and crawl statistics can make a small URL set look more significant than it is. The point is to compare the crawl footprint with the number of URLs you actually intend customers and search engines to use.

Canonicals are a configuration choice, not a default safety net

Magento’s canonical URL settings for categories and products should be treated as a deliberate configuration. They are not a universal guarantee that every generated URL will resolve to the correct canonical destination. The settings also need to be tested against the URL rewrite data already in the catalogue.

A canonical implementation should answer three separate questions: which URL represents the category, which URL represents the product, and what happens when the same content is available through another store, a different URL suffix or a filtered category path. A single global setting cannot make those decisions safely.

For products, confirm that the canonical target is the product’s preferred visible URL rather than a URL containing an unwanted filter, tracking parameter or store identifier. For categories, choose the clean category URL that excludes filters unless a filtered view is deliberately the primary destination.

Use a crawl or a simple request to inspect the returned HTML. The canonical tag must be present on the intended page, point to an absolute, indexable URL, and not point to a redirect. Its target should return a successful response. A canonical pointing at a redirect is an avoidable chain of instructions.

Self-referencing canonical tags are generally the least confusing arrangement for ordinary category and product pages. Canonical tags do not solve the case where two genuinely different products have been given the same product URL, because search engines need a distinct canonical for each product.

Canonical is not the same as duplicate content

Duplicate content can arise from bad URL rewrites, copied descriptions, product variants, store views, temporary catalogue states and genuine filter combinations. Canonicalisation can consolidate signals when the pages are genuinely alternatives, but it cannot make two distinct products semantically identical.

Do not use canonical tags to hide a catalogue error. If a product is assigned the URL of another product, repair the catalogue and URL rewrite rules first. If a discontinued product redirects to a category, decide whether that redirect is useful to customers and search engines, then ensure it does not replace a better destination with a chain.

URL suffixes look cosmetic until the catalogue changes

Magento stores commonly use suffixes such as .html on product URLs. They may also use different suffix settings for categories. The suffix itself is not the main issue. The issue is the cost of changing it after products, categories and rewrite rules have accumulated around the old format.

Changing the suffix can create redirects for every affected product and category. If rewrite rules are not updated consistently, Magento may generate the old URL, the new URL or both. Search engines can then encounter a mixed set of links, redirect chains and duplicate destinations.

The cost is not only the time spent changing configuration. Products can have custom URL rewrites, imported rewrite rules, category assignments and external links pointing to the previous URL. A careless catalogue import can add another layer of rewrite records or overwrite the intended mapping, leaving the catalogue with several URLs for the same item.

Before changing suffixes, export and inventory the current rewrite rules. Record the old URLs, their destinations and their response codes. Then test a small set of products and categories, including products with custom rewrites, products in multiple categories and URLs already receiving traffic.

If the old format has external value, a single, tested redirect from the old URL to the new one is usually safer than allowing both versions to remain discoverable. Avoid redirecting a URL to itself, creating loops, or chaining several Magento-generated redirects to reach the final product page.

Pagination and rel attributes

Category pagination is necessary for large catalogues, but it creates another set of URLs. The first page should generally be the canonical category URL, with later pages linked as paginated views. Search engines need to understand that page two is part of the same category rather than a separate category competing for the same terms.

Magento’s pagination links should remain crawlable. Blocking them can make deeper products harder to discover, while generating every page for every filter combination can multiply the crawl problem. Use the pagination controls to expose a coherent sequence and keep the most useful links within it.

The rel="prev" and rel="next" attributes have limited value as a search-engine instruction. Google has said it does not use these attributes as a signal, and relying on them as the only pagination strategy is not enough. Crawlable links, clean canonicals and sensible internal linking do more practical work.

Do not canonicalise every paginated page back to page one if that makes deeper products unreachable from the category’s paginated URLs. The preferred arrangement is usually page one as the category’s canonical URL and each subsequent page as a legitimate paginated URL, with the product pages themselves remaining the destinations that can be indexed.

Watch out for combinations of pagination and filters. A filter URL may point to a paginated sequence of its own. Decide which filtered views are useful enough to crawl, and make sure the canonical and pagination logic does not contradict each other.

The URL rewrite table is part of the SEO system

Magento’s URL rewrite table is the mapping between request paths and catalogue destinations. It is used to resolve product and category URLs, and it also records rewrites for things such as old product paths, custom URLs and generated filter combinations. Its size and quality affect both the storefront and administration.

As the catalogue changes, the table can grow with every new product, category URL, imported rewrite and old URL retained for redirection. A large table does not automatically mean poor SEO, but it makes URL behaviour harder to reason about. It can slow category or product URL resolution, especially when many rules are evaluated against a request.

That administrative cost matters when a merchant imports new products regularly. The import may be correct at the product-data level and still create unnecessary rewrite records. Duplicate SKU imports, products being recreated rather than updated, and repeated custom URL rewrites can all increase the table without adding a new search result.

Look for rewrite records that point to the same destination through several paths. Determine which path should be canonical and which old paths need a redirect. Do not delete a rewrite record simply because it looks redundant until the redirect requirement has been checked.

For a catalogue import, use a staging or non-production environment where possible. Compare the rewrite count before and after, inspect sample records and verify that old product URLs still resolve to the intended product. Keep a record of removed and replaced SKUs so URL redirects can be tested rather than assumed.

Multi-store URLs need a planned relationship

Magento can run several stores from one installation, with each store having its own URL configuration, catalogue scope and potentially its own products or categories. This can be useful for markets, languages or store concepts, but it creates duplicate-looking URLs unless the relationships are handled deliberately.

Each store needs a stable, self-referencing URL for its own pages. The same product in two stores should not automatically canonicalise to whichever store happens to be indexed first. If the stores target different countries or languages, use the appropriate language and regional relationships rather than treating one as a duplicate of the other.

Hreflang annotations describe equivalent or corresponding pages across locales. They should be implemented with reciprocal references: each page lists the relevant alternates, and each alternate points back to the set. The annotations should describe genuine locale or market versions, not every store combination that happens to show the same product.

Do not use hreflang to solve an incorrect canonical setup. If one URL is canonicalised to a different store’s page, the hreflang annotations will be sending mixed signals. Decide the canonical model first, then map alternates.

For international stores, also test the server response and redirects outside the page. A locale path that redirects according to IP address can make a crawler receive a different page from the one linked in the HTML. The URL, response and annotations should agree for the store being requested.

A diagnostic order that works without an agency

Start with a URL inventory. Use the sitemap, crawl data, server logs and the URL rewrite table to build a list of the URLs being generated, requested or redirected. Separate pages that should be indexed from URLs that are parameters, duplicates, historical paths or unwanted filter combinations.

  1. Fix crawl exposure. Decide which filter and parameter URLs should be crawlable, which should be redirected or consolidated, and which should not be linked. Use robots.txt for crawl control only where blocking a crawler will not prevent a needed canonical or noindex instruction being read.
  2. Configure canonicals. Enable and test category and product canonical settings. Check the tag on representative pages, including filtered pages, paginated pages, multi-store pages and redirected URLs. Confirm that the target is a successful response and that the rules agree with the catalogue structure.
  3. Audit the rewrite table. Look for duplicate destinations, loops, chains, obsolete product paths and rules created by imports. Test the customer-facing response for a sample of each rule type before changing bulk data.
  4. Check pagination. Confirm that the first page is clean and canonical, later pages are linked and crawlable, and paginated URLs do not compete with the wrong category or filter destination.
  5. Map stores and locales. Document each store’s canonical policy, then add hreflang only where the same content is intentionally available to different language or regional audiences. Test reciprocal references and redirects.
  6. Re-crawl and compare. After changes, inspect the response codes, canonical tags, internal links and crawl distribution again. A configuration that looks right in the admin panel is not proof that the crawler receives it.

This process usually reveals that Magento’s technical SEO is a set of linked decisions: URL generation, crawl access, canonical selection, rewrite storage and store relationships. Changing one part without checking the others can create new contradictions.

The content half is not a Magento problem

Technical configuration determines whether search engines can reach, understand and consolidate the right URLs. It cannot decide whether a product description is useful, whether a category page explains the right buying choices, or whether the copy is distinctive enough to be worth retaining in results.

Content work belongs in the same audit, but it should not be confused with a platform repair. Product names, attributes, category introductions, internal links and editorial landing pages may need attention even when every response code and canonical tag is correct.

The practical distinction is straightforward: if the URL is not generated, redirected or indexed correctly, fix the Magento configuration. If the URL is technically sound but the page is weak, fix the page. Both halves are necessary, and only one of them is a Magento implementation problem.

Questions this page answers

How do filtered category URLs create structural SEO problems for Magento?
Every exposed filter combination can become a crawlable URL, potentially multiplying crawl activity and diluting internal link signals. Filtered views with demand can still deserve indexing.
Does robots.txt remove filtered URLs from search indexes?
No. It controls crawling, not indexing, and a blocked URL can still be indexed if discovered through internal or external links.
When is canonicalisation the cleaner SEO approach?
It is generally cleaner when several URLs represent the same content, because it identifies the preferred version. It does not guarantee that version will be crawled, selected or indexed.
How should category pagination be handled for search engines?
The first page should generally be the canonical category URL, with later pages linked as paginated views. Pagination links should remain crawlable, while avoiding generating every page for every filter combination.

More in Magento SEO

Editorial & corrections: [email protected]