What this guide covers
This guide covers crawl and index control for filtered and sorted category pages on an ecommerce site. It assumes you already have category pages that work and focuses on which URL variants should exist and be indexable.
Assumed knowledge
- A working category page structure
- Access to change robots and meta directives
Add five filters with four options each to a category page and you have created over a thousand URL combinations from one page of products. Add sorting and pagination on top and the number keeps climbing.
Most of those pages are worthless. A few are genuinely valuable. The job is telling them apart and then being disciplined about it.
Work out which facets have real demand
A filtered page deserves to be indexable when people actually search for that combination and the page answers it well.
- Worth indexing: a facet that matches how people search, such as a brand within a category, or a size that is a real buying constraint
- Not worth indexing: sort order, page size, view mode, and most multi-facet combinations
- Judgement call: price bands and colour, which depends on the category
The test is simple. Would someone type this, and does the resulting page give them a better answer than the unfiltered category? If not, it should not be in the index.
Give the valuable ones a real page
If a filtered combination is worth indexing, it needs to behave like a page rather than a filtered view.
- A stable, clean address that does not depend on parameter order
- A self-referencing canonical
- A heading and introduction specific to that combination
- Enough products that the page is not nearly empty
A page that fails those tests is thin content with a keyword in the title, and treating it as a landing page tends to hurt rather than help.
Control everything else
For the combinations you do not want indexed, there are three tools and they do different jobs.
| Control | Effect | Use it for |
|---|---|---|
| noindex meta tag | Page can be crawled, will not be indexed | Filter combinations you want followed but not indexed |
| robots.txt disallow | Page is not crawled at all | High volume parameter patterns with no value |
| rel canonical | Suggests consolidation to another URL | Near-duplicate variants such as sort order |
Normalise the parameters
Decide one canonical form and stick to it.
- Fix the parameter order, so ?colour=blue&size=10 and ?size=10&colour=blue cannot both exist
- Reject unknown parameters rather than rendering a page for them
- Strip tracking parameters from internal links and from canonical values
- Cap the number of simultaneous facets, typically at two
That last one matters more than it sounds. An uncapped facet system is how a site ends up with hundreds of thousands of crawlable URLs and a few hundred products.
Keep pagination ordinary
Numbered pages, with ordinary links a crawler can follow.
- Page one is the category address itself, never ?page=1
- Later pages canonical to themselves, not back to page one
- An out of range page number returns 404 rather than an empty 200
Infinite scroll is acceptable as an enhancement, provided real paginated links exist underneath it. If the only way to reach page four is to scroll, then for a crawler page four does not exist.
Check what you actually built
After any change to filtering, crawl the category section and count the distinct URLs. If the number is much larger than you expected, something is generating combinations you did not intend. That is far easier to fix in the week you built it than a year later.
Key takeaways
- A filtered page earns indexing only if people search for it and it beats the unfiltered category
- Sort order, page size and most multi-facet combinations should never be indexed
- Fix parameter order and reject unknown parameters so duplicates cannot exist
- Cap simultaneous facets, usually at two
- A disallowed URL can still be indexed, and noindex cannot be read on a disallowed URL
- Page one is the category address itself, never a page parameter
Frequently asked
Questions readers ask most
Should filtered category pages be indexed?
Some of them. A brand within a category or a size that is a real buying constraint often deserves indexing. Sort order, page size and most multi-facet combinations do not, because they produce near-duplicate pages with no distinct demand.
Is robots.txt or noindex better for filter URLs?
Use noindex when you want the page crawled but not indexed, which is most cases. Use robots only for high volume parameter patterns with no value. A URL blocked in robots can still be known from external links, and a noindex tag cannot be read on a URL that is disallowed.
How do I know if faceted navigation is hurting my crawl budget?
Crawl your category section and count distinct URLs against your product count. If the first number is several times the second, filters are generating pages nobody asked for. Server logs will show how much crawl is going to them.



