The Surprisingly Secret SEO Recipe:
PRG Pattern For Advanced Layered Navigation SEO
It’s now a few weeks since we released our Magento 2 SEO Extension “JaJuMa PRG Pattern Link Masking” and since it was published we received some first valuable and interesting feedback. Based on this first feedback we had to notice:
Usage of PRG Pattern in SEO is still surprisingly unknown.
This insight came as a not small surprise to us, because actually PRG Pattern as SEO measure is not exactly a secret or a new recipe to tackle SEO issues related to Layered Navigations. In fact, it has been around and discussed in the SEO scene for some time and we have been using it for our Magento 1 clients before and have been seeing great results with it!
So What Is PRG Pattern & This SEO Issue With Layered Navigation About Exactly?
Almost every online store features a so-called layered navigation, also known as facetted or filter navigation. This function allows customers to narrow down product collections, e. g. on a category page, to find the suitable product based on product attributes and criteria’s like color, price, material, size… you name it. But this neat little function we see everywhere can also be a beast – technically and from an SEO perspective.
Of course, while technically for sure not the easiest part for an online store to implement in a good way, decent layered navigation are readily available for anyone using standard online store software like Magento - It’s just a matter of configuration and that’s it.
But wait, just after setting up, happy about providing this awesome and handy tool to the customer, there comes the second thought:
What about SEO with layered navigation? And this is where the challenge begins…
SEO Challenges With Layered Navigations
The biggest challenge with layered navigation when it comes to SEO is the sheer number of URLs created. For each available filterable attribute combination, in each category and each also combined with sort options, number of products shown, pagination parameters etc., a new URL is created. Try to calculate the number for your or any online store, the more filterable attributes you have the worse it gets. With literally countless possible combinations and new URLs created and exposed to as well as crawled by search engines.
(Near-) Duplicate Content & Waste Of Crawl Budget
The problem with these URLs: They represent no new, no valuable content for search engines, but only (Near-) Duplicate Content already present and found on other URLs.
In fact, these (from an SEO perspective) “useless” URLs easily outnumber pages that have more SEO value and real content, causing Search Engine Crawlers and Bots to waste a major part of the limited Crawling Budget on useless content, while new or valuable pages remain un-crawled. Additionally, this unlimited number of useless pages also pass their Link Juice to other linked URLs and pages, which are again these useless URLs you don`t even want to be indexed. In other words:
Layered Navigations create Duplicate Content as well as a pretty mess in your online store. With a messed up internal link structure causing limited crawl budget for your site to be wasted and valuable Link Juice being lost to irrelevant pages.
How NOT To Fix SEO Issues With Layered Navigations…
There are many recommendations and advice out there on how to fix the issues described above. Usually, it`s about using the typical suspects utilized in SEO, e. g. “Meta Robots”, Canonical URL, robots.txt, “rel NoFollow” etc.
Don`t want to go into detail regarding each of these in this post. But none of these traditional SEO methods address the described issues sufficiently and satisfactory. We described the limitations and downsides of each of these SEO techniques regarding problems discussed in more detail in another blog post regarding SEO optimization for Layered Navigation.
Just one more idea that we were approached with recently:
How about we simply hide the whole filter navigation block from Google Bot by using display:none or not render it at all based on User-Agent?
Uh, no, please never do something like this, never ever, never. It’s really not a good idea trying to cheat Google! And yes, they will figure it out, Google also visits your site using “regular” User-Agents. So chances are good your little trick will be discovered.
…And HOW TO Fix SEO Issues With Layered Navigations (using PRG Pattern)
But, the idea of “hiding” is actually not completely bad. At least if we could do this in a smarter, less hacky way. Now the question is, how can we do it better?
That’s where PRG Pattern (Post – Redirect – Get) comes into play.
It’s actually nothing new, Wikipedia defines it as follows:
Post/Redirect/Get (PRG) is a web development design pattern that prevents some duplicate form submissions, creating a more intuitive interface for user agents (users).
Wait, “duplicate form submissions” and “intuitive user interface”… how does that relate to a technical SEO issue as discussed here with Layered Navigations?
That’s not too hard to understand when looking at the details on how PRG Pattern works. The interesting part for our problem is the HTTP POST request. For those not familiar with different HTTP request methods, simply explained, POST requests are used for submitting forms to a server, e. g. contact forms, login forms, orders etc.. For obvious reasons search engine bots and crawlers never follow/submit POST requests. Never, because they don’t know the credentials to log in, but also never because otherwise, they would spam sites with wrong contact messages and false orders. No legit crawler or bot will ever want to do that.
Here we go, with POST requests we have the perfect tool for making certain parts of an online store or a website inaccessible for bots and crawlers. Not in a hacky way, just by using a technique already present and used on almost any website, just for a different purpose. Think of a customer account area, only accessible after submitting valid user credentials, it remains simply invisible or hidden to any search engine.
Guess you see where this is going. By simply dropping in the POST request for Filter Navigation before the normally used GET request, we solve all our problems and issues as outlined above. We simply move the mess of unlimited and useless URLs to be in a for search engines and bots unreachable and invisible space. All issues with Duplicate Content, wasted Crawl-Budget and cluttered internal link structure caused by Layered Navigations, fixed in one go by applying this simple trick.
Best of all, it works without giving up on the benefits of a Layered Navigation. It all happens in the background, completely transparent to the normal user. Filter Navigation simply work as before from user perspective.
Sounds too good to be true?
Yes right, certainly too good to be not considered by any online store taking SEO for Layered Navigation serious ;-)
The PRG Pattern is an awesome and elegant way to solve and avoid SEO issues that come with Layered Navigations. Since we (and others) are already using it for some time with great results and seeing the huge optimization potential this elegant and advanced SEO technique provides for technical on-site SEO, it remains a miracle to us why it has not been adopted on a wider range yet. True, it might require a bit of technical understanding to get the idea of how and why it works. And it might be a bit more difficult to implement than just taking care of Metadata and tags, canonicals, robots.txt and other SEO tips found in the average SEO guides.
On the other side, in competing for good rankings, basically everyone is already optimizing regarding these more traditional SEO measures. In order to achieve the decisive advantages and make the difference in comparison to the competition, it is usually not enough to do exactly the same thing. For the goal of achieving top ranking positions, one should always think ahead and move on. PRG Pattern might be exactly this missing building block in your SEO strategy for reaching the next level.
The good news for Magento 2 merchants:
Applying and improving your SEO with PRG Pattern is now as easy as installing and enabling an Extension.
Don’t wait and miss this chance any longer!