Is it fine when category filter reaches less than 15 products, we set a meta tag to robots, no index, no follow?

Is it okay when category filter reaches less than 15 products, we set a meta tag to robots, no index, no follow?

Webmaster Guru Asked on November 11, 2016 in Sitemaps.
Add Comment
1 Answer(s)

JOHN MEULLER (Webmaster Analyst from Google):

That sounds pretty good to me.Let’s see how that goes.And then a question about the crawl rateof no index, no follow URLs.So Google bot will probably be crawling a lot of these pages.We’ll find that no index, no follow.And that’s fine for us.So usually what happens is we have enough capacity to crawl your website anyway.So we can dig through a lot of these pages and just ignore them if they’re no index.You can also guide us a little bit if you know ahead of time which links lead to no index pages,and maybe put in rel no follow on those links.So depending on the way you have your site structure set up,that might be an option.If crawling these no index pages is a critical burden on your server that causes problems for normal users,then I would consider using a robot’s text file.And really, blocking crawling of those pages completely– it doesn’t prevent them from being indexed completely,because we don’t know if there’s a no index there or not.But at least the content isn’t there.And usually they don’t rank for any of the content that might be on that page.But that’s something I’d really only do if it’s really critically hard to actually crawl all of those pages.

If your server is overloaded and really can’t handle that load.The second part here is we have some brand temporarily without products.Can we redirect users using a 302 to a category with the same products,or almost the same products, or do we need show a 404 page?If you essentially have a replacement page for the existing page that’s currently not existent,then a redirect might be an option there.I think a 302 helps users, because that’s not something that gets cached on their side.So what would generally happen is they would go to that URL again. And if you suddenly do have content there,then that would be visible.Otherwise, they would follow that redirect


Webmaster Guru Answered on November 11, 2016.
Add Comment

Your Answer

By posting your answer, you agree to the privacy policy and terms of service.