What would be the cause of some discrepancies in a number of indexed pages in my site Search Console?

I’m seeing some discrepancies in a number of indexed pages in my site Search Console, which is a single page app. What would be the cause of this? Would pages receive the index quickly if there was a temporary problem crawling HTML snapshots?

Webmaster Guru Asked on September 15, 2016 in Indexing.
Add Comment
1 Answer(s)

JOHN MEULLER (Webmaster Analyst from Google):

So there are two things, mostly, here. On the one hand, the index status report has been stuck for a while. I think it’s now getting data again. So that’s something that should be catching up again. So if you’ve made significant changes in the last couple of weeks, then I’d double check that report again to see that it’s actually being shown. On the other hand, specifically if you’re looking to see that your important pages are being indexed, then I’d recommend setting up a sitemap file where you can see the number of indexed URLs per sitemap file in Search Console and focus on those numbers rather than the broader index status or site query numbers. With regards to single page apps, sometimes that’s tricky. So in general, with single page apps, we have one HTML page. And the JavaScript generates the content that’s actually shown. It pulls it separately from the server. So when we crawl the page and just see that one HTML page, all of those pages look the same to us at first. We have to render them before we can actually recognize the content there. And one of the issues that might happen there is if the JavaScript is blocked by robots.txt, blocked in other ways, then maybe we won’t be able to pick up that JavaScript, process it properly. And in the end, it will look like all of these pages are just the same HTML page. And then we say, well, we can fold them all into one page and just index them like that. So specifically for single page apps, really make sure that your JavaScript is working well that it doesn’t crash when Googlebot renders it that Google is able to actually fetch it properly. That’s really important there, because we can’t just look at the HTML page. We have to actually render the full content to actually get what you’re providing. So that’s the main issue that I’ve seen where people have had problems with regards to single page apps or JavaScript frameworks in general is that we can’t process the JavaScript properly. Therefore, we can’t actually index the content properly. In some cases, you might be able to look into prerendering these pages and serving us a prerendered version or using, like I mentioned before, isomorphic JavaScript techniques to render this on the server for all users when they come for the first visit, which can provide a significant speed-up as well.

Webmaster Guru Answered on September 15, 2016.
Add Comment

Your Answer

By posting your answer, you agree to the privacy policy and terms of service.