.Google.com's John Mueller responded to a question regarding why Google.com indexes pages that are actually disallowed from creeping through robots.txt and why the it is actually safe to dismiss the relevant Search Console records about those crawls.Crawler Visitor Traffic To Question Criterion URLs.The person asking the concern recorded that robots were actually creating links to non-existent question criterion Links (? q= xyz) to web pages along with noindex meta tags that are actually additionally blocked in robots.txt. What motivated the inquiry is that Google.com is creeping the links to those webpages, receiving blocked out by robots.txt (without watching a noindex robots meta tag) at that point acquiring turned up in Google Look Console as "Indexed, though blocked by robots.txt.".The person asked the complying with concern:." But below's the major question: why would Google index pages when they can not even find the web content? What is actually the conveniences in that?".Google.com's John Mueller validated that if they can't crawl the page they can't view the noindex meta tag. He likewise produces an interesting acknowledgment of the web site: search driver, suggesting to ignore the results since the "average" customers will not find those results.He composed:." Yes, you're right: if our company can't crawl the web page, our experts can't observe the noindex. That claimed, if we can't creep the webpages, at that point there is actually certainly not a lot for our company to mark. Thus while you might view a few of those webpages with a targeted website:- concern, the ordinary user won't view all of them, so I wouldn't bother it. Noindex is likewise alright (without robots.txt disallow), it simply suggests the URLs will end up being actually crawled (and end up in the Explore Console report for crawled/not indexed-- neither of these conditions cause concerns to the rest of the website). The essential part is actually that you don't create all of them crawlable + indexable.".Takeaways:.1. Mueller's solution verifies the constraints in using the Web site: search advanced search operator for diagnostic causes. Some of those main reasons is actually due to the fact that it is actually certainly not attached to the normal search mark, it's a different trait altogether.Google.com's John Mueller discussed the internet site hunt driver in 2021:." The quick solution is that a website: question is not indicated to be comprehensive, neither made use of for diagnostics functions.An internet site query is a specific sort of search that restricts the end results to a certain website. It is actually generally simply words internet site, a bowel, and afterwards the web site's domain.This query limits the results to a details site. It's certainly not indicated to be a complete collection of all the web pages from that internet site.".2. Noindex tag without utilizing a robots.txt is actually great for these type of scenarios where a bot is actually connecting to non-existent web pages that are actually obtaining discovered through Googlebot.3. Links with the noindex tag will certainly produce a "crawled/not recorded" item in Browse Console which those will not have a bad result on the remainder of the web site.Review the question and also address on LinkedIn:.Why would Google.com mark web pages when they can't also find the information?Featured Image through Shutterstock/Krakenimages. com.