Google notifying webmasters to remove noindex from robots.txt | GK Web Agency
GK Web Agency
Last Update
30 July 2019
Reading Time: 3 minutes

Google notifying webmasters to remove noindex from robots.txt

Google is now sending reminders that you need to stop depending on your noindex in your robots.txt file.

As part of Google fully removing support for the noindex directive in robots.txt files, Google is now sending notifications to those that have such directives. This morning, many within the SEO community started getting notifications from Google Search Console with the subject line “Remove “noindex” statements from the robots.txt of…”

Google is now sending reminders that you need to stop depending on your noindex in your robots.txt file.

As part of Google fully removing support for the noindex directive in robots.txt files, Google is now sending notifications to those that have such directives. This morning, many within the SEO community started getting notifications from Google Search Console with the subject line “Remove “noindex” statements from the robots.txt of…”

September 1, 2019.

That is the date that you need to no longer depend on the noindex mention in your robots.txt file. This is something Google announced earlier this month and is now sending out messaging to help spread the word of this change.

Why we should care.

If you get this notice, make sure to ensure whatever you mentioned in this noindex directive is supported a different way. The most important thing is to make sure that you are not using the noindex directive in the robots.txt file. If you are, you will want to make the suggested changes above before September 1. Also, look to see if you are using the nofollow or crawl-delay commands and, if so, look to use the true supported method for those directives going forward.

What are the alternatives?

Google listed the following options, the ones you probably should have been using anyway:

  1. Noindex in robots meta tags: Supported both in the HTTP response headers and in HTML, the noindex directive is the most effective way to remove URLs from the index when crawling is allowed.
  2. 404 and 410 HTTP status codes: Both status codes mean that the page does not exist, which will drop such URLs from Google’s index once they’re crawled and processed.
  3. Password protection: Unless markup is used to indicate subscription or paywalled content, hiding a page behind a login will generally remove it from Google’s index.
  4. Disallow in robots.txt: Search engines can only index pages that they know about, so blocking the page from being crawled often means its content won’t be indexed. While the search engine may also index a URL based on links from other pages, without seeing the content itself, we aim to make such pages less visible in the future.
  5. Search Console Remove URL tool: The tool is a quick and easy method to remove a URL temporarily from Google’s search results.
Wordpress Developer Brisbane • Brisbane Website Designers

CHOOSE YOUR INDUSTRY

GK DIGITAL

GK Web Agency is a web agency that designs, builds and manages Website & eCommerce worldwide. Based in Australia, we are dedicated to serving our customers with personalized campaigns that focus on losing the big data to meet big goals.

Contact

GooGle Partner


Brisbane Website Designer | Brisbane Online Marketing | GK Web Agency

Copyright © 2021 - GK Web Agency. Brisbane Website Designer All Rights Reserved.

Follow me around the web