You must Sign In to post a response.
  • Category: General Questions

    How to stop search engines from accessing internal search results pages ?


    Wondering if it is possible to prevent search engines from accessing internal search results pages of your website? Know about this possibility and how to get it done from our experts.

    How to stop Google bot, Bing bot and other search engine bot from indexing the internal search results pages of my website with robot.txt file. I don't want some internal pages of my website to appear on search engines. Please explain how to create such robot.txt file for blogger platform.
  • #8789
    You can stop Google or Bing from crawling some pages of your blog by following the steps below :

    1. Login to your Blogger Dashboard and select your blog.

    2. Now goto settings->Search preferences.

    3. Then, on the new page, you will see 'Custom robots.txt' option, enable it.

    4. Once enabled, click on edit button and add Disallow: /2015/04/my-low-quality-page.html. Now Google, Bing or any other search engine will not crawl this page. Similarly, you have to add this code for each page, you do not want to see in the search results.

    5. Now click on save button and you are done.

    Note : Don't add Disallow: / unless you want to de-index you whole site from search engines.


  • Sign In to post your comments