Ask Experts » Blogging and Websites » General Questions »

How to stop search engines from accessing internal search results pages ?


Date: 17 Apr 2015   Posted By: Mukhtar     Group: Blogging and Websites    Category: General Questions   

How to stop Google bot, Bing bot and other search engine bot from indexing the internal search results pages of my website with robot.txt file. I don't want some internal pages of my website to appear on search engines. Please explain how to create such robot.txt file for blogger platform.

Wondering if it is possible to prevent search engines from accessing internal search results pages of your website? Know about this possibility and how to get it done from our experts.


Author: Muddassir    20 Apr 2015      Member Level: Diamond     Points : 4  (Rs 4)    Voting Score: 0

You can stop Google or Bing from crawling some pages of your blog by following the steps below :

1. Login to your Blogger Dashboard and select your blog.

2. Now goto settings->Search preferences.

3. Then, on the new page, you will see 'Custom robots.txt' option, enable it.

4. Once enabled, click on edit button and add Disallow: /2015/04/my-low-quality-page.html. Now Google, Bing or any other search engine will not crawl this page. Similarly, you have to add this code for each page, you do not want to see in the search results.

5. Now click on save button and you are done.

Note : Don't add Disallow: / unless you want to de-index you whole site from search engines.





Post Answer



You must Sign In to post a response.


Post Answer        





Related Questions
  • Return to Question Index
  • Ask Question to Experts


  • Top Contributors
    TodayLast 7 Daysmore...