To stay away from undesirable information inside the look for indexes, webmasters can instruct spiders to not crawl selected files or directories throughout the typical robots.txt file in the foundation directory of the area. Moreover, a site might be explicitly excluded from the search engine's database through the use of https://news.rafeeg.ae