In order to avoid unwanted articles in the research indexes, website owners can instruct spiders to not crawl sure documents or directories with the conventional robots.txt file in the basis directory of your area. In addition, a web site could be explicitly excluded from the search engine's databases by using https://news.rafeeg.ae