Though You should utilize the robots.txt file to tell a crawler in which it could’t go on your site, you Decrease the probability of specific webpages getting crawled, such as indexing and showing in the search results. A virtual Net host distinguishes utilizing the domain name of various web pages https://rankingseotoolz.com/robots-txt-generator