If you've ever go through an 'revenue report' from a favorite blogger, you can detect that 'sponsored posts' make up a large proportion in their revenue. txt to disallow every little thing or precise endpoints (hackers can however search robots.txt for endpoints) which stops Google bots from crawling delicate endpoints https://daltonidwsl.verybigblog.com/30574465/a-review-of-used-google-accounts