#============================================================================= # Please don't mess... Thank you! #============================================================================= #***************************************************************************** #>>> ROBOTS.TXT FILE #***************************************************************************** # # # Synopsis: keeps things from crawling the site. # # # REFERENCES # ---------- # http://www.robotstxt.org/wc/robots.html -- good entry point # http://www.robotstxt.org/wc/norobots.html -- file syntax spec # # # LAWS OF ROBOTICS # ---------------- # 1. A robot may not injure a human being, or, through inaction, allow # a human to come to harm. # # 2. A robot must obey all orders from human beings except where # such orders would conflict with the First Law. # # 3. A robot must protect its own existence as long as such protection # does not conflict with the First or Second Law. # #***************************************************************************** User-Agent: * Disallow: /personal/ Disallow: /the_hodgi/ Disallow: /clients/ #*****************************************************************************