How Robot Text Files Can Reveal More Than You Want
Thursday, June 7, 2007A robot text file is used to disallow specific or all search engine spider’s access to folders or pages that you do not want indexed. You may have created a’ directory or web’ page for employees or a paid members’ area that you don’t want listed. The case may be you simply created a private page just for your own personal... [Read more...]
