I just saw someone mention Google's robot.txt on twitter. If you go to ggogle.com/robots.txt and scroll to the bottom you'll see this:
User-agent: Kids
Disallow: /tricks
Allow: /treats
:)
Printable View
I just saw someone mention Google's robot.txt on twitter. If you go to ggogle.com/robots.txt and scroll to the bottom you'll see this:
User-agent: Kids
Disallow: /tricks
Allow: /treats
:)
Yeah, but everyone knows that kids, just like bad spiders, ignore the robots.txt file