# This file tells web crawlers to avoid searching certain directories User-agent: * Disallow: /