Adding Robots.txt to Sites

You can use Brightspot to create and manage a robots.txt file on a site. If you are running multiple sites, you can add a robots.txt file to each site.

  1. Using the Dashboard, create a Raw HTML object.

  2. In the Main tab, enter the robots.txt directives.

    ../../../_images/robots-disallow.png
  3. In the Advanced tab, create a new header called Content-Type with the value text/plain.

    ../../../_images/robots-header.png
  4. In the URL widget, enter robots.txt as the URL. Specifying only the name will place robots.txt at the root of your site.

  5. To add robots.txt to other Brightspot sites, repeat the above steps.