Adding Robots.txt to Sites

You can use Brightspot to create and manage a robots.txt file on a site. If you are running multiple sites, you can add a robots.txt file to each site.

  1. Using the dashboard, create a Raw HTML object. For details, see Creating any type of content.

  2. Under Main, do the following:

    1. In the Name field, enter a name for the file.

    2. In the HTML field, enter the robots.txt directives.

      ../../../_images/robots-disallow.png
  3. Under Advanced, create a new header called Content-Type with the value text/plain.

    ../../../_images/robots-header.png
  4. In the URL widget, enter robots.txt as the URL. Specifying only the name will place robots.txt at the root of your site.

  5. To add robots.txt to other Brightspot sites, repeat the above steps.