Support and Documentation

Adding robots.txt to sites

You can use Brightspot to create and manage a robots.txt file on a site. If you are publishing to multiple sites, you can add a robots.txt file to each site.

Procedure. To add a robots.txt file to a site:

  1. Using the dashboard, create a Raw HTML object. For details, see Creating any type of content.

  2. In the Name field, enter a name for the file.

  3. In the HTML field, enter the robots.txt directives.

    robots-disallow.png

  4. Toward the right of the widget, select |mi-more_horiz|> Advanced.

  5. Create a new header called Content-Type with the value text/plain.

    robots-header.png

  6. In the URLs widget, enter robots.txt as the URL. Specifying only the name places robots.txt at the root of your site.

  7. To add robots.txt to other Brightspot sites, repeat the above steps.

. See also: