AI Audit
The AI Audit widget helps you track AI usage across multiple dimensions, enabling more effective monitoring and troubleshooting.

Enabling permissions
To enable permissions for the AI Audit widget:
- From the left navigation, under Admin, click Users & Roles.
- From the Roles list, select the role to which you want to add AI audit permissions, or create a new one.
- Under Additional Permissions, select AI Dashboard Widgets Permission.
- Toggle on the View AI Dashboard Widgets? setting.
- Click Save.
Once you have configured the role, ensure it is applied to the users whom you want to have access to the AI Audit widget.
Configuring the AI Audit widget
To configure the AI Audit widget:
-
Create or open a shared or one-off dashboard.
tipBrightspot recommends adding an additional dashboard tab for the AI Audit widget so that you can view it more easily.
-
From either Main Tab Columns or Additional Tabs, click Add Dashboard Column.
-
Under Widgets, select AI Audit Dashboard Widget.
-
Click Save.
Using the AI Audit widget
To use the AI Audit widget:
-
Navigate to your dashboard and locate the widget. The widget and its CSV export show only assets from sites you can access; assets created outside a site require access to global content.
-
Use the following filters to find assets:
- All Completion Types — View assets based on completion type (Complete, Error, Stop). Interactions with no completion status show as Incomplete and appear only under All Completion Types.
- All Time — Filter assets based on when they were submitted.
The widget header shows the Error Rate and Average Response Time for the filtered interactions.
-
View the individual asset line items. Each contains the following column headers:
- ID — The ID of the audit record.
- User — The user who initiated the request to the model.
- Timestamp — The time when the audit record was created (when the request was initiated).
- Completion Type — The completion status of the given asset.
- Model ID — The model name reported for the response. For new Ask and Create responses in 4.0, this comes from the chat client configured for the site selected when viewing the audit, and can differ from the model used during generation.
- Response Time — How long the AI model took to respond to the submitted prompt for this asset.
- Token Count — Every token the provider counted for the final response, including input served from its prompt cache and any reasoning tokens.
-
(Optional) Click an asset's ID to view the AI Audit Metadata pop-up, which shows read-only information about the AI interaction. This information includes more granular details about the asset than the widget's overview, such as the specific request made to the model and its response, and the content that was referenced for the response.
Changing the configured model or selected site can change Model ID for these responses in both the widget and CSV exports. Historical provider snapshots are not rewritten. See the 4.0 audit changes when upgrading a custom audit integration.
Exporting data from the AI Audit widget
To export data from the AI Audit widget:
- Navigate to your dashboard and locate the widget.
- In the top-right corner, click Export CSV.
The export applies the widget's current filters and contains one row per interaction with the columns ID, User, Timestamp, Completion Type, Model ID, Response Time, Token Count, Request Text, Response Text, Feature (Ask Esca or Create with Esca), and Reference (the asset the interaction was made from, if any). Rows are ordered newest first, and Request Text and Response Text longer than 200 characters are cut to their first 200 characters followed by .... The file downloads directly and, unlike in 3.x, is not saved to storage.
Tracking usage against an allowance
The AI Usage widget charts how many Ask Esca and Create with Esca actions a site has used over the past year against the site's allowance. The allowance is for reporting only; reaching it does not block AI requests.
To set a site's allowance:
- From the left navigation, under Admin, click Sites & Settings.
- From the Integrations tab, expand the AI cluster.
- In the Total Allocated Actions field, type the number of AI actions the site may use per year. The default is 75000.
- Click Save.
To add the AI Usage widget:
- Create or open a shared or one-off dashboard.
- From either Main Tab Columns or Additional Tabs, click Add Dashboard Column.
- Under Widgets, select AI Usage Widget.
- (Optional) In the Period field, select the time range for the Usage Over Time chart. If not set, it shows the past year.
- Click Save.
The widget shows a Usage Over Time line chart of Ask Esca Actions and Create with Esca Actions, and a stacked bar of both against Remaining Actions. Viewing it requires the same AI Dashboard Widgets Permission as the AI Audit widget.