HTTP endpoint database

HttpEndpointDatabase allows you to integrate with a third-party service that uses an HTTP API. A subclass of HttpEndpointDatabase can convert Data queries to HTTP API calls and return the results as if they were retrieved from the Dari underlying database. The search experience is the same for the end user, whether a query is on the underlying database or converted to an HTTP request to a third-party API.

HttpEndpointDatabase includes methods for extracting Dari query predicates and values for constructing HTTP API calls. It also includes methods that you can override to open and close connections to a third-party service, and to save objects to the underlying Dari database.

An HttpEndpointDatabase subclass must implement the doInitialize and readPartial methods inherited from the Dari AbstractDatabase and Database classes, respectively.

The following example shows a simple implementation of HttpEndpointDatabase that leverages the Getty REST Search API to retrieve images from that service. The GettyDatabase class gets the predicate and associated values from a Dari query, constructs an HTTP request that is sent to the Getty API, and returns the Getty response as a Dari PaginatedResult object.

showLineNumbers {4,27,36,42,43,45,47,54,62,65}

1public class GettyDatabase extends HttpEndpointDatabase {
2
3    private static final Logger LOGGER = LoggerFactory.getLogger(GettyDatabase.class);
4    public static final String API_KEY_SUB_SETTING = "apiKey";
5    public static final String API_URL_SUB_SETTING = "apiUrl";
6    public static final String DEFAULT_API_URL = "https://api.gettyimages.com/v3";
7
8    private String apiUrl;
9    private String apiKey;
10
11    public String getApiUrl() {
12        return apiUrl;
13    }
14
15    public void setApiUrl(String apiUrl) {
16        this.apiUrl = apiUrl;
17    }
18
19    public String getApiKey() {
20        return apiKey;
21    }
22
23    public void setApiKey(String apiKey) {
24        this.apiKey = apiKey;
25    }
26
27    @Override
28    protected void doInitialize(String settingsKey, Map<String, Object> settings) {
29        setApiUrl(ObjectUtils.to(String.class, settings.get(API_URL_SUB_SETTING)));
30        setApiKey(ObjectUtils.to(String.class, settings.get(API_KEY_SUB_SETTING)));
31    }
32
33    @Override /* Must be implemented */
34    public <T> PaginatedResult<T> readPartial(Query<T> query, long offset, int limit) {
35
36        String apiKey = getApiKey();
37        if (apiKey == null) {
38            return PaginatedResult.empty();
39        }
40
41        try {
42            String apiUrl = ObjectUtils.firstNonBlank(getApiUrl(), DEFAULT_API_URL);
43            Predicate predicate = query.getPredicate();
44
45            final String fields = "id,title,display_set";
46
47            URL url = new URL(StringUtils.addQueryParameters(apiUrl + "/search/images",
48                     "phrase", findQueryStringPredicateValue(predicate),
49                     "fields", fields,
50                     "page", offset + 1,
51                     "page_size", limit));
52
53            HttpClient httpClient = HttpClientBuilder.create().build();
54            HttpGet get = new HttpGet(url.toString());
55            get.addHeader("Api-Key", apiKey);
56            HttpResponse response = httpClient.execute(get);
57
58            if (response == null) return PaginatedResult.empty();
59
60            Map<String, Object> responseMap = (Map<String, Object>) ObjectUtils
61                .fromJson(EntityUtils.toString(response.getEntity()));
62            List<Object> images = ((List<Object>) responseMap.get("images"));
63
64            if (images != null) {
65                List<Object> items = images.stream()
66                     .map(image -> (Map<String, Object>) image)
67                     .filter(Objects::nonNull)
68                     .collect(Collectors.toList());
69
70                return Optional.ofNullable((Long) responseMap.get("result_count"))
71                     .map(count -> new PaginatedResult(offset, limit, count, items))
72                     .orElse(new PaginatedResult(offset, limit, items));
73            }
74            else return PaginatedResult.empty();
75
76        } catch (IOException error) {
77            LOGGER.error("Unable to perform Getty request!", error);
78        }
79        return PaginatedResult.empty();
80    }
81}

4. Final variables that refer to the URL and security key for the Getty REST API. The API_URL_SUB_SETTING specifies the host for the API, and the API_KEY_SUB_SETTING specifies the security key issued by the service. These values are set in the client. A default host site is set on DEFAULT_API_URL in the event that the Getty API URL is not set in the client.
27. Implementation for the doInitialize method, which sets the Getty URL and security key. Instead of calling doInitialize, a client can set the Getty URL and security key using the public setter methods setApiUrl and setApiKey.
36. Gets the Getty security key.
42. Gets the URL for the Getty REST API.
43. Gets the predicate from the Dari query passed to the method.
45. Initializes a fields variable that specifies the data to be retrieved from the Getty service.
47. Creates a URL object to be passed in the HTTP request to the Getty service. The URL specifies the HTTP endpoint (/search/images) and the fields variable. Also included in the URL is phrase, the value of the Dari predicate. This value is retrieved with the HttpEndpointDatabase#findQueryStringPredicateValue method.
54. Creates HTTP-related objects and submits the request to the Getty service. JSON data returned from the service is set on response.
62. Creates a list of objects from the JSON image data.
65. Creates a list of objects to construct a PaginatedResult object returned by the method.

SourceDatabaseProvider implementation

Objects retrieved from a third-party service are represented as ExternalItem objects in Brightspot. External items are associated with a source database provider, which specifies the Database object that retrieves data from a third-party service.

For GettyDatabase that integrates with the Getty third-party service, the GettyDatabaseProvider is the source database provider. It provides the glue between the ExternalItem objects that represent retrieved Getty images and GettyDatabase. GettyDatabaseProvider implements SourceDatabaseProvider#get, which creates the GettyDatabase instance and sets the Getty API key on the instance. The key is retrieved from the Brightspot UI.

showLineNumbers

1public class GettyDatabaseProvider implements SourceDatabaseProvider {
2
3    @Override
4    public Database get(ObjectType type) {
5        GettyDatabase db = new GettyDatabase();
6        db.setEnvironment(Database.Static.getDefault().getEnvironment());
7        db.setApiKey(Application.Static.getInstance(CmsTool.class).as(GettySettings.class).getApiKey());
8        return db;
9    }
10}

SourceDatabaseProvider implementation​

SourceDatabaseProvider implementation