Migrating from Solr to OpenSearch

This topic walks through moving an existing, Solr-backed Brightspot project to OpenSearch without losing content or downtime. It assumes a project already running Solr that wants to add OpenSearch as a new search back end and cut over to it—not a new project choosing a back end from scratch.

Solr keeps serving all production reads for the entire process described here. OpenSearch is added as a second delegate that receives every matching write from the moment it is configured, but serves no reads until step 5—there is no window where search depends on a partially populated index, and nothing about adding the delegate changes search behavior before the explicit read cutover.

note

This requires BOM 4.8.17 or later on the 4.8 line, or 5.0.0 or later on the 5.0 line—the versions where OpenSearch became available as a full alternative to Solr for text search, not only as a vector database. Provisioning the OpenSearch cluster itself is an infrastructure prerequisite handled outside the application; this guide assumes a reachable OpenSearch endpoint already exists before step 1.

Relevance and ranking in OpenSearch are tuned independently of Solr. A different result order after cutover is expected and is not, by itself, evidence of a problem with the migration—investigate functional issues (missing results, errors, wrong content) rather than ranking differences alone.

Step 1: Provision the OpenSearch index

OpenSearchDatabase reads and validates an existing index's mapping the first time it connects—it does not create the index or infer a mapping for it. Pointing it at an index that does not exist yet, or one built with an incompatible mapping, fails immediately with OpenSearch index <indexName> not configured!.

warning

Provision the index before step 2, not after. On Brightspot Cloud, Ops creates it with the required mapping as part of cluster setup—confirm this is already in place, or request it, before configuring the database. If you are managing OpenSearch yourself, create an equivalent index first: dynamic templates keyed by Dari's indexed-field-type prefixes, full-text analyzers for the combined all text field, and, if the opt-in search-as-you-type feature will ever be enabled, its prefix dynamic template. Get the exact mapping from Ops rather than improvising it—an incompatible mapping surfaces as query or indexing failures that are hard to trace back to the schema.

If the project also uses OpenSearch for vector search (Ask AI/Ask Esca embeddings), that is a separate index with its own mapping—provision it the same way before configuring a vector database against it.

Step 2: Add the dependency and configure OpenSearch as a new delegate

Add the OpenSearch module alongside the existing Solr dependency—do not remove Solr yet:

1
api 'com.brightspot.db:opensearch'

If the cluster is a managed AWS OpenSearch domain, also add:

1
api 'com.brightspot.db:opensearch-aws'

Add OpenSearch as a new delegate on the existing aggregate database, alongside the Solr delegate already configured there—do not configure it as a separate, top-level database. Every delegate on an aggregate database receives writes; only the readers setting (set in step 5) controls which delegate actually serves reads, which is what makes deploying this configuration safe before cutover.

1
dari/database/brightspot/delegate/opensearch/class=com.brightspot.opensearch.OpenSearchDatabase
2
dari/database/brightspot/delegate/opensearch/index=<indexName>
3
dari/database/brightspot/delegate/opensearch/groups=-* +cms.content.searchable
4
dari/database/brightspot/delegate/opensearch/client/class=com.brightspot.opensearch.aws.AwsOpenSearchClientSupplier
5
dari/database/brightspot/delegate/opensearch/client/host=<opensearch-endpoint>
6
dari/database/brightspot/delegate/opensearch/client/service=es
7
dari/database/brightspot/delegate/opensearch/client/region=<region>

Replace brightspot above with whatever name the project's own aggregate database actually uses—check the existing Solr delegate's settings for it. index must name the index provisioned in step 1. groups scopes which content types get written to this delegate at all. -* +cms.content.searchable is the standard Brightspot convention—exclude everything by default, then include only the searchable content group—and should match what the existing Solr delegate already uses.

For a self-hosted (non-AWS) OpenSearch endpoint, use com.brightspot.opensearch.common.HttpOpenSearchClientSupplier instead, with client/baseServerUrl in place of client/host, client/service, and client/region. An optional client/queryClient/class can point at a separate client used only for queries, if reads and writes need different credentials or endpoints.

Deploying this configuration is itself safe: until readers changes in step 5, OpenSearch receives every matching write but nothing reads from it, so search behavior is unaffected.

Step 3: Backfill existing content

The delegate configured in step 2 only captures writes going forward—back it up with a one-time copy of everything that already existed, using the Copy tool at /_debug/db-bulk. Do a small test copy before committing to a full run: scope Type to a single, representative content type, leave Parallel Execution set to This host, and confirm the copy completes cleanly and the record count matches before copying everything.

note

If your default database is an aggregate of a primary database (typically SQL) plus one or more search delegates, the Copy tool restricts Source to that primary database and refuses to let Destination be blank or the same database—both would otherwise silently copy into the database everything else reads from. In that configuration, Source is your SQL database and Destination is the new delegate configured in step 2, selectable as brightspot/delegate/opensearch (adjusted for the project's actual aggregate database name).

For the full copy:

• Leave Type unset to copy every type, or select one to scope the copy.
• Leave Delete before copy unchecked on a first run into an empty index.
• Set # Of Writers and Commit Size based on data volume—higher values copy faster but use more memory and put more load on the destination.
• Set Parallel Execution to distribute the copy across the cluster instead of a single host:

Mode	Behavior
This host	Runs entirely on the host handling this request.
All hosts	Every host in the cluster participates.
Limited hosts	Up to a fixed number of hosts participate.
Specific hosts	Only the listed host names participate.
Dynamic hosts	The number of participating hosts grows or shrinks automatically based on measured throughput, optionally capped by Max Hosts.

Every mode besides This host requires the cluster's replication cache (update notifiers) to be active. If it is not, the tool disables every mode but This host and states why.

A running copy appears in the Parallel Bulk Copy Executions table with live progress, an ETA, per-host throughput, and Pause, Resume, and Cancel controls; a finished execution can be dismissed from the same table.

warning

Combining Delete before copy with an Only After/Only Before date range is refused outright: the delete clears the full type scope, but the copy only restores the date-scoped window, permanently losing everything outside it.

Only After and Only Before (shown only when the source supports them) are not a normal part of this flow—since the delegate already receives every write going forward once step 2 is deployed, there is no ongoing gap for a second full copy to close. They matter for one specific situation: a delegate that was configured and backfilled ahead of time but then went a stretch without receiving production writes—for example, a tenant prepared during a cluster migration before it went live. In that case, a date-scoped copy closes the gap without re-running a full copy or writing a one-off script.

Step 4: Validate before cutover

Compare the new delegate against SQL—the source of truth—before changing which delegate serves reads. Solr is not a reliable baseline for this comparison on its own: it can accumulate stale or orphaned documents that SQL does not have, so comparing OpenSearch only against Solr can hide a real gap or report a fictitious one. Compare against both SQL and Solr if you want confirmation that nothing was lost in either direction, but treat SQL as authoritative wherever they disagree.

Query a specific database by name:

1
Database searchDb = Database.Static.getInstance("search");
2

3
Query<Article> query = Query.from(Article.class)
4
    .using(searchDb);

Pass in Database.Static.getInstance("brightspot/delegate/opensearch") to target the new delegate directly (again, adjusted for the project's actual aggregate database name). Compare record counts per type, and spot-check a handful of real queries before relying on the new delegate.

Step 5: Cut over

Switch reads from Solr to OpenSearch by updating the readers setting on the aggregate database, then deploy:

1
dari/database/brightspot/readers=sql opensearch

Both delegates keep receiving every write throughout, so rollback is just as safe as cutover itself: set readers back to sql solr and deploy again. Neither direction touches stored data—this is a configuration change, not a data recovery problem.

After cutover

Remove Solr only in this order:

1. Cut a release with the Solr dependency and configuration removed from the project.
2. Deploy that release—this is what actually removes the Solr delegate and drops it from readers.
3. Only once that release is deployed and confirmed stable, decommission the Solr infrastructure itself.

Decommissioning the infrastructure ahead of the release that stops referencing it is the one step in this process that is not easily reversible—do not skip ahead to it. As noted above, do not treat a difference in result ranking alone as a sign the migration went wrong—OpenSearch's relevance scoring is tuned independently of Solr's.