You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Although the Bulk API supports the if_seq_no and if_primary_term options for each document in the request below, the OpenSearch output plugin doesn't provide an option to set those values for the request.
Describe the solution you'd like
Adding two new options to support Optimistic Concurrency Control for the output plugin. It could look like this.
opensearch {
id => "os-output"
hosts => "..."
user => "..."
password => "..."
index => "some-index"
document_id => "..."
sequence_number => "[@metadata][_seq_no]"
primary_term => "[@metadata][_seq_no]"
}
Describe alternatives you've considered
I had to figure out a workaround, where I fetch the _version field alongside the document from OpenSearch, manually increment it by 1, and configure my output plugin as shown below.
opensearch {
id => "os-output"
hosts => "..."
user => "..."
password => "..."
index => "some-index"
document_id => "..."
version => "%{[@metadata][_version]}"
version_type => "external"
}
While this does work, I would rather not have to manage my version externally (and introduce unnecessary points of failure), as the only logic I need is to update the document in OpenSearch if there hasn't been any updates while the pipeline is running.
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
Although the Bulk API supports the
if_seq_no
andif_primary_term
options for each document in the request below, the OpenSearch output plugin doesn't provide an option to set those values for the request.Describe the solution you'd like
Adding two new options to support Optimistic Concurrency Control for the output plugin. It could look like this.
Describe alternatives you've considered
I had to figure out a workaround, where I fetch the
_version
field alongside the document from OpenSearch, manually increment it by 1, and configure my output plugin as shown below.While this does work, I would rather not have to manage my version externally (and introduce unnecessary points of failure), as the only logic I need is to update the document in OpenSearch if there hasn't been any updates while the pipeline is running.
The text was updated successfully, but these errors were encountered: