-
Notifications
You must be signed in to change notification settings - Fork 25.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
New vector_rescore
parameter as a quantized index type option
#124581
base: main
Are you sure you want to change the base?
New vector_rescore
parameter as a quantized index type option
#124581
Conversation
Pinging @elastic/es-search-relevance (Team:Search Relevance) |
Hi @benwtrent, I've created a changelog YAML for you. |
…wtrent/elasticsearch into feature/add-rescore-to-index-options
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM. 🎉 Left some small doc comments
Float oversample = indexOptions instanceof QuantizedIndexOptions quantizedIndexOptions | ||
? quantizedIndexOptions.rescoreVector != null ? quantizedIndexOptions.rescoreVector.oversample() : null | ||
: null; | ||
if (queryOversample != null) { | ||
oversample = queryOversample; | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit - I find hard to follow multiple ternary operators
Float oversample = indexOptions instanceof QuantizedIndexOptions quantizedIndexOptions | |
? quantizedIndexOptions.rescoreVector != null ? quantizedIndexOptions.rescoreVector.oversample() : null | |
: null; | |
if (queryOversample != null) { | |
oversample = queryOversample; | |
} | |
Float oversample = null; | |
if (queryOversample != null) { | |
oversample = queryOversample; | |
} else if (queryOversample indexOptions instanceof QuantizedIndexOptions quantizedIndexOptions) { | |
oversample = quantizedIndexOptions.rescoreVector != null ? quantizedIndexOptions.rescoreVector.oversample() : null | |
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will refactor
Co-authored-by: Carlos Delgado <[email protected]>
This adds a new parameter to the quantized index mapping that allows default oversampling and rescoring to occur.
This doesn't adjust any of the defaults. It allows it to be configured. When the user provides
rescore_vector: {oversample: <number>}
in the query it will overwrite it.For example, here is how to use it with bbq:
Then, when querying, it will auto oversample the
k
by3x
and rerank with the raw vectors.