Skip to content

Commit e128a39

Browse files
waileongilayaperumalg
waileong
authored and
ilayaperumalg
committed
Updates the SearchRequest class to be non-final and adds a MilvusSearchRequest subclass that includes Milvus-specific fields for native expressions and search parameters.
- Updated `SearchRequest.java` to make the class non-final. - Added `MilvusSearchRequest` with specific Milvus parameters such as `nativeExpression` and `searchParamsJson`. - Modified `doSimilaritySearch` method in `MilvusVectorStore` to handle these new fields from `MilvusSearchRequest`. Add unit tests for MilvusVectorStore and MilvusSearchRequest Introduce comprehensive unit tests to validate the functionality of MilvusVectorStore and MilvusSearchRequest, including scenarios for native and filter expressions. Refactor MilvusVectorStore to improve filter expression handling by introducing a helper method for converted expressions. Add detailed documentation for MilvusSearchRequest usage Introduced sections explaining MilvusSearchRequest's parameters, `nativeExpression`, and `searchParamsJson`, with examples for enhanced clarity. This update provides guidance on leveraging Milvus-specific features for precise filtering and optimal search performance. Signed-off-by: waileong <[email protected]>
1 parent bb7c52c commit e128a39

File tree

6 files changed

+453
-3
lines changed

6 files changed

+453
-3
lines changed

Diff for: ‎spring-ai-core/src/main/java/org/springframework/ai/vectorstore/SearchRequest.java

+11-1
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@
3333
* @author Thomas Vitale
3434
* @author Ilayaperumal Gopinathan
3535
*/
36-
public final class SearchRequest {
36+
public class SearchRequest {
3737

3838
/**
3939
* Similarity threshold that accepts all search scores. A threshold value of 0.0 means
@@ -71,6 +71,16 @@ public static Builder from(SearchRequest originalSearchRequest) {
7171
.filterExpression(originalSearchRequest.getFilterExpression());
7272
}
7373

74+
public SearchRequest() {
75+
}
76+
77+
protected SearchRequest(SearchRequest original) {
78+
this.query = original.query;
79+
this.topK = original.topK;
80+
this.similarityThreshold = original.similarityThreshold;
81+
this.filterExpression = original.filterExpression;
82+
}
83+
7484
public String getQuery() {
7585
return this.query;
7686
}

Diff for: ‎spring-ai-docs/src/main/antora/modules/ROOT/pages/api/vectordbs/milvus.adoc

+52
Original file line numberDiff line numberDiff line change
@@ -161,6 +161,58 @@ vectorStore.similaritySearch(SearchRequest.builder()
161161

162162
NOTE: These filter expressions are converted into the equivalent Milvus filters.
163163

164+
== Using MilvusSearchRequest
165+
166+
MilvusSearchRequest extends SearchRequest, allowing you to use Milvus-specific search parameters such as native expressions and search parameter JSON.
167+
168+
[source,java]
169+
----
170+
MilvusSearchRequest request = MilvusSearchRequest.milvusBuilder()
171+
.query("sample query")
172+
.topK(5)
173+
.similarityThreshold(0.7)
174+
.nativeExpression("metadata[\"age\"] > 30") // Overrides filterExpression if both are set
175+
.filterExpression("age <= 30") // Ignored if nativeExpression is set
176+
.searchParamsJson("{\"nprobe\":128}")
177+
.build();
178+
List results = vectorStore.similaritySearch(request);
179+
----
180+
This allows greater flexibility when using Milvus-specific search features.
181+
182+
== Importance of `nativeExpression` and `searchParamsJson` in `MilvusSearchRequest`
183+
184+
These two parameters enhance Milvus search precision and ensure optimal query performance:
185+
186+
*nativeExpression*: Enables additional filtering capabilities using Milvus' native filtering expressions.
187+
https://milvus.io/docs/boolean.md[Milvus Filtering]
188+
189+
Example:
190+
[source,java]
191+
----
192+
MilvusSearchRequest request = MilvusSearchRequest.milvusBuilder()
193+
.query("sample query")
194+
.topK(5)
195+
.nativeExpression("metadata['category'] == 'science'")
196+
.build();
197+
----
198+
199+
*searchParamsJson*: Essential for tuning search behavior when using IVF_FLAT, Milvus' default index.
200+
https://milvus.io/docs/index.md?tab=floating[Milvus Vector Index]
201+
202+
By default, `IVF_FLAT` requires `nprobe` to be set for accurate results. If not specified, `nprobe` defaults to `1`, which can lead to poor recall or even zero search results.
203+
204+
Example:
205+
[source,java]
206+
----
207+
MilvusSearchRequest request = MilvusSearchRequest.milvusBuilder()
208+
.query("sample query")
209+
.topK(5)
210+
.searchParamsJson("{\"nprobe\":128}")
211+
.build();
212+
----
213+
214+
Using `nativeExpression` ensures advanced filtering, while `searchParamsJson` prevents ineffective searches caused by a low default `nprobe` value.
215+
164216
[[milvus-properties]]
165217
== Milvus VectorStore properties
166218

Diff for: ‎vector-stores/spring-ai-milvus-store/src/main/java/org/springframework/ai/vectorstore/milvus/MilvusSearchRequest.java

+164
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
package org.springframework.ai.vectorstore.milvus;
2+
3+
import org.springframework.ai.vectorstore.SearchRequest;
4+
import org.springframework.ai.vectorstore.filter.Filter;
5+
import org.springframework.lang.Nullable;
6+
7+
/**
8+
* A specialized {@link SearchRequest} for Milvus vector search, extending the base
9+
* request with Milvus-specific parameters.
10+
* <p>
11+
* This class introduces two additional fields:
12+
* <ul>
13+
* <li>{@code nativeExpression} - A native Milvus filter expression (e.g.,
14+
* {@code "city LIKE
15+
* 'New%'"}).</li>
16+
* <li>{@code searchParamsJson} - A JSON string containing search parameters (e.g.,
17+
* {@code "{\"nprobe\":128}"}).</li>
18+
* </ul>
19+
* <p>
20+
* Use the {@link MilvusBuilder} to construct instances of this class.
21+
*
22+
* @author waileong
23+
*/
24+
public final class MilvusSearchRequest extends SearchRequest {
25+
26+
@Nullable
27+
private final String nativeExpression;
28+
29+
@Nullable
30+
private final String searchParamsJson;
31+
32+
/**
33+
* Private constructor to initialize a MilvusSearchRequest using the base request and
34+
* builder.
35+
* @param baseRequest The base {@link SearchRequest} containing standard search
36+
* fields.
37+
* @param builder The {@link MilvusBuilder} containing Milvus-specific parameters.
38+
*/
39+
private MilvusSearchRequest(SearchRequest baseRequest, MilvusBuilder builder) {
40+
super(baseRequest); // Copy all standard fields
41+
this.nativeExpression = builder.nativeExpression;
42+
this.searchParamsJson = builder.searchParamsJson;
43+
}
44+
45+
/**
46+
* Retrieves the native Milvus filter expression.
47+
* @return A string representing the native Milvus expression, or {@code null} if not
48+
* set.
49+
*/
50+
@Nullable
51+
public String getNativeExpression() {
52+
return this.nativeExpression;
53+
}
54+
55+
/**
56+
* Retrieves the JSON-encoded search parameters.
57+
* @return A JSON string containing search parameters, or {@code null} if not set.
58+
*/
59+
@Nullable
60+
public String getSearchParamsJson() {
61+
return this.searchParamsJson;
62+
}
63+
64+
/**
65+
* Creates a new {@link MilvusBuilder} for constructing a {@link MilvusSearchRequest}.
66+
* @return A new {@link MilvusBuilder} instance.
67+
*/
68+
public static MilvusBuilder milvusBuilder() {
69+
return new MilvusBuilder();
70+
}
71+
72+
/**
73+
* Builder class for constructing instances of {@link MilvusSearchRequest}.
74+
*/
75+
public static class MilvusBuilder {
76+
77+
private final SearchRequest.Builder baseBuilder = SearchRequest.builder();
78+
79+
@Nullable
80+
private String nativeExpression;
81+
82+
@Nullable
83+
private String searchParamsJson;
84+
85+
/**
86+
* {@link Builder#query(java.lang.String)}
87+
*/
88+
public MilvusBuilder query(String query) {
89+
this.baseBuilder.query(query);
90+
return this;
91+
}
92+
93+
/**
94+
* {@link Builder#topK(int)}
95+
*/
96+
public MilvusBuilder topK(int topK) {
97+
this.baseBuilder.topK(topK);
98+
return this;
99+
}
100+
101+
/**
102+
* {@link Builder#similarityThreshold(double)}
103+
*/
104+
public MilvusBuilder similarityThreshold(double threshold) {
105+
this.baseBuilder.similarityThreshold(threshold);
106+
return this;
107+
}
108+
109+
/**
110+
* {@link Builder#similarityThresholdAll()}
111+
*/
112+
public MilvusBuilder similarityThresholdAll() {
113+
this.baseBuilder.similarityThresholdAll();
114+
return this;
115+
}
116+
117+
/**
118+
* {@link Builder#filterExpression(String)}
119+
*/
120+
public MilvusBuilder filterExpression(String textExpression) {
121+
this.baseBuilder.filterExpression(textExpression);
122+
return this;
123+
}
124+
125+
/**
126+
* {@link Builder#filterExpression(Filter.Expression)}
127+
*/
128+
public MilvusBuilder filterExpression(Filter.Expression expression) {
129+
this.baseBuilder.filterExpression(expression);
130+
return this;
131+
}
132+
133+
/**
134+
* Sets the native Milvus filter expression.
135+
* @param nativeExpression The native Milvus expression string.
136+
* @return This builder instance.
137+
*/
138+
public MilvusBuilder nativeExpression(String nativeExpression) {
139+
this.nativeExpression = nativeExpression;
140+
return this;
141+
}
142+
143+
/**
144+
* Sets the JSON-encoded search parameters.
145+
* @param searchParamsJson A JSON string containing search parameters.
146+
* @return This builder instance.
147+
*/
148+
public MilvusBuilder searchParamsJson(String searchParamsJson) {
149+
this.searchParamsJson = searchParamsJson;
150+
return this;
151+
}
152+
153+
/**
154+
* Builds and returns a {@link MilvusSearchRequest} instance.
155+
* @return A new {@link MilvusSearchRequest} object with the specified parameters.
156+
*/
157+
public MilvusSearchRequest build() {
158+
SearchRequest parentRequest = this.baseBuilder.build();
159+
return new MilvusSearchRequest(parentRequest, this);
160+
}
161+
162+
}
163+
164+
}

Diff for: ‎vector-stores/spring-ai-milvus-store/src/main/java/org/springframework/ai/vectorstore/milvus/MilvusVectorStore.java

+20-2
Original file line numberDiff line numberDiff line change
@@ -323,9 +323,18 @@ protected void doDelete(Filter.Expression filterExpression) {
323323

324324
@Override
325325
public List<Document> doSimilaritySearch(SearchRequest request) {
326+
String nativeFilterExpressions = "";
327+
String searchParamsJson = null;
328+
if (request instanceof MilvusSearchRequest milvusReq) {
329+
nativeFilterExpressions = StringUtils.hasText(milvusReq.getNativeExpression())
330+
? milvusReq.getNativeExpression() : getConvertedFilterExpression(request);
326331

327-
String nativeFilterExpressions = (request.getFilterExpression() != null)
328-
? this.filterExpressionConverter.convertExpression(request.getFilterExpression()) : "";
332+
searchParamsJson = StringUtils.hasText(milvusReq.getSearchParamsJson()) ? milvusReq.getSearchParamsJson()
333+
: null;
334+
}
335+
else {
336+
nativeFilterExpressions = getConvertedFilterExpression(request);
337+
}
329338

330339
Assert.notNull(request.getQuery(), "Query string must not be null");
331340
List<String> outFieldNames = new ArrayList<>();
@@ -348,6 +357,10 @@ public List<Document> doSimilaritySearch(SearchRequest request) {
348357
searchParamBuilder.withExpr(nativeFilterExpressions);
349358
}
350359

360+
if (StringUtils.hasText(searchParamsJson)) {
361+
searchParamBuilder.withParams(searchParamsJson);
362+
}
363+
351364
R<SearchResults> respSearch = this.milvusClient.search(searchParamBuilder.build());
352365

353366
if (respSearch.getException() != null) {
@@ -385,6 +398,11 @@ public List<Document> doSimilaritySearch(SearchRequest request) {
385398
.toList();
386399
}
387400

401+
private String getConvertedFilterExpression(SearchRequest request) {
402+
return (request.getFilterExpression() != null)
403+
? this.filterExpressionConverter.convertExpression(request.getFilterExpression()) : "";
404+
}
405+
388406
private float getResultSimilarity(RowRecord rowRecord) {
389407
Float score = (Float) rowRecord.get(SIMILARITY_FIELD_NAME);
390408
return (this.metricType == MetricType.IP || this.metricType == MetricType.COSINE) ? score : (1 - score);

Diff for: ‎vector-stores/spring-ai-milvus-store/src/test/java/org/springframework/ai/vectorstore/milvus/MilvusSearchRequestTest.java

+66
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,66 @@
1+
package org.springframework.ai.vectorstore.milvus;
2+
3+
import org.junit.jupiter.api.Test;
4+
5+
import static org.assertj.core.api.Assertions.assertThat;
6+
import static org.springframework.ai.vectorstore.SearchRequest.DEFAULT_TOP_K;
7+
import static org.springframework.ai.vectorstore.SearchRequest.SIMILARITY_THRESHOLD_ACCEPT_ALL;
8+
9+
/**
10+
* Test class for verifying the functionality of the {@link MilvusSearchRequest} class.
11+
*
12+
* @author waileong
13+
*/
14+
class MilvusSearchRequestTest {
15+
16+
@Test
17+
void shouldBuildMilvusSearchRequestWithNativeExpression() {
18+
String query = "sample query";
19+
int topK = 10;
20+
double similarityThreshold = 0.8;
21+
String nativeExpression = "city LIKE 'New%'";
22+
String searchParamsJson = "{\"nprobe\":128}";
23+
24+
MilvusSearchRequest request = MilvusSearchRequest.milvusBuilder()
25+
.query(query)
26+
.topK(topK)
27+
.similarityThreshold(similarityThreshold)
28+
.nativeExpression(nativeExpression)
29+
.searchParamsJson(searchParamsJson)
30+
.build();
31+
32+
assertThat(request.getQuery()).isEqualTo(query);
33+
assertThat(request.getTopK()).isEqualTo(topK);
34+
assertThat(request.getSimilarityThreshold()).isEqualTo(similarityThreshold);
35+
assertThat(request.getNativeExpression()).isEqualTo(nativeExpression);
36+
assertThat(request.getSearchParamsJson()).isEqualTo(searchParamsJson);
37+
}
38+
39+
@Test
40+
void shouldBuildMilvusSearchRequestWithDefaults() {
41+
MilvusSearchRequest request = MilvusSearchRequest.milvusBuilder().build();
42+
43+
assertThat(request.getQuery()).isEmpty();
44+
assertThat(request.getTopK()).isEqualTo(DEFAULT_TOP_K);
45+
assertThat(request.getSimilarityThreshold()).isEqualTo(SIMILARITY_THRESHOLD_ACCEPT_ALL);
46+
assertThat(request.getNativeExpression()).isNull();
47+
assertThat(request.getSearchParamsJson()).isNull();
48+
}
49+
50+
@Test
51+
void shouldAllowSettingNativeExpressionIndependently() {
52+
String nativeExpression = "age > 30";
53+
MilvusSearchRequest request = MilvusSearchRequest.milvusBuilder().nativeExpression(nativeExpression).build();
54+
55+
assertThat(request.getNativeExpression()).isEqualTo(nativeExpression);
56+
}
57+
58+
@Test
59+
void shouldAllowSettingSearchParamsJsonIndependently() {
60+
String searchParamsJson = "{\"metric_type\": \"IP\"}";
61+
MilvusSearchRequest request = MilvusSearchRequest.milvusBuilder().searchParamsJson(searchParamsJson).build();
62+
63+
assertThat(request.getSearchParamsJson()).isEqualTo(searchParamsJson);
64+
}
65+
66+
}

0 commit comments

Comments
 (0)