Skip to main content

Distance Range Filtering

Distance range filtering selects records by a distance threshold. The complete range-query examples below use PostgreSQL pgvector. Milvus implements range searches through ANN search, so it does not guarantee finding every matching entity; threshold directions and supported combinations also depend on the metric. See Milvus Vector Operations. dbVisitor uses vectorBy* to generate vector-distance predicates in the SQL WHERE clause.

Suitable For​

  • The query needs all records that satisfy a similarity threshold.
  • The result count is decided by the threshold and data distribution.
  • Vector predicates need to compose with ordinary field predicates.

Not Suitable For​

  • The query needs a fixed number of nearest records; use KNN Ordering.
  • The threshold is unknown and a ranked candidate list is preferred.
  • The data source does not support vector range filtering SQL.

Range Filter Pattern​

Range filtering
All records or candidate records
        |
        | Compute distance between embedding and query vector
        v
Keep records where distance < threshold
        |
        v
Return all matching records

vectorBy* is itself a query predicate. It can compose with ordinary predicates such as eq, likeRight, and gt in the same WHERE clause.

Basic Usage​

L2 distance filter
List<Float> target = Arrays.asList(0.1f, 0.2f, 0.3f);

List<ProductVector> rows = lambda.query(ProductVector.class)
        .vectorByL2(ProductVector::getEmbedding, target, 5.0)
        .queryForList();

SQL shape with pgvector:

SELECT * FROM product_vector
WHERE embedding <-> ? < ?

vectorBy* vector arguments are converted through the TypeHandler in entity mapping, so List<Float> can usually be passed directly.

Available Filter Methods​

GoalMethodNotes
L2 distancevectorByL2Smaller distance is more similar.
Cosine distancevectorByCosineCommon for text semantic vectors.
Inner productvectorByIPCommon for recommendation and ranking.
Hamming distancevectorByHammingCommon for binary vectors.
Jaccard distancevectorByJaccardCommon for set similarity.
BM25vectorByBM25Applies to data sources that support BM25.

Enable Filtering Dynamically​

All vectorBy* methods support a first boolean parameter that controls whether the predicate is emitted.

Conditional vector filter
boolean enableVectorFilter = request.hasVector();
List<Float> target = request.getVector();

List<ProductVector> rows = lambda.query(ProductVector.class)
        .eq(ProductVector::getCategory, request.getCategory())
        .vectorByL2(enableVectorFilter, ProductVector::getEmbedding, target, 5.0)
        .queryForList();

When enableVectorFilter is false, the vector-distance predicate is not emitted into SQL.

Choose A Threshold​

The threshold is not a fixed value computed by dbVisitor. It is a business condition chosen from the vector model, distance metric, and data distribution. A common approach is to inspect distance distributions offline and choose a threshold that reaches the target recall.

Threshold effect
Smaller threshold -> fewer matches, closer results
Larger threshold  -> more matches, looser results

Different metrics have different value meanings, so the same threshold cannot be reused blindly.

Further Reading​