Multiple target vectors

Added in v1.26

In a multi-target vector search, Weaviate searches multiple target vector spaces concurrently and combines the results. A multi-target vector search produce a single set of search results.

There are multiple ways to specify the target vectors and query vectors, such as:

• Specify target vector names only
• Specify query vectors
• Specify target vector names and join strategy
• Weight raw vector distances
• Weight normalized vector distances

Combining search results

A multi-target vector search involves multiple, concurrent, single-target vector searches. These searches will produce multiple sets of results, each with a vector distance score.

These distances are combined using a "join strategy".

How Weaviate combines search results

• If an object is within the search limit or the distance threshold of any of the target vectors, it will be included in the search results.
• If an object does not contain vectors for any selected target vector, Weaviate ignores that object and does not include it in the search results.

Available join strategies.

These are the available join strategies:

• minimum (default) Use the minimum of all vector distances.
• sum Use the sum of the vector distances.
• average Use the average of the vector distances.
• manual weights Use the sum of weighted distances, where the weight is provided for each target vector.
• relative score Use the sum of weighted normalized distances, where the weight is provided for each target vector.

Specify target vector names only

As a minimum, specify the target vector names as an array of named vectors. This will use the default join strategy.

Python Client v4

from weaviate.classes.query import MetadataQuery

collection = client.collections.get("JeopardyTiny")

response = collection.query.near_text(
    query="a wild animal",
    limit=2,
    target_vector=["jeopardy_questions_vector", "jeopardy_answers_vector"],  # Specify the target vectors
    return_metadata=MetadataQuery(distance=True)
)

for o in response.objects:
    print(o.properties)
    return_metadata=MetadataQuery(distance=True)

JS/TS Client v3

var jeopardy, result;

jeopardy = client.collections.get('JeopardyTiny');

result = await jeopardy.query.nearText('a wild animal', {
  limit: 2,
  targetVector: ['jeopardy_questions_vector', 'jeopardy_answers_vector'],
  returnMetadata: ['distance'],
});

result.objects.forEach((item) => {
  console.log(JSON.stringify(item.properties, null, 2));
  console.log(JSON.stringify(item.metadata?.distance, null, 2));
});

Go

concepts := []string{"a wild animal"}
nearText := client.GraphQL().NearTextArgBuilder().
WithConcepts(concepts).
WithTargetVectors("jeopardy_questions_vector", "jeopardy_answers_vector")

ctx := context.Background()

result, err := client.GraphQL().Get().
WithClassName(className).
WithFields(graphql.Field{Name: "question"}, graphql.Field{Name: "answer"}).
WithNearText(nearText).
Do(ctx)

Complete code

package main

import (
"context"
"fmt"
"os"

"github.com/weaviate/weaviate-go-client/v4/weaviate"
"github.com/weaviate/weaviate-go-client/v4/weaviate/graphql"
)

func main() {
cfg := weaviate.Config{
Host:   "localhost:8080",
Scheme: "http",
Headers: map[string]string{
  "X-Openai-Api-Key": os.Getenv("OPENAI_API_KEY"),
},
}

client, err := weaviate.NewClient(cfg)
if err != nil {
panic(err)
}

className := "JeopardyTiny"
concepts := []string{"a wild animal"}
nearText := client.GraphQL().NearTextArgBuilder().
WithConcepts(concepts).
WithTargetVectors("jeopardy_questions_vector", "jeopardy_answers_vector")

ctx := context.Background()

result, err := client.GraphQL().Get().
WithClassName(className).
WithFields(graphql.Field{Name: "question"}, graphql.Field{Name: "answer"}).
WithNearText(nearText).
Do(ctx)

if err != nil {
panic(err)
}

fmt.Printf("%+v", result)
}

Specify query vectors

You can specify multiple query vectors in the search query with a nearVector search. This allows use of a different query vector for each corresponding target vector.

Python Client v4

from weaviate.classes.query import MetadataQuery

collection = client.collections.get("JeopardyTiny")

response = collection.query.near_vector(
    # Specify the query vectors for each target vector
    near_vector={
        "jeopardy_questions_vector": v1,
        "jeopardy_answers_vector": v2,
    },
    limit=2,
    target_vector=["jeopardy_questions_vector", "jeopardy_answers_vector"],  # Specify the target vectors
    return_metadata=MetadataQuery(distance=True)
)

for o in response.objects:
    print(o.properties)
    return_metadata=MetadataQuery(distance=True)

JS/TS Client v3

var jeopardy, result;

jeopardy = client.collections.get('JeopardyTiny');

result = await jeopardy.query.nearVector({
  'jeopardy_questions_vector': v1,
  'jeopardy_answers_vector': v2
}, {
  limit: 2,
  targetVector: ['jeopardy_questions_vector', 'jeopardy_answers_vector'],
  returnMetadata: ['distance'],
});

result.objects.forEach((item) => {
  console.log(JSON.stringify(item.properties, null, 2));
  console.log(JSON.stringify(item.metadata?.distance, null, 2));
});

You can also specify the query vectors as an array of vectors. The array will be parsed according to the order of the specified target vectors.

Specify array(s) of query vectors

Added in v1.27

Additionally to the above, you can specify the same target vector multiple times with different query vectors. In other words, you can use multiple query vectors for the same target vector.

The query vectors in this case are specified as an array of vectors. There are multiple ways to specify the target vectors in this case:

Target vector names only

The target vectors can be specified as an array as shown here.

Python Client v4

from weaviate.classes.query import MetadataQuery

collection = client.collections.get("JeopardyTiny")

response = collection.query.near_vector(
    # Specify the query vectors for each target vector
    near_vector={
        "jeopardy_questions_vector": v1,
        "jeopardy_answers_vector": [v2, v3]
    },
    limit=2,
    # Specify the target vectors as a list
    target_vector=[
        "jeopardy_questions_vector",
        "jeopardy_answers_vector",
    ],
    return_metadata=MetadataQuery(distance=True)
)

for o in response.objects:
    print(o.properties)
    return_metadata=MetadataQuery(distance=True)

JS/TS Client v3

var jeopardy, result;

jeopardy = client.collections.get('JeopardyTiny');

result = await jeopardy.query.nearVector({
  // Specify the query vectors for each target vector. where v1, v2.. are vectors
  'jeopardy_questions_vector': v1,
  'jeopardy_answers_vector': [v2, v3]
}, {
  limit: 2,
  // Specify the target vectors as a list
  targetVector: ['jeopardy_questions_vector', 'jeopardy_answers_vector'],
  returnMetadata: ['distance'],
});

result.objects.forEach((item) => {
  console.log(JSON.stringify(item.properties, null, 2));
  console.log(JSON.stringify(item.metadata?.distance, null, 2));
});

Target vectors and weights

If you want to provide weights for each target vector you can do it as shown here.

Python Client v4

from weaviate.classes.query import MetadataQuery

collection = client.collections.get("JeopardyTiny")

response = collection.query.near_vector(
    # Specify the query vectors for each target vector
    near_vector={
        "jeopardy_questions_vector": v1,
        "jeopardy_answers_vector": [v2, v3]
    },
    limit=2,
    # Specify the target vectors and weights
    target_vector=TargetVectors.manual_weights({
        "jeopardy_questions_vector": 10,
        "jeopardy_answers_vector": [30, 30],  # Matches the order of the vectors above
    }),
    return_metadata=MetadataQuery(distance=True)
)

for o in response.objects:
    print(o.properties)
    return_metadata=MetadataQuery(distance=True)

JS/TS Client v3

var jeopardy, result;

jeopardy = client.collections.get('JeopardyTiny');

result = await jeopardy.query.nearVector({
  'jeopardy_questions_vector': v1,
  'jeopardy_answers_vector': [v2, v3]
}, {
  limit: 2,
  // Specify the target vectors as a list
  targetVector: jeopardy.multiTargetVector.manualWeights({
    "jeopardy_questions_vector": 10,
    "jeopardy_answers_vector": [30, 30], // Matches the order of the vectors above
  }),
  returnMetadata: ['distance'],
});

result.objects.forEach((item) => {
  console.log(JSON.stringify(item.properties, null, 2));
  console.log(JSON.stringify(item.metadata?.distance, null, 2));
});

Specify target vector names and join strategy

Specify target vectors as an array of named vectors and how to join the result sets.

The sum, average, minimum join strategies only require the name of the strategy and the target vectors.

Python Client v4

from weaviate.classes.query import TargetVectors, MetadataQuery

collection = client.collections.get("JeopardyTiny")

response = collection.query.near_text(
    query="a wild animal",
    limit=2,
    target_vector=TargetVectors.average(["jeopardy_questions_vector", "jeopardy_answers_vector"]),  # Specify the target vectors and the join strategy
    # .sum(), .minimum(), .manual_weights(), .relative_score() also available
    return_metadata=MetadataQuery(distance=True)
)

for o in response.objects:
    print(o.properties)
    print(o.metadata.distance)

JS/TS Client v3

var jeopardy, result;

jeopardy = client.collections.get('JeopardyTiny');

result = await jeopardy.query.nearText('a wild animal', {
  limit: 2,
  targetVector: jeopardy.multiTargetVector.average(['jeopardy_questions_vector', 'jeopardy_answers_vector']),
  returnMetadata: ['distance'],
});

result.objects.forEach((item) => {
  console.log(JSON.stringify(item.properties, null, 2));
  console.log(JSON.stringify(item.metadata?.distance, null, 2));
});

Weight raw vector distances

Search by sums of weighted, raw distances to each target vector.

The weighting in detail

Each distance between the query vector and the target vector is multiplied by the specified weight, then the resulting weighted distances are summed for each object to produce a combined distance. The search results are sorted by this combined distance.

Python Client v4

from weaviate.classes.query import TargetVectors, MetadataQuery

collection = client.collections.get("JeopardyTiny")

response = collection.query.near_text(
    query="a wild animal",
    limit=2,
    target_vector=TargetVectors.manual_weights({
        "jeopardy_questions_vector": 10,
        "jeopardy_answers_vector": 50
    }),
    return_metadata=MetadataQuery(distance=True)
)

for o in response.objects:
    print(o.properties)
    print(o.metadata.distance)

JS/TS Client v3

var jeopardy, result;

jeopardy = client.collections.get('JeopardyTiny');

result = await jeopardy.query.nearText('a wild animal', {
  limit: 2,
  targetVector: jeopardy.multiTargetVector.manualWeights({
    jeopardy_questions_vector: 10,
    jeopardy_answers_vector: 50,
  }),
  returnMetadata: ['distance'],
});

result.objects.forEach((item) => {
  console.log(JSON.stringify(item.properties, null, 2));
  console.log(JSON.stringify(item.metadata?.distance, null, 2));
});

Weight normalized vector distances

Search by sums of weighted, normalized distances to each target vector.

The weighting in detail

Each distance is normalized against other results for that target vector. Each normalized distance between the query vector and the target vector is multiplied by the specified weight. The resulting weighted distances are summed for each object to produce a combined distance. The search results are sorted by this combined distance.

For a more detailed explanation of how scores are normalized, see the blog post on hybrid relative score fusion

Python Client v4

from weaviate.classes.query import TargetVectors, MetadataQuery

collection = client.collections.get("JeopardyTiny")

response = collection.query.near_text(
    query="a wild animal",
    limit=2,
    target_vector=TargetVectors.relative_score({
        "jeopardy_questions_vector": 10,
        "jeopardy_answers_vector": 10
    }),
    return_metadata=MetadataQuery(distance=True)
)

for o in response.objects:
    print(o.properties)
    print(o.metadata.distance)

JS/TS Client v3

var jeopardy, result;

jeopardy = client.collections.get('JeopardyTiny');

result = await jeopardy.query.nearText('a wild animal', {
  limit: 2,
  targetVector: jeopardy.multiTargetVector.relativeScore({
    jeopardy_questions_vector: 10,
    jeopardy_answers_vector: 10,
  }),
  returnMetadata: ['distance'],
});

result.objects.forEach((item) => {
  console.log(JSON.stringify(item.properties, null, 2));
  console.log(JSON.stringify(item.metadata?.distance, null, 2));
});

Related pages

• Connect to Weaviate

Questions and feedback

If you have any questions or feedback, let us know in the user forum.

• Technical questions
• Documentation feedback