text2vec-aws

Added in v1.22.5

Starting in v1.22.5, AWS Bedrock is supported.

Added in v1.24

Starting in v1.24.1, AWS Sagemaker is supported.

The text2vec-aws module allows Weaviate to access AWS Bedrock and AWS Sagemaker services.

If you need to run your own embedding service, use Sagemaker. Bedrock uses AWS models.

Considerations

• This module is available on Weaviate Cloud Services (WCS).
• Bedrock and Sagemaker are third party APIs. You must provide AWS API credentials.
• Bedrock requires a model.
  • There is no default Bedrock model set for this module.
  • You must request access from AWS Bedrock to make models available in your account.
  • Not all AWS Bedrock models are supported. See Supported models.
  • If you set "service": "bedrock"``, set a model as well. For example, "model": "amazon.titan-embed-text-v1"`
  • Set a distance metric in your class configuration that corresponds to the model you use.
• These APIs may incur costs. Before vectorizing large amounts of data, check the pricing pages.
  • AWS Bedrock pricing
  • AWS Sagemaker
• Enabling this module enables the nearText search operator.

Where to set module parameters

The module accepts parameters through the request header, collection configuration, or environment variables. Some parameters (such as the API key) can be set in multiple ways.

Where the same parameter can be set in multiple ways, setting it at query-time through the HTTP request header (if possible) will have the highest precedence.

We suggest you only set any given parameter in one place to avoid confusion.

API Authentication

You must provide access key based AWS credentials to use the API, including an AWS access key and a corresponding AWS secret access key. You can set them as environment variables, or provide them at query time.

Weaviate instance configuration

Docker Compose file

To use text2vec-aws, enable it in your Docker Compose file (docker-compose.yml).

Parameters

Parameter	Type	Optional	Default	Description
AWS_ACCESS_KEY	string	yes	none	Your AWS access key. An alternative for AWS_ACCESS_KEY_ID.
AWS_ACCESS_KEY_ID	string	yes	none	Your AWS access key. An alternative for AWS_ACCESS_KEY.
AWS_SECRET_KEY	string	yes	none	Your AWS secret access key. An alternative for AWS_SECRET_ACCESS_KEY.
AWS_SECRET_ACCESS_KEY	string	yes	none	Your AWS secret access key. An alternative for AWS_SECRET_KEY.
DEFAULT_VECTORIZER_MODULE	string	yes	none	The default vectorizer module. To make text2vec-aws the default vectorizer, set this parameter to text2vec-aws.
ENABLE_MODULES	string	no	none	Set text2vec-aws to enable the module.
SERVICE	string	yes	bedrock	Must be bedrock or sagemaker.

Example

This configuration does the following:

• Enables the text2vec-aws vectorizer
• Sets text2vec-aws as the default vectorizer
• Sets AWS authentication credentials

---
version: '3.4'
services:
  weaviate:
    image: cr.weaviate.io/semitechnologies/weaviate:1.24.10
    restart: on-failure:0
    ports:
     - 8080:8080
     - 50051:50051
    environment:
      QUERY_DEFAULTS_LIMIT: 20
      AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
      PERSISTENCE_DATA_PATH: "./data"
      ENABLE_MODULES: text2vec-aws
      DEFAULT_VECTORIZER_MODULE: text2vec-aws
      AWS_ACCESS_KEY: sk-foobar  # Optional. Can be set at query time.
      AWS_SECRET_KEY: sk-foobar  # Optional. Can be set at query time.
      CLUSTER_HOSTNAME: 'node1'
...

Class configuration

To configure module behavior for a collection, set collection level values in the Weaviate schema.

API settings

Parameters

Parameter	Required	Default	Purpose
model	No	NONE	The model to use with Bedrock. See supported models.
region	Yes	NONE	AWS region name. For example, us-east-1.

Example

The following example configures the Document class to set the following parameters:

• vectorizer: text2vec-aws
• distance metric: cosine
• service: "sagemaker"
• endpoint: AWS provides this value when you configure sagemaker
• AWS region: us-east-1

{
  "classes": [
    {
      "class": "Document",
      "description": "A class called document",
      "vectorizer": "text2vec-aws",
      "vectorIndexConfig": {
        "distance": "cosine"
      },
      "moduleConfig": {
        "text2vec-aws": {
          "model": "amazon.titan-embed-text-v1",   // REQUIRED
          "region": "us-east-1"                    // REQUIRED
        }
      },
    }
  ]
}

Vectorizer settings

To configure vectorization for collections and properties, use the moduleConfig section in the collection definition.

Setting	Level	Default	Description
vectorizer	collection	none	The module that vectorizes the data.
vectorizeClassName	collection	true	When true, vectorize the class name.
skip	property	false	When true, does not vectorize the property.
vectorizePropertyName	property	false	When true, does not vectorize the property name.

Example

{
  "classes": [
    {
      "class": "Document",
      "description": "A class called document",
      "vectorizer": "text2vec-aws",
      "vectorIndexConfig": {
        "distance": "cosine"
      },
      "moduleConfig": {
        "text2vec-aws": {
          "model": "amazon.titan-embed-text-v1",    // REQUIRED
          "region": "us-east-1"                     // REQUIRED
          "vectorizeClassName": false
        }
      },
      "properties": [
        {
          "name": "content",
          "dataType": ["text"],
          "description": "Content that will be vectorized",
          "moduleConfig": {
            "text2vec-aws": {
              "skip": false,
              "vectorizePropertyName": false
            }
          }
        }
      ]
    }
  ]
}

Query-time parameters

To supply parameters at query time, adding them to the HTTP header.

HTTP Header	Value	Purpose
"X-AWS-Access-Key"	"YOUR-AWS-API-ACCESS-KEY"	Your AWS access key.
"X-AWS-Secret-Key"	"YOUR-AWS-API-SECRET-KEY"	Your AWS secret access key

Additional information

Supported models

text2vec-aws supports these models:

• amazon.titan-embed-text-v1
• cohere.embed-english-v3
• cohere.embed-multilingual-v3

Distance metric

You can change the distance metric used for vector searches. cosine distance is a good starting point. For supported distance metrics, see Distance metrics.

Consult the documentation of the model you are using to see which distance metrics are appropriate.

API rate limits

This module uses your AWS credentials. API limits that restrict your AWS account also apply to the module. When the API returns a rate limited error, Weaviate returns an error message.

Import throttling

If your API access is rate limited, consider throttling imports within your application.

See code example

Python

from weaviate import Client
import time

def configure_batch(client: Client, batch_size: int, batch_target_rate: int):
    """
    Configure the weaviate client's batch so it creates objects at `batch_target_rate`.

    Parameters
    ----------
    client : Client
        The Weaviate client instance.
    batch_size : int
        The batch size.
    batch_target_rate : int
        The batch target rate as # of objects per second.
    """

    def callback(batch_results: dict) -> None:

        # you could print batch errors here
        time_took_to_create_batch = batch_size * (client.batch.creation_time/client.batch.recommended_num_objects)
        time.sleep(
            max(batch_size/batch_target_rate - time_took_to_create_batch + 1, 0)
        )

    client.batch.configure(
        batch_size=batch_size,
        timeout_retries=5,
        callback=callback,
    )

Go

package main

import (
    "context"
    "time"

    "github.com/weaviate/weaviate-go-client/v4/weaviate"
    "github.com/weaviate/weaviate/entities/models"
)

var (
    // adjust to your liking
    targetRatePerMin = 600
    batchSize        = 50
)

func main() {
    cfg := weaviate.Config{
        Host:   "localhost:8080",
        Scheme: "http",
    }
    client, err := weaviate.NewClient(cfg)
    if err != nil {
        panic(err)
    }

    // replace those 10000 empty objects with your actual data
    objects := make([]*models.Object, 10000)

    // we aim to send one batch every tickInterval second.
    tickInterval := time.Duration(batchSize/targetRatePerMinute) * time.Minute
    t := time.NewTicker(tickInterval)
    before := time.Now()

    for i := 0; i < len(objects); i += batchSize {

        // create a fresh batch
        batch := client.Batch().ObjectsBatcher()

        // add batchSize objects to the batch
        for j := i; j < i+batchSize; j++ {
            batch = batch.WithObject(objects[i+j])
        }

        // send off batch
        res, err := batch.Do(context.Background())
        // TODO: inspect result for individual errors
        _ = res
        // TODO: check request error
        _ = err

        // we wait for the next tick. If the previous batch took longer than
        // tickInterval, we won't need to wait, effectively making this an
        // unthrottled import.
        <-t.C
    }
}

Usage example

This is an example of a nearText query with text2vec-aws.

Python (v4)

import weaviate
from weaviate.classes.query import MetadataQuery, Move
import os

client = weaviate.connect_to_local(
    headers={
        "X-AWS-Access-Key": "YOUR_ACCESS_KEY",
        "X-AWS-Secret-Key": "YOUR_SECRET_KEY",
    }
)

publications = client.collections.get("Publication")

response = publications.query.near_text(
    query="fashion",
    distance=0.6,
    move_to=Move(force=0.85, concepts="haute couture"),
    move_away=Move(force=0.45, concepts="finance"),
    return_metadata=MetadataQuery(distance=True),
    limit=2
)

for o in response.objects:
    print(o.properties)
    print(o.metadata)

client.close()

Python (v3)

import weaviate

client = weaviate.Client(
    url="http://localhost:8080",
    additional_headers={
        "X-AWS-Access-Key": "YOUR_ACCESS_KEY",
        "X-AWS-Secret-Key": "YOUR_SECRET_KEY",
    }
)

nearText = {
  "concepts": ["fashion"],
  "distance": 0.6, # prior to v1.14 use "certainty" instead of "distance"
  "moveAwayFrom": {
    "concepts": ["finance"],
    "force": 0.45
  },
  "moveTo": {
    "concepts": ["haute couture"],
    "force": 0.85
  }
}

result = (
  client.query
  .get("Publication", "name")
  .with_additional(["certainty OR distance"]) # note that certainty is only supported if distance==cosine
  .with_near_text(nearText)
  .do()
)

print(result)

JavaScript/TypeScript

import weaviate, { ApiKey } from 'weaviate-ts-client';

const client = weaviate.client({
  scheme: 'https',
  host: 'WEAVIATE_INSTANCE_URL',  // Replace WEAVIATE_INSTANCE_URL with your instance URL
  apiKey: new ApiKey('YOUR-API-KEY'),
  headers: {
    'X-AWS-Access-Key': 'YOUR_ACCESS_KEY',
    'X-AWS-Secret-Key': 'YOUR_SECRET_KEY'
  },
});

const response = await client.graphql
  .get()
  .withClassName('Publication')
  .withFields('name _additional{certainty distance}') // note that certainty is only supported if distance==cosine
  .withNearText({
    concepts: ['fashion'],
    distance: 0.6,  // prior to v1.14 use certainty instead of distance
    moveAwayFrom: {
      concepts: ['finance'],
      force: 0.45,
    },
    moveTo: {
      concepts: ['haute couture'],
      force: 0.85,
    },
  })
  .do();
console.log(response);

Go

package main

import (
  "context"
  "fmt"

  "github.com/weaviate/weaviate-go-client/v4/weaviate"
  "github.com/weaviate/weaviate-go-client/v4/weaviate/graphql"
)

func main() {
  cfg := weaviate.Config{
    Host:    "WEAVIATE_INSTANCE_URL",   // Replace with your instance URL
    Scheme:  "https",
    Headers: map[string]string{
      "X-AWS-Access-Key": "YOUR_ACCESS_KEY",
      "X-AWS-Secret-Key": "YOUR_SECRET_KEY",
    },
  }
  client, err := weaviate.NewClient(cfg)
  if err != nil {
    panic(err)
  }

  className := "Publication"

  name := graphql.Field{Name: "name"}
  _additional := graphql.Field{
    Name: "_additional", Fields: []graphql.Field{
      {Name: "certainty"}, // only supported if distance==cosine
      {Name: "distance"},  // always supported
    },
  }

  concepts := []string{"fashion"}
  distance := float32(0.6)
  moveAwayFrom := &graphql.MoveParameters{
    Concepts: []string{"finance"},
    Force:    0.45,
  }
  moveTo := &graphql.MoveParameters{
    Concepts: []string{"haute couture"},
    Force:    0.85,
  }
  nearText := client.GraphQL().NearTextArgBuilder().
    WithConcepts(concepts).
    WithDistance(distance). // use WithCertainty(certainty) prior to v1.14
    WithMoveTo(moveTo).
    WithMoveAwayFrom(moveAwayFrom)

  ctx := context.Background()

  result, err := client.GraphQL().Get().
    WithClassName(className).
    WithFields(name, _additional).
    WithNearText(nearText).
    Do(ctx)

  if err != nil {
    panic(err)
  }
  fmt.Printf("%v", result)
}

Java

package io.weaviate;

import io.weaviate.client.Config;
import io.weaviate.client.WeaviateClient;
import io.weaviate.client.base.Result;
import io.weaviate.client.v1.graphql.model.GraphQLResponse;
import io.weaviate.client.v1.graphql.query.argument.NearTextArgument;
import io.weaviate.client.v1.graphql.query.argument.NearTextMoveParameters;
import io.weaviate.client.v1.graphql.query.fields.Field;

import java.util.HashMap;
import java.util.Map;

public class App {
  public static void main(String[] args) {
    Map<String, String> headers = new HashMap<String, String>() { {
      put("X-AWS-Access-Key", "YOUR_ACCESS_KEY");
      put("X-AWS-Secret-Key", "YOUR_SECRET_KEY");
    } };
    Config config = new Config("https", "WEAVIATE_INSTANCE_URL", headers);
    // Replace with your instance URL

    WeaviateClient client = new WeaviateClient(config);

    NearTextMoveParameters moveTo = NearTextMoveParameters.builder()
      .concepts(new String[]{ "haute couture" }).force(0.85f).build();

    NearTextMoveParameters moveAway = NearTextMoveParameters.builder()
      .concepts(new String[]{ "finance" }).force(0.45f)
      .build();

    NearTextArgument nearText = client.graphQL().arguments().nearTextArgBuilder()
      .concepts(new String[]{ "fashion" })
      .distance(0.6f) // use .certainty(0.7f) prior to v1.14
      .moveTo(moveTo)
      .moveAwayFrom(moveAway)
      .build();

    Field name = Field.builder().name("name").build();
    Field _additional = Field.builder()
      .name("_additional")
      .fields(new Field[]{
        Field.builder().name("certainty").build(), // only supported if distance==cosine
        Field.builder().name("distance").build(),  // always supported
      }).build();

    Result<GraphQLResponse> result = client.graphQL().get()
      .withClassName("Publication")
      .withFields(name, _additional)
      .withNearText(nearText)
      .run();

    if (result.hasErrors()) {
      System.out.println(result.getError());
      return;
    }
    System.out.println(result.getResult());
  }
}

Curl

# Note: Under nearText, use `certainty` instead of distance prior to v1.14
# Under _additional, `certainty` is only supported if distance==cosine, but `distance` is always supported
echo '{
  "query": "{
    Get {
      Publication(
        nearText: {
          concepts: [\"fashion\"],
          distance: 0.6,
          moveAwayFrom: {
            concepts: [\"finance\"],
            force: 0.45
          },
          moveTo: {
            concepts: [\"haute couture\"],
            force: 0.85
          }
        }
      ) {
        name
        _additional {
          certainty
          distance
        }
      }
    }
  }"
}' | curl \
    -X POST \
    -H 'Content-Type: application/json' \
    -H "Authorization: Bearer $WEAVIATE_API_KEY" \
    -H "X-AWS-Access-Key: $YOUR_ACCESS_KEY" \
    -H "X-AWS-Secret-Key: $YOUR_SECRET_KEY" \
    -d @- \
    https://WEAVIATE_INSTANCE_URL/v1/graphql  # Replace WEAVIATE_INSTANCE_URL with your instance URL

GraphQL

{
  Get{
    Publication(
      nearText: {
        concepts: ["fashion"],
        distance: 0.6 # prior to v1.14 use "certainty" instead of "distance"
        moveAwayFrom: {
          concepts: ["finance"],
          force: 0.45
        },
        moveTo: {
          concepts: ["haute couture"],
          force: 0.85
        }
      }
    ){
      name
      _additional {
        certainty # only supported if distance==cosine.
        distance  # always supported
      }
    }
  }
}

Questions and feedback

If you have any questions or feedback, let us know in our user forum.

• Technical questions
• Documentation feedback