text2vec-openai

     

In short

• This module uses a third-party API and may incur costs.
• Check the vendor pricing (e.g. OpenAI pricing page) before vectorizing large amounts of data.
• Weaviate automatically parallelizes requests to the API when using the batch endpoint.
• Check out the text2vec-openai demo.
• You will need an API key from OpenAI or Azure OpenAI to use this module.
• The default OpenAI model is text-embedding-ada-002.

Overview

The text2vec-openai module enables you to use OpenAI in Weaviate or Azure embeddings to represent data objects.

Azure OpenAI or OpenAI?

tip

This module is compatible with both OpenAI and Azure OpenAI.

The instructions vary slightly based on whether you are using OpenAI directly or Azure OpenAI. Please make sure that you are following the right instructions for your service provider.

The differences are in:

• Parameter names used in the schema, and
• Names of the API key to be used.

Module configuration

Not applicable to WCS

This module is enabled and pre-configured on Weaviate Cloud Services.

Configuration file (Weaviate open source only)

You can enable the text2vec-openai module in your configuration file (e.g. docker-compose.yaml).

• This configuration will start Weaviate with the OpenAI module enabled, and set as the default vectorizer module.
• Optionally, you can specify the required API key in the file..
  • If you do not, you must specify the API key at runtime.

---
version: '3.4'
services:
  weaviate:
    image: semitechnologies/weaviate:1.19.6
    restart: on-failure:0
    ports:
     - "8080:8080"
    environment:
      QUERY_DEFAULTS_LIMIT: 20
      AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
      PERSISTENCE_DATA_PATH: "./data"
      DEFAULT_VECTORIZER_MODULE: text2vec-openai
      ENABLE_MODULES: text2vec-openai
      OPENAI_APIKEY: sk-foobar  # For use with OpenAI. Setting this parameter is optional; you can also provide the key at runtime.
      AZURE_APIKEY: sk-foobar  # For use with Azure OpenAI. Setting this parameter is optional; you can also provide the key at runtime.
      CLUSTER_HOSTNAME: 'node1'
...

note

• You can also use the Weaviate configuration tool to create a Weaviate setup with this module.
• The OPENAI_APIKEY environment variable is optional and you can instead provide the key at insert/query time as an HTTP header (see the 'usage' section for instructions)

Schema configuration

You can provide additional module configurations through the schema. You can learn about schemas here.

For text2vec-openai, you can set the vectorizer model and vectorizer behavior.

OpenAI settings

Set the vectorizer model using parameters model, modelVersion and type in the moduleConfig section of your schema:

{
  "classes": [
    {
      "class": "Document",
      "description": "A class called document",
      "vectorizer": "text2vec-openai",
      "moduleConfig": {
        "text2vec-openai": {
          "model": "ada",
          "modelVersion": "002",
          "type": "text"
        }
      },
    }
  ]
}

Azure OpenAI settings

Set the parameters resourceName and deploymentId in the moduleConfig section of your schema:

{
  "classes": [
    {
      "class": "Document",
      "description": "A class called document",
      "vectorizer": "text2vec-openai",
      "moduleConfig": {
        "text2vec-openai": {
          "resourceName": "<YOUR-RESOURCE-NAME>",
          "deploymentId": "<YOUR-MODEL-NAME>",
        }
      }
    }
  ]
}

Vectorizer behavior

Set property-level vectorizer behavior using the moduleConfig section under each property:

{
  "classes": [
    {
      "class": "Document",
      "description": "A class called document",
      "vectorizer": "text2vec-openai",
      "moduleConfig": {
        "text2vec-openai": {
          "model": "ada",
          "modelVersion": "002",
          "type": "text"
        }
      },
      "properties": [
        {
          "dataType": ["text"],
          "description": "Content that will be vectorized",
          "moduleConfig": {
            "text2vec-openai": {
              "skip": false,
              "vectorizePropertyName": false
            }
          },
          "name": "content"
        }
      ]
    }
  ]
}

Usage

Enabling this module will make GraphQL vector search operators available.

Provide the API key

If the API key is not set in the text2vec-openai configuration, you can supply it when making a query.

You can achieve this by adding the appropriate key to the HTTP header:

• X-OpenAI-Api-Key: YOUR-OPENAI-API-KEY for OpenAI, and
• X-Azure-Api-Key: YOUR-AZURE-API-KEY for Azure OpenAI, and

Example

GraphQL

{
  Get{
    Publication(
      nearText: {
        concepts: ["fashion"],
        distance: 0.6 # prior to v1.14 use "certainty" instead of "distance"
        moveAwayFrom: {
          concepts: ["finance"],
          force: 0.45
        },
        moveTo: {
          concepts: ["haute couture"],
          force: 0.85
        }
      }
    ){
      name
      _additional {
        certainty # only supported if distance==cosine.
        distance  # always supported
      }
    }
  }
}

Python

import weaviate

client = weaviate.Client(
    url="http://localhost:8080",
    additional_headers={
        "X-OpenAI-Api-Key": "YOUR-OPENAI-API-KEY"
    }
)

nearText = {
  "concepts": ["fashion"],
  "distance": 0.6, # prior to v1.14 use "certainty" instead of "distance"
  "moveAwayFrom": {
    "concepts": ["finance"],
    "force": 0.45
  },
  "moveTo": {
    "concepts": ["haute couture"],
    "force": 0.85
  }
}

result = (
  client.query
  .get("Publication", "name")
  .with_additional(["certainty OR distance"]) # note that certainty is only supported if distance==cosine
  .with_near_text(nearText)
  .do()
)

print(result)

JavaScript

const weaviate = require('weaviate-client');

const client = weaviate.client({
  scheme: 'http',
  host: 'localhost:8080',
  headers: {'X-OpenAI-Api-Key': 'YOUR-OPENAI-API-KEY'},
});

client.graphql
  .get()
  .withClassName('Publication')
  .withFields('name _additional{certainty distance}') // note that certainty is only supported if distance==cosine
  .withNearText({
    concepts: ['fashion'],
    distance: 0.6, // prior to v1.14 use certainty instead of distance
    moveAwayFrom: {
      concepts: ['finance'],
      force: 0.45
    },
    moveTo: {
      concepts: ['haute couture'],
      force: 0.85
    }
  })
  .do()
  .then(console.log)
  .catch(console.error);

Go

package main

import (
  "context"
  "fmt"

  "github.com/weaviate/weaviate-go-client/v4/weaviate"
  "github.com/weaviate/weaviate-go-client/v4/weaviate/graphql"
)

func main() {
  cfg := weaviate.Config{
    Host:    "localhost:8080",
    Scheme:  "http",
    Headers: map[string]string{"X-OpenAI-Api-Key": "YOUR-OPENAI-API-KEY"},
  }
  client, err := weaviate.NewClient(cfg)
  if err != nil {
    panic(err)
  }

  className := "Publication"

  name := graphql.Field{Name: "name"}
  _additional := graphql.Field{
    Name: "_additional", Fields: []graphql.Field{
      {Name: "certainty"}, // only supported if distance==cosine
      {Name: "distance"},  // always supported
    },
  }

  concepts := []string{"fashion"}
  distance := float32(0.6)
  moveAwayFrom := &graphql.MoveParameters{
    Concepts: []string{"finance"},
    Force:    0.45,
  }
  moveTo := &graphql.MoveParameters{
    Concepts: []string{"haute couture"},
    Force:    0.85,
  }
  nearText := client.GraphQL().NearTextArgBuilder().
    WithConcepts(concepts).
    WithDistance(distance). // use WithCertainty(certainty) prior to v1.14
    WithMoveTo(moveTo).
    WithMoveAwayFrom(moveAwayFrom)

  ctx := context.Background()

  result, err := client.GraphQL().Get().
    WithClassName(className).
    WithFields(name, _additional).
    WithNearText(nearText).
    Do(ctx)

  if err != nil {
    panic(err)
  }
  fmt.Printf("%v", result)
}

Java

package io.weaviate;

import io.weaviate.client.Config;
import io.weaviate.client.WeaviateClient;
import io.weaviate.client.base.Result;
import io.weaviate.client.v1.graphql.model.GraphQLResponse;
import io.weaviate.client.v1.graphql.query.argument.NearTextArgument;
import io.weaviate.client.v1.graphql.query.argument.NearTextMoveParameters;
import io.weaviate.client.v1.graphql.query.fields.Field;

import java.util.HashMap;
import java.util.Map;

public class App {
  public static void main(String[] args) {
    Map<String, String> headers = new HashMap<String, String>() { {
      put("X-OpenAI-Api-Key", "YOUR-OPENAI-API-KEY");
    } };
    Config config = new Config("http", "localhost:8080", headers);
    WeaviateClient client = new WeaviateClient(config);

    NearTextMoveParameters moveTo = NearTextMoveParameters.builder()
      .concepts(new String[]{ "haute couture" }).force(0.85f).build();

    NearTextMoveParameters moveAway = NearTextMoveParameters.builder()
      .concepts(new String[]{ "finance" }).force(0.45f)
      .build();

    NearTextArgument nearText = client.graphQL().arguments().nearTextArgBuilder()
      .concepts(new String[]{ "fashion" })
      .distance(0.6f) // use .certainty(0.7f) prior to v1.14
      .moveTo(moveTo)
      .moveAwayFrom(moveAway)
      .build();

    Field name = Field.builder().name("name").build();
    Field _additional = Field.builder()
      .name("_additional")
      .fields(new Field[]{
        Field.builder().name("certainty").build(), // only supported if distance==cosine
        Field.builder().name("distance").build(),  // always supported
      }).build();

    Result<GraphQLResponse> result = client.graphQL().get()
      .withClassName("Publication")
      .withFields(name, _additional)
      .withNearText(nearText)
      .run();

    if (result.hasErrors()) {
      System.out.println(result.getError());
      return;
    }
    System.out.println(result.getResult());
  }
}

Curl

$ echo '{
  "query": "{
    Get{
      Publication(
        nearText: {
          concepts: [\"fashion\"],
          distance: 0.6, // use certainty instead of distance prior to v1.14
          moveAwayFrom: {
            concepts: [\"finance\"],
            force: 0.45
          },
          moveTo: {
            concepts: [\"haute couture\"],
            force: 0.85
          }
        }
      ){
        name
        _additional {
          certainty // only supported if distance==cosine
          distance  // always supported
        }
      }
    }
  }"
}' | curl \
    -X POST \
    -H 'Content-Type: application/json' \
    -H "X-OpenAI-Api-Key: YOUR-OPENAI-API-KEY" \
    -d @- \
    http://localhost:8080/v1/graphql

Additional information

Available models (OpenAI)

OpenAI has multiple models available with different trade-offs. All the models offered by OpenAI can be used within Weaviate. Note that the more dimensions a model produces, the larger your data footprint will be. To estimate the total size of your dataset use this calculation.

The default model is text-embedding-ada-002 but you can also specify it in your schema. An example as part of a class definition:

{
  "classes": [
    {
      "class": "Document",
      "vectorizer": "text2vec-openai",
      "moduleConfig": {
        "text2vec-openai": {
          "model": "ada",
          "modelVersion": "002",
          "type": "text"
        }
      }
    }
  ]
}

For document embeddings, choose from the following models:

• ada
• babbage
• curie
• davinci

For code embeddings, see the Codex models.

In the moduleConfig inside a class, you need to set two values:

1. model – one of the models mentioned above, e.g. davinci.
2. modelVersion – version string, e.g. 003.
3. type – text or code.

Example (as part of a class definition):

{
  "classes": [
    {
      "class": "Document",
      "vectorizer": "text2vec-openai",
      "moduleConfig": {
        "text2vec-openai": {
          "model": "ada",
          "modelVersion": "002",
          "type": "text"
        }
      }
    }
  ]
}

OpenAI rate limits

Since you will obtain embeddings using your own API key, any corresponding rate limits related to your account will apply to your use with Weaviate also.

If you exceed your rate limit, Weaviate will output the error message generated by the OpenAI API. You can request to increase your rate limit by emailing OpenAI at support@openai.com describing your use case with Weaviate.

Throttle the import inside your application

One way of dealing with rate limits is to throttle the import within your application. For example, when using the Weaviate client in Python or Java:

Python

from weaviate import Client
import time

def configure_batch(client: Client, batch_size: int, batch_target_rate: int):
    """
    Configure the weaviate client's batch so it creates objects at `batch_target_rate`.

    Parameters
    ----------
    client : Client
        The Weaviate client instance.
    batch_size : int
        The batch size.
    batch_target_rate : int
        The batch target rate as # of objects per second.
    """

    def callback(batch_results: dict) -> None:

        # you could print batch errors here
        time_took_to_create_batch = batch_size * (client.batch.creation_time/client.batch.recommended_num_objects)
        time.sleep(
            max(batch_size/batch_target_rate - time_took_to_create_batch + 1, 0)
        )

    client.batch.configure(
        batch_size=batch_size,
        timeout_retries=5,
        callback=callback,
    )

Go

package main

import (
    "context"
    "time"

    "github.com/weaviate/weaviate-go-client/v4/weaviate"
    "github.com/weaviate/weaviate/entities/models"
)

var (
    // adjust to your liking
    targetRatePerMin = 600
    batchSize        = 50
)

func main() {
    cfg := weaviate.Config{
        Host:   "localhost:8080",
        Scheme: "http",
    }
    client, err := weaviate.NewClient(cfg)
    if err != nil {
        panic(err)
    }

    // replace those 10000 empty objects with your actual data
    objects := make([]*models.Object, 10000)

    // we aim to send one batch every tickInterval second.
    tickInterval := time.Duration(batchSize/targetRatePerMinute) * time.Minute
    t := time.NewTicker(tickInterval)
    before := time.Now()

    for i := 0; i < len(objects); i += batchSize {

        // create a fresh batch
        batch := client.Batch().ObjectsBatcher()

        // add batchSize objects to the batch
        for j := i; j < i+batchSize; j++ {
            batch = batch.WithObject(objects[i+j])
        }

        // send off batch
        res, err := batch.Do(context.Background())
        // TODO: inspect result for individual errors
        _ = res
        // TODO: check request error
        _ = err

        // we wait for the next tick. If the previous batch took longer than
        // tickInterval, we won't need to wait, effectively making this an
        // unthrottled import.
        <-t.C
    }
}

The current rate limit will appear in the error message, as shown below:

{
  "message": "Rate limit reached for requests. Limit: 600.000000 / min. Current: 1024.000000 / min. Contact support@openai.com if you continue to have issues."
}

More resources

If you can't find the answer to your question here, please look at the:

1. Frequently Asked Questions. Or,
2. Knowledge base of old issues. Or,
3. For questions: Stackoverflow. Or,
4. For more involved discussion: Weaviate Community Forum. Or,
5. We also have a Slack channel.