text2vec-openai

Overview

The text2vec-openai module enables Weaviate to obtain vectors using OpenAI or Azure OpenAI.

• OpenAI
• Azure OpenAI

Key notes:

• As it uses a third-party API, you will need an API key.
• Its usage may incur costs.
  • Please check the vendor pricing (e.g. OpenAI pricing page), especially before vectorizing large amounts of data.
• This module is available on Weaviate Cloud Services (WCS).
• Enabling this module will enable the nearText search operator.
• The default model is text-embedding-ada-002.

Azure OpenAI or OpenAI?

The module usage instructions may vary based on whether you are using OpenAI directly or Azure OpenAI. Please make sure that you are following the right instructions for your service provider.

For example, the following may vary:

• Parameter names used in the schema, and
• Names of the API key to be used.

Where to set module parameters

The module accepts parameters through the request header, collection configuration, or environment variables. Some parameters (such as the API key) can be set in multiple ways.

Where the same parameter can be set in multiple ways, setting it at query-time through the HTTP request header (if possible) will have the highest precedence.

We suggest you only set any given parameter in one place to avoid confusion.

Weaviate instance configuration

Not applicable to WCS

This module is enabled and pre-configured on Weaviate Cloud Services.

Docker Compose file

To use text2vec-openai, you must enable it in your Docker Compose file (docker-compose.yml). You can do so manually, or create one using the Weaviate configuration tool.

Parameters

Parameter	Required	Purpose
ENABLE_MODULES	Required	The modules to enable. Include text2vec-openai to enable the module.
DEFAULT_VECTORIZER_MODULE	Optional	The default vectorizer module. You can set this to text2vec-openai to make it the default for all classes.
OPENAI_APIKEY	Optional	Your OpenAI API key (if using OpenAI). You can also provide the key at query time.
AZURE_APIKEY	Optional	Your Azure OpenAI API key (if using Azure OpenAI). You can also provide the key at query time.

Example

This configuration enables text2vec-openai, sets it as the default vectorizer, and sets various other parameters such as the API key as environment variables.

---
version: '3.4'
services:
  weaviate:
    image: semitechnologies/weaviate:1.22.5
    restart: on-failure:0
    ports:
     - "8080:8080"
    environment:
      QUERY_DEFAULTS_LIMIT: 20
      AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
      PERSISTENCE_DATA_PATH: "./data"
      ENABLE_MODULES: text2vec-openai
      DEFAULT_VECTORIZER_MODULE: text2vec-openai
      OPENAI_APIKEY: sk-foobar  # For use with OpenAI. Setting this parameter is optional; you can also provide the key at query time.
      OPENAI_ORGANIZATION: your-orgname  # For use with OpenAI. Setting this parameter is optional; you can also provide the key at runtime.
      AZURE_APIKEY: sk-foobar  # For use with Azure OpenAI. Setting this parameter is optional; you can also provide the key at query time.
      CLUSTER_HOSTNAME: 'node1'
...

Class configuration

You can configure how the module will behave in each class through the Weaviate schema.

API settings (OpenAI)

Parameters

Parameter	Required	Default	Purpose
model	Optional	text-embedding-ada-002	A model family, e.g. davinci.
modelVersion	Optional		Version string, e.g. 003.
type	Optional		Model type. Can be text or code.
baseURL	Optional	https://api.openai.com	Sets a proxy or other URL instead of the default OpenAI URL. Use a the protocol domain format: https://your.domain.com.

Example

The following example configures the Document class by setting the vectorizer to text2vec-openai, model to ada, the model version to 002 and the type to text:

{
  "classes": [
    {
      "class": "Document",
      "description": "A class called document",
      "vectorizer": "text2vec-openai",
      "moduleConfig": {
        "text2vec-openai": {
          "model": "ada",
          "modelVersion": "002",
          "type": "text",
          "baseURL": "https://proxy.yourcompanydomain.com"  // Optional. Can be overridden by one set in the HTTP header.
        }
      },
    }
  ]
}

API settings (Azure OpenAI)

Parameters

Parameter
resourceName	Azure resource name
deploymentId	Azure deployment ID (your model name)

Example

{
  "classes": [
    {
      "class": "Document",
      "description": "A class called document",
      "vectorizer": "text2vec-openai",
      "moduleConfig": {
        "text2vec-openai": {
          "resourceName": "<YOUR-RESOURCE-NAME>",
          "deploymentId": "<YOUR-MODEL-NAME>",
        }
      }
    }
  ]
}

Vectorization settings

You can set vectorizer behavior using the moduleConfig section under each class and property:

Collection level settings

Parameter	Type	Default
vectorizer	string	-
vectorizeClassName	boolean	true

Property level settings

Parameter	Type	Default
skip	boolean	false
vectorizePropertyName	boolean	false

Example

{
  "classes": [
    {
      "class": "Document",
      "description": "A class called document",
      "vectorizer": "text2vec-openai",
      "moduleConfig": {
        "text2vec-openai": {
          "model": "ada",
          "modelVersion": "002",
          "type": "text",
          "vectorizeClassName": false
        }
      },
      "properties": [
        {
          "name": "content",
          "dataType": ["text"],
          "description": "Content that will be vectorized",
          "moduleConfig": {
            "text2vec-openai": {
              "skip": false,
              "vectorizePropertyName": false
            }
          }
        }
      ]
    }
  ]
}

Query-time parameters

You can supply parameters at query time by adding it to the HTTP header.

HTTP Header	Value	Purpose	Note
"X-OpenAI-Api-Key"	"YOUR-OPENAI-API-KEY"	OpenAI API key
"X-Azure-Api-Key"	"YOUR-AZURE-API-KEY"	Azure OpenAI API key
"X-OpenAI-Organization"	"YOUR-OPENAI-ORGANIZATION"	OpenAI organization name	Available from version v1.21.1
"X-OpenAI-BaseURL"	"YOUR-OPENAI-BASE-URL"	OpenAI base URL	Available from version v1.22.3 Use the protocol domain format: https://your.domain.com. If specified, this will have precedence over the class-level setting.

Additional information

Available models (OpenAI)

You can use any OpenAI embedding model with text2vec-openai.

For document embeddings, choose from the following models:

• ada
• babbage
• curie
• davinci

For code embeddings, see the Codex models.

Model size vs resource requirements

The more dimensions a model produces, the larger your data footprint will be. You can estimate the total size of your dataset here.

API rate limits

Since this module uses your API key, your account's corresponding rate limits will also apply to the module. Weaviate will output any rate-limit related error messages generated by the API.

You can request to increase your rate limit by emailing OpenAI at support@openai.com describing your use case with Weaviate.

The current rate limit will appear in the error message, as shown below:

{
  "message": "Rate limit reached for requests. Limit: 600.000000 / min. Current: 1024.000000 / min. Contact support@openai.com if you continue to have issues."
}

Import throttling

One potential solution to rate limiting would be to throttle the import within your application. We include an example below.

See code example

Python

from weaviate import Client
import time

def configure_batch(client: Client, batch_size: int, batch_target_rate: int):
    """
    Configure the weaviate client's batch so it creates objects at `batch_target_rate`.

    Parameters
    ----------
    client : Client
        The Weaviate client instance.
    batch_size : int
        The batch size.
    batch_target_rate : int
        The batch target rate as # of objects per second.
    """

    def callback(batch_results: dict) -> None:

        # you could print batch errors here
        time_took_to_create_batch = batch_size * (client.batch.creation_time/client.batch.recommended_num_objects)
        time.sleep(
            max(batch_size/batch_target_rate - time_took_to_create_batch + 1, 0)
        )

    client.batch.configure(
        batch_size=batch_size,
        timeout_retries=5,
        callback=callback,
    )

Go

package main

import (
    "context"
    "time"

    "github.com/weaviate/weaviate-go-client/v4/weaviate"
    "github.com/weaviate/weaviate/entities/models"
)

var (
    // adjust to your liking
    targetRatePerMin = 600
    batchSize        = 50
)

func main() {
    cfg := weaviate.Config{
        Host:   "localhost:8080",
        Scheme: "http",
    }
    client, err := weaviate.NewClient(cfg)
    if err != nil {
        panic(err)
    }

    // replace those 10000 empty objects with your actual data
    objects := make([]*models.Object, 10000)

    // we aim to send one batch every tickInterval second.
    tickInterval := time.Duration(batchSize/targetRatePerMinute) * time.Minute
    t := time.NewTicker(tickInterval)
    before := time.Now()

    for i := 0; i < len(objects); i += batchSize {

        // create a fresh batch
        batch := client.Batch().ObjectsBatcher()

        // add batchSize objects to the batch
        for j := i; j < i+batchSize; j++ {
            batch = batch.WithObject(objects[i+j])
        }

        // send off batch
        res, err := batch.Do(context.Background())
        // TODO: inspect result for individual errors
        _ = res
        // TODO: check request error
        _ = err

        // we wait for the next tick. If the previous batch took longer than
        // tickInterval, we won't need to wait, effectively making this an
        // unthrottled import.
        <-t.C
    }
}

Usage example

This is an example of a nearText query with text2vec-openai.

Python

import weaviate

client = weaviate.Client(
    url="http://localhost:8080",
    additional_headers={
        "X-OpenAI-Api-Key": "YOUR-OPENAI-API-KEY"
    }
)

nearText = {
  "concepts": ["fashion"],
  "distance": 0.6, # prior to v1.14 use "certainty" instead of "distance"
  "moveAwayFrom": {
    "concepts": ["finance"],
    "force": 0.45
  },
  "moveTo": {
    "concepts": ["haute couture"],
    "force": 0.85
  }
}

result = (
  client.query
  .get("Publication", "name")
  .with_additional(["certainty OR distance"]) # note that certainty is only supported if distance==cosine
  .with_near_text(nearText)
  .do()
)

print(result)

JavaScript/TypeScript

import weaviate from 'weaviate-ts-client';

const client = weaviate.client({
  scheme: 'http',
  host: 'localhost:8080',
  headers: { 'X-OpenAI-Api-Key': process.env['OPENAI_API_KEY'] },  // Replace with your API key
});

const response = await client.graphql
  .get()
  .withClassName('Publication')
  .withFields('name _additional { certainty distance }') // note that certainty is only supported if distance==cosine
  .withNearText({
    concepts: ['fashion'],
    distance: 0.6, // prior to v1.14 use certainty instead of distance
    moveAwayFrom: {
      concepts: ['finance'],
      force: 0.45,
    },
    moveTo: {
      concepts: ['haute couture'],
      force: 0.85,
    },
  })
  .do();
console.log(response);

Go

package main

import (
  "context"
  "fmt"

  "github.com/weaviate/weaviate-go-client/v4/weaviate"
  "github.com/weaviate/weaviate-go-client/v4/weaviate/graphql"
)

func main() {
  cfg := weaviate.Config{
    Host:    "localhost:8080",
    Scheme:  "http",
    Headers: map[string]string{"X-OpenAI-Api-Key": "YOUR-OPENAI-API-KEY"},
  }
  client, err := weaviate.NewClient(cfg)
  if err != nil {
    panic(err)
  }

  className := "Publication"

  name := graphql.Field{Name: "name"}
  _additional := graphql.Field{
    Name: "_additional", Fields: []graphql.Field{
      {Name: "certainty"}, // only supported if distance==cosine
      {Name: "distance"},  // always supported
    },
  }

  concepts := []string{"fashion"}
  distance := float32(0.6)
  moveAwayFrom := &graphql.MoveParameters{
    Concepts: []string{"finance"},
    Force:    0.45,
  }
  moveTo := &graphql.MoveParameters{
    Concepts: []string{"haute couture"},
    Force:    0.85,
  }
  nearText := client.GraphQL().NearTextArgBuilder().
    WithConcepts(concepts).
    WithDistance(distance). // use WithCertainty(certainty) prior to v1.14
    WithMoveTo(moveTo).
    WithMoveAwayFrom(moveAwayFrom)

  ctx := context.Background()

  result, err := client.GraphQL().Get().
    WithClassName(className).
    WithFields(name, _additional).
    WithNearText(nearText).
    Do(ctx)

  if err != nil {
    panic(err)
  }
  fmt.Printf("%v", result)
}

Java

package io.weaviate;

import io.weaviate.client.Config;
import io.weaviate.client.WeaviateClient;
import io.weaviate.client.base.Result;
import io.weaviate.client.v1.graphql.model.GraphQLResponse;
import io.weaviate.client.v1.graphql.query.argument.NearTextArgument;
import io.weaviate.client.v1.graphql.query.argument.NearTextMoveParameters;
import io.weaviate.client.v1.graphql.query.fields.Field;

import java.util.HashMap;
import java.util.Map;

public class App {
  public static void main(String[] args) {
    Map<String, String> headers = new HashMap<String, String>() { {
      put("X-OpenAI-Api-Key", "YOUR-OPENAI-API-KEY");
    } };
    Config config = new Config("http", "localhost:8080", headers);
    WeaviateClient client = new WeaviateClient(config);

    NearTextMoveParameters moveTo = NearTextMoveParameters.builder()
      .concepts(new String[]{ "haute couture" }).force(0.85f).build();

    NearTextMoveParameters moveAway = NearTextMoveParameters.builder()
      .concepts(new String[]{ "finance" }).force(0.45f)
      .build();

    NearTextArgument nearText = client.graphQL().arguments().nearTextArgBuilder()
      .concepts(new String[]{ "fashion" })
      .distance(0.6f) // use .certainty(0.7f) prior to v1.14
      .moveTo(moveTo)
      .moveAwayFrom(moveAway)
      .build();

    Field name = Field.builder().name("name").build();
    Field _additional = Field.builder()
      .name("_additional")
      .fields(new Field[]{
        Field.builder().name("certainty").build(), // only supported if distance==cosine
        Field.builder().name("distance").build(),  // always supported
      }).build();

    Result<GraphQLResponse> result = client.graphQL().get()
      .withClassName("Publication")
      .withFields(name, _additional)
      .withNearText(nearText)
      .run();

    if (result.hasErrors()) {
      System.out.println(result.getError());
      return;
    }
    System.out.println(result.getResult());
  }
}

Curl

# Note: Under nearText, use `certainty` instead of distance prior to v1.14
# Under _additional, `certainty` is only supported if distance==cosine, but `distance` is always supported

echo '{
  "query": "{
    Get{
      Publication(
        nearText: {
          concepts: [\"fashion\"],
          distance: 0.6,
          moveAwayFrom: {
            concepts: [\"finance\"],
            force: 0.45
          },
          moveTo: {
            concepts: [\"haute couture\"],
            force: 0.85
          }
        }
      ){
        name
        _additional {
          certainty
          distance
        }
      }
    }
  }"
}' | curl \
    -X POST \
    -H 'Content-Type: application/json' \
    -H 'Authorization: Bearer learn-weaviate' \
    -H "X-OpenAI-Api-Key: $OPENAI_API_KEY" \
    -d @- \
    https://edu-demo.weaviate.network/v1/graphql

GraphQL

{
  Get{
    Publication(
      nearText: {
        concepts: ["fashion"],
        distance: 0.6 # prior to v1.14 use "certainty" instead of "distance"
        moveAwayFrom: {
          concepts: ["finance"],
          force: 0.45
        },
        moveTo: {
          concepts: ["haute couture"],
          force: 0.85
        }
      }
    ){
      name
      _additional {
        certainty # only supported if distance==cosine.
        distance  # always supported
      }
    }
  }
}