text2vec-openai

Weaviate on Stackoverflow badge Weaviate issues on Github badge Weaviate total Docker pulls badge

💡 You are looking at older or release candidate documentation. The current Weaviate version is v1.15.1

💡 This is the most recent Weaviate module that leverages an external API. We would love to know how you use it, especially on large-scale implementations. Connect with us on Slack here.


Introduction

The text2vec-​openai module allows you to use the OpenAI embeddings directly in the Weaviate vector search engine as a vectorization module. ​When you create a Weaviate class that is set to use this module, it will automatically vectorize your data using OpenAI’s Ada, Babbage, Curie, or Davinci models.

💡 Check-out the demo dataset

Note I: make sure to check the OpenAI pricing page before vectorizing large amounts of data

Note II: Weaviate automatically parallelizes requests to the OpenAI-API when using the batch endpoint, see the previous note

How to use

​Using the OpenAI module within Weaviate requires you to create an OpenAI API key. You can create a key here.

To test if your key has access to the embedding API, you can run the following command in your terminal.

$ curl https://api.openai.com/v1/engines/text-search-ada-query-001/embeddings \
  -X POST \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"input": "​Is Weaviate the coolest vector search engine?"}'

OpenAI Rate Limits

Because you will be getting embeddings based on your own API key, you will be dealing with rate limits applied to your account. If you have a low rate limit set, Weaviate will output the error message generated by the OpenAI API.

Request a higher rate or removal of your limit

You can request to increase your rate limit by emailing OpenAI directly on support@openai.com describing your use case with Weaviate.

Throttle the import inside your application

If you run into rate limits, you can also decide to throttle the import in your application.

E.g., in Python and Java using the Weaviate client.

  from weaviate import Client
import time


def configure_batch(client: Client, batch_size: int, batch_tagret_rate: int):
    """
    Configure the weaviate client's batch so it creates objects at `batch_tagret_rate`.

    Parameters
    ----------
    client : Client
        The Weaviate client instance.
    batch_size : int
        The batch size.
    batch_tagret_rate : int
        The batch target rate as # objects per second.
    """

    def callback(batch_results: dict) -> None:

        # you could print batch errors here
        time_took_to_create_batch = batch_size * (client.batch.creation_time/client.batch.recommended_num_objects)
        time.sleep(
            max(batch_size/batch_tagret_rate - time_took_to_create_batch + 1, 0)
        )

    client.batch.configure(
        batch_size=batch_size,
        timeout_retries=5,
        callback=callback,
    )

  package main

import (
	"context"
	"time"

	"github.com/semi-technologies/weaviate-go-client/v4/weaviate"
	"github.com/semi-technologies/weaviate/entities/models"
)

var (
	// adjust to your liking
	targetRatePerMin = 600
	batchSize        = 50
)

var cfg = weaviate.Config{
	Host:   "localhost:8080",
	Scheme: "http",
}
var client = weaviate.New(cfg)

// replace those 10000 empty objects with your actual data
var objects = make([]*models.Object, 10000)

func main() {
	// we aim to send one batch every tickInterval second.
	tickInterval := time.Duration(batchSize/targetRatePerMinute) * time.Minute
	t := time.NewTicker(tickInterval)
	before := time.Now()

	for i := 0; i < len(objects); i += batchSize {

		// create a fresh batch
		batch := client.Batch().ObjectsBatcher()

		// add batchSize objects to the batch
		for j := i; j < i+batchSize; j++ {
			batch = batch.WithObject(objects[i+j])
		}

		// send off batch
		res, err := batch.Do(context.Background())
		// TODO: inspect result for individual errors
		_ = res
		// TODO: check request error
		_ = err

		// we wait for the next tick. If the previous batch took longer than
		// tickInterval, we won't need to wait, effectively making this an
		// unthrottled import.
		<-t.C
	}
}

The current rate limit will be displayed in the error message like:

{
  "message": "Rate limit reached for requests. Limit: 600.000000 / min. Current: 1024.000000 / min. Contact support@openai.com if you continue to have issues."
}

Example Docker-compose file

You can find an example Docker-compose file below, which will spin up Weaviate with the OpenAI module.

version: '3.4'
services:
  weaviate:
    image: semitechnologies/weaviate:v1.15.1
    restart: on-failure:0
    ports:
     - "8080:8080"
    environment:
      QUERY_DEFAULTS_LIMIT: 20
      AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
      PERSISTENCE_DATA_PATH: "./data"
      DEFAULT_VECTORIZER_MODULE: text2vec-openai
      ENABLE_MODULES: text2vec-openai
      OPENAI_APIKEY: sk-foobar # request a key on openai.com

Note: you can also use the Weaviate configuration tool to create a Weaviate setup with this module.

Model selection on class level

If you want to use the OpenAI embeddings for your data objects, you need to set the text2vec-openai vectorizer on a class level. Working with Weaviate classes in a schema is described in detail here. Weaviate automatically determines which endpoints need to be used for vectorizing and querying data.

Available models

OpenAI has multiple models available with different trade-offs. All the models offered by OpenAI can be used within Weaviate. Note that the more dimensions a model produces, the larger your data footprint will be. To estimate the total size of your dataset use this calculation.

  • For document embeddings you can choose one of the following models:
  • For code embeddings you can choose one of the following models:

In the moduleConfig inside a class, you need to set two values:

  1. model – one of the models mentioned above. E.g., babbage.
  2. typetext or code.

Example class configuration

The following schema configuration uses the babbage model.

{
  "classes": [
    {
      "class": "Document",
      "description": "A class called document",
      "moduleConfig": {
        "text2vec-openai": {
          "model": "babbage",
          "type": "text"
        }
      },
      "properties": [
        {
          "dataType": [
            "text"
          ],
          "description": "Content that will be vectorized",
          "moduleConfig": {
            "text2vec-openai": {
              "skip": false,
              "vectorizePropertyName": false
            }
          },
          "name": "content"
        }
      ]
    }
  ]
}

Additional GraphQL API parameters

nearText

The text2vec-openai vectorizer module adds one search operator for Get {} and Explore {} GraphQL functions: nearText: {}. This operator can be used for semantically searching text in your dataset.

Note: You cannot use multiple 'near' filters, or a 'near' operators along with an 'ask' filter!

Example GraphQL Get(nearText{}) operator

  {
  Get{
    Publication(
      nearText: {
        concepts: ["fashion"],
        distance: 0.6 # prior to v1.14 use "certainty" instead of "distance"
        moveAwayFrom: {
          concepts: ["finance"],
          force: 0.45
        },
        moveTo: {
          concepts: ["haute couture"],
          force: 0.85
        }
      }
    ){
      name
      _additional {
        certainty # only supported if distance==cosine.
        distance  # always supported
      }
    }
  }
}
  import weaviate

client = weaviate.Client("http://localhost:8080")

nearText = {
  "concepts": ["fashion"],
  "distance": 0.6, # prior to v1.14 use "certainty" instead of "distance"
  "moveAwayFrom": {
    "concepts": ["finance"],
    "force": 0.45
  },
  "moveTo": {
    "concepts": ["haute couture"],
    "force": 0.85
  }
}

result = (
  client.query
  .get("Publication", ["name", "_additional {certainty distance} "]) # note that certainty is only supported if distance==cosine
  .with_near_text(nearText)
  .do()
)

print(result)
  const weaviate = require("weaviate-client");

const client = weaviate.client({
  scheme: 'http',
  host: 'localhost:8080',
});

client.graphql
  .get()
  .withClassName('Publication')
  .withFields('name _additional{certainty distance}') // note that certainty is only supported if distance==cosine
  .withNearText({
    concepts: ["fashion"],
    distance: 0.6, // prior to v1.14 use certainty instead of distance
    moveAwayFrom: {
      concepts: ["finance"],
      force: 0.45
    },
    moveTo: {
      concepts: ["haute couture"],
      force: 0.85
    }
  })
  .do()
  .then(console.log)
  .catch(console.error);
  package main

import (
  "context"
  "fmt"

  "github.com/semi-technologies/weaviate-go-client/v4/weaviate"
  "github.com/semi-technologies/weaviate-go-client/v4/weaviate/graphql"
)

func main() {
  cfg := weaviate.Config{
    Host:   "localhost:8080",
    Scheme: "http",
  }
  client := weaviate.New(cfg)

  className := "Publication"

  name := graphql.Field{Name: "name"}
  _additional := graphql.Field{
    Name: "_additional", Fields: []graphql.Field{
      {Name: "certainty"}, // only supported if distance==cosine
      {Name: "distance"},  // always supported
    },
  }

  concepts := []string{"fashion"}
  distance := float32(0.6)
  moveAwayFrom := &graphql.MoveParameters{
    Concepts: []string{"finance"},
    Force:    0.45,
  }
  moveTo := &graphql.MoveParameters{
    Concepts: []string{"haute couture"},
    Force:    0.85,
  }
  nearText := client.GraphQL().NearTextArgBuilder().
    WithConcepts(concepts).
    WithDistance(distance). // use WithCertainty(certainty) prior to v1.14
    WithMoveTo(moveTo).
    WithMoveAwayFrom(moveAwayFrom)

  ctx := context.Background()

  result, err := client.GraphQL().Get().
    WithClassName(className).
    WithFields(name, _additional).
    WithNearText(nearText).
    Do(ctx)

  if err != nil {
    panic(err)
  }
  fmt.Printf("%v", result)
}
  package technology.semi.weaviate;

import technology.semi.weaviate.client.Config;
import technology.semi.weaviate.client.WeaviateClient;
import technology.semi.weaviate.client.base.Result;
import technology.semi.weaviate.client.v1.graphql.model.GraphQLResponse;
import technology.semi.weaviate.client.v1.graphql.query.argument.NearTextArgument;
import technology.semi.weaviate.client.v1.graphql.query.argument.NearTextMoveParameters;
import technology.semi.weaviate.client.v1.graphql.query.fields.Field;

public class App {
  public static void main(String[] args) {
    Config config = new Config("http", "localhost:8080");
    WeaviateClient client = new WeaviateClient(config);

    NearTextMoveParameters moveTo = NearTextMoveParameters.builder()
      .concepts(new String[]{ "haute couture" }).force(0.85f).build();

    NearTextMoveParameters moveAway = NearTextMoveParameters.builder()
      .concepts(new String[]{ "finance" }).force(0.45f)
      .build();

    NearTextArgument nearText = client.graphQL().arguments().nearTextArgBuilder()
      .concepts(new String[]{ "fashion" })
      .distance(0.6f) // use .certainty(0.7f) prior to v1.14
      .moveTo(moveTo)
      .moveAwayFrom(moveAway)
      .build();

    Field name = Field.builder().name("name").build();
    Field _additional = Field.builder()
      .name("_additional")
      .fields(new Field[]{
        Field.builder().name("certainty").build(), // only supported if distance==cosine
        Field.builder().name("distance").build(),  // always supported
      }).build();

    Result<GraphQLResponse> result = client.graphQL().get()
      .withClassName("Publication")
      .withFields(name, _additional)
      .withNearText(nearText)
      .run();

    if (result.hasErrors()) {
      System.out.println(result.getError());
      return;
    }
    System.out.println(result.getResult());
  }
}
  $ echo '{
  "query": "{
    Get{
      Publication(
        nearText: {
          concepts: [\"fashion\"],
          distance: 0.6, // use certainty instead of distance prior to v1.14
          moveAwayFrom: {
            concepts: [\"finance\"],
            force: 0.45
          },
          moveTo: {
            concepts: [\"haute couture\"],
            force: 0.85
          }
        }
      ){
        name
        _additional {
          certainty // only supported if distance==cosine
          distance  // always supported
        }
      }
    }
  }"
}' | curl \
    -X POST \
    -H 'Content-Type: application/json' \
    -d @- \
    http://localhost:8080/v1/graphql

🟢 Click here to try out this graphql example in the Weaviate Console.

Example GraphQL Explore(nearText{}) operator

  {
  Explore (
    nearText: {
      concepts: ["New Yorker"],
      distance: 0.1,  # use certainty instead of distance prior to v1.14
      moveAwayFrom: {
        concepts: ["fashion", "shop"],
        force: 0.2
      }
      moveTo: {
        concepts: ["publisher", "articles"],
        force: 0.5
      },
    }
  ) {
    beacon
    certainty # only supported if distance==cosine. 
    distance  # always supported
    className
  }
}
  import weaviate

client = weaviate.Client("http://localhost:8080")

explore_articles_query = """
  {
    Explore (
      nearText: {
        concepts: ["New Yorker"],
        distance: 0.1, // use certainty instead of distance prior to v1.14
        moveAwayFrom: {
          concepts: ["fashion", "shop"],
          force: 0.2
        }
        moveTo: {
          concepts: ["publisher", "articles"],
          force: 0.5
        },
      }
    ) {
      beacon
      certainty # only supported if distance==cosine. 
      distance  # always supported
      className
    }
  }
"""

query_result = client.query.raw(explore_articles_query)
print(query_result)
  const weaviate = require("weaviate-client");

const client = weaviate.client({
  scheme: 'http',
  host: 'localhost:8080',
});

client.graphql
      .explore()
      .withNearText({concepts:['New Yorkers'])}, moveTo: {concepts: ['publisher', 'articles'], force: 0.5}, moveAwayFrom: {concepts: ['fashion', 'shop'], force: 0.2})})
      .withDistance(0.95) // prior to v1.14, use withCertainty()
      .withFields('beacon certainty distance className') // note that certainty is only supported if distance==cosine
      .do()
      .then(res => {
        console.log(res)
      })
      .catch(err => {
        console.error(err)
      });
  package main

import (
  "context"
  "fmt"

  "github.com/semi-technologies/weaviate-go-client/v4/weaviate"
  "github.com/semi-technologies/weaviate-go-client/v4/weaviate/graphql"
)

func main() {
  cfg := weaviate.Config{
    Host:   "localhost:8080",
    Scheme: "http",
  }
  client := weaviate.New(cfg)

  concepts := []string{"New Yorker"}
  distance := float32(0.95)
  moveTo := &graphql.MoveParameters{
    Concepts: []string{"publisher", "articles"},
    Force:    0.5,
  }
  moveAwayFrom := &graphql.MoveParameters{
    Concepts: []string{"fashion", "shop"},
    Force:    0.2,
  }
  withNearText := client.GraphQL().NearTextArgBuilder().
    WithConcepts(concepts).
    WithDistance(0.6). // prior to v1.14, use WithCertainty()
    WithMoveTo(moveTo).
    WithMoveAwayFrom(moveAwayFrom)

  result, err := client.GraphQL().Explore().
    WithFields(
      graphql.Beacon, 
      graphql.Certainty, // only supported if distance==cosine
      graphql.Distance,  // always supported
      graphql.ClassName).
    WithNearText(withNearText).
    Do(context.Background())

  if err != nil {
    panic(err)
  }
  fmt.Printf("%v", result)
}
  package technology.semi.weaviate;

import technology.semi.weaviate.client.Config;
import technology.semi.weaviate.client.WeaviateClient;
import technology.semi.weaviate.client.base.Result;
import technology.semi.weaviate.client.v1.graphql.model.ExploreFields;
import technology.semi.weaviate.client.v1.graphql.model.GraphQLResponse;
import technology.semi.weaviate.client.v1.graphql.query.argument.NearTextArgument;
import technology.semi.weaviate.client.v1.graphql.query.argument.NearTextMoveParameters;

public class App {
  public static void main(String[] args) {
    Config config = new Config("http", "localhost:8080");
    WeaviateClient client = new WeaviateClient(config);

    NearTextMoveParameters moveTo = NearTextMoveParameters.builder()
      .concepts(new String[]{ "publisher", "articles" }).force(0.5f).build();

    NearTextMoveParameters moveAway = NearTextMoveParameters.builder()
      .concepts(new String[]{ "fashion", "shop" }).force(0.2f)
      .build();
    
    NearTextArgument nearText = client.graphQL().arguments().nearTextArgBuilder()
      .concepts(new String[]{ "New Yorker" })
      .moveTo(moveTo)
      .moveAwayFrom(moveAway)
      .distance(0.95f)  // prior to v1.14 use .certainty()
      .build();

    Result<GraphQLResponse> result = client.graphQL().explore()
      .withFields(ExploreFields.BEACON, 
        ExploreFields.CERTAINTY, // only supported if distance==cosine
        ExploreFields.DISTANCE,  // always supported
        ExploreFields.CLASS_NAME)
      .withNearText(nearText)
      .run();

    if (result.hasErrors()) {
      System.out.println(result.getError());
      return;
    }
    System.out.println(result.getResult());
  }
}
  $ echo '{ 
  "query": "{
    Explore (
      nearText: {
        concepts: [\"New Yorker\"],
        distance: 0.95, # prior to v1.14 use "certainty" instead of "distance"
        moveAwayFrom: {
          concepts: [\"fashion\", \"shop\"],
          force: 0.2
        }
        moveTo: {
          concepts: [\"publisher\", \"articles\"],
          force: 0.5
        },
      }
    ) {
      beacon
      certainty # only supported if distance==cosine. 
      distance  # always supported
      className
    }
  }"
}' | curl \
    -X POST \
    -H 'Content-Type: application/json' \
    -d @- \
    http://localhost:8080/v1/graphql

🟢 Click here to try out this graphql example in the Weaviate Console.

Certainty

You can set a minimum required certainty, which will be used to determine which data results to return. The value is a float between 0.0 (return all data objects, regardless of similarity) and 1.0 (only return data objects that match completely, without any uncertainty). The certainty of a query result is computed by normalized distance of the fuzzy query and the data object in the vector space.

Moving

Because pagination is not possible in multidimensional storage, you can improve your results with additional explore functions which can move away from semantic concepts or towards semantic concepts. E.g., if you look for the concept ‘New York Times’ but don’t want to find the city New York, you can use the moveAwayFrom{} function by using the words ‘New York’. This is also a way to exclude concepts and to deal with negations (not operators in similar query languages). Concepts in the moveAwayFrom{} filter are not per definition excluded from the result, but the resulting concepts are further away from the concepts in this filter.

Moving can be done based on concepts and/or objects.

  • concepts requires a list of one or more words
  • objects requires a list of one or more objects, given by their id or beacon. For example:
{
  Get{
    Publication(
      nearText: {
        concepts: ["fashion"],
        certainty: 0.7,
        moveTo: {
            objects: [{
                beacon: "weaviate://localhost/e5dc4a4c-ef0f-3aed-89a3-a73435c6bbcf"
            }, {
                id: "9f0c7463-8633-30ff-99e9-fd84349018f5" 
            }],
            concepts: ["summer"],
            force: 0.9
        }
      }
    ){
      name
      _additional {
        certainty
        id
      }
    }
  }
}

More resources

If you can’t find the answer to your question here, please look at the:

  1. Frequently Asked Questions. Or,
  2. Knowledge base of old issues. Or,
  3. For questions: Stackoverflow. Or,
  4. For issues: Github. Or,
  5. Ask your question in the Slack channel: Slack.