Skip to main content

text2vec-openai

LICENSEΒ Weaviate on Stackoverflow badgeΒ Weaviate issues on Github badgeΒ Weaviate version badgeΒ Weaviate total Docker pulls badgeΒ Go Report Card

Introduction​

The text2vec-​openai module allows you to use the OpenAI embeddings directly in the Weaviate vector search engine as a vectorization module. ​When you create a Weaviate class that is set to use this module, it will automatically vectorize your data using OpenAI's text-embedding-ada-002 model (legacy Ada, Babbage, Curie, or Davinci models are also supported).

  • Note: this module uses a third-party API and may incur costs.
  • Note: make sure to check the OpenAI pricing page before vectorizing large amounts of data.
  • Note: Weaviate automatically parallelizes requests to the OpenAI-API when using the batch endpoint, see the previous note.
  • Note: Check-out the demo dataset.

How to enable​

Request an OpenAI API-key via their website.

Weaviate Cloud Service​

This module is enabled by default on the WCS

Weaviate open source​

You can find an example Docker-compose file below, which will spin up Weaviate with the OpenAI module.

---
version: '3.4'
services:
weaviate:
image: semitechnologies/weaviate:1.17.2
restart: on-failure:0
ports:
- "8080:8080"
environment:
QUERY_DEFAULTS_LIMIT: 20
AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
PERSISTENCE_DATA_PATH: "./data"
DEFAULT_VECTORIZER_MODULE: text2vec-openai
ENABLE_MODULES: text2vec-openai
OPENAI_APIKEY: sk-foobar # request a key on openai.com, setting this parameter is optional, you can also provide the API key on runtime
CLUSTER_HOSTNAME: 'node1'
...
  • Note: you can also use the Weaviate configuration tool to create a Weaviate setup with this module.
  • Note: Starting with v1.11.0 the OPENAI_APIKEY variable is now optional and you can instead provide the key at insert/query time as an HTTP header.

How to configure​

​In your Weaviate schema, you must define how you want this module to vectorize your data. If you are new to Weaviate schemas, you might want to check out the quickstart tutorial on the Weaviate schema first.

For example, the following schema configuration will set Weaviate to vectorize the Document class with text2vec-openai using the babbage model.

{
"classes": [
{
"class": "Document",
"description": "A class called document",
"vectorizer": "text2vec-openai",
"moduleConfig": {
"text2vec-openai": {
"model": "ada",
"modelVersion": "002",
"type": "text"
}
},
"properties": [
{
"dataType": [
"text"
],
"description": "Content that will be vectorized",
"moduleConfig": {
"text2vec-openai": {
"skip": false,
"vectorizePropertyName": false
}
},
"name": "content"
}
],
"vectorizer": "text2vec-openai"
}
]
}

How to use​

  • If the OpenAI API key is not set in the text2vec-​openai module, you can set the API key on query time by adding the following to the HTTP header: X-OpenAI-Api-Key: <openai-api-key>.
  • Using this module will enable GraphQL vector search parameters in Weaviate. They can be found here.

Example​

{
Get{
Publication(
nearText: {
concepts: ["fashion"],
distance: 0.6 # prior to v1.14 use "certainty" instead of "distance"
moveAwayFrom: {
concepts: ["finance"],
force: 0.45
},
moveTo: {
concepts: ["haute couture"],
force: 0.85
}
}
){
name
_additional {
certainty # only supported if distance==cosine.
distance # always supported
}
}
}
}

🟒 Try out this GraphQL example in the Weaviate Console.

Additional information​

Available models​

OpenAI has multiple models available with different trade-offs. All the models offered by OpenAI can be used within Weaviate. Note that the more dimensions a model produces, the larger your data footprint will be. To estimate the total size of your dataset use this calculation.

The default model is: text-embedding-ada-002 but you can also specify it in your schema. An example as part of a class definition:

{
"classes": [
{
"class": "Document",
"vectorizer": "text2vec-openai",
"moduleConfig": {
"text2vec-openai": {
"model": "ada",
"modelVersion": "002",
"type": "text"
}
}
}
]
}

Legacy models​

The default model is: text-embedding-ada-002 but you can also specify it in your schema. An example as part of a class definition:

{
"classes": [
{
"class": "Document",
"vectorizer": "text2vec-openai",
"moduleConfig": {
"text2vec-openai": {
"model": "ada",
"modelVersion": "002",
"type": "text"
}
}
}
]
}

Legacy models​

  • For document embeddings you can choose one of the following models:
  • For code embeddings you can choose one of the following models:

In the moduleConfig inside a class, you need to set two values:

  1. model – one of the models mentioned above. E.g., babbage.
  2. modelVersion – one of the model version as mentioned above. E.g., babbage.
  3. type – text or code.

Example (as part of a class definition):

{
"classes": [
{
"class": "Document",
"vectorizer": "text2vec-openai",
"moduleConfig": {
"text2vec-openai": {
"model": "babbage",
"modelVersion": "001",
"type": "text"
}
}
}
]
}

OpenAI Rate Limits​

Because you will be getting embeddings based on your own API key, you will be dealing with rate limits applied to your account. If you have a low rate limit set, Weaviate will output the error message generated by the OpenAI API. You can request to increase your rate limit by emailing OpenAI directly on support@openai.com describing your use case with Weaviate.

Throttle the import inside your application​

If you run into rate limits, you can also decide to throttle the import in your application.

E.g., in Python and Java using the Weaviate client.

from weaviate import Client
import time

def configure_batch(client: Client, batch_size: int, batch_target_rate: int):
"""
Configure the weaviate client's batch so it creates objects at `batch_target_rate`.

Parameters
----------
client : Client
The Weaviate client instance.
batch_size : int
The batch size.
batch_target_rate : int
The batch target rate as # of objects per second.
"""

def callback(batch_results: dict) -> None:

# you could print batch errors here
time_took_to_create_batch = batch_size * (client.batch.creation_time/client.batch.recommended_num_objects)
time.sleep(
max(batch_size/batch_target_rate - time_took_to_create_batch + 1, 0)
)

client.batch.configure(
batch_size=batch_size,
timeout_retries=5,
callback=callback,
)

The current rate limit will be displayed in the error message like:

{
"message": "Rate limit reached for requests. Limit: 600.000000 / min. Current: 1024.000000 / min. Contact support@openai.com if you continue to have issues."
}

More resources​

If you can't find the answer to your question here, please look at the:

  1. Frequently Asked Questions. Or,
  2. Knowledge base of old issues. Or,
  3. For questions: Stackoverflow. Or,
  4. For issues: Github. Or,
  5. Ask your question in the Slack channel: Slack.