Skip to main content

text2vec-openai

LICENSEย Weaviate on Stackoverflow badgeย Weaviate issues on GitHub badgeย Weaviate version badgeย Weaviate total Docker pulls badgeย Go Report Card

In shortโ€‹

  • This module uses a third-party API and may incur costs.
  • Check the vendor pricing (e.g. OpenAI pricing page) before vectorizing large amounts of data.
  • Weaviate automatically parallelizes requests to the API when using the batch endpoint.
  • Check out the text2vec-openai demo.
  • You will need an API key from OpenAI or Azure OpenAI to use this module.
  • The default OpenAI model is text-embedding-ada-002.

Overviewโ€‹

The text2vec-openai module enables you to use OpenAI in Weaviate or Azure embeddings to represent data objects.

Azure OpenAI or OpenAI?โ€‹

tip

This module is compatible with both OpenAI and Azure OpenAI.

The instructions vary slightly based on whether you are using OpenAI directly or Azure OpenAI. Please make sure that you are following the right instructions for your service provider.

The differences are in:

  • Parameter names used in the schema, and
  • Names of the API key to be used.

Module configurationโ€‹

Not applicable to WCS

This module is enabled and pre-configured on Weaviate Cloud Services.

Configuration file (Weaviate open source only)โ€‹

You can enable the text2vec-openai module in your configuration file (e.g. docker-compose.yaml).

  • This configuration will start Weaviate with the OpenAI module enabled, and set as the default vectorizer module.
  • Optionally, you can specify the required API key in the file..
    • If you do not, you must specify the API key at runtime.
---
version: '3.4'
services:
weaviate:
image: semitechnologies/weaviate:1.19.6
restart: on-failure:0
ports:
- "8080:8080"
environment:
QUERY_DEFAULTS_LIMIT: 20
AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
PERSISTENCE_DATA_PATH: "./data"
DEFAULT_VECTORIZER_MODULE: text2vec-openai
ENABLE_MODULES: text2vec-openai
OPENAI_APIKEY: sk-foobar # For use with OpenAI. Setting this parameter is optional; you can also provide the key at runtime.
AZURE_APIKEY: sk-foobar # For use with Azure OpenAI. Setting this parameter is optional; you can also provide the key at runtime.
CLUSTER_HOSTNAME: 'node1'
...
note

Schema configurationโ€‹

You can provide additional module configurations through the schema. You can learn about schemas here.

For text2vec-openai, you can set the vectorizer model and vectorizer behavior.

OpenAI settingsโ€‹

Set the vectorizer model using parameters model, modelVersion and type in the moduleConfig section of your schema:

{
"classes": [
{
"class": "Document",
"description": "A class called document",
"vectorizer": "text2vec-openai",
"moduleConfig": {
"text2vec-openai": {
"model": "ada",
"modelVersion": "002",
"type": "text"
}
},
}
]
}

Azure OpenAI settingsโ€‹

Set the parameters resourceName and deploymentId in the moduleConfig section of your schema:

{
"classes": [
{
"class": "Document",
"description": "A class called document",
"vectorizer": "text2vec-openai",
"moduleConfig": {
"text2vec-openai": {
"resourceName": "<YOUR-RESOURCE-NAME>",
"deploymentId": "<YOUR-MODEL-NAME>",
}
}
}
]
}

Vectorizer behaviorโ€‹

Set property-level vectorizer behavior using the moduleConfig section under each property:

{
"classes": [
{
"class": "Document",
"description": "A class called document",
"vectorizer": "text2vec-openai",
"moduleConfig": {
"text2vec-openai": {
"model": "ada",
"modelVersion": "002",
"type": "text"
}
},
"properties": [
{
"dataType": ["text"],
"description": "Content that will be vectorized",
"moduleConfig": {
"text2vec-openai": {
"skip": false,
"vectorizePropertyName": false
}
},
"name": "content"
}
]
}
]
}

Usageโ€‹

Enabling this module will make GraphQL vector search operators available.

Provide the API keyโ€‹

If the API key is not set in the text2vec-openai configuration, you can supply it when making a query.

You can achieve this by adding the appropriate key to the HTTP header:

  • X-OpenAI-Api-Key: YOUR-OPENAI-API-KEY for OpenAI, and
  • X-Azure-Api-Key: YOUR-AZURE-API-KEY for Azure OpenAI, and

Exampleโ€‹

{
Get{
Publication(
nearText: {
concepts: ["fashion"],
distance: 0.6 # prior to v1.14 use "certainty" instead of "distance"
moveAwayFrom: {
concepts: ["finance"],
force: 0.45
},
moveTo: {
concepts: ["haute couture"],
force: 0.85
}
}
){
name
_additional {
certainty # only supported if distance==cosine.
distance # always supported
}
}
}
}

Additional informationโ€‹

Available models (OpenAI)โ€‹

OpenAI has multiple models available with different trade-offs. All the models offered by OpenAI can be used within Weaviate. Note that the more dimensions a model produces, the larger your data footprint will be. To estimate the total size of your dataset use this calculation.

The default model is text-embedding-ada-002 but you can also specify it in your schema. An example as part of a class definition:

{
"classes": [
{
"class": "Document",
"vectorizer": "text2vec-openai",
"moduleConfig": {
"text2vec-openai": {
"model": "ada",
"modelVersion": "002",
"type": "text"
}
}
}
]
}

For document embeddings, choose from the following models:

For code embeddings, see the Codex models.

In the moduleConfig inside a class, you need to set two values:

  1. model โ€“ one of the models mentioned above, e.g. davinci.
  2. modelVersion โ€“ version string, e.g. 003.
  3. type โ€“ text or code.

Example (as part of a class definition):

{
"classes": [
{
"class": "Document",
"vectorizer": "text2vec-openai",
"moduleConfig": {
"text2vec-openai": {
"model": "ada",
"modelVersion": "002",
"type": "text"
}
}
}
]
}

OpenAI rate limitsโ€‹

Since you will obtain embeddings using your own API key, any corresponding rate limits related to your account will apply to your use with Weaviate also.

If you exceed your rate limit, Weaviate will output the error message generated by the OpenAI API. You can request to increase your rate limit by emailing OpenAI at support@openai.com describing your use case with Weaviate.

Throttle the import inside your applicationโ€‹

One way of dealing with rate limits is to throttle the import within your application. For example, when using the Weaviate client in Python or Java:

from weaviate import Client
import time

def configure_batch(client: Client, batch_size: int, batch_target_rate: int):
"""
Configure the weaviate client's batch so it creates objects at `batch_target_rate`.

Parameters
----------
client : Client
The Weaviate client instance.
batch_size : int
The batch size.
batch_target_rate : int
The batch target rate as # of objects per second.
"""

def callback(batch_results: dict) -> None:

# you could print batch errors here
time_took_to_create_batch = batch_size * (client.batch.creation_time/client.batch.recommended_num_objects)
time.sleep(
max(batch_size/batch_target_rate - time_took_to_create_batch + 1, 0)
)

client.batch.configure(
batch_size=batch_size,
timeout_retries=5,
callback=callback,
)

The current rate limit will appear in the error message, as shown below:

{
"message": "Rate limit reached for requests. Limit: 600.000000 / min. Current: 1024.000000 / min. Contact support@openai.com if you continue to have issues."
}

More resourcesโ€‹

If you can't find the answer to your question here, please look at the:

  1. Frequently Asked Questions. Or,
  2. Knowledge base of old issues. Or,
  3. For questions: Stackoverflow. Or,
  4. For more involved discussion: Weaviate Community Forum. Or,
  5. We also have a Slack channel.