Skip to main content

text2vec-openai

Overview

The text2vec-openai module enables Weaviate to obtain vectors using OpenAI or Azure OpenAI.

Key notes:

  • As it uses a third-party API, you will need an API key.
  • Its usage may incur costs.
    • Please check the vendor pricing (e.g. OpenAI pricing page), especially before vectorizing large amounts of data.
  • This module is available on Weaviate Cloud Services (WCS).
  • Enabling this module will enable the nearText search operator.
  • The default model is text-embedding-ada-002.
Azure OpenAI or OpenAI?

The module usage instructions may vary based on whether you are using OpenAI directly or Azure OpenAI. Please make sure that you are following the right instructions for your service provider.


For example, the following may vary:

  • Parameter names used in the schema, and
  • Names of the API key to be used.

Where to set module parameters

The module accepts parameters through the request header, collection configuration, or environment variables. Some parameters (such as the API key) can be set in multiple ways.

Where the same parameter can be set in multiple ways, setting it at query-time through the HTTP request header (if possible) will have the highest precedence.

We suggest you only set any given parameter in one place to avoid confusion.

Weaviate instance configuration

Not applicable to WCS

This module is enabled and pre-configured on Weaviate Cloud Services.

Docker Compose file

To use text2vec-openai, you must enable it in your Docker Compose file (docker-compose.yml). You can do so manually, or create one using the Weaviate configuration tool.

Parameters

ParameterRequiredPurpose
ENABLE_MODULESRequiredThe modules to enable. Include text2vec-openai to enable the module.
DEFAULT_VECTORIZER_MODULEOptionalThe default vectorizer module. You can set this to text2vec-openai to make it the default for all classes.
OPENAI_APIKEYOptionalYour OpenAI API key (if using OpenAI). You can also provide the key at query time.
AZURE_APIKEYOptionalYour Azure OpenAI API key (if using Azure OpenAI). You can also provide the key at query time.

Example

This configuration enables text2vec-openai, sets it as the default vectorizer, and sets various other parameters such as the API key as environment variables.

---
version: '3.4'
services:
weaviate:
image: cr.weaviate.io/semitechnologies/weaviate:1.24.6
restart: on-failure:0
ports:
- 8080:8080
- 50051:50051
environment:
QUERY_DEFAULTS_LIMIT: 20
AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
PERSISTENCE_DATA_PATH: "./data"
ENABLE_MODULES: text2vec-openai
DEFAULT_VECTORIZER_MODULE: text2vec-openai
OPENAI_APIKEY: sk-foobar # For use with OpenAI. Setting this parameter is optional; you can also provide the key at query time.
OPENAI_ORGANIZATION: your-orgname # For use with OpenAI. Setting this parameter is optional; you can also provide the key at runtime.
AZURE_APIKEY: sk-foobar # For use with Azure OpenAI. Setting this parameter is optional; you can also provide the key at query time.
CLUSTER_HOSTNAME: 'node1'
...

Class configuration

You can configure how the module will behave in each class through the Weaviate schema.

API settings (OpenAI)

Parameters

ParameterData typeRequiredDefaultPurpose
modelstringOptionaladaFor v3 OpenAI embedding models, the model name. For earlier models, model family, e.g. ada.
dimensionsintOptional1536 for text-embedding-3-small
3072 for text-embedding-3-large
Number of dimensions. Applicable to v3 OpenAI models only.
modelVersionstringOptionalVersion string, e.g. 003.
typestringOptionalModel type. Can be text or code.
baseURLstringOptionalhttps://api.openai.comSets a proxy or other URL instead of the default OpenAI URL.

Use a the protocol domain format: https://your.domain.com.

Example

The following example configures the Document class by setting the vectorizer to text2vec-openai, model to ada, the model version to 002 and the type to text:

{
"classes": [
{
"class": "Document",
"description": "A class called document",
"vectorizer": "text2vec-openai",
"moduleConfig": {
"text2vec-openai": {
// "model": "ada",
// "modelVersion": "002", // Parameter only applicable for `ada` model family and older
"model": "text-embedding-3-large",
"dimensions": 3072, // Parameter only applicable for `v3` model family and newer
"type": "text",
"baseURL": "https://proxy.yourcompanydomain.com" // Optional. Can be overridden by one set in the HTTP header.
}
},
}
]
}

API settings (Azure OpenAI)

Parameters

Parameter
resourceNameAzure resource name
deploymentIdAzure deployment ID (your model name)
baseURLSet the endpoint of your Azure Open AI Deployments e.g. https://eastus.api.cognitive.microsoft.com

Example

{
"classes": [
{
"class": "Document",
"description": "A class called document",
"vectorizer": "text2vec-openai",
"moduleConfig": {
"text2vec-openai": {
"resourceName": "<YOUR-RESOURCE-NAME>",
"deploymentId": "<YOUR-MODEL-NAME>",
}
}
}
]
}

Vectorization settings

You can set vectorizer behavior using the moduleConfig section under each class and property:

Collection level settings

ParameterTypeDefault
vectorizerstring-
vectorizeClassNamebooleantrue

Property level settings

ParameterTypeDefault
skipbooleanfalse
vectorizePropertyNamebooleanfalse

Example

{
"classes": [
{
"class": "Document",
"description": "A class called document",
"vectorizer": "text2vec-openai",
"moduleConfig": {
"text2vec-openai": {
// "model": "ada",
// "modelVersion": "002", // Parameter only applicable for `ada` model family and older
"model": "text-embedding-3-large",
"dimensions": 3072, // Parameter only applicable for `v3` model family and newer
"type": "text",
"vectorizeClassName": false
}
},
"properties": [
{
"name": "content",
"dataType": ["text"],
"description": "Content that will be vectorized",
"moduleConfig": {
"text2vec-openai": {
"skip": false,
"vectorizePropertyName": false
}
}
}
]
}
]
}

Query-time parameters

You can supply parameters at query time by adding it to the HTTP header.

HTTP HeaderValuePurposeNote
"X-OpenAI-Api-Key""YOUR-OPENAI-API-KEY"OpenAI API key
"X-Azure-Api-Key""YOUR-AZURE-API-KEY"Azure OpenAI API key
"X-OpenAI-Organization""YOUR-OPENAI-ORGANIZATION"OpenAI organization nameAvailable from version v1.21.1
"X-OpenAI-BaseURL""YOUR-OPENAI-BASE-URL"OpenAI base URLAvailable from version v1.22.3

Use the protocol domain format: https://your.domain.com.

If specified, this will have precedence over the class-level setting.

Additional information

Available models (OpenAI)

You can use any OpenAI embedding model with text2vec-openai.

For document embeddings, choose from the following embedding models:

  • text-embedding-3
    • Available dimensions:
      • text-embedding-3-large: 256, 1024, 3072 (default)
      • text-embedding-3-small: 512, 1536 (default)
  • ada
  • babbage
  • davinci

The following models are now deprecated

Source

  • Codex
  • babbage-001
  • davinci-001
  • curie
Model size vs resource requirements

The more dimensions a model produces, the larger your data footprint will be. You can estimate the total size of your dataset here.

API rate limits

Since this module uses your API key, your account's corresponding rate limits will also apply to the module. Weaviate will output any rate-limit related error messages generated by the API.

You can request to increase your rate limit by emailing OpenAI at support@openai.com describing your use case with Weaviate.

The current rate limit will appear in the error message, as shown below:

{
"message": "Rate limit reached for requests. Limit: 600.000000 / min. Current: 1024.000000 / min. Contact support@openai.com if you continue to have issues."
}

Import throttling

One potential solution to rate limiting would be to throttle the import within your application. We include an example below.

See code example
from weaviate import Client
import time

def configure_batch(client: Client, batch_size: int, batch_target_rate: int):
"""
Configure the weaviate client's batch so it creates objects at `batch_target_rate`.

Parameters
----------
client : Client
The Weaviate client instance.
batch_size : int
The batch size.
batch_target_rate : int
The batch target rate as # of objects per second.
"""

def callback(batch_results: dict) -> None:

# you could print batch errors here
time_took_to_create_batch = batch_size * (client.batch.creation_time/client.batch.recommended_num_objects)
time.sleep(
max(batch_size/batch_target_rate - time_took_to_create_batch + 1, 0)
)

client.batch.configure(
batch_size=batch_size,
timeout_retries=5,
callback=callback,
)

Usage example

This is an example of a nearText query with text2vec-openai.

import weaviate
from weaviate.classes.query import MetadataQuery, Move
import os

client = weaviate.connect_to_local(
headers={
"X-OpenAI-Api-Key": os.getenv("OPENAI_APIKEY"),
}
)

publications = client.collections.get("Publication")

response = publications.query.near_text(
query="fashion",
distance=0.6,
move_to=Move(force=0.85, concepts="haute couture"),
move_away=Move(force=0.45, concepts="finance"),
return_metadata=MetadataQuery(distance=True),
limit=2
)

for o in response.objects:
print(o.properties)
print(o.metadata)

client.close()