text2vec-openai
Overview
The text2vec-openai
module enables Weaviate to obtain vectors using OpenAI or Azure OpenAI.
Key notes:
- As it uses a third-party API, you will need an API key.
- Its usage may incur costs.
- Please check the vendor pricing (e.g. OpenAI pricing page), especially before vectorizing large amounts of data.
- This module is available on Weaviate Cloud Services (WCS).
- Enabling this module will enable the
nearText
search operator. - The default model is
text-embedding-ada-002
.
Azure OpenAI or OpenAI?
The module usage instructions may vary based on whether you are using OpenAI directly or Azure OpenAI. Please make sure that you are following the right instructions for your service provider.
For example, the following may vary:
- Parameter names used in the schema, and
- Names of the API key to be used.
Where to set module parameters
The module accepts parameters through the request header, collection configuration, or environment variables. Some parameters (such as the API key) can be set in multiple ways.
Where the same parameter can be set in multiple ways, setting it at query-time through the HTTP request header (if possible) will have the highest precedence.
We suggest you only set any given parameter in one place to avoid confusion.
Weaviate instance configuration
This module is enabled and pre-configured on Weaviate Cloud Services.
Docker Compose file
To use text2vec-openai
, you must enable it in your Docker Compose file (docker-compose.yml
). You can do so manually, or create one using the Weaviate configuration tool.
Parameters
Parameter | Required | Purpose |
---|---|---|
ENABLE_MODULES | Required | The modules to enable. Include text2vec-openai to enable the module. |
DEFAULT_VECTORIZER_MODULE | Optional | The default vectorizer module. You can set this to text2vec-openai to make it the default for all classes. |
OPENAI_APIKEY | Optional | Your OpenAI API key (if using OpenAI). You can also provide the key at query time. |
AZURE_APIKEY | Optional | Your Azure OpenAI API key (if using Azure OpenAI). You can also provide the key at query time. |
Example
This configuration enables text2vec-openai
, sets it as the default vectorizer, and sets various other parameters such as the API key as environment variables.
---
version: '3.4'
services:
weaviate:
image: semitechnologies/weaviate:1.22.5
restart: on-failure:0
ports:
- "8080:8080"
environment:
QUERY_DEFAULTS_LIMIT: 20
AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
PERSISTENCE_DATA_PATH: "./data"
ENABLE_MODULES: text2vec-openai
DEFAULT_VECTORIZER_MODULE: text2vec-openai
OPENAI_APIKEY: sk-foobar # For use with OpenAI. Setting this parameter is optional; you can also provide the key at query time.
OPENAI_ORGANIZATION: your-orgname # For use with OpenAI. Setting this parameter is optional; you can also provide the key at runtime.
AZURE_APIKEY: sk-foobar # For use with Azure OpenAI. Setting this parameter is optional; you can also provide the key at query time.
CLUSTER_HOSTNAME: 'node1'
...
Class configuration
You can configure how the module will behave in each class through the Weaviate schema.
API settings (OpenAI)
Parameters
Parameter | Required | Default | Purpose |
---|---|---|---|
model | Optional | text-embedding-ada-002 | A model family, e.g. davinci . |
modelVersion | Optional | Version string, e.g. 003 . | |
type | Optional | Model type. Can be text or code . | |
baseURL | Optional | https://api.openai.com | Sets a proxy or other URL instead of the default OpenAI URL. Use a the protocol domain format: https://your.domain.com . |
Example
The following example configures the Document
class by setting the vectorizer to text2vec-openai
, model to ada
, the model version to 002
and the type to text
:
{
"classes": [
{
"class": "Document",
"description": "A class called document",
"vectorizer": "text2vec-openai",
"moduleConfig": {
"text2vec-openai": {
"model": "ada",
"modelVersion": "002",
"type": "text",
"baseURL": "https://proxy.yourcompanydomain.com" // Optional. Can be overridden by one set in the HTTP header.
}
},
}
]
}
API settings (Azure OpenAI)
Parameters
Parameter | |
---|---|
resourceName | Azure resource name |
deploymentId | Azure deployment ID (your model name) |
Example
{
"classes": [
{
"class": "Document",
"description": "A class called document",
"vectorizer": "text2vec-openai",
"moduleConfig": {
"text2vec-openai": {
"resourceName": "<YOUR-RESOURCE-NAME>",
"deploymentId": "<YOUR-MODEL-NAME>",
}
}
}
]
}
Vectorization settings
You can set vectorizer behavior using the moduleConfig
section under each class and property:
Collection level settings
Parameter | Type | Default |
---|---|---|
vectorizer | string | - |
vectorizeClassName | boolean | true |
Property level settings
Parameter | Type | Default |
---|---|---|
skip | boolean | false |
vectorizePropertyName | boolean | false |
Example
{
"classes": [
{
"class": "Document",
"description": "A class called document",
"vectorizer": "text2vec-openai",
"moduleConfig": {
"text2vec-openai": {
"model": "ada",
"modelVersion": "002",
"type": "text",
"vectorizeClassName": false
}
},
"properties": [
{
"name": "content",
"dataType": ["text"],
"description": "Content that will be vectorized",
"moduleConfig": {
"text2vec-openai": {
"skip": false,
"vectorizePropertyName": false
}
}
}
]
}
]
}
Query-time parameters
You can supply parameters at query time by adding it to the HTTP header.
HTTP Header | Value | Purpose | Note |
---|---|---|---|
"X-OpenAI-Api-Key" | "YOUR-OPENAI-API-KEY" | OpenAI API key | |
"X-Azure-Api-Key" | "YOUR-AZURE-API-KEY" | Azure OpenAI API key | |
"X-OpenAI-Organization" | "YOUR-OPENAI-ORGANIZATION" | OpenAI organization name | Available from version v1.21.1 |
"X-OpenAI-BaseURL" | "YOUR-OPENAI-BASE-URL" | OpenAI base URL | Available from version v1.22.3 Use the protocol domain format: https://your.domain.com . If specified, this will have precedence over the class-level setting. |
Additional information
Available models (OpenAI)
You can use any OpenAI embedding model with text2vec-openai
.
For document embeddings, choose from the following models:
For code embeddings, see the Codex models.
The more dimensions a model produces, the larger your data footprint will be. You can estimate the total size of your dataset here.
API rate limits
Since this module uses your API key, your account's corresponding rate limits will also apply to the module. Weaviate will output any rate-limit related error messages generated by the API.
You can request to increase your rate limit by emailing OpenAI at support@openai.com
describing your use case with Weaviate.
The current rate limit will appear in the error message, as shown below:
{
"message": "Rate limit reached for requests. Limit: 600.000000 / min. Current: 1024.000000 / min. Contact support@openai.com if you continue to have issues."
}
Import throttling
One potential solution to rate limiting would be to throttle the import within your application. We include an example below.
See code example
- Python
- Go
from weaviate import Client
import time
def configure_batch(client: Client, batch_size: int, batch_target_rate: int):
"""
Configure the weaviate client's batch so it creates objects at `batch_target_rate`.
Parameters
----------
client : Client
The Weaviate client instance.
batch_size : int
The batch size.
batch_target_rate : int
The batch target rate as # of objects per second.
"""
def callback(batch_results: dict) -> None:
# you could print batch errors here
time_took_to_create_batch = batch_size * (client.batch.creation_time/client.batch.recommended_num_objects)
time.sleep(
max(batch_size/batch_target_rate - time_took_to_create_batch + 1, 0)
)
client.batch.configure(
batch_size=batch_size,
timeout_retries=5,
callback=callback,
)
package main
import (
"context"
"time"
"github.com/weaviate/weaviate-go-client/v4/weaviate"
"github.com/weaviate/weaviate/entities/models"
)
var (
// adjust to your liking
targetRatePerMin = 600
batchSize = 50
)
func main() {
cfg := weaviate.Config{
Host: "localhost:8080",
Scheme: "http",
}
client, err := weaviate.NewClient(cfg)
if err != nil {
panic(err)
}
// replace those 10000 empty objects with your actual data
objects := make([]*models.Object, 10000)
// we aim to send one batch every tickInterval second.
tickInterval := time.Duration(batchSize/targetRatePerMinute) * time.Minute
t := time.NewTicker(tickInterval)
before := time.Now()
for i := 0; i < len(objects); i += batchSize {
// create a fresh batch
batch := client.Batch().ObjectsBatcher()
// add batchSize objects to the batch
for j := i; j < i+batchSize; j++ {
batch = batch.WithObject(objects[i+j])
}
// send off batch
res, err := batch.Do(context.Background())
// TODO: inspect result for individual errors
_ = res
// TODO: check request error
_ = err
// we wait for the next tick. If the previous batch took longer than
// tickInterval, we won't need to wait, effectively making this an
// unthrottled import.
<-t.C
}
}
Usage example
This is an example of a nearText
query with text2vec-openai
.
- Python
- JavaScript/TypeScript
- Go
- Java
- Curl
- GraphQL
import weaviate
client = weaviate.Client(
url="http://localhost:8080",
additional_headers={
"X-OpenAI-Api-Key": "YOUR-OPENAI-API-KEY"
}
)
nearText = {
"concepts": ["fashion"],
"distance": 0.6, # prior to v1.14 use "certainty" instead of "distance"
"moveAwayFrom": {
"concepts": ["finance"],
"force": 0.45
},
"moveTo": {
"concepts": ["haute couture"],
"force": 0.85
}
}
result = (
client.query
.get("Publication", "name")
.with_additional(["certainty OR distance"]) # note that certainty is only supported if distance==cosine
.with_near_text(nearText)
.do()
)
print(result)
import weaviate from 'weaviate-ts-client';
const client = weaviate.client({
scheme: 'http',
host: 'localhost:8080',
headers: { 'X-OpenAI-Api-Key': process.env['OPENAI_API_KEY'] }, // Replace with your API key
});
const response = await client.graphql
.get()
.withClassName('Publication')
.withFields('name _additional { certainty distance }') // note that certainty is only supported if distance==cosine
.withNearText({
concepts: ['fashion'],
distance: 0.6, // prior to v1.14 use certainty instead of distance
moveAwayFrom: {
concepts: ['finance'],
force: 0.45,
},
moveTo: {
concepts: ['haute couture'],
force: 0.85,
},
})
.do();
console.log(response);
package main
import (
"context"
"fmt"
"github.com/weaviate/weaviate-go-client/v4/weaviate"
"github.com/weaviate/weaviate-go-client/v4/weaviate/graphql"
)
func main() {
cfg := weaviate.Config{
Host: "localhost:8080",
Scheme: "http",
Headers: map[string]string{"X-OpenAI-Api-Key": "YOUR-OPENAI-API-KEY"},
}
client, err := weaviate.NewClient(cfg)
if err != nil {
panic(err)
}
className := "Publication"
name := graphql.Field{Name: "name"}
_additional := graphql.Field{
Name: "_additional", Fields: []graphql.Field{
{Name: "certainty"}, // only supported if distance==cosine
{Name: "distance"}, // always supported
},
}
concepts := []string{"fashion"}
distance := float32(0.6)
moveAwayFrom := &graphql.MoveParameters{
Concepts: []string{"finance"},
Force: 0.45,
}
moveTo := &graphql.MoveParameters{
Concepts: []string{"haute couture"},
Force: 0.85,
}
nearText := client.GraphQL().NearTextArgBuilder().
WithConcepts(concepts).
WithDistance(distance). // use WithCertainty(certainty) prior to v1.14
WithMoveTo(moveTo).
WithMoveAwayFrom(moveAwayFrom)
ctx := context.Background()
result, err := client.GraphQL().Get().
WithClassName(className).
WithFields(name, _additional).
WithNearText(nearText).
Do(ctx)
if err != nil {
panic(err)
}
fmt.Printf("%v", result)
}
package io.weaviate;
import io.weaviate.client.Config;
import io.weaviate.client.WeaviateClient;
import io.weaviate.client.base.Result;
import io.weaviate.client.v1.graphql.model.GraphQLResponse;
import io.weaviate.client.v1.graphql.query.argument.NearTextArgument;
import io.weaviate.client.v1.graphql.query.argument.NearTextMoveParameters;
import io.weaviate.client.v1.graphql.query.fields.Field;
import java.util.HashMap;
import java.util.Map;
public class App {
public static void main(String[] args) {
Map<String, String> headers = new HashMap<String, String>() { {
put("X-OpenAI-Api-Key", "YOUR-OPENAI-API-KEY");
} };
Config config = new Config("http", "localhost:8080", headers);
WeaviateClient client = new WeaviateClient(config);
NearTextMoveParameters moveTo = NearTextMoveParameters.builder()
.concepts(new String[]{ "haute couture" }).force(0.85f).build();
NearTextMoveParameters moveAway = NearTextMoveParameters.builder()
.concepts(new String[]{ "finance" }).force(0.45f)
.build();
NearTextArgument nearText = client.graphQL().arguments().nearTextArgBuilder()
.concepts(new String[]{ "fashion" })
.distance(0.6f) // use .certainty(0.7f) prior to v1.14
.moveTo(moveTo)
.moveAwayFrom(moveAway)
.build();
Field name = Field.builder().name("name").build();
Field _additional = Field.builder()
.name("_additional")
.fields(new Field[]{
Field.builder().name("certainty").build(), // only supported if distance==cosine
Field.builder().name("distance").build(), // always supported
}).build();
Result<GraphQLResponse> result = client.graphQL().get()
.withClassName("Publication")
.withFields(name, _additional)
.withNearText(nearText)
.run();
if (result.hasErrors()) {
System.out.println(result.getError());
return;
}
System.out.println(result.getResult());
}
}
# Note: Under nearText, use `certainty` instead of distance prior to v1.14
# Under _additional, `certainty` is only supported if distance==cosine, but `distance` is always supported
echo '{
"query": "{
Get{
Publication(
nearText: {
concepts: [\"fashion\"],
distance: 0.6,
moveAwayFrom: {
concepts: [\"finance\"],
force: 0.45
},
moveTo: {
concepts: [\"haute couture\"],
force: 0.85
}
}
){
name
_additional {
certainty
distance
}
}
}
}"
}' | curl \
-X POST \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer learn-weaviate' \
-H "X-OpenAI-Api-Key: $OPENAI_API_KEY" \
-d @- \
https://edu-demo.weaviate.network/v1/graphql
{
Get{
Publication(
nearText: {
concepts: ["fashion"],
distance: 0.6 # prior to v1.14 use "certainty" instead of "distance"
moveAwayFrom: {
concepts: ["finance"],
force: 0.45
},
moveTo: {
concepts: ["haute couture"],
force: 0.85
}
}
){
name
_additional {
certainty # only supported if distance==cosine.
distance # always supported
}
}
}
}