text2vec-openai
In shortโ
- This module uses a third-party API and may incur costs.
- Check the vendor pricing (e.g. OpenAI pricing page) before vectorizing large amounts of data.
- Weaviate automatically parallelizes requests to the API when using the batch endpoint.
- Check out the text2vec-openai demo.
- You will need an API key from OpenAI or Azure OpenAI to use this module.
- The default OpenAI model is
text-embedding-ada-002
.
Overviewโ
The text2vec-openai
module enables you to use OpenAI in Weaviate or Azure embeddings to represent data objects.
Azure OpenAI or OpenAI?โ
This module is compatible with both OpenAI and Azure OpenAI.
The instructions vary slightly based on whether you are using OpenAI directly or Azure OpenAI. Please make sure that you are following the right instructions for your service provider.
The differences are in:
- Parameter names used in the schema, and
- Names of the API key to be used.
Module configurationโ
This module is enabled and pre-configured on Weaviate Cloud Services.
Configuration file (Weaviate open source only)โ
You can enable the text2vec-openai
module in your configuration file (e.g. docker-compose.yaml
).
- This configuration will start Weaviate with the OpenAI module enabled, and set as the default vectorizer module.
- Optionally, you can specify the required API key in the file..
- If you do not, you must specify the API key at runtime.
---
version: '3.4'
services:
weaviate:
image: semitechnologies/weaviate:1.19.6
restart: on-failure:0
ports:
- "8080:8080"
environment:
QUERY_DEFAULTS_LIMIT: 20
AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
PERSISTENCE_DATA_PATH: "./data"
DEFAULT_VECTORIZER_MODULE: text2vec-openai
ENABLE_MODULES: text2vec-openai
OPENAI_APIKEY: sk-foobar # For use with OpenAI. Setting this parameter is optional; you can also provide the key at runtime.
AZURE_APIKEY: sk-foobar # For use with Azure OpenAI. Setting this parameter is optional; you can also provide the key at runtime.
CLUSTER_HOSTNAME: 'node1'
...
- You can also use the Weaviate configuration tool to create a Weaviate setup with this module.
- The
OPENAI_APIKEY
environment variable is optional and you can instead provide the key at insert/query time as an HTTP header (see the 'usage' section for instructions)
Schema configurationโ
You can provide additional module configurations through the schema. You can learn about schemas here.
For text2vec-openai
, you can set the vectorizer model and vectorizer behavior.
OpenAI settingsโ
Set the vectorizer model using parameters model
, modelVersion
and type
in the moduleConfig
section of your schema:
{
"classes": [
{
"class": "Document",
"description": "A class called document",
"vectorizer": "text2vec-openai",
"moduleConfig": {
"text2vec-openai": {
"model": "ada",
"modelVersion": "002",
"type": "text"
}
},
}
]
}
Azure OpenAI settingsโ
Set the parameters resourceName
and deploymentId
in the moduleConfig
section of your schema:
{
"classes": [
{
"class": "Document",
"description": "A class called document",
"vectorizer": "text2vec-openai",
"moduleConfig": {
"text2vec-openai": {
"resourceName": "<YOUR-RESOURCE-NAME>",
"deploymentId": "<YOUR-MODEL-NAME>",
}
}
}
]
}
Vectorizer behaviorโ
Set property-level vectorizer behavior using the moduleConfig
section under each property:
{
"classes": [
{
"class": "Document",
"description": "A class called document",
"vectorizer": "text2vec-openai",
"moduleConfig": {
"text2vec-openai": {
"model": "ada",
"modelVersion": "002",
"type": "text"
}
},
"properties": [
{
"dataType": ["text"],
"description": "Content that will be vectorized",
"moduleConfig": {
"text2vec-openai": {
"skip": false,
"vectorizePropertyName": false
}
},
"name": "content"
}
]
}
]
}
Usageโ
Enabling this module will make GraphQL vector search operators available.
Provide the API keyโ
If the API key is not set in the text2vec-openai
configuration, you can supply it when making a query.
You can achieve this by adding the appropriate key to the HTTP header:
X-OpenAI-Api-Key: YOUR-OPENAI-API-KEY
for OpenAI, andX-Azure-Api-Key: YOUR-AZURE-API-KEY
for Azure OpenAI, and
Exampleโ
- GraphQL
- Python
- JavaScript
- Go
- Java
- Curl
{
Get{
Publication(
nearText: {
concepts: ["fashion"],
distance: 0.6 # prior to v1.14 use "certainty" instead of "distance"
moveAwayFrom: {
concepts: ["finance"],
force: 0.45
},
moveTo: {
concepts: ["haute couture"],
force: 0.85
}
}
){
name
_additional {
certainty # only supported if distance==cosine.
distance # always supported
}
}
}
}
import weaviate
client = weaviate.Client(
url="http://localhost:8080",
additional_headers={
"X-OpenAI-Api-Key": "YOUR-OPENAI-API-KEY"
}
)
nearText = {
"concepts": ["fashion"],
"distance": 0.6, # prior to v1.14 use "certainty" instead of "distance"
"moveAwayFrom": {
"concepts": ["finance"],
"force": 0.45
},
"moveTo": {
"concepts": ["haute couture"],
"force": 0.85
}
}
result = (
client.query
.get("Publication", "name")
.with_additional(["certainty OR distance"]) # note that certainty is only supported if distance==cosine
.with_near_text(nearText)
.do()
)
print(result)
const weaviate = require('weaviate-client');
const client = weaviate.client({
scheme: 'http',
host: 'localhost:8080',
headers: {'X-OpenAI-Api-Key': 'YOUR-OPENAI-API-KEY'},
});
client.graphql
.get()
.withClassName('Publication')
.withFields('name _additional{certainty distance}') // note that certainty is only supported if distance==cosine
.withNearText({
concepts: ['fashion'],
distance: 0.6, // prior to v1.14 use certainty instead of distance
moveAwayFrom: {
concepts: ['finance'],
force: 0.45
},
moveTo: {
concepts: ['haute couture'],
force: 0.85
}
})
.do()
.then(console.log)
.catch(console.error);
package main
import (
"context"
"fmt"
"github.com/weaviate/weaviate-go-client/v4/weaviate"
"github.com/weaviate/weaviate-go-client/v4/weaviate/graphql"
)
func main() {
cfg := weaviate.Config{
Host: "localhost:8080",
Scheme: "http",
Headers: map[string]string{"X-OpenAI-Api-Key": "YOUR-OPENAI-API-KEY"},
}
client, err := weaviate.NewClient(cfg)
if err != nil {
panic(err)
}
className := "Publication"
name := graphql.Field{Name: "name"}
_additional := graphql.Field{
Name: "_additional", Fields: []graphql.Field{
{Name: "certainty"}, // only supported if distance==cosine
{Name: "distance"}, // always supported
},
}
concepts := []string{"fashion"}
distance := float32(0.6)
moveAwayFrom := &graphql.MoveParameters{
Concepts: []string{"finance"},
Force: 0.45,
}
moveTo := &graphql.MoveParameters{
Concepts: []string{"haute couture"},
Force: 0.85,
}
nearText := client.GraphQL().NearTextArgBuilder().
WithConcepts(concepts).
WithDistance(distance). // use WithCertainty(certainty) prior to v1.14
WithMoveTo(moveTo).
WithMoveAwayFrom(moveAwayFrom)
ctx := context.Background()
result, err := client.GraphQL().Get().
WithClassName(className).
WithFields(name, _additional).
WithNearText(nearText).
Do(ctx)
if err != nil {
panic(err)
}
fmt.Printf("%v", result)
}
package io.weaviate;
import io.weaviate.client.Config;
import io.weaviate.client.WeaviateClient;
import io.weaviate.client.base.Result;
import io.weaviate.client.v1.graphql.model.GraphQLResponse;
import io.weaviate.client.v1.graphql.query.argument.NearTextArgument;
import io.weaviate.client.v1.graphql.query.argument.NearTextMoveParameters;
import io.weaviate.client.v1.graphql.query.fields.Field;
import java.util.HashMap;
import java.util.Map;
public class App {
public static void main(String[] args) {
Map<String, String> headers = new HashMap<String, String>() { {
put("X-OpenAI-Api-Key", "YOUR-OPENAI-API-KEY");
} };
Config config = new Config("http", "localhost:8080", headers);
WeaviateClient client = new WeaviateClient(config);
NearTextMoveParameters moveTo = NearTextMoveParameters.builder()
.concepts(new String[]{ "haute couture" }).force(0.85f).build();
NearTextMoveParameters moveAway = NearTextMoveParameters.builder()
.concepts(new String[]{ "finance" }).force(0.45f)
.build();
NearTextArgument nearText = client.graphQL().arguments().nearTextArgBuilder()
.concepts(new String[]{ "fashion" })
.distance(0.6f) // use .certainty(0.7f) prior to v1.14
.moveTo(moveTo)
.moveAwayFrom(moveAway)
.build();
Field name = Field.builder().name("name").build();
Field _additional = Field.builder()
.name("_additional")
.fields(new Field[]{
Field.builder().name("certainty").build(), // only supported if distance==cosine
Field.builder().name("distance").build(), // always supported
}).build();
Result<GraphQLResponse> result = client.graphQL().get()
.withClassName("Publication")
.withFields(name, _additional)
.withNearText(nearText)
.run();
if (result.hasErrors()) {
System.out.println(result.getError());
return;
}
System.out.println(result.getResult());
}
}
$ echo '{
"query": "{
Get{
Publication(
nearText: {
concepts: [\"fashion\"],
distance: 0.6, // use certainty instead of distance prior to v1.14
moveAwayFrom: {
concepts: [\"finance\"],
force: 0.45
},
moveTo: {
concepts: [\"haute couture\"],
force: 0.85
}
}
){
name
_additional {
certainty // only supported if distance==cosine
distance // always supported
}
}
}
}"
}' | curl \
-X POST \
-H 'Content-Type: application/json' \
-H "X-OpenAI-Api-Key: YOUR-OPENAI-API-KEY" \
-d @- \
http://localhost:8080/v1/graphql
Additional informationโ
Available models (OpenAI)โ
OpenAI has multiple models available with different trade-offs. All the models offered by OpenAI can be used within Weaviate. Note that the more dimensions a model produces, the larger your data footprint will be. To estimate the total size of your dataset use this calculation.
The default model is text-embedding-ada-002
but you can also specify it in your schema. An example as part of a class definition:
{
"classes": [
{
"class": "Document",
"vectorizer": "text2vec-openai",
"moduleConfig": {
"text2vec-openai": {
"model": "ada",
"modelVersion": "002",
"type": "text"
}
}
}
]
}
For document embeddings, choose from the following models:
For code embeddings, see the Codex models.
In the moduleConfig
inside a class, you need to set two values:
model
โ one of the models mentioned above, e.g.davinci
.modelVersion
โ version string, e.g.003
.type
โtext
orcode
.
Example (as part of a class definition):
{
"classes": [
{
"class": "Document",
"vectorizer": "text2vec-openai",
"moduleConfig": {
"text2vec-openai": {
"model": "ada",
"modelVersion": "002",
"type": "text"
}
}
}
]
}
OpenAI rate limitsโ
Since you will obtain embeddings using your own API key, any corresponding rate limits related to your account will apply to your use with Weaviate also.
If you exceed your rate limit, Weaviate will output the error message generated by the OpenAI API. You can request to increase your rate limit by emailing OpenAI at support@openai.com
describing your use case with Weaviate.
Throttle the import inside your applicationโ
One way of dealing with rate limits is to throttle the import within your application. For example, when using the Weaviate client in Python or Java:
- Python
- Go
from weaviate import Client
import time
def configure_batch(client: Client, batch_size: int, batch_target_rate: int):
"""
Configure the weaviate client's batch so it creates objects at `batch_target_rate`.
Parameters
----------
client : Client
The Weaviate client instance.
batch_size : int
The batch size.
batch_target_rate : int
The batch target rate as # of objects per second.
"""
def callback(batch_results: dict) -> None:
# you could print batch errors here
time_took_to_create_batch = batch_size * (client.batch.creation_time/client.batch.recommended_num_objects)
time.sleep(
max(batch_size/batch_target_rate - time_took_to_create_batch + 1, 0)
)
client.batch.configure(
batch_size=batch_size,
timeout_retries=5,
callback=callback,
)
package main
import (
"context"
"time"
"github.com/weaviate/weaviate-go-client/v4/weaviate"
"github.com/weaviate/weaviate/entities/models"
)
var (
// adjust to your liking
targetRatePerMin = 600
batchSize = 50
)
func main() {
cfg := weaviate.Config{
Host: "localhost:8080",
Scheme: "http",
}
client, err := weaviate.NewClient(cfg)
if err != nil {
panic(err)
}
// replace those 10000 empty objects with your actual data
objects := make([]*models.Object, 10000)
// we aim to send one batch every tickInterval second.
tickInterval := time.Duration(batchSize/targetRatePerMinute) * time.Minute
t := time.NewTicker(tickInterval)
before := time.Now()
for i := 0; i < len(objects); i += batchSize {
// create a fresh batch
batch := client.Batch().ObjectsBatcher()
// add batchSize objects to the batch
for j := i; j < i+batchSize; j++ {
batch = batch.WithObject(objects[i+j])
}
// send off batch
res, err := batch.Do(context.Background())
// TODO: inspect result for individual errors
_ = res
// TODO: check request error
_ = err
// we wait for the next tick. If the previous batch took longer than
// tickInterval, we won't need to wait, effectively making this an
// unthrottled import.
<-t.C
}
}
The current rate limit will appear in the error message, as shown below:
{
"message": "Rate limit reached for requests. Limit: 600.000000 / min. Current: 1024.000000 / min. Contact support@openai.com if you continue to have issues."
}
More resourcesโ
If you can't find the answer to your question here, please look at the:
- Frequently Asked Questions. Or,
- Knowledge base of old issues. Or,
- For questions: Stackoverflow. Or,
- For more involved discussion: Weaviate Community Forum. Or,
- We also have a Slack channel.