Question Answering - OpenAI
In shortโ
- The OpenAI Question and Answer (Q&A) module is a Weaviate module for answer extraction from data through the OpenAI completions endpoint or the Azure OpenAI equivalent.
- The module depends on a text vectorization module that should be running with Weaviate.
- The module adds an
ask {}
parameter to the GraphQLGet {}
queries - The module returns a max. of 1 answer in the GraphQL
_additional {}
field. - The answer with the highest
certainty
(confidence level) will be returned. - Added in Weaviate
v1.16.6
Azure OpenAI or OpenAI?โ
This module is compatible with both OpenAI and Azure OpenAI.
The instructions vary slightly based on whether you are using OpenAI directly or Azure OpenAI. Please make sure that you are following the right instructions for your service provider.
The differences are in:
- Parameter names used in the schema, and
- Names of the API key to be used.
Introductionโ
The Question and Answer (Q&A) OpenAI module is a Weaviate module for answer extraction from data. It uses an OpenAI completions endpoint to try and extract an answer from the most relevant docs.
This module can be used in GraphQL Get{...}
queries, as a search operator. The qna-openai
module tries to find an answer in the data objects of the specified class. If an answer is found within the given certainty
range, it will be returned in the GraphQL _additional { answer { ... } }
field. There will be a maximum of 1 answer returned, if this is above the optionally set certainty
. The answer with the highest certainty
(confidence level) will be returned.
Inference API keyโ
qna-openai
requires an API key from OpenAI or Azure OpenAI.
You only need to provide one of the two keys, depending on which service (OpenAI or Azure OpenAI) you are using.
Providing the key to Weaviateโ
You can provide your API key in two ways:
During the configuration of your Docker instance, by adding
OPENAI_APIKEY
orAZURE_APIKEY
as appropriate underenvironment
to yourdocker-compose
file, like this:environment:
OPENAI_APIKEY: 'your-key-goes-here' # For use with OpenAI. Setting this parameter is optional; you can also provide the key at runtime.
AZURE_APIKEY: 'your-key-goes-here' # For use with Azure OpenAI. Setting this parameter is optional; you can also provide the key at runtime.
...At run-time (recommended), by providing
"X-OpenAI-Api-Key"
or"X-Azure-Api-Key"
through the request header. You can provide it using the Weaviate client, like this:
- Python
- JavaScript
- Go
- Java
import weaviate
client = weaviate.Client(
url = "https://some-endpoint.weaviate.network/",
additional_headers = {
"X-OpenAI-Api-Key": "YOUR-OPENAI-API-KEY", # Replace with your API key
"X-Azure-Api-Key": "YOUR-AZURE-API-KEY", # Replace with your API key
}
)
const weaviate = require("weaviate-ts-client");
const client = weaviate.client({
scheme: 'https',
host: 'some-endpoint.weaviate.network',
// Replace with your API key
headers: {
'X-OpenAI-Api-Key': 'YOUR-OPENAI-API-KEY',
'X-Azure-Api-Key': 'YOUR-AZURE-API-KEY',
},
});
package main
import (
"context"
"fmt"
"github.com/weaviate/weaviate-go-client/v4/weaviate"
"github.com/weaviate/weaviate/entities/models"
)
func main() {
cfg := weaviate.Config{
Host: "some-endpoint.weaviate.network/", // Replace with your endpoint
Scheme: "https",
// Replace with your API key
Headers: map[string]string{
"X-OpenAI-Api-Key": "YOUR-OPENAI-API-KEY", // Replace with your API key
"X-Azure-Api-Key": "YOUR-AZURE-API-KEY", // Replace with your API key
}
}
client, err := weaviate.NewClient(cfg)
if err != nil {
panic(err)
}
}
package io.weaviate;
import java.util.ArrayList;
import io.weaviate.client.Config;
import io.weaviate.client.WeaviateClient;
import io.weaviate.client.base.Result;
public class App {
public static void main(String[] args) {
Map<String, String> headers = new HashMap<String, String>() { {
// Replace with your API key
put("X-OpenAI-Api-Key", "YOUR-OPENAI-API-KEY"); // Replace with your API key
put("X-Azure-Api-Key", "YOUR-AZURE-API-KEY"); // Replace with your API key
} };
Config config = new Config("https", "some-endpoint.weaviate.network/", headers);
WeaviateClient client = new WeaviateClient(config);
}
}
Module configurationโ
This module is enabled and pre-configured on Weaviate Cloud Services.
Configuration file (Weaviate open source only)โ
You can enable the OpenAI Q&A module in your configuration file (e.g. docker-compose.yaml
). Add the qna-openai
module (alongside any other module you may need) to the ENABLE_MODULES
property, like this:
ENABLE_MODULES: 'text2vec-openai,qna-openai'
Here is a full example of a Docker configuration, which uses the qna-openai
module in combination with text2vec-openai
:
---
version: '3.4'
services:
weaviate:
command:
- --host
- 0.0.0.0
- --port
- '8080'
- --scheme
- http
image:
semitechnologies/weaviate:1.19.6
ports:
- 8080:8080
restart: on-failure:0
environment:
QUERY_DEFAULTS_LIMIT: 25
AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
PERSISTENCE_DATA_PATH: '/var/lib/weaviate'
DEFAULT_VECTORIZER_MODULE: 'text2vec-openai'
ENABLE_MODULES: 'text2vec-openai,qna-openai'
OPENAI_APIKEY: sk-foobar # For use with OpenAI. Setting this parameter is optional; you can also provide the key at runtime.
AZURE_APIKEY: sk-foobar # For use with Azure OpenAI. Setting this parameter is optional; you can also provide the key at runtime.
CLUSTER_HOSTNAME: 'node1'
Schema configurationโ
You can define settings for this module in the schema.
OpenAI vs Azure OpenAIโ
- OpenAI users can optionally set the
model
parameter. - Azure OpenAI users must set the parameters
resourceName
anddeploymentId
.
Model parametersโ
You can also configure additional parameters for the model through the parameters shown below.
Example schemaโ
For example, the following schema configuration will set Weaviate to use the qna-openai
model with the Document
class.
The following schema configuration uses the text-davinci-002
model.
{
"classes": [
{
"class": "Document",
"description": "A class called document",
"vectorizer": "text2vec-openai",
"moduleConfig": {
"qna-openai": {
"model": "text-davinci-002", // For OpenAI
"resourceName": "<YOUR-RESOURCE-NAME>", // For Azure OpenAI
"deploymentId": "<YOUR-MODEL-NAME>", // For Azure OpenAI
"maxTokens": 16, // Applicable to both OpenAI and Azure OpenAI
"temperature": 0.0, // Applicable to both OpenAI and Azure OpenAI
"topP": 1, // Applicable to both OpenAI and Azure OpenAI
"frequencyPenalty": 0.0, // Applicable to both OpenAI and Azure OpenAI
"presencePenalty": 0.0 // Applicable to both OpenAI and Azure OpenAI
}
},
"properties": [
{
"dataType": [
"text"
],
"description": "Content that will be vectorized",
"name": "content"
}
]
}
]
}
For information on how to use the individual parameters you can check here
How to useโ
This module adds a search parameter to GraphQL Get{...}
queries: ask{}
. This new search parameter takes the following arguments:
Field | Data Type | Required | Example value | Description |
---|---|---|---|---|
question | string | yes | "What is the name of the Dutch king?" | The question to be answered. |
properties | list of strings | no | ["summary"] | The properties of the queries Class which contains text. If no properties are set, all are considered. |
Notes:
- The GraphQL
Explore { }
function does support theask
searcher, but the result is only a beacon to the object containing the answer. It is thus not any different from performing a nearText semantic search with the question. No extraction is happening. - You cannot use the
'ask'
parameter along with a'near'
parameter!
Example queryโ
- GraphQL
- Python
- JavaScript
- Go
- Java
- Curl
{
Get {
Article(
ask: {
question: "Who is Stanley Kubrick?",
properties: ["summary"]
},
limit: 1
) {
title
_additional {
answer {
hasAnswer
property
result
startPosition
endPosition
}
}
}
}
}
import weaviate
client = weaviate.Client(
url="http://localhost:8080",
additional_headers={
"X-OpenAI-Api-Key": "YOUR-OPENAI-API-KEY"
}
)
ask = {
"question": "Who is Stanley Kubrick?",
"properties": ["summary"]
}
result = (
client.query
.get("Article", ["title", "_additional {answer {hasAnswer property result startPosition endPosition} }"])
.with_ask(ask)
.with_limit(1)
.do()
)
print(result)
const weaviate = require('weaviate-client');
const client = weaviate.client({
scheme: 'http',
host: 'localhost:8080',
headers: {'X-OpenAI-Api-Key': 'YOUR-OPENAI-API-KEY'},
});
client.graphql
.get()
.withClassName('Article')
.withAsk({
question: 'Who is Stanley Kubrick?',
properties: ['summary'],
})
.withFields('title _additional { answer { hasAnswer property result startPosition endPosition } }')
.withLimit(1)
.do()
.then(res => {
console.log(res)
})
.catch(err => {
console.error(err)
});
package main
import (
"context"
"fmt"
"github.com/weaviate/weaviate-go-client/v4/weaviate"
"github.com/weaviate/weaviate-go-client/v4/weaviate/graphql"
)
func main() {
cfg := weaviate.Config{
Host: "localhost:8080",
Scheme: "http",
Headers: map[string]string{"X-OpenAI-Api-Key": "YOUR-OPENAI-API-KEY"},
}
client, err := weaviate.NewClient(cfg)
if err != nil {
panic(err)
}
className := "Article"
fields := []graphql.Field{
{Name: "title"},
{Name: "_additional", Fields: []graphql.Field{
{Name: "answer", Fields: []graphql.Field{
{Name: "hasAnswer"},
{Name: "property"},
{Name: "result"},
{Name: "startPosition"},
{Name: "endPosition"},
}},
}},
}
ask := client.GraphQL().AskArgBuilder().
WithQuestion("Who is Stanley Kubrick?").
WithProperties([]string{"summary"})
ctx := context.Background()
result, err := client.GraphQL().Get().
WithClassName(className).
WithFields(fields...).
WithAsk(ask).
WithLimit(1).
Do(ctx)
if err != nil {
panic(err)
}
fmt.Printf("%v", result)
}
package io.weaviate;
import io.weaviate.client.Config;
import io.weaviate.client.WeaviateClient;
import io.weaviate.client.base.Result;
import io.weaviate.client.v1.graphql.model.GraphQLResponse;
import io.weaviate.client.v1.graphql.query.argument.AskArgument;
import io.weaviate.client.v1.graphql.query.fields.Field;
public class App {
public static void main(String[] args) {
Map<String, String> headers = new HashMap<String, String>() { {
put("X-OpenAI-Api-Key", "YOUR-OPENAI-API-KEY");
} };
Config config = new Config("http", "localhost:8080", headers);
WeaviateClient client = new WeaviateClient(config);
Field title = Field.builder().name("title").build();
Field _additional = Field.builder()
.name("_additional")
.fields(new Field[]{
Field.builder()
.name("answer")
.fields(new Field[]{
Field.builder().name("hasAnswer").build(),
Field.builder().name("property").build(),
Field.builder().name("result").build(),
Field.builder().name("startPosition").build(),
Field.builder().name("endPosition").build()
}).build()
}).build();
AskArgument ask = AskArgument.builder()
.question("Who is Stanley Kubrick?")
.properties(new String[]{ "summary" })
.build();
Result<GraphQLResponse> result = client.graphQL().get()
.withClassName("Article")
.withFields(title, _additional)
.withAsk(ask)
.withLimit(1)
.run();
if (result.hasErrors()) {
System.out.println(result.getError());
return;
}
System.out.println(result.getResult());
}
}
$ echo '{
"query": "{
Get {
Article(
ask: {
question: \"Who is Stanley Kubrick?\",
properties: [\"summary\"]
},
limit: 1
) {
title
_additional {
answer {
hasAnswer
property
result
startPosition
endPosition
}
}
}
}
}
"
}' | curl \
-X POST \
-H 'Content-Type: application/json' \
-H "X-OpenAI-Api-Key: YOUR-OPENAI-API-KEY" \
-d @- \
http://localhost:8080/v1/graphql
GraphQL responseโ
The answer is contained in a new GraphQL _additional
property called answer
. It contains the following fields:
hasAnswer
(boolean
): could an answer be found?result
(nullablestring
): An answer if one could be found.null
ifhasAnswer==false
property
(nullablestring
): The property which contains the answer.null
ifhasAnswer==false
startPosition
(int
): The character offset where the answer starts.0
ifhasAnswer==false
endPosition
(int
): The character offset where the answer ends0
ifhasAnswer==false
Note: startPosition
, endPosition
and property
in the response are not guaranteed to be present. They are calculated by a case-insensitive string matching function against the input text. If the transformer model formats the output differently (e.g. by introducing spaces between tokens which were not present in the original input), the calculation of the position and determining the property fails.
Example responseโ
{
"data": {
"Get": {
"Document": [
{
"_additional": {
"answer": {
"hasAnswer": true,
"result": " Stanley Kubrick is an American filmmaker who is best known for his films, including \"A Clockwork Orange,\" \"Eyes Wide Shut,\" and \"The Shining.\""
}
}
}
]
}
}
}
How it works (under the hood)โ
Under the hood, the model uses a two-step approach. First it performs a semantic search with k=1
to find the document (e.g. a Sentence, Paragraph, Article, etc.) which is most likely to contain the answer. This step has no certainty threshold and as long as at least one document is present, it will be fetched and selected as the one most likely containing the answer. In a second step, Weaviate creates the required prompt as an input to an external call made to the OpenAI Completions endpoint. Weaviate uses the most relevant documents to establish a prompt for which OpenAI extracts the answer. There are three possible outcomes:
- No answer was found because the question can not be answered,
- An answer was found, but did not meet the user-specified minimum certainty, so it was discarded (typically the case when the document is on topic, but does not contain an actual answer to the question), and
- An answer was found that matches the desired certainty. It is returned to the user.
The module performs a semantic search under the hood, so a text2vec-...
module is required. It does not need to be transformers-based and you can also combine it with text2vec-contextionary
. However, we expect that you will receive the best results by combining it with a well-fitting transformers model by using the appropriate configured text2vec-transformers
module.
Additional informationโ
Available modelsโ
OpenAI has multiple models available for the extraction of answers from a given context.
These models can be configured
More resourcesโ
If you can't find the answer to your question here, please look at the:
- Frequently Asked Questions. Or,
- Knowledge base of old issues. Or,
- For questions: Stackoverflow. Or,
- For more involved discussion: Weaviate Community Forum. Or,
- We also have a Slack channel.