Question Answering - OpenAI

     

In short

• The OpenAI Question and Answer (Q&A) module is a Weaviate module for answer extraction from data through the OpenAI completions endpoint.
• The module depends on a text vectorization module that should be running with Weaviate.
• The module adds an ask {} parameter to the GraphQL Get {} queries
• The module returns a max. of 1 answer in the GraphQL _additional {} field.
• The answer with the highest certainty (confidence level) will be returned.
• Added in Weaviate v1.16.6

Introduction

The Question and Answer (Q&A) OpenAI module is a Weaviate module for answer extraction from data. It uses the completions endpoint, created by OpenAI, to try and extract an answer from the most relevant docs. This module can be used in GraphQL Get{...} queries, as a search operator. The qna-openai module tries to find an answer in the data objects of the specified class. If an answer is found within the given certainty range, it will be returned in the GraphQL _additional { answer { ... } } field. There will be a maximum of 1 answer returned, if this is above the optionally set certainty. The answer with the highest certainty (confidence level) will be returned.

How to enable

Request an OpenAI API-key via their website.

How to enable (module configuration)

Docker-compose

• The Q&A module can be added as a service to the Docker-compose file.
• You must have a text vectorizer running.
• An example Docker-compose file for using the qna-openai module in combination with the text2vec-openai is as follows:

---
version: '3.4'
services:
  weaviate:
    command:
      - --host
      - 0.0.0.0
      - --port
      - '8080'
      - --scheme
      - http
    image:
      semitechnologies/weaviate:1.17.2
    ports:
      - 8080:8080
    restart: on-failure:0
    environment:
      QUERY_DEFAULTS_LIMIT: 25
      AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
      PERSISTENCE_DATA_PATH: '/var/lib/weaviate'
      DEFAULT_VECTORIZER_MODULE: 'text2vec-openai'
      ENABLE_MODULES: 'text2vec-openai,qna-openai'
      OPENAI_APIKEY: sk-foobar # this parameter is optional, as you can also provide it at insert/query time
      CLUSTER_HOSTNAME: 'node1'

Variable explanations:

• Note: Starting with v1.11.0 the OPENAI_APIKEY variable is now optional and you can instead provide the key at insert/query time as an HTTP header.

How to configure

​In your Weaviate schema, you must define how you want this module to interact with the OpenAI endpoint. If you are new to Weaviate schemas, you might want to check out the getting started guide on the Weaviate schema first.

The following schema configuration uses the ada model.

{
  "classes": [
    {
      "class": "Document",
      "description": "A class called document",
      "vectorizer": "text2vec-openai",
      "moduleConfig": {
        "qna-openai": {
          "model": "text-davinci-002",
          "maxTokens": 16,
          "temperature": 0.0,
          "topP": 1,
          "frequencyPenalty": 0.0,
          "presencePenalty": 0.0
        }
      },
      "properties": [
        {
          "dataType": [
            "text"
          ],
          "description": "Content that will be vectorized",
          "name": "content"
        }
      ]
    }
  ]
}

For information on how to use the individual parameters you can check here

How to use (GraphQL)

GraphQL Ask search

This module adds a search parameter to GraphQL Get{...} queries: ask{}. This new search parameter takes the following arguments:

Field	Data Type	Required	Example value	Description
question	string	yes	"What is the name of the Dutch king?"	The question to be answered.
properties	list of strings	no	["summary"]	The properties of the queries Class which contains text. If no properties are set, all are considered.

Notes:

• The GraphQL Explore { } function does support the ask searcher, but the result is only a beacon to the object containing the answer. It is thus not any different from performing a nearText semantic search with the question. No extraction is happening.
• You cannot use the 'ask' parameter along with a 'near' parameter!

Example query

GraphQL

{
  Get {
    Article(
      ask: {
        question: "Who is Stanley Kubrick?",
        properties: ["summary"]
      },
      limit: 1
    ) {
      title
      _additional {
        answer {
          hasAnswer
          property
          result
          startPosition
          endPosition
        }
      }
    }
  }
}

Python

import weaviate

client  = weaviate.Client(
    url="http://localhost:8080",
    additional_headers={
        "X-OpenAI-Api-Key": "<THE-KEY>"
    }
)

ask = {
  "question": "Who is Stanley Kubrick?",
  "properties": ["summary"]
}

result = (
  client.query
  .get("Article", ["title", "_additional {answer {hasAnswer property result startPosition endPosition} }"])
  .with_ask(ask)
  .with_limit(1)
  .do()
)

print(result)

JavaScript

const weaviate = require('weaviate-client');

const client = weaviate.client({
  scheme: 'http',
  host: 'localhost:8080',
  headers: {'X-OpenAI-Api-Key': '<THE-KEY>'},
});

client.graphql
      .get()
      .withClassName('Article')
      .withAsk({
        question: 'Who is Stanley Kubrick?',
        properties: ['summary'],
      })
      .withFields('title _additional { answer { hasAnswer property result startPosition endPosition } }')
      .withLimit(1)
      .do()
      .then(res => {
        console.log(res)
      })
      .catch(err => {
        console.error(err)
      });

Go

package main

import (
  "context"
  "fmt"

  "github.com/weaviate/weaviate-go-client/v4/weaviate"
  "github.com/weaviate/weaviate-go-client/v4/weaviate/graphql"
)

func main() {
  cfg := weaviate.Config{
    Host:    "localhost:8080",
    Scheme:  "http",
    Headers: map[string]string{"X-OpenAI-Api-Key": "<THE-KEY>"},
  }
  client := weaviate.New(cfg)

  className := "Article"
  fields := []graphql.Field{
    {Name: "title"},
    {Name: "_additional", Fields: []graphql.Field{
      {Name: "answer", Fields: []graphql.Field{
        {Name: "hasAnswer"},
        {Name: "property"},
        {Name: "result"},
        {Name: "startPosition"},
        {Name: "endPosition"},
      }},
    }},
  }
  ask := client.GraphQL().AskArgBuilder().
    WithQuestion("Who is Stanley Kubrick?").
    WithProperties([]string{"summary"})
  ctx := context.Background()

  result, err := client.GraphQL().Get().
    WithClassName(className).
    WithFields(fields...).
    WithAsk(ask).
    WithLimit(1).
    Do(ctx)

  if err != nil {
    panic(err)
  }
  fmt.Printf("%v", result)
}

Java

package technology.semi.weaviate;

import technology.semi.weaviate.client.Config;
import technology.semi.weaviate.client.WeaviateClient;
import technology.semi.weaviate.client.base.Result;
import technology.semi.weaviate.client.v1.graphql.model.GraphQLResponse;
import technology.semi.weaviate.client.v1.graphql.query.argument.AskArgument;
import technology.semi.weaviate.client.v1.graphql.query.fields.Field;

public class App {
  public static void main(String[] args) {
    Map<String, String> headers = new HashMap<String, String>() { {
      put("X-OpenAI-Api-Key", "<THE-KEY>");
    } };
    Config config = new Config("http", "localhost:8080", headers);
    WeaviateClient client = new WeaviateClient(config);

    Field title = Field.builder().name("title").build();
    Field _additional = Field.builder()
      .name("_additional")
      .fields(new Field[]{
        Field.builder()
          .name("answer")
          .fields(new Field[]{
            Field.builder().name("hasAnswer").build(),
            Field.builder().name("property").build(),
            Field.builder().name("result").build(),
            Field.builder().name("startPosition").build(),
            Field.builder().name("endPosition").build()
          }).build()
      }).build();

    AskArgument ask = AskArgument.builder()
      .question("Who is Stanley Kubrick?")
      .properties(new String[]{ "summary" })
      .build();

    Result<GraphQLResponse> result = client.graphQL().get()
      .withClassName("Article")
      .withFields(title, _additional)
      .withAsk(ask)
      .withLimit(1)
      .run();

    if (result.hasErrors()) {
      System.out.println(result.getError());
      return;
    }
    System.out.println(result.getResult());
  }
}

Curl

$ echo '{
  "query": "{
    Get {
      Article(
        ask: {
          question: \"Who is Stanley Kubrick?\",
          properties: [\"summary\"]
        },
        limit: 1
      ) {
        title
        _additional {
          answer {
            hasAnswer
            property
            result
            startPosition
            endPosition
          }
        }
      }
    }
  }
  "
}' | curl \
    -X POST \
    -H 'Content-Type: application/json' \
    -H "X-OpenAI-Api-Key: <THE-KEY>" \
    -d @- \
    http://localhost:8080/v1/graphql

🟢 Try out this GraphQL example in the Weaviate Console.

🟠 Whoops, the demo service is temporarily offline.

GraphQL response

The answer is contained in a new GraphQL _additional property called answer. It contains the following fields:

• hasAnswer (boolean): could an answer be found?
• result (nullable string): An answer if one could be found. null if hasAnswer==false
• property (nullable string): The property which contains the answer. null if hasAnswer==false
• startPosition (int): The character offset where the answer starts. 0 if hasAnswer==false
• endPosition (int): The character offset where the answer ends 0 if hasAnswer==false

Note: startPosition, endPosition and property in the response are not guaranteed to be present. They are calculated by a case-insensitive string matching function against the input text. If the transformer model formats the output differently (e.g. by introducing spaces between tokens which were not present in the original input), the calculation of the position and determining the property fails.

Example response

{
  "data": {
    "Get": {
      "Document": [
        {
          "_additional": {
            "answer": {
              "hasAnswer": true,
              "result": " Stanley Kubrick is an American filmmaker who is best known for his films, including \"A Clockwork Orange,\" \"Eyes Wide Shut,\" and \"The Shining.\""
            }
          }
        }
      ]
    }
  }
}

How it works (under the hood)

Under the hood, the model uses a two-step approach. First it performs a semantic search with k=1 to find the document (e.g. a Sentence, Paragraph, Article, etc.) which is most likely to contain the answer. This step has no certainty threshold and as long as at least one document is present, it will be fetched and selected as the one most likely containing the answer. In a second step, Weaviate creates the required prompt as an input to an external call made to the OpenAI Completions endpoint. Weaviate uses the most relevant documents to establish a prompt for which OpenAI extracts the answer. There are three possible outcomes:

1. No answer was found because the question can not be answered,
2. An answer was found, but did not meet the user-specified minimum certainty, so it was discarded (typically the case when the document is on topic, but does not contain an actual answer to the question), and
3. An answer was found that matches the desired certainty. It is returned to the user.

The module performs a semantic search under the hood, so a text2vec-... module is required. It does not need to be transformers-based and you can also combine it with text2vec-contextionary. However, we expect that you will receive the best results by combining it with a well-fitting transformers model by using the appropriate configured text2vec-transformers module.

Additional information

Available models

OpenAI has multiple models available for the extraction of answers from a given context.

• For document embeddings you can choose one of the following models:
  • ada
  • babbage
  • curie
  • davinci

These models can be configured

More resources

If you can't find the answer to your question here, please look at the:

1. Frequently Asked Questions. Or,
2. Knowledge base of old issues. Or,
3. For questions: Stackoverflow. Or,
4. For issues: Github. Or,
5. Ask your question in the Slack channel: Slack.