Generative search

Overview

This page shows you how to perform generative searches.

A generative search uses a large language model (LLM) to generate text based on the search results and a user-provided prompt. This technique is also called retrieval augmented generation, or RAG.

Requirements

To use the generative search feature, you must:

Configure Weaviate to use a generator module (generative-openai, generative-cohere, generative-palm),
Configure the parameters for the generative-* module in the target class,
Specify a query to retrieve one or more objects, and
Provide a single prompt or a grouped task to generate text from.

How do I configure Weaviate with a generator module?

You must enable the desired generative search module and (optionally) specify the corresponding inference service (OpenAI, Cohere, PaLM) API key in the relevant Docker Compose file (e.g. docker-compose.yml), or (recommended) request that client code provide it with every request. You can generate this file using the Weaviate configuration tool.

Here are the relevant settings from the Docker Compose file. Ensure the corresponding environment variable is set (i.e. $OPENAI_APIKEY, $COHERE_APIKEY, or $PALM_APIKEY), unless you want the client to supply the API key (recommended).

OpenAI
Cohere
PaLM

services:
  weaviate:
    environment:
      OPENAI_APIKEY: $OPENAI_APIKEY
      ENABLE_MODULES: '...,generative-openai,...'

services:
  weaviate:
    environment:
      COHERE_APIKEY: $COHERE_APIKEY
      ENABLE_MODULES: '...,generative-cohere,...'

services:
  weaviate:
    environment:
      PALM_APIKEY: $PALM_APIKEY
      ENABLE_MODULES: '...,generative-palm,...'

How do I set the generative module in the target class?

Where multiple generative modules are enabled, you must specify the generative module to be used in the moduleConfig section of the schema. For example, this configures the Article class to use the generative-openai module:

{
  "classes": [
    {
      "class": "Article",
      ...,
      "moduleConfig": {
        "generative-openai": {},  // This will configure the 'Article' class to use the 'generative-openai' module
      }
    }
  ]
}

You can configure additional module parameters here also. Please refer to the "Schema configuration" section in the relevant module page.

Single prompt

A single prompt generative search returns a generated response for each object in the query results. For single prompt generative searches, you must specify which object properties to use in the prompt.

In this example, the query:

Retrieves two JeopardyQuestion objects related to World history,
Prepares a prompt for each object, based on the prompt "Convert the following into a question for twitter. Include emojis for fun, but do not include the answer: {question}.", where {question} is an object property, and
Retrieves a generated text for each object (2 total), and
Returns the generated text as a part of each object, along with the question property.

Python (v4)
Python (v3)
JavaScript/TypeScript
GraphQL

prompt = "Convert the following into a question for twitter. Include emojis for fun, but do not include the answer: {question}."

jeopardy = client.collections.get("JeopardyQuestion")
response = jeopardy.generate.near_text(
    query="World history",
    limit=2,
    single_prompt=prompt
)

# print source questions and generated responses
for o in response.objects:
    print(o.properties["question"])
    print(o.generated)

generate_prompt = "Convert the following into a question for twitter. Include emojis for fun, but do not include the answer: {question}."

response = (
  client.query
  .get("JeopardyQuestion", ["question"])
  .with_generate(single_prompt=generate_prompt)
  .with_near_text({
    "concepts": ["World history"]
  })
  .with_limit(2)
).do()

print(json.dumps(response, indent=2))

generatePrompt = `Convert the following into a question for twitter.
Include emojis for fun, but do not include the answer: {question}.`;

result = await client.graphql
  .get()
  .withClassName('JeopardyQuestion')
  .withGenerate({
    singlePrompt: generatePrompt,
  })
  .withNearText({
    concepts: ['World history'],
  })
  .withLimit(2)
  .withFields('question')
  .do();

console.log(JSON.stringify(result, null, 2));

{
  Get {
    JeopardyQuestion (
      nearText: {
        concepts: ["World history"]
      }
      limit: 2
    ) {
      question
      _additional {
        generate(
          singleResult: {
            prompt: """
              Convert the following into a question for twitter. Include emojis for fun, but do not include the answer: {question}.
            """
          }
        ) {
          singleResult
          error
        }
      }
    }
  }
}

Example response

It should produce a response like the one below:

{
  "data": {
    "Get": {
      "JeopardyQuestion": [
        {
          "_additional": {
            "generate": {
              "error": null,
              "singleResult": "\ud83c\udf0e\ud83c\udf1e Did you know that in the 19th century, one quarter of the world's land and people were part of an empire where the sun never set? #history #funfact"
            }
          },
          "question": "Including, in 19th century, one quarter of world's land & people, the sun never set on it"
        },
        {
          "_additional": {
            "generate": {
              "error": null,
              "singleResult": "\ud83e\udd14 Which country had more kings than any other in ancient history, from Menes to the Ptolemys? \ud83d\udc51\ud83c\udfdb\ufe0f #history #ancientworld"
            }
          },
          "question": "From Menes to the Ptolemys, this country had more kings than any other in ancient history"
        }
      ]
    }
  }
}

Single prompt property selection

When using generative search with single prompts, you must specify which object properties to use in the prompt.

The properties to use as a part of the prompt do not need to be among the properties retrieved in the query.

In this example, the query:

Retrieves two JeopardyQuestion objects related to World history,
Prepares a prompt for each object, based on the prompt "Convert this quiz question: {question} and answer: {answer} into a trivia tweet. where {question} and {answer} are object properties, and
Retrieves a generated text for each object (2 total), and
Returns the generated text as a part of each object.

Note that the question and answer properties are not retrieved in the query, but are used in the prompt.

Python (v4)
Python (v3)
JavaScript/TypeScript
GraphQL

prompt = "Convert this quiz question: {question} and answer: {answer} into a trivia tweet."

jeopardy = client.collections.get("JeopardyQuestion")
response = jeopardy.generate.near_text(
    query="World history",
    limit=2,
    single_prompt=prompt
)

# print source properties and generated responses
for o in response.objects:
    print(o.properties)
    print(o.generated)

generate_prompt = "Convert this quiz question: {question} and answer: {answer} into a trivia tweet."

response = (
  client.query
  .get("JeopardyQuestion")
  .with_generate(single_prompt=generate_prompt)
  .with_near_text({
    "concepts": ["World history"]
  })
  .with_limit(2)
).do()

print(json.dumps(response, indent=2))

generatePrompt = 'Convert this quiz question: {question} and answer: {answer} into a trivia tweet.';

result = await client.graphql
  .get()
  .withClassName('JeopardyQuestion')
  .withGenerate({
    singlePrompt: generatePrompt,
  })
  .withNearText({
    concepts: ['World history'],
  })
  .withFields('round')
  .withLimit(2)
  .do();

console.log(JSON.stringify(result, null, 2));

{
  Get {
    JeopardyQuestion (
      nearText: {
        concepts: ["World history"]
      }
      limit: 2
    ) {
      _additional {
        generate(
          singleResult: {
            prompt: """
              Convert this quiz question: {question} and answer: {answer} into a trivia tweet.
            """
          }
        ) {
          singleResult
          error
        }
      }
    }
  }
}

Example response

It should produce a response like the one below:

{
  "data": {
    "Get": {
      "JeopardyQuestion": [
        {
          "_additional": {
            "generate": {
              "error": null,
              "singleResult": "Did you know that in the 19th century, the British Empire included one quarter of the world's land and people? The sun never set on it! #BritishEmpire #TriviaTuesday"
            }
          }
        },
        {
          "_additional": {
            "generate": {
              "error": null,
              "singleResult": "Did you know that Egypt had more kings than any other country in ancient history? From Menes to the Ptolemys, they ruled the land of the Nile. #Egypt #AncientHistory #Kings"
            }
          }
        }
      ]
    }
  }
}

Grouped task

A grouped task works by generating a response for the entire query results set.

When using generative search with a grouped task, the required parameter is the user prompt. By default, the entire set of properties are included in the combined prompt unless specified otherwise.

Example

In this example, the query:

Retrieves three JeopardyQuestion objects related to cute animals,
Combines the user prompt with the set of retrieved objects to build the grouped task,
Retrieves one generated text using the grouped task, and
Returns the generated text as a part of the first object returned, as well as the requested points property.

Note that the prompt includes information about the type of the animal (from the answer property), even though the answer property is not explicitly retrieved.

Python (v4)
Python (v3)
JavaScript/TypeScript
GraphQL

task = "What do these animals have in common, if anything?"

jeopardy = client.collections.get("JeopardyQuestion")
response = jeopardy.generate.near_text(
    query="Cute animals",
    limit=3,
    grouped_task=task
)

# print the generated response
print(response.generated)

generate_prompt = "What do these animals have in common, if anything?"

response = (
  client.query
  .get("JeopardyQuestion", ["points"])
  .with_generate(grouped_task=generate_prompt)
  .with_near_text({
    "concepts": ["Cute animals"]
  })
  .with_limit(3)
).do()

print(json.dumps(response, indent=2))

generatePrompt = 'What do these animals have in common, if anything?';

result = await client.graphql
  .get()
  .withClassName('JeopardyQuestion')
  .withGenerate({
    groupedTask: generatePrompt,
  })
  .withNearText({
    concepts: ['Cute animals'],
  })
  .withFields('points')
  .withLimit(3)
  .do();

console.log(JSON.stringify(result, null, 2));

{
  Get {
    JeopardyQuestion (
      nearText: {
        concepts: ["Cute animals"]
      }
      limit: 3
    ) {
      points
      _additional {
        generate(
          groupedResult: {
            task: """
              What do these animals have in common, if anything?
            """
          }
        ) {
          groupedResult
          error
        }
      }
    }
  }
}

Example response

It should produce a response like the one below:

{
  "data": {
    "Get": {
      "JeopardyQuestion": [
        {
          "_additional": {
            "generate": {
              "error": null,
              "groupedResult": "All of these animals are mammals."
            }
          },
          "points": 400
        },
        {
          "_additional": {
            "generate": null
          },
          "points": 300
        },
        {
          "_additional": {
            "generate": null
          },
          "points": 400
        }
      ]
    }
  }
}

Grouped task property selection

Added in v1.18.3

You can specify which properties will be included in the grouped task prompt. Use this to limit the information provided in the prompt, and to reduce the prompt length.

In this example, the prompt only includes the question and answer properties. Note that the answer property is not explicitly retrieved in the query, but it is used by the prompt.

Python (v4)
Python (v3)
JavaScript/TypeScript
GraphQL

task = "What do these animals have in common, if anything?"

jeopardy = client.collections.get("JeopardyQuestion")
response = jeopardy.generate.near_text(
    query="Australian animals",
    limit=3,
    grouped_task=task,
    grouped_properties=["answer", "question"]
)

# print source properties
for o in response.objects:
    print(o.properties)

# print the generated response
print(response.generated)

generate_prompt = 'What do these animals have in common, if anything?'

response = (
  client.query
  .get('JeopardyQuestion', ['question points'])
  .with_generate(
      grouped_task=generate_prompt,
      grouped_properties=['answer', 'question']  # available since client version 3.19.2
  )
  .with_near_text({
    'concepts': ['Australian animals']
  })
  .with_limit(3)
).do()

print(json.dumps(response, indent=2))

generatePrompt = 'What do these animals have in common, if anything?';

result = await client.graphql
  .get()
  .withClassName('JeopardyQuestion')
  .withGenerate({
    groupedTask: generatePrompt,
    groupedProperties: ['answer', 'question'],  // available since client version 1.3.2
  })
  .withNearText({
    concepts: ['Australian animals'],
  })
  .withFields('question points')
  .withLimit(3)
  .do();

console.log(JSON.stringify(result, null, 2));

{
  Get {
    JeopardyQuestion (
      nearText: {
        concepts: ["Australian animals"]
      }
      limit: 3
    ) {
      question
      points
      _additional {
        generate(
          groupedResult: {
            task: """
              What do these animals have in common, if anything?
            """
            properties: ["answer", "question"]
          }
        ) {
          groupedResult
          error
        }
      }
    }
  }
}

Example response

It should produce a response like the one below:

{
  "data": {
    "Get": {
      "JeopardyQuestion": [
        {
          "_additional": {
            "generate": {
              "error": null,
              "groupedResult": "All of the animals mentioned are native to Australia."
            }
          },
          "points": 800,
          "question": "Australians call this animal a jumbuck or a monkey"
        },
        {
          "_additional": {
            "generate": null
          },
          "points": 100,
          "question": "An island named for the animal seen <a href=\"http://www.j-archive.com/media/2000-03-10_J_01.jpg\" target=\"_blank\">here</a> belongs to this country [kangaroo]"
        },
        {
          "_additional": {
            "generate": null
          },
          "points": 300,
          "question": "Found chiefly in Australia, the wallaby is a smaller type of this marsupial"
        }
      ]
    }
  }
}

Overview​

Requirements​

Single prompt​

Single prompt property selection​

Grouped task​

Example​

Grouped task property selection​

Overview

Requirements

Single prompt

Single prompt property selection

Grouped task

Example

Grouped task property selection