Generative search
Overview
This page shows you how to perform generative
searches.
A generative search uses a large language model (LLM) to generate text based on the search results and a user-provided prompt. This technique is also called retrieval augmented generation, or RAG.
Requirements
To use the generative search feature, you must:
- Configure Weaviate to use a generator module (
generative-openai
,generative-cohere
,generative-palm
), - Configure the parameters for the
generative-*
module in the target class, - Specify a query to retrieve one or more objects, and
- Provide a
single prompt
or agrouped task
to generate text from.
How do I configure Weaviate with a generator module?
You must enable the desired generative search module and (optionally) specify the corresponding inference service (OpenAI, Cohere, PaLM) API key in the relevant Docker Compose file (e.g. docker-compose.yml
), or (recommended) request that client code provide it with every request. You can generate this file using the Weaviate configuration tool.
Here are the relevant settings from the Docker Compose file. Ensure the corresponding environment variable is set (i.e. $OPENAI_APIKEY
, $COHERE_APIKEY
, or $PALM_APIKEY
), unless you want the client to supply the API key (recommended).
- OpenAI
- Cohere
- PaLM
services:
weaviate:
environment:
OPENAI_APIKEY: $OPENAI_APIKEY
ENABLE_MODULES: '...,generative-openai,...'
services:
weaviate:
environment:
COHERE_APIKEY: $COHERE_APIKEY
ENABLE_MODULES: '...,generative-cohere,...'
services:
weaviate:
environment:
PALM_APIKEY: $PALM_APIKEY
ENABLE_MODULES: '...,generative-palm,...'
How do I set the generative module in the target class?
Where multiple generative
modules are enabled, you must specify the generative module to be used in the moduleConfig
section of the schema. For example, this configures the Article
class to use the generative-openai
module:
{
"classes": [
{
"class": "Article",
...,
"moduleConfig": {
"generative-openai": {}, // This will configure the 'Article' class to use the 'generative-openai' module
}
}
]
}
You can configure additional module parameters here also. Please refer to the "Schema configuration" section in the relevant module page.
Single prompt
A single prompt generative search returns a generated response for each object in the query results. For single prompt generative searches, you must specify which object properties to use in the prompt.
In this example, the query:
- Retrieves two
JeopardyQuestion
objects related toWorld history
, - Prepares a prompt for each object, based on the prompt
"Convert the following into a question for twitter. Include emojis for fun, but do not include the answer: {question}."
, where{question}
is an object property, and - Retrieves a generated text for each object (2 total), and
- Returns the generated text as a part of each object, along with the
question
property.
- Python (v4)
- Python (v3)
- JavaScript/TypeScript
- GraphQL
prompt = "Convert the following into a question for twitter. Include emojis for fun, but do not include the answer: {question}."
jeopardy = client.collections.get("JeopardyQuestion")
response = jeopardy.generate.near_text(
query="World history",
limit=2,
single_prompt=prompt
)
# print source questions and generated responses
for o in response.objects:
print(o.properties["question"])
print(o.generated)
generate_prompt = "Convert the following into a question for twitter. Include emojis for fun, but do not include the answer: {question}."
response = (
client.query
.get("JeopardyQuestion", ["question"])
.with_generate(single_prompt=generate_prompt)
.with_near_text({
"concepts": ["World history"]
})
.with_limit(2)
).do()
print(json.dumps(response, indent=2))
generatePrompt = `Convert the following into a question for twitter.
Include emojis for fun, but do not include the answer: {question}.`;
result = await client.graphql
.get()
.withClassName('JeopardyQuestion')
.withGenerate({
singlePrompt: generatePrompt,
})
.withNearText({
concepts: ['World history'],
})
.withLimit(2)
.withFields('question')
.do();
console.log(JSON.stringify(result, null, 2));
{
Get {
JeopardyQuestion (
nearText: {
concepts: ["World history"]
}
limit: 2
) {
question
_additional {
generate(
singleResult: {
prompt: """
Convert the following into a question for twitter. Include emojis for fun, but do not include the answer: {question}.
"""
}
) {
singleResult
error
}
}
}
}
}
Example response
It should produce a response like the one below:
{
"data": {
"Get": {
"JeopardyQuestion": [
{
"_additional": {
"generate": {
"error": null,
"singleResult": "\ud83c\udf0e\ud83c\udf1e Did you know that in the 19th century, one quarter of the world's land and people were part of an empire where the sun never set? #history #funfact"
}
},
"question": "Including, in 19th century, one quarter of world's land & people, the sun never set on it"
},
{
"_additional": {
"generate": {
"error": null,
"singleResult": "\ud83e\udd14 Which country had more kings than any other in ancient history, from Menes to the Ptolemys? \ud83d\udc51\ud83c\udfdb\ufe0f #history #ancientworld"
}
},
"question": "From Menes to the Ptolemys, this country had more kings than any other in ancient history"
}
]
}
}
}
Single prompt property selection
When using generative search with single prompts, you must specify which object properties to use in the prompt.
The properties to use as a part of the prompt do not need to be among the properties retrieved in the query.
In this example, the query:
- Retrieves two
JeopardyQuestion
objects related toWorld history
, - Prepares a prompt for each object, based on the prompt
"Convert this quiz question: {question} and answer: {answer} into a trivia tweet.
where{question}
and{answer}
are object properties, and - Retrieves a generated text for each object (2 total), and
- Returns the generated text as a part of each object.
Note that the question
and answer
properties are not retrieved in the query, but are used in the prompt.
- Python (v4)
- Python (v3)
- JavaScript/TypeScript
- GraphQL
prompt = "Convert this quiz question: {question} and answer: {answer} into a trivia tweet."
jeopardy = client.collections.get("JeopardyQuestion")
response = jeopardy.generate.near_text(
query="World history",
limit=2,
single_prompt=prompt
)
# print source properties and generated responses
for o in response.objects:
print(o.properties)
print(o.generated)
generate_prompt = "Convert this quiz question: {question} and answer: {answer} into a trivia tweet."
response = (
client.query
.get("JeopardyQuestion")
.with_generate(single_prompt=generate_prompt)
.with_near_text({
"concepts": ["World history"]
})
.with_limit(2)
).do()
print(json.dumps(response, indent=2))
generatePrompt = 'Convert this quiz question: {question} and answer: {answer} into a trivia tweet.';
result = await client.graphql
.get()
.withClassName('JeopardyQuestion')
.withGenerate({
singlePrompt: generatePrompt,
})
.withNearText({
concepts: ['World history'],
})
.withFields('round')
.withLimit(2)
.do();
console.log(JSON.stringify(result, null, 2));
{
Get {
JeopardyQuestion (
nearText: {
concepts: ["World history"]
}
limit: 2
) {
_additional {
generate(
singleResult: {
prompt: """
Convert this quiz question: {question} and answer: {answer} into a trivia tweet.
"""
}
) {
singleResult
error
}
}
}
}
}
Example response
It should produce a response like the one below:
{
"data": {
"Get": {
"JeopardyQuestion": [
{
"_additional": {
"generate": {
"error": null,
"singleResult": "Did you know that in the 19th century, the British Empire included one quarter of the world's land and people? The sun never set on it! #BritishEmpire #TriviaTuesday"
}
}
},
{
"_additional": {
"generate": {
"error": null,
"singleResult": "Did you know that Egypt had more kings than any other country in ancient history? From Menes to the Ptolemys, they ruled the land of the Nile. #Egypt #AncientHistory #Kings"
}
}
}
]
}
}
}
Grouped task
A grouped task works by generating a response for the entire query results set.
When using generative search with a grouped task, the required parameter is the user prompt. By default, the entire set of properties are included in the combined prompt unless specified otherwise.
Example
In this example, the query:
- Retrieves three
JeopardyQuestion
objects related tocute animals
, - Combines the user prompt with the set of retrieved objects to build the grouped task,
- Retrieves one generated text using the grouped task, and
- Returns the generated text as a part of the first object returned, as well as the requested
points
property.
Note that the prompt includes information about the type of the animal (from the answer
property), even though the answer
property is not explicitly retrieved.
- Python (v4)
- Python (v3)
- JavaScript/TypeScript
- GraphQL
task = "What do these animals have in common, if anything?"
jeopardy = client.collections.get("JeopardyQuestion")
response = jeopardy.generate.near_text(
query="Cute animals",
limit=3,
grouped_task=task
)
# print the generated response
print(response.generated)
generate_prompt = "What do these animals have in common, if anything?"
response = (
client.query
.get("JeopardyQuestion", ["points"])
.with_generate(grouped_task=generate_prompt)
.with_near_text({
"concepts": ["Cute animals"]
})
.with_limit(3)
).do()
print(json.dumps(response, indent=2))
generatePrompt = 'What do these animals have in common, if anything?';
result = await client.graphql
.get()
.withClassName('JeopardyQuestion')
.withGenerate({
groupedTask: generatePrompt,
})
.withNearText({
concepts: ['Cute animals'],
})
.withFields('points')
.withLimit(3)
.do();
console.log(JSON.stringify(result, null, 2));
{
Get {
JeopardyQuestion (
nearText: {
concepts: ["Cute animals"]
}
limit: 3
) {
points
_additional {
generate(
groupedResult: {
task: """
What do these animals have in common, if anything?
"""
}
) {
groupedResult
error
}
}
}
}
}
Example response
It should produce a response like the one below:
{
"data": {
"Get": {
"JeopardyQuestion": [
{
"_additional": {
"generate": {
"error": null,
"groupedResult": "All of these animals are mammals."
}
},
"points": 400
},
{
"_additional": {
"generate": null
},
"points": 300
},
{
"_additional": {
"generate": null
},
"points": 400
}
]
}
}
}
Grouped task property selection
v1.18.3
You can specify which properties will be included in the grouped task
prompt. Use this to limit the information provided in the prompt, and to reduce the prompt length.
In this example, the prompt only includes the question
and answer
properties. Note that the answer
property is not explicitly retrieved in the query, but it is used by the prompt.
- Python (v4)
- Python (v3)
- JavaScript/TypeScript
- GraphQL
task = "What do these animals have in common, if anything?"
jeopardy = client.collections.get("JeopardyQuestion")
response = jeopardy.generate.near_text(
query="Australian animals",
limit=3,
grouped_task=task,
grouped_properties=["answer", "question"]
)
# print source properties
for o in response.objects:
print(o.properties)
# print the generated response
print(response.generated)
generate_prompt = 'What do these animals have in common, if anything?'
response = (
client.query
.get('JeopardyQuestion', ['question points'])
.with_generate(
grouped_task=generate_prompt,
grouped_properties=['answer', 'question'] # available since client version 3.19.2
)
.with_near_text({
'concepts': ['Australian animals']
})
.with_limit(3)
).do()
print(json.dumps(response, indent=2))
generatePrompt = 'What do these animals have in common, if anything?';
result = await client.graphql
.get()
.withClassName('JeopardyQuestion')
.withGenerate({
groupedTask: generatePrompt,
groupedProperties: ['answer', 'question'], // available since client version 1.3.2
})
.withNearText({
concepts: ['Australian animals'],
})
.withFields('question points')
.withLimit(3)
.do();
console.log(JSON.stringify(result, null, 2));
{
Get {
JeopardyQuestion (
nearText: {
concepts: ["Australian animals"]
}
limit: 3
) {
question
points
_additional {
generate(
groupedResult: {
task: """
What do these animals have in common, if anything?
"""
properties: ["answer", "question"]
}
) {
groupedResult
error
}
}
}
}
}
Example response
It should produce a response like the one below:
{
"data": {
"Get": {
"JeopardyQuestion": [
{
"_additional": {
"generate": {
"error": null,
"groupedResult": "All of the animals mentioned are native to Australia."
}
},
"points": 800,
"question": "Australians call this animal a jumbuck or a monkey"
},
{
"_additional": {
"generate": null
},
"points": 100,
"question": "An island named for the animal seen <a href=\"http://www.j-archive.com/media/2000-03-10_J_01.jpg\" target=\"_blank\">here</a> belongs to this country [kangaroo]"
},
{
"_additional": {
"generate": null
},
"points": 300,
"question": "Found chiefly in Australia, the wallaby is a smaller type of this marsupial"
}
]
}
}
}