Skip to main content

Generative search

Generative search, also known as "Retrieval Augmented Generation" (RAG), is a multi-stage process.
First Weaviate performs a query, then it passes the retrieved results and a prompt to a large language model (LLM), to generate a new output.

Additional information

  1. Configure Weaviate to use a generator module. For details, see the module reference page:

  2. Configure the target collection to use the generator module. For details, see schema configuration on the module reference page.

  3. Query your database to retrieve one or more objects.

  4. Use the query results to generate a new result.

Named vectors​

Added in v1.24

Any vector-based search on collections with named vectors configured must include a target vector name in the query. This allows Weaviate to find the correct vector to compare with the query vector.

from weaviate.classes.query import MetadataQuery

reviews = client.collections.get("WineReviewNV")
response = reviews.generate.near_text(
    query="a sweet German white wine",
    limit=2,
    target_vector="title_country",  # Specify the target vector for named vector collections
    single_prompt="Translate this into German: {review_body}",
    grouped_task="Summarize these review",
    return_metadata=MetadataQuery(distance=True)
)

print(response.generated)
for o in response.objects:
    print(o.properties)
    print(o.generated)

Single prompt search returns a generated response for each object in the query results.
Define object properties – using {prop-name} syntax – to interpolate retrieved content in the prompt.
The properties you use in the prompt do not have to be among the properties you retrieve in the query.

prompt = "Convert this quiz question: {question} and answer: {answer} into a trivia tweet."

jeopardy = client.collections.get("JeopardyQuestion")
response = jeopardy.generate.near_text(
    query="World history",
    limit=2,
    single_prompt=prompt
)

# print source properties and generated responses
for o in response.objects:
    print(o.properties)
    print(o.generated)
Example response

The output is like this:

{
  "data": {
    "Get": {
      "JeopardyQuestion": [
        {
          "_additional": {
            "generate": {
              "error": null,
              "singleResult": "Did you know that in the 19th century, the British Empire included one quarter of the world's land and people? The sun never set on it! #BritishEmpire #TriviaTuesday"
            }
          }
        },
        {
          "_additional": {
            "generate": {
              "error": null,
              "singleResult": "Did you know that Egypt had more kings than any other country in ancient history? From Menes to the Ptolemys, they ruled the land of the Nile. #Egypt #AncientHistory #Kings"
            }
          }
        }
      ]
    }
  }
}

Grouped task search returns one response that includes all of the query results. By default grouped task search uses all object properties in the prompt.

task = "What do these animals have in common, if anything?"

jeopardy = client.collections.get("JeopardyQuestion")
response = jeopardy.generate.near_text(
    query="Cute animals",
    limit=3,
    grouped_task=task
)

# print the generated response
print(response.generated)
Example response

The output is like this:

{
  "data": {
    "Get": {
      "JeopardyQuestion": [
        {
          "_additional": {
            "generate": {
              "error": null,
              "groupedResult": "All of these animals are mammals."
            }
          },
          "points": 400
        },
        {
          "_additional": {
            "generate": null
          },
          "points": 300
        },
        {
          "_additional": {
            "generate": null
          },
          "points": 400
        }
      ]
    }
  }
}

Set grouped task prompt properties​

Added in v1.18.3

Define object properties to use in the prompt. This limits the information in the prompt and reduces prompt length.

task = "What do these animals have in common, if anything?"

jeopardy = client.collections.get("JeopardyQuestion")
response = jeopardy.generate.near_text(
    query="Australian animals",
    limit=3,
    grouped_task=task,
    grouped_properties=["answer", "question"]
)

# print the generated response
print(response.generated)
Example response

The output is like this:

{
  "data": {
    "Get": {
      "JeopardyQuestion": [
        {
          "_additional": {
            "generate": {
              "error": null,
              "groupedResult": "All of the animals mentioned are native to Australia."
            }
          },
          "points": 800,
          "question": "Australians call this animal a jumbuck or a monkey"
        },
        {
          "_additional": {
            "generate": null
          },
          "points": 100,
          "question": "An island named for the animal seen <a href=\"http://www.j-archive.com/media/2000-03-10_J_01.jpg\" target=\"_blank\">here</a> belongs to this country [kangaroo]"
        },
        {
          "_additional": {
            "generate": null
          },
          "points": 300,
          "question": "Found chiefly in Australia, the wallaby is a smaller type of this marsupial"
        }
      ]
    }
  }
}

Questions and feedback​

If you have any questions or feedback, please let us know on our forum. For example, you can: