Generative Search - AWS
In short
v1.22.5
- The Generative AWS (
generative-aws
) module performs retrieval augmented generation, or RAG, based on the data stored in your Weaviate instance. - The module can generate a response for each object returned from Weaviate, or a combined response for a group of objects.
- The module enables generative search operations on the Weaviate instance.
API Authentication
You must provide access key based AWS credentials to use the API, including an AWS access key and a corresponding AWS secret access key. You can set them as environment variables, or provide them at query time.
Introduction
generative-aws
performs retrieval augmented generation, or RAG, based on the data stored in your Weaviate instance.
The module works in two steps:
- (Weaviate) Run a search query in Weaviate to find relevant objects.
- (AWS Bedrock) Use a Large Language Model to generate a response based on the results (from the previous step) and the provided prompt or task.
You can use the Generative AWS module with any other upstream modules. For example, you could use text2vec-cohere
, text2vec-huggingface
or text2vec-openai
to vectorize and query your data, but then rely on the generative-aws
module to generate a response.
The generative module can perform RAG for:
- each returned object -
singlePrompt
- the group of all results together –
groupedTask
You need to input both a query and a prompt (for individual responses) or a task (for all responses).
Weaviate instance configuration
If you use Weaviate Cloud Services (WCS), this module is already enabled and pre-configured. You cannot edit the configuration in WCS.
Docker Compose file
To use generative-aws
, you must enable it in your Docker Compose file (docker-compose.yml
). You can do so manually, or create one using the Weaviate configuration tool.
Parameters
ENABLE_MODULES
(Required): The modules to enable. Includegenerative-aws
to enable the module.AWS_ACCESS_KEY
orAWS_ACCESS_KEY_ID
(Optional): Your AWS access key. You can also provide the key at query time.AWS_SECRET_KEY
orAWS_SECRET_ACCESS_KEY
(Optional): Your AWS secret access key. You can also provide the key at query time.
Example
This configuration enables generative-aws
and sets the AWS authentication credentials.
---
version: '3.4'
services:
weaviate:
command:
- --host
- 0.0.0.0
- --port
- '8080'
- --scheme
- http
image: cr.weaviate.io/semitechnologies/weaviate:1.24.10
ports:
- 8080:8080
- 50051:50051
volumes:
- weaviate_data:/var/lib/weaviate
restart: on-failure:0
environment:
QUERY_DEFAULTS_LIMIT: 25
AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
PERSISTENCE_DATA_PATH: '/var/lib/weaviate'
DEFAULT_VECTORIZER_MODULE: text2vec-aws
AWS_ACCESS_KEY: sk-foobar # Optional. Can be set at query time.
AWS_SECRET_KEY: sk-foobar # Optional. Can be set at query time.
ENABLE_MODULES: 'text2vec-aws,generative-aws'
CLUSTER_HOSTNAME: 'node1'
volumes:
weaviate_data:
...
Class configuration
You can configure how the module will behave in each class through the Weaviate schema.
See this page for code examples on how to specify a generative module.
API settings
Parameters
Parameter | Required | Default | Purpose |
---|---|---|---|
model | No | NONE | The model to use. You must provide an available & supported model name. |
region | Yes | NONE | AWS region name, e.g. us-east-1 . |
Example
The following example configures the Document
class to use the generative-aws
module with the Document
class, with the command-xlarge-nightly
model and the AWS region to us-east-1
.
{
"classes": [
{
"class": "Document",
"description": "A class called document",
...,
"moduleConfig": {
"generative-aws": {
"model": "cohere.command-text-v14", // REQUIRED
"region": "us-east-1" // REQUIRED
},
}
}
]
}
New to Weaviate Schemas?
If you are new to Weaviate, check out the Weaviate schema tutorial.
How to use
Query-time parameters
You can supply parameters at query time by adding it to the HTTP header.
HTTP Header | Value | Purpose | Note |
---|---|---|---|
"X-AWS-Access-Key" | "YOUR-AWS-API-ACCESS-KEY" | Your AWS access key. | |
"X-AWS-Secret-Key" | "YOUR-AWS-API-SECRET-KEY" | Your AWS secret access key |
Queries
This module extends the _additional {...}
property with a generate
operator.
generate
takes the following arguments:
Field | Data Type | Required | Example | Description |
---|---|---|---|---|
singleResult {prompt} | string | no | Summarize the following in a tweet: {summary} | Generates a response for each individual search result. You need to include at least one result field in the prompt, between braces. |
groupedResult {task} | string | no | Explain why these results are similar to each other | Generates a single response for all search results |
Example of properties in the prompt
When piping the results to the prompt, at least one field returned by the query must be added to the prompt. If you don't add any fields, Weaviate will throw an error.
For example, assume your schema looks like this:
{
Article {
title
summary
}
}
You can add both title
and summary
to the prompt by enclosing them in curly brackets:
{
Get {
Article {
title
summary
_additional {
generate(
singleResult: {
prompt: """
Summarize the following in a tweet:
{title} - {summary}
"""
}
) {
singleResult
error
}
}
}
}
}
Example - single result
Here is an example of a query where:
- we run a vector search (with
nearText
) to find articles about "Italian food" - then we ask the generator module to describe each result as a Facebook ad.
- the query asks for the
summary
field, which it then includes in theprompt
argument of thegenerate
operator.
- the query asks for the
- Python (v4)
- Python (v3)
- JavaScript/TypeScript
- Go
- Java
- GraphQL
- Curl
import weaviate
import os
client = weaviate.connect_to_local(
headers={
"X-AWS-Access-Key": "YOUR_ACCESS_KEY",
"X-AWS-Secret-Key": "YOUR_SECRET_KEY",
}
)
try:
reviews = client.collections.get("WineReview")
# instruction for the generative module
generate_prompt = "Describe the following as a Facebook Ad: {review_body}"
response = reviews.generate.near_text(
query="fruity white wine",
single_prompt=generate_prompt,
limit=3
)
for o in response.objects:
print(o.generated) # "Single prompt" generations are attributes of each object
print(o.properties) # To inspect the retrieved object
finally:
client.close()
import weaviate
client = weaviate.Client(
url="https://WEAVIATE_INSTANCE_URL/", # Replace WEAVIATE_INSTANCE_URL with your instance URL
additional_headers={
"X-AWS-Access-Key": "YOUR_ACCESS_KEY",
"X-AWS-Secret-Key": "YOUR_SECRET_KEY",
}
)
# instruction for the generative module
generatePrompt = "Describe the following as a Facebook Ad: {summary}"
result = (
client.query
.get("Article", ["title", "summary"])
.with_generate(single_prompt=generatePrompt)
.with_near_text({
"concepts": ["Italian food"]
})
.with_limit(5)
).do()
print(result)
import weaviate, { ApiKey } from 'weaviate-ts-client';
const client = weaviate.client({
scheme: 'https',
host: 'edu-demo.weaviate.network',
apiKey: new ApiKey('learn-weaviate'),
headers: {
'X-AWS-Access-Key': 'YOUR_ACCESS_KEY',
'X-AWS-Secret-Key': 'YOUR_SECRET_KEY'
},
});
// instruction for the generative module
const generatePrompt = 'Describe the following as a Facebook Ad: {summary}';
const response = await client.graphql
.get()
.withClassName('Article')
.withFields('title summary')
.withNearText({
concepts: ['Italian food'],
})
.withGenerate({
singlePrompt: generatePrompt,
})
.withLimit(5)
.do();
console.log(JSON.stringify(response, null, 2));
package main
import (
"context"
"fmt"
"github.com/weaviate/weaviate-go-client/v4/weaviate"
"github.com/weaviate/weaviate-go-client/v4/weaviate/graphql"
)
func main() {
cfg := weaviate.Config{
Host: "WEAVIATE_INSTANCE_URL", // Replace with your instance URL
Scheme: "https",
Headers: map[string]string{
"X-AWS-Access-Key": "YOUR_ACCESS_KEY",
"X-AWS-Secret-Key": "YOUR_SECRET_KEY",
},
}
client, err := weaviate.NewClient(cfg)
if err != nil {
panic(err)
}
ctx := context.Background()
fields := []graphql.Field{
{Name: "title"},
{Name: "summary"},
}
concepts := []string{"Italian food"}
nearText := client.GraphQL().NearTextArgBuilder().
WithConcepts(concepts)
gs := graphql.NewGenerativeSearch().SingleResult("\"Describe the following as a Facebook Ad: {summary}\"")
result, err := client.GraphQL().Get().
WithClassName("Article").
WithFields(fields...).
WithNearText(nearText).
withGenerativeSearch(generativeSearch).
WithLimit(5).
Do(ctx)
if err != nil {
panic(err)
}
fmt.Printf("%v", result)
}
package io.weaviate;
import java.util.HashMap;
import java.util.Map;
import io.weaviate.client.Config;
import io.weaviate.client.WeaviateClient;
import io.weaviate.client.base.Result;
import io.weaviate.client.v1.graphql.model.GraphQLResponse;
import io.weaviate.client.v1.graphql.query.argument.NearTextArgument;
import io.weaviate.client.v1.graphql.query.fields.Field;
public class App {
public static void main(String[] args) {
Map<String, String> headers = new HashMap<String, String>() { {
put("X-AWS-Access-Key", "YOUR_ACCESS_KEY");
put("X-AWS-Secret-Key", "YOUR_SECRET_KEY");
} };
Config config = new Config("https", "WEAVIATE_INSTANCE_URL", headers);
// Replace with your instance URL
WeaviateClient client = new WeaviateClient(config);
// instruction for the generative module
GenerativeSearchBuilder generativeSearch = GenerativeSearchBuilder.builder()
.singleResultPrompt("\"Describe the following as a Facebook Ad: {summary}\"")
.build();
Field title = Field.builder().name("title").build();
Field summary = Field.builder().name("summary").build();
NearTextArgument nearText = client.graphQL().arguments().nearTextArgBuilder()
.concepts(new String[]{ "Italian food" })
.build();
Result<GraphQLResponse> result = client.graphQL().get()
.withClassName("Article")
.withFields(title, summary)
.withGenerativeSearch(generativeSearch)
.withNearText(nearText)
.withLimit(5)
.run();
if (result.hasErrors()) {
System.out.println(result.getError());
return;
}
System.out.println(result.getResult());
}
}
{
Get {
Article(
nearText: {
concepts: ["Italian food"]
}
limit: 1
) {
title
summary
_additional {
generate(
singleResult: {
prompt: """
Describe the following as a Facebook Ad: {summary}
"""
}
) {
singleResult
error
}
}
}
}
}
echo '{
"query": "{
Get {
Article(
nearText: {
concepts: [\"Italian food\"]
}
limit: 5
) {
title
summary
_additional {
generate(
singleResult: {
prompt: \"\"\"
Describe the following as a Facebook Ad: {summary}
\"\"\"
}
) {
singleResult
error
}
}
}
}
}
"
}' | curl \
-X POST \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer $WEAVIATE_API_KEY" \
-H "X-AWS-Access-Key: $YOUR_ACCESS_KEY" \
-H "X-AWS-Secret-Key: $YOUR_SECRET_KEY" \
-d @- \
https://WEAVIATE_INSTANCE_URL/v1/graphql # Replace WEAVIATE_INSTANCE_URL with your instance URL
Example response - single result
{
"data": {
"Get": {
"Article": [
{
"_additional": {
"generate": {
"error": null,
"singleResult": "Italian food, as we know it today, might be a relatively modern concept. But it's hard to deny that there's something special about it. It could be the way the pasta tastes or the way the sauce smells. It could be the way the cheese stretches or the way the bread soaks up the sauce. Whatever it is, Italian food has a way of capturing our hearts and our stomachs. So if you're looking for a way to spice up your meal routine, why not try Italian? You might just find that it's your new favorite cuisine."
}
},
"summary": "Even the emoji for pasta isn't just pasta -- it's a steaming plate of spaghetti heaped with tomato sauce on top. But while today we think of tomatoes as inextricably linked to Italian food, that hasn't always been the case. \"People tend to think Italian food was always as it is now -- that Dante was eating pizza,\" says Dr Eva Del Soldato , associate professor of romance languages at the University of Pennsylvania, who leads courses on Italian food history. In fact, she says, Italy's complex history -- it wasn't unified until 1861 -- means that what we think of Italian food is, for the most part, a relatively modern concept. Diego Zancani, emeritus professor of medieval and modern languages at Oxford University and author of \"How We Fell in Love with Italian Food,\" agrees.",
"title": "How this fruit became the star of Italian cooking"
}
]
}
}
}
Example - grouped result
Here is an example of a query where:
- we run a vector search (with
nearText
) to find publications about finance, - then we ask the generator module to explain why these articles are about finance.
- Python (v4)
- Python (v3)
- JavaScript/TypeScript
- Go
- Java
- GraphQL
- Curl
import weaviate
import os
client = weaviate.connect_to_local(
headers={
"X-AWS-Access-Key": "YOUR_ACCESS_KEY",
"X-AWS-Secret-Key": "YOUR_SECRET_KEY",
}
)
try:
reviews = client.collections.get("WineReview")
# instruction for the generative module
generate_prompt = "Explain what occasion these wines might be good for."
response = reviews.generate.near_text(
query="dry red wine",
grouped_task=generate_prompt,
limit=5
)
print(response.generated) # "Grouped task" generations are attributes of the entire response
for o in response.objects:
print(o.properties) # To inspect the retrieved object
finally:
client.close()
import weaviate
client = weaviate.Client(
url = "https://WEAVIATE_INSTANCE_URL/", # Replace WEAVIATE_INSTANCE_URL with your instance URL
additional_headers={
"X-AWS-Access-Key": "YOUR_ACCESS_KEY",
"X-AWS-Secret-Key": "YOUR_SECRET_KEY",
}
)
# instruction for the generative module
generateTask = "Explain why these magazines or newspapers are about finance"
result = (
client.query
.get("Publication", ["name"])
.with_generate(grouped_task=generateTask)
.with_near_text({
"concepts": ["magazine or newspaper about finance"]
})
.with_limit(5)
).do()
print(result)
import weaviate, { ApiKey } from 'weaviate-ts-client';
const client = weaviate.client({
scheme: 'https',
host: 'edu-demo.weaviate.network',
apiKey: new ApiKey('learn-weaviate'),
headers: {
'X-AWS-Access-Key': 'YOUR_ACCESS_KEY',
'X-AWS-Secret-Key': 'YOUR_SECRET_KEY'
},
});
// instruction for the generative module
const generateTask = 'Explain why these magazines or newspapers are about finance';
const response = await client.graphql
.get()
.withClassName('Article')
.withFields('title')
.withNearText({
concepts: ['magazine or newspaper about finance'],
})
.withGenerate({
groupedTask: generateTask,
})
.withLimit(5)
.do();
console.log(JSON.stringify(response, null, 2));
package main
import (
"context"
"fmt"
"github.com/weaviate/weaviate-go-client/v4/weaviate"
"github.com/weaviate/weaviate-go-client/v4/weaviate/graphql"
)
func main() {
cfg := weaviate.Config{
Host: "WEAVIATE_INSTANCE_URL", // Replace with your instance URL
Scheme: "https",
Headers: map[string]string{
"X-AWS-Access-Key": "YOUR_ACCESS_KEY",
"X-AWS-Secret-Key": "YOUR_SECRET_KEY",
},
}
client, err := weaviate.NewClient(cfg)
if err != nil {
panic(err)
}
ctx := context.Background()
name := graphql.Field{Name: "name"}
concepts := []string{"magazine or newspaper about finance"}
nearText := client.GraphQL().NearTextArgBuilder().
WithConcepts(concepts)
gs := graphql.NewGenerativeSearch().GroupedResult("Explain why these magazines or newspapers are about finance")
result, err := client.GraphQL().Get().
WithClassName("Publication").
WithFields(name).
WithGenerativeSearch(gs).
WithNearText(nearText).
WithLimit(5).
Do(ctx)
if err != nil {
panic(err)
}
fmt.Printf("%v", result)
}
package io.weaviate;
import java.util.HashMap;
import java.util.Map;
import io.weaviate.client.Config;
import io.weaviate.client.WeaviateClient;
import io.weaviate.client.base.Result;
import io.weaviate.client.v1.graphql.model.GraphQLResponse;
import io.weaviate.client.v1.graphql.query.argument.NearTextArgument;
import io.weaviate.client.v1.graphql.query.fields.Field;
public class App {
public static void main(String[] args) {
Map<String, String> headers = new HashMap<String, String>() { {
put("X-AWS-Access-Key", "YOUR_ACCESS_KEY");
put("X-AWS-Secret-Key", "YOUR_SECRET_KEY");
} };
Config config = new Config("https", "WEAVIATE_INSTANCE_URL", headers);
// Replace with your instance URL
WeaviateClient client = new WeaviateClient(config);
// instruction for the generative module
GenerativeSearchBuilder generativeSearch = GenerativeSearchBuilder.builder()
.groupedResultTask("Explain why these magazines or newspapers are about finance")
.build();
Field name = Field.builder().name("name").build();
NearTextArgument nearText = client.graphQL().arguments().nearTextArgBuilder()
.concepts(new String[]{ "magazine or newspaper about finance" })
.build();
Result<GraphQLResponse> result = client.graphQL().get()
.withClassName("Publication")
.withFields(name)
.withGenerativeSearch(generativeSearch)
.withNearText(nearText)
.withLimit(5)
.run();
if (result.hasErrors()) {
System.out.println(result.getError());
return;
}
System.out.println(result.getResult());
}
}
{
Get {
Publication(
nearText: {
concepts: ["magazine or newspaper about finance"]
certainty: 0.75
}
) {
name
_additional {
generate(
groupedResult: {
task: "Explain why these magazines or newspapers are about finance"
}
) {
groupedResult
error
}
}
}
}
}
echo '{
"query": "{
Get {
Publication(
nearText: {
concepts: [\"magazine or newspaper about finance\"]
}
limit: 5
) {
name
_additional {
generate(
groupedResult: {
task: \"Explain why these magazines or newspapers are about finance\"
}
) {
groupedResult
error
}
}
}
}
}
"
}' | curl \
-X POST \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer $WEAVIATE_API_KEY" \
-H "X-AWS-Access-Key: $YOUR_ACCESS_KEY" \
-H "X-AWS-Secret-Key: $YOUR_SECRET_KEY" \
-d @- \
https://WEAVIATE_INSTANCE_URL/v1/graphql # Replace WEAVIATE_INSTANCE_URL with your instance URL
Example response - grouped result
{
"data": {
"Get": {
"Publication": [
{
"_additional": {
"generate": {
"error": null,
"groupedResult": "These magazines or newspapers are about finance because they cover topics related to finance, such as business news, financial markets, and economic trends. They also often feature articles about personal finance, such as investing, budgeting, and retirement planning."
}
},
"name": "Financial Times"
},
{
"_additional": {
"generate": null
},
"name": "Wall Street Journal"
},
{
"_additional": {
"generate": null
},
"name": "The New York Times Company"
}
]
}
}
}
Additional information
Supported models
You can use any of the following models with generative-aws
:
- cohere.command-text-v14
- cohere.command-light-text-v14
Questions and feedback
If you have any questions or feedback, let us know in our user forum.