Generative Search - Mistral
In short
v1.24.2
- The Generative Mistral (
generative-mistral
) module performs retrieval augmented generation, or RAG, based on the data stored in your Weaviate instance. - The module can generate a response for each object returned from Weaviate, or a combined response for a group of objects.
- The module enables generative search operations on the Weaviate instance.
- The default model is
open-mistral-7b
. - The module requires an API key for Mistral inference endpoints to perform the generation task.
Introduction
generative-mistral
performs retrieval augmented generation, or RAG, based on the data stored in your Weaviate instance.
The module works in two steps:
- (Weaviate) Run a search query in Weaviate to find relevant objects.
- (Mistral Inference API) Use a Large Language Model to generate a response based on the results (from the previous step) and the provided prompt or task.
You can use the Generative Mistral module with any other upstream modules. For example, you could use text2vec-cohere
, text2vec-huggingface
or text2vec-openai
to vectorize and query your data, but then rely on the generative-mistral
module to generate a response.
The generative module can perform RAG for:
- each returned object -
singlePrompt
- the group of all results together –
groupedTask
You need to input both a query and a prompt (for individual responses) or a task (for all responses).
Weaviate instance configuration
If you use Weaviate Cloud Services (WCS), this module is already enabled and pre-configured. You cannot edit the configuration in WCS.
Docker Compose file
To use generative-mistral
, you must enable it in your Docker Compose file (docker-compose.yml
). You can do so manually, or create one using the Weaviate configuration tool.
Parameters
ENABLE_MODULES
(Required): The modules to enable. Includegenerative-mistral
to enable the module.MISTRAL_APIKEY
Your Mistral API key. You can also provide the key at query time.
Example
This configuration enables generative-mistral
and sets the Mistral authentication credentials.
---
version: '3.4'
services:
weaviate:
command:
- --host
- 0.0.0.0
- --port
- '8080'
- --scheme
- http
image: cr.weaviate.io/semitechnologies/weaviate:1.24.10
ports:
- 8080:8080
- 50051:50051
volumes:
- weaviate_data:/var/lib/weaviate
restart: on-failure:0
environment:
QUERY_DEFAULTS_LIMIT: 25
AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
PERSISTENCE_DATA_PATH: '/var/lib/weaviate'
MISTRAL_APIKEY: escecret-foobar # Optional. Can be set at query time.
ENABLE_MODULES: 'text2vec-cohere,generative-mistral' # Can include any modules
CLUSTER_HOSTNAME: 'node1'
volumes:
weaviate_data:
...
Class configuration
You can configure how the module will behave in each class through the Weaviate schema.
API settings
Parameters
Parameter | Required | Default | Purpose |
---|---|---|---|
model | No | "open-mistral-7b" | The model to use. |
temperature | No | 0 | Control of LLM stochasticity. |
maxTokens | No | 2048 | Maximum number of tokens to generate. |
Supported models
You can use any of the following models with generative-mistral
:
open-mistral-7b
(akamistral-tiny-2312
)open-mixtral-8x7b
(akamistral-small-2312
)mistral-tiny
mistral-small
mistral-small-latest
(akamistral-small-2402
)mistral-medium
mistral-medium-latest
(akamistral-medium-2312
)mistral-large
mistral-large-latest
(akamistral-large-2402
)
Example
The following example configures the Document
class to use the generative-mistral
module with the Document
class, with the mistral-medium-latest
model.
{
"classes": [
{
"class": "Document",
"description": "A class called document",
...,
"moduleConfig": {
"generative-mistral": {
"model": "mistral-medium-latest",
},
}
}
]
}
New to Weaviate Schemas?
If you are new to Weaviate, check out the Weaviate schema tutorial.
How to use
Query-time parameters
You can supply parameters at query time by adding them to the HTTP header.
HTTP Header | Value | Purpose | Note |
---|---|---|---|
"X-Mistral-Api-Key" | Your Mistral API key. | Authentication | Learn more |
Queries
This module enables generative search queries.
generate
takes the following arguments:
Field | Data Type | Required | Example | Description |
---|---|---|---|---|
singleResult {prompt} | string | no | Summarize the following in a tweet: {summary} | Generates a response for each individual search result. You need to include at least one result field in the prompt, between braces. |
groupedResult {task} | string | no | Explain why these results are similar to each other | Generates a single response for all search results |
Example of properties in the prompt
When piping the results to the prompt, at least one field returned by the query must be added to the prompt. If you don't add any fields, Weaviate will throw an error.
For example, assume your schema looks like this:
{
Article {
title
summary
}
}
You can add both title
and summary
to the prompt by enclosing them in curly brackets:
{
Get {
Article {
title
summary
_additional {
generate(
singleResult: {
prompt: """
Summarize the following in a tweet:
{title} - {summary}
"""
}
) {
singleResult
error
}
}
}
}
}
Example - single result
Here is an example of a query where:
- we get a podcast clip (with limit 1)
- then we ask the generator module to summarize the content into one sentence.
- the query asks for the
speaker
andcontent
fields, which are then included in theprompt
argument of thegenerate
operator.
- the query asks for the
- GraphQL
- Python
- JavaScript/TypeScript
- Go
- Java
- Curl
{
Get {
PodClip (limit:1){
speaker
content
_additional {
generate(
singleResult:{
prompt: """
Please summarize the following podcast clip into one sentence.
Speaker: {speaker} said {content}.
"""
}
){
singleResult
error
}
}
}
}
}
import weaviate
client = weaviate.Client(
url = "https://WEAVIATE_INSTANCE_URL/", # Replace WEAVIATE_INSTANCE_URL with your instance URL
additional_headers={
"X-Mistral-Api-Key": "YOUR-MISTRAL-API-KEY"
}
)
# instruction for the generative module
generatePrompt = "Please summarize the following podcast clip into one sentence. Speaker: {speaker} said {content}."
result = (
client.query
.get("PodClip", ["speaker", "content"])
.with_generate(single_prompt=generatePrompt)
.with_limit(1)
).do()
print(result)
import weaviate, { ApiKey } from 'weaviate-ts-client';
const client = weaviate.client({
scheme: 'https',
host: 'edu-demo.weaviate.network',
apiKey: new ApiKey('learn-weaviate'),
headers: { 'X-Mistral-Api-Key': process.env['MISTRAL_API_KEY'] },
});
// instruction for the generative module
const generatePrompt = 'Please summarize the following podcast clip into one sentence. Speaker: {speaker} said {content}.';
const response = await client.graphql
.get()
.withClassName('PodClip')
.withFields('speaker content')
.withGenerate({
singlePrompt: generatePrompt,
})
.withLimit(1)
.do();
console.log(JSON.stringify(response, null, 2));
package main
import (
"context"
"fmt"
"github.com/weaviate/weaviate-go-client/v4/weaviate"
"github.com/weaviate/weaviate-go-client/v4/weaviate/graphql"
)
func main() {
cfg := weaviate.Config{
Host: "WEAVIATE_INSTANCE_URL", // Replace with your instance URL
Scheme: "https",
Headers: map[string]string{"X-Mistral-Api-Key": "YOUR-MISTRAL-API-KEY"},
}
client, err := weaviate.NewClient(cfg)
if err != nil {
panic(err)
}
ctx := context.Background()
fields := []graphql.Field{
{Name: "speaker"},
{Name: "content"},
}
gs := graphql.NewGenerativeSearch().SingleResult("\"Please summarize the following podcast clip into one sentence. Speaker: {speaker} said {content}.\"")
result, err := client.GraphQL().Get().
WithClassName("PodClip").
WithFields(fields...).
withGenerativeSearch(generativeSearch).
WithLimit(1).
Do(ctx)
if err != nil {
panic(err)
}
fmt.Printf("%v", result)
}
package io.weaviate;
import java.util.HashMap;
import java.util.Map;
import io.weaviate.client.Config;
import io.weaviate.client.WeaviateClient;
import io.weaviate.client.base.Result;
import io.weaviate.client.v1.graphql.model.GraphQLResponse;
import io.weaviate.client.v1.graphql.query.argument.NearTextArgument;
import io.weaviate.client.v1.graphql.query.fields.Field;
public class App {
public static void main(String[] args) {
Map<String, String> headers = new HashMap<String, String>() {
{put("X-Cohere-Api-Key", "YOUR-COHERE-API-KEY");}
};
Config config = new Config("https", "WEAVIATE_INSTANCE_URL", headers);
// Replace with your instance URL
WeaviateClient client = new WeaviateClient(config);
// instruction for the generative module
GenerativeSearchBuilder generativeSearch = GenerativeSearchBuilder.builder()
.singleResultPrompt("\"Please summarize the following podcast clip into one sentence. Speaker: {speaker} said {content}.\"")
.build();
Field speaker = Field.builder().name("speaker").build();
Field content = Field.builder().name("content").build();
Result<GraphQLResponse> result = client.graphQL().get()
.withClassName("PodClip")
.withFields(speaker, content)
.withGenerativeSearch(generativeSearch)
.withLimit(1)
.run();
if (result.hasErrors()) {
System.out.println(result.getError());
return;
}
System.out.println(result.getResult());
}
}
echo '{
"query": "{
Get {
PodClip(
limit: 1
) {
title
summary
_additional {
generate(
singleResult: {
prompt: \"\"\"
Please summarize the following podcast clip into one sentence.
Speaker: {speaker} said {content}.
\"\"\"
}
) {
singleResult
error
}
}
}
}
}
"
}' | curl \
-X POST \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer $WEAVIATE_API_KEY" \
-H "X-Mistral-Api-Key: $MISTRAL_API_KEY" \
-d @- \
https://WEAVIATE_INSTANCE_URL/v1/graphql # Replace WEAVIATE_INSTANCE_URL with your instance URL
Example - grouped result
Here is an example of a query where:
- we run a vector search (with
nearText
) to find podcast clips semantically similar to"What is ref2vec?"
- then we ask the generator module to answer the question:
"What is ref2vec?"
based on the search results.
- GraphQL
- Python
- JavaScript/TypeScript
- Go
- Java
- Curl
{
Get {
PodClip(
nearText: {
concepts: ["What is ref2vec?"]
},
limit: 5
) {
speaker
content
_additional {
generate(
groupedResult: {
task: "Please answer the question: What is ref2vec? based on the following search results. IMPORTANT! Please make sure your answer is supported by the information in the search results and provide references to the provided content when possible."
}
) {
groupedResult
error
}
}
}
}
}
import weaviate
client = weaviate.Client(
url = "https://WEAVIATE_INSTANCE_URL/", # Replace WEAVIATE_INSTANCE_URL with your instance URL
additional_headers={
"X-Mistral-Api-Key": "YOUR-MISTRAL-API-KEY"
}
)
# instruction for the generative module
generateTask = "Please answer the question: What is ref2vec? based on the following search results. IMPORTANT! Please make sure your answer is supported by the information in the search results and provide references to the provided content when possible."
result = (
client.query
.get("PodClip", ["speaker", "content"])
.with_generate(grouped_task=generateTask)
.with_near_text({
"concepts": ["What is ref2vec?"]
})
.with_limit(5)
).do()
print(result)
import weaviate, { ApiKey } from 'weaviate-ts-client';
const client = weaviate.client({
scheme: 'https',
host: 'edu-demo.weaviate.network',
apiKey: new ApiKey('learn-weaviate'),
headers: { 'X-Mistral-Api-Key': process.env['MISTRAL_API_KEY'] },
});
// instruction for the generative module
const generateTask = 'Please answer the question: What is ref2vec? based on the following search results. IMPORTANT! Please make sure your answer is supported by the information in the search results and provide references to the provided content when possible.';
const response = await client.graphql
.get()
.withClassName('PodClip')
.withFields('speaker content')
.withNearText({
concepts: ['What is ref2vec?'],
})
.withGenerate({
groupedTask: generateTask,
})
.withLimit(5)
.do();
console.log(JSON.stringify(response, null, 2));
package main
import (
"context"
"fmt"
"github.com/weaviate/weaviate-go-client/v4/weaviate"
"github.com/weaviate/weaviate-go-client/v4/weaviate/graphql"
)
func main() {
cfg := weaviate.Config{
Host: "WEAVIATE_INSTANCE_URL", // Replace with your instance URL
Scheme: "https",
Headers: map[string]string{"X-Mistral-Api-Key": "YOUR-MISTRAL-API-KEY"},
}
client, err := weaviate.NewClient(cfg)
if err != nil {
panic(err)
}
ctx := context.Background()
fields := []graphql.Field{
{Name: "speaker"},
{Name: "content"},
}
concepts := []string{"What is ref2vec?"}
nearText := client.GraphQL().NearTextArgBuilder().
WithConcepts(concepts)
gs := graphql.NewGenerativeSearch().GroupedResult("Please answer the question: What is ref2vec? based on the following search results. IMPORTANT! Please make sure your answer is supported by the information in the search results and provide references to the provided content when possible.")
result, err := client.GraphQL().Get().
WithClassName("PodClip").
WithFields(fields...).
WithGenerativeSearch(gs).
WithNearText(nearText).
WithLimit(5).
Do(ctx)
if err != nil {
panic(err)
}
fmt.Printf("%v", result)
}
package io.weaviate;
import java.util.HashMap;
import java.util.Map;
import io.weaviate.client.Config;
import io.weaviate.client.WeaviateClient;
import io.weaviate.client.base.Result;
import io.weaviate.client.v1.graphql.model.GraphQLResponse;
import io.weaviate.client.v1.graphql.query.argument.NearTextArgument;
import io.weaviate.client.v1.graphql.query.fields.Field;
public class App {
public static void main(String[] args) {
Map<String, String> headers = new HashMap<String, String>() { {
put("X-Mistral-Api-Key", "YOUR-MISTRAL-API-KEY");
} };
Config config = new Config("https", "WEAVIATE_INSTANCE_URL", headers);
// Replace with your instance URL
WeaviateClient client = new WeaviateClient(config);
// instruction for the generative module
GenerativeSearchBuilder generativeSearch = GenerativeSearchBuilder.builder()
.groupedResultTask("Please answer the question: What is ref2vec? based on the following search results. IMPORTANT! Please make sure your answer is supported by the information in the search results and provide references to the provided content when possible.")
.build();
Field speaker = Field.builder().name("speaker").build();
Field content = Field.builder().name("content").build();
NearTextArgument nearText = client.graphQL().arguments().nearTextArgBuilder()
.concepts(new String[]{ "What is ref2vec?" })
.build();
Result<GraphQLResponse> result = client.graphQL().get()
.withClassName("PodClip")
.withFields(speaker, content)
.withGenerativeSearch(generativeSearch)
.withNearText(nearText)
.withLimit(5)
.run();
if (result.hasErrors()) {
System.out.println(result.getError());
return;
}
System.out.println(result.getResult());
}
}
echo '{
"query": "{
Get {
PodClip(
nearText: {
concepts: [\"What is ref2vec?\"]
}
limit: 5
) {
speaker
content
_additional {
generate(
groupedResult: {
task: \"Please answer the question: What is ref2vec? based on the following search results. IMPORTANT! Please make sure your answer is supported by the information in the search results and provide references to the provided content when possible.\"
}
) {
groupedResult
error
}
}
}
}
}
"
}' | curl \
-X POST \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer $WEAVIATE_API_KEY" \
-H "X-Mistral-Api-Key: $MISTRAL_API_KEY" \
-d @- \
https://WEAVIATE_INSTANCE_URL/v1/graphql # Replace WEAVIATE_INSTANCE_URL with your instance URL
Further examples
For further usage examples, please see the how-to search: generative page.
Questions and feedback
If you have any questions or feedback, let us know in our user forum.