Generative Search - Cohere
In shortโ
- The Generative Cohere (
generative-cohere
) module generates responses based on the data stored in your Weaviate instance. - The module can generate a response for each object returned from Weaviate, or a combined response for a group of objects.
- The module adds a
generate {}
parameter to the GraphQL_additional {}
property of theGet {}
queries - Added in Weaviate
v1.19.0
- The default model is
command-xlarge-nightly
, which the Cohere team trains nightly and pushes updates.
Introductionโ
generative-cohere
generates responses based on the data stored in your Weaviate instance.
The module works in two steps:
- (Weaviate) Run a search query in Weaviate to find relevant objects.
- (Cohere) Use a Cohere Large Language Model to generate a response based on the results (from the previous step) and the provided prompt or task.
You can use the Generative Cohere module with non-Cohere upstream modules. For example, you could use text2vec-openai
or text2vec-huggingface
to vectorize and query your data, but then rely on the generative-cohere
module to generate a response.
The generative module can provide results for:
- each returned object -
singleResult{ prompt }
- the group of all results together โ
groupedResult{ task }
You need to input both a query and a prompt (for individual responses) or a task (for all responses).
Cohere API keyโ
generative-cohere
requires an Cohere API key to perform the generation task.
Providing the key to Weaviateโ
You can provide your Cohere API key in two ways:
During the configuration of your Docker instance, by adding
COHERE_APIKEY
underenvironment
to yourdocker-compose
file, like this:environment:
COHERE_APIKEY: 'your-key-goes-here'
...At run-time (recommended), by providing
"X-Cohere-Api-Key"
to the Weaviate client, like this:
- Python
- JavaScript
- Go
- Java
import weaviate
client = weaviate.Client(
url = "https://some-endpoint.weaviate.network/",
additional_headers = {
"X-Cohere-Api-Key": "YOUR-COHERE-API-KEY" # Replace with your API key
}
)
const weaviate = require('weaviate-ts-client');
const client = weaviate.client({
scheme: 'https',
host: 'some-endpoint.weaviate.network',
// Replace with your API key
headers: {'X-Cohere-Api-Key': 'YOUR-COHERE-API-KEY'},
});
package main
import (
"context"
"fmt"
"github.com/weaviate/weaviate-go-client/v4/weaviate"
"github.com/weaviate/weaviate/entities/models"
)
func main() {
cfg := weaviate.Config{
Host: "some-endpoint.weaviate.network/", // Replace with your endpoint
Scheme: "https",
// Replace with your API key
Headers: map[string]string{"X-Cohere-Api-Key": "YOUR-COHERE-API-KEY"}
}
client, err := weaviate.NewClient(cfg)
if err != nil {
panic(err)
}
}
package io.weaviate;
import java.util.ArrayList;
import io.weaviate.client.Config;
import io.weaviate.client.WeaviateClient;
import io.weaviate.client.base.Result;
public class App {
public static void main(String[] args) {
Map<String, String> headers = new HashMap<String, String>() { {
// Replace with your API key
put("X-Cohere-Api-Key", "YOUR-COHERE-API-KEY");
} };
Config config = new Config("https", "some-endpoint.weaviate.network/", headers);
WeaviateClient client = new WeaviateClient(config);
}
}
Module configurationโ
This module is enabled and pre-configured on Weaviate Cloud Services.
Your Weaviate instance must be on 1.19.0
or newer.
If your instance is older than 1.19.0
then you need to migrate or upgrade it to a newer version.
Configuration file (Weaviate open source only)โ
You can enable the Generative Cohere module in your configuration file (e.g. docker-compose.yaml
). Add the generative-cohere
module (alongside any other module you may need) to the ENABLE_MODULES
property, like this:
ENABLE_MODULES: 'text2vec-cohere,generative-cohere'
Here is a full example of a Docker configuration, which uses the generative-cohere
module in combination with text2vec-cohere
:
---
version: '3.4'
services:
weaviate:
command:
- --host
- 0.0.0.0
- --port
- '8080'
- --scheme
- http
image:
semitechnologies/weaviate:1.19.6
ports:
- 8080:8080
restart: on-failure:0
environment:
QUERY_DEFAULTS_LIMIT: 25
AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
PERSISTENCE_DATA_PATH: '/var/lib/weaviate'
DEFAULT_VECTORIZER_MODULE: 'text2vec-cohere'
ENABLE_MODULES: 'text2vec-cohere,generative-cohere'
COHERE_APIKEY: sk-foobar # this parameter is optional, as you can also provide it through the client
CLUSTER_HOSTNAME: 'node1'
Schema configurationโ
In your Weaviate schema, you can define settings for this module.
For example, the following schema configuration will set Weaviate to use the generative-cohere
module with the Document
class, with the command-xlarge-nightly
model. You can also configure additional parameters for the Cohere endpoint through the parameters shown below. Other models you can use from Cohere are command-xlarge-beta
and command-xlarge
.
{
"classes": [
{
"class": "Document",
"description": "A class called document",
...,
"moduleConfig": {
"generative-cohere": {
"model": "command-xlarge-nightly", // Optional - Defaults to `command-xlarge-nightly`. Can also use`command-xlarge-beta` and `command-xlarge`
"temperatureProperty": <temperature>, // Optional
"maxTokensProperty": <maxTokens>, // Optional
"kProperty": <k>, // Optional
"stopSequencesProperty": <stopSequences>, // Optional
"returnLikelihoodsProperty": <returnLikelihoods>, // Optional
}
}
}
]
}
New to Weaviate Schemas?
If you are new to Weaviate, check out the Weaviate schema tutorial.
How to useโ
This module extends the _additional {...}
property with a generate
operator.
generate
takes the following arguments:
Field | Data Type | Required | Example | Description |
---|---|---|---|---|
singleResult {prompt} | string | no | Summarize the following in a tweet: {summary} | Generates a response for each individual search result. You need to include at least one result field in the prompt, between braces. |
groupedResult {task} | string | no | Explain why these results are similar to each other | Generates a single response for all search results |
Example of properties in the promptโ
When piping the results to the prompt, at least one field returned by the query must be added to the prompt. If you don't add any fields, Weaviate will throw an error.
For example, assume your schema looks like this:
{
Article {
title
summary
}
}
You can add both title
and summary
to the prompt by enclosing them in curly brackets:
{
Get {
Article {
title
summary
_additional {
generate(
singleResult: {
prompt: """
Summarize the following in a tweet:
{title} - {summary}
"""
}
) {
singleResult
error
}
}
}
}
}
Example - single resultโ
Here is an example of a query where:
- we run a vector search (with
nearText
) to find articles about "Italian food" - then we ask the generator module to describe each result as a Facebook ad.
- the query asks for the
summary
field, which it then includes in theprompt
argument of thegenerate
operator.
- the query asks for the
- GraphQL
- Python
- JavaScript
- TypeScript
- Go
- Java
- Curl
{
Get {
Article(
nearText: {
concepts: ["Italian food"]
}
limit: 1
) {
title
summary
_additional {
generate(
singleResult: {
prompt: """
Describe the following as a Facebook Ad: {summary}
"""
}
) {
singleResult
error
}
}
}
}
}
import weaviate
client = weaviate.Client(
url = "https://some-endpoint.weaviate.network/",
additional_headers={
"X-Cohere-Api-Key": "YOUR-COHERE-API-KEY"
}
)
# instruction for the generative module
generatePrompt = "Describe the following as a Facebook Ad: {summary}"
result = (
client.query
.get("Article", ["title", "summary"])
.with_generate(single_prompt=generatePrompt)
.with_near_text({
"concepts": ["Italian food"]
})
.with_limit(5)
).do()
print(result)
const weaviate = require('weaviate-ts-client');
const client = weaviate.client({
scheme: 'https',
host: 'some-endpoint.weaviate.network',
headers: {'X-Cohere-Api-Key': 'YOUR-COHERE-API-KEY'},
});
// instruction for the generative module
const generatePrompt = 'Describe the following as a Facebook Ad: {summary}';
client.graphql
.get()
.withClassName('Article')
.withFields('title summary')
.withNearText({
concepts: ['Italian food']
})
.withGenerate({
singlePrompt: generatePrompt,
})
.withLimit(5)
.do()
.then(res => {
console.log(res)
})
.catch(err => {
console.error(err)
});
import weaviate, { WeaviateClient } from 'weaviate-ts-client';
const client: WeaviateClient = weaviate.client({
scheme: 'https',
host: 'some-endpoint.weaviate.network',
headers: {'X-Cohere-Api-Key': 'YOUR-COHERE-API-KEY'},
});
// instruction for the generative module
const generatePrompt = 'Describe the following as a Facebook Ad: {summary}';
client.graphql
.get()
.withClassName('Article')
.withFields('title summary')
.withNearText({
concepts: ['Italian food']
})
.withGenerate({
singlePrompt: generatePrompt,
})
.withLimit(5)
.do()
.then((res: any) => {
console.log(res)
})
.catch((err: Error) => {
console.error(err)
});
package main
import (
"context"
"fmt"
"github.com/weaviate/weaviate-go-client/v4/weaviate"
"github.com/weaviate/weaviate-go-client/v4/weaviate/graphql"
)
func main() {
cfg := weaviate.Config{
Host: "some-endpoint.weaviate.network",
Scheme: "https",
Headers: map[string]string{"X-Cohere-Api-Key": "YOUR-COHERE-API-KEY"},
}
client, err := weaviate.NewClient(cfg)
if err != nil {
panic(err)
}
ctx := context.Background()
fields := []graphql.Field{
{Name: "title"},
{Name: "summary"},
}
concepts := []string{"Italian food"}
nearText := client.GraphQL().NearTextArgBuilder().
WithConcepts(concepts)
gs := graphql.NewGenerativeSearch().SingleResult("\"Describe the following as a Facebook Ad: {summary}\"")
result, err := client.GraphQL().Get().
WithClassName("Article").
WithFields(fields...).
WithNearText(nearText).
withGenerativeSearch(generativeSearch).
WithLimit(5).
Do(ctx)
if err != nil {
panic(err)
}
fmt.Printf("%v", result)
}
package io.weaviate;
import java.util.HashMap;
import java.util.Map;
import io.weaviate.client.Config;
import io.weaviate.client.WeaviateClient;
import io.weaviate.client.base.Result;
import io.weaviate.client.v1.graphql.model.GraphQLResponse;
import io.weaviate.client.v1.graphql.query.argument.NearTextArgument;
import io.weaviate.client.v1.graphql.query.fields.Field;
public class App {
public static void main(String[] args) {
Map<String, String> headers = new HashMap<String, String>() {
{put("X-Cohere-Api-Key", "YOUR-COHERE-API-KEY");}
};
Config config = new Config("https", "some-endpoint.weaviate.network", headers);
WeaviateClient client = new WeaviateClient(config);
// instruction for the generative module
GenerativeSearchBuilder generativeSearch = GenerativeSearchBuilder.builder()
.singleResultPrompt("\"Describe the following as a Facebook Ad: {summary}\"")
.build();
Field title = Field.builder().name("title").build();
Field summary = Field.builder().name("summary").build();
NearTextArgument nearText = client.graphQL().arguments().nearTextArgBuilder()
.concepts(new String[]{ "Italian food" })
.build();
Result<GraphQLResponse> result = client.graphQL().get()
.withClassName("Article")
.withFields(title, summary)
.withGenerativeSearch(generativeSearch)
.withNearText(nearText)
.withLimit(5)
.run();
if (result.hasErrors()) {
System.out.println(result.getError());
return;
}
System.out.println(result.getResult());
}
}
$ echo '{
"query": "{
Get {
Article(
nearText: {
concepts: [\"Italian food\"]
}
limit: 5
) {
title
summary
_additional {
generate(
singleResult: {
prompt: \"\"\"
Describe the following as a Facebook Ad: {summary}
\"\"\"
}
) {
singleResult
error
}
}
}
}
}
"
}' | tr -d "\n" | curl \
-X POST \
-H 'Content-Type: application/json' \
-H "X-Cohere-Api-Key: YOUR-COHERE-API-KEY" \
-d @- \
https://some-endpoint.weaviate.network/v1/graphql
Example response - single resultโ
{
"data": {
"Get": {
"Article": [
{
"_additional": {
"generate": {
"error": null,
"singleResult": "Italian food, as we know it today, might be a relatively modern concept. But it's hard to deny that there's something special about it. It could be the way the pasta tastes or the way the sauce smells. It could be the way the cheese stretches or the way the bread soaks up the sauce. Whatever it is, Italian food has a way of capturing our hearts and our stomachs. So if you're looking for a way to spice up your meal routine, why not try Italian? You might just find that it's your new favorite cuisine."
}
},
"summary": "Even the emoji for pasta isn't just pasta -- it's a steaming plate of spaghetti heaped with tomato sauce on top. But while today we think of tomatoes as inextricably linked to Italian food, that hasn't always been the case. \"People tend to think Italian food was always as it is now -- that Dante was eating pizza,\" says Dr Eva Del Soldato , associate professor of romance languages at the University of Pennsylvania, who leads courses on Italian food history. In fact, she says, Italy's complex history -- it wasn't unified until 1861 -- means that what we think of Italian food is, for the most part, a relatively modern concept. Diego Zancani, emeritus professor of medieval and modern languages at Oxford University and author of \"How We Fell in Love with Italian Food,\" agrees.",
"title": "How this fruit became the star of Italian cooking"
}
]
}
}
}
Example - grouped resultโ
Here is an example of a query where:
- we run a vector search (with
nearText
) to find publications about finance, - then we ask the generator module to explain why these articles are about finance.
- GraphQL
- Python
- JavaScript
- TypeScript
- Go
- Java
- Curl
{
Get {
Publication(
nearText: {
concepts: ["magazine or newspaper about finance"]
certainty: 0.75
}
) {
name
_additional {
generate(
groupedResult: {
task: "Explain why these magazines or newspapers are about finance"
}
) {
groupedResult
error
}
}
}
}
}
import weaviate
client = weaviate.Client(
url = "https://some-endpoint.weaviate.network/",
additional_headers={
"X-Cohere-Api-Key": "YOUR-COHERE-API-KEY"
}
)
# instruction for the generative module
generateTask = "Explain why these magazines or newspapers are about finance"
result = (
client.query
.get("Publication", ["name"])
.with_generate(grouped_task=generateTask)
.with_near_text({
"concepts": ["magazine or newspaper about finance"]
})
.with_limit(5)
).do()
print(result)
const weaviate = require('weaviate-ts-client');
const client = weaviate.client({
scheme: 'https',
host: 'some-endpoint.weaviate.network',
headers: {'X-Cohere-Api-Key': 'YOUR-COHERE-API-KEY'},
});
// instruction for the generative module
const generateTask = 'Explain why these magazines or newspapers are about finance';
client.graphql
.get()
.withClassName('Article')
.withFields('name')
.withNearText({
concepts: ['magazine or newspaper about finance']
})
.withGenerate({
groupedTask: generateTask,
})
.withLimit(5)
.do()
.then(res => {
console.log(res)
})
.catch(err => {
console.error(err)
});
import weaviate, { WeaviateClient } from 'weaviate-ts-client';
const client: WeaviateClient = weaviate.client({
scheme: 'https',
host: 'some-endpoint.weaviate.network',
headers: {'X-Cohere-Api-Key': 'YOUR-COHERE-API-KEY'},
});
// instruction for the generative module
const generateTask = 'Explain why these magazines or newspapers are about finance';
client.graphql
.get()
.withClassName('Article')
.withFields('name')
.withNearText({
concepts: ['magazine or newspaper about finance']
})
.withGenerate({
groupedTask: generateTask,
})
.withLimit(5)
.do()
.then((res: any) => {
console.log(res)
})
.catch((err: Error) => {
console.error(err)
});
package main
import (
"context"
"fmt"
"github.com/weaviate/weaviate-go-client/v4/weaviate"
"github.com/weaviate/weaviate-go-client/v4/weaviate/graphql"
)
func main() {
cfg := weaviate.Config{
Host: "some-endpoint.weaviate.network",
Scheme: "https",
Headers: map[string]string{"X-Cohere-Api-Key": "YOUR-COHERE-API-KEY"},
}
client, err := weaviate.NewClient(cfg)
if err != nil {
panic(err)
}
ctx := context.Background()
name := graphql.Field{Name: "name"}
concepts := []string{"magazine or newspaper about finance"}
nearText := client.GraphQL().NearTextArgBuilder().
WithConcepts(concepts)
gs := graphql.NewGenerativeSearch().GroupedResult("Explain why these magazines or newspapers are about finance")
result, err := client.GraphQL().Get().
WithClassName("Publication").
WithFields(name).
WithGenerativeSearch(gs).
WithNearText(nearText).
WithLimit(5).
Do(ctx)
if err != nil {
panic(err)
}
fmt.Printf("%v", result)
}
package io.weaviate;
import java.util.HashMap;
import java.util.Map;
import io.weaviate.client.Config;
import io.weaviate.client.WeaviateClient;
import io.weaviate.client.base.Result;
import io.weaviate.client.v1.graphql.model.GraphQLResponse;
import io.weaviate.client.v1.graphql.query.argument.NearTextArgument;
import io.weaviate.client.v1.graphql.query.fields.Field;
public class App {
public static void main(String[] args) {
Map<String, String> headers = new HashMap<String, String>() { {
put("X-Cohere-Api-Key", "YOUR-COHERE-API-KEY");
} };
Config config = new Config("https", "some-endpoint.weaviate.network", headers);
WeaviateClient client = new WeaviateClient(config);
// instruction for the generative module
GenerativeSearchBuilder generativeSearch = GenerativeSearchBuilder.builder()
.groupedResultTask("Explain why these magazines or newspapers are about finance")
.build();
Field name = Field.builder().name("name").build();
NearTextArgument nearText = client.graphQL().arguments().nearTextArgBuilder()
.concepts(new String[]{ "magazine or newspaper about finance" })
.build();
Result<GraphQLResponse> result = client.graphQL().get()
.withClassName("Publication")
.withFields(name)
.withGenerativeSearch(generativeSearch)
.withNearText(nearText)
.withLimit(5)
.run();
if (result.hasErrors()) {
System.out.println(result.getError());
return;
}
System.out.println(result.getResult());
}
}
$ echo '{
"query": "{
Get {
Publication(
nearText: {
concepts: [\"magazine or newspaper about finance\"]
}
limit: 5
) {
name
_additional {
generate(
groupedResult: {
task: \"Explain why these magazines or newspapers are about finance\"
}
) {
groupedResult
error
}
}
}
}
}
"
}' | tr -d "\n" | curl \
-X POST \
-H 'Content-Type: application/json' \
-H "X-Cohere-Api-Key: YOUR-COHERE-API-KEY" \
-d @- \
https://some-endpoint.weaviate.network/v1/graphql
Example response - grouped resultโ
{
"data": {
"Get": {
"Publication": [
{
"_additional": {
"generate": {
"error": null,
"groupedResult": "These magazines or newspapers are about finance because they cover topics related to finance, such as business news, financial markets, and economic trends. They also often feature articles about personal finance, such as investing, budgeting, and retirement planning."
}
},
"name": "Financial Times"
},
{
"_additional": {
"generate": null
},
"name": "Wall Street Journal"
},
{
"_additional": {
"generate": null
},
"name": "The New York Times Company"
}
]
}
}
}
Additional informationโ
Supported modelsโ
You can use any of
command-xlarge-nightly
(default)command-xlarge-beta
command-xlarge
More resourcesโ
If you can't find the answer to your question here, please look at the:
- Frequently Asked Questions. Or,
- Knowledge base of old issues. Or,
- For questions: Stackoverflow. Or,
- For more involved discussion: Weaviate Community Forum. Or,
- We also have a Slack channel.