text2vec-cohere

Overview

The text2vec-cohere module enables Weaviate to obtain vectors using Cohere.

Key notes:

• As it uses a third-party API, you will need an API key.
• Its usage may incur costs.
  • Please check the Cohere pricing page, especially before vectorizing large amounts of data.
• This module is available on Weaviate Cloud Services (WCS).
• Enabling this module will enable the nearText search operator.
• The default model is embed-multilingual-v3.0.
• Set the appropriate distance metric in your class configuration, depending on the model used.

Where to set module parameters

The module accepts parameters through the request header, collection configuration, or environment variables. Some parameters (such as the API key) can be set in multiple ways.

Where the same parameter can be set in multiple ways, setting it at query-time through the HTTP request header (if possible) will have the highest precedence.

We suggest you only set any given parameter in one place to avoid confusion.

Weaviate instance configuration

Not applicable to WCS

This module is enabled and pre-configured on Weaviate Cloud Services.

Docker Compose file

To use text2vec-cohere, you must enable it in your Docker Compose file (docker-compose.yml). You can do so manually, or create one using the Weaviate configuration tool.

Parameters

• ENABLE_MODULES (Required): The modules to enable. Include text2vec-cohere to enable the module.
• DEFAULT_VECTORIZER_MODULE (Optional): The default vectorizer module. You can set this to text2vec-cohere to make it the default for all classes.
• COHERE_APIKEY (Optional): Your Cohere API key. You can also provide the key at query time.

Example

This configuration enables text2vec-cohere, sets it as the default vectorizer, and sets the API keys.

---
version: '3.4'
services:
  weaviate:
    image: semitechnologies/weaviate:1.22.6
    restart: on-failure:0
    ports:
     - "8080:8080"
    environment:
      QUERY_DEFAULTS_LIMIT: 20
      AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
      PERSISTENCE_DATA_PATH: "./data"
      ENABLE_MODULES: text2vec-cohere
      DEFAULT_VECTORIZER_MODULE: text2vec-cohere
      COHERE_APIKEY: sk-foobar # Setting this parameter is optional, you can also provide the API key at query time.
      CLUSTER_HOSTNAME: 'node1'
...

Class configuration

You can configure how the module will behave in each class through the Weaviate schema.

API settings

Parameters

Parameter	Required	Default	Purpose
model	No	embed-multilingual-v3.0	The model to use.
truncate	No	END	Sets Cohere API input truncation behavior (NONE, START or END - read more).
baseURL	No	https://api.cohere.ai	Sets a proxy or other URL instead of the default URL. Use a the protocol domain format: https://your.domain.com.

Example

The following example configures the Document class by setting the vectorizer to text2vec-cohere, distance metric to cosine, model to embed-multilingual-v3.0 and without input truncation by the Cohere API.

info

Different Cohere models use different distance metrics. Make sure to set this accordingly. See the distance metric section for more information.

{
  "classes": [
    {
      "class": "Document",
      "description": "A class called document",
      "vectorizer": "text2vec-cohere",
      "vectorIndexConfig": {
        "distance": "cosine"
      },
      "moduleConfig": {
        "text2vec-cohere": {
          "model": "embed-multilingual-v3.0", // Defaults to embed-multilingual-v3.0 if not set
          "truncate": "END", // Defaults to END if not set
          "baseURL": "https://proxy.yourcompanydomain.com"  // Optional. Can be overridden by one set in the HTTP header.
        }
      },
    }
  ]
}

Vectorization settings

You can set vectorizer behavior using the moduleConfig section under each class and property:

Class-level

• vectorizer - what module to use to vectorize the data.
• vectorizeClassName – whether to vectorize the class name. Default: true.

Property-level

• skip – whether to skip vectorizing the property altogether. Default: false
• vectorizePropertyName – whether to vectorize the property name. Default: false

Example

{
  "classes": [
    {
      "class": "Document",
      "description": "A class called document",
      "vectorizer": "text2vec-cohere",
      "vectorIndexConfig": {
        "distance": "cosine"
      },
      "moduleConfig": {
        "text2vec-cohere": {
          "model": "embed-multilingual-v3.0", // Defaults to embed-multilingual-v3.0 if not set
          "truncate": "END", // Defaults to END if not set
          "vectorizeClassName": false
        }
      },
      "properties": [
        {
          "name": "content",
          "dataType": ["text"],
          "description": "Content that will be vectorized",
          "moduleConfig": {
            "text2vec-cohere": {
              "skip": false,
              "vectorizePropertyName": false
            }
          }
        }
      ]
    }
  ]
}

Query-time parameters

You can supply parameters at query time by adding it to the HTTP header.

HTTP Header	Value	Purpose	Note
"X-Cohere-Api-Key"	"YOUR-COHERE-API-KEY"	Cohere API key
"X-Cohere-BaseURL"	"YOUR-COHERE-BASE-URL"	Cohere base URL	Available from version v1.22.3 Use the protocol domain format: https://your.domain.com. If specified, this will have precedence over the class-level setting.

Additional information

Available models

You can use any of the following models with text2vec-cohere (source):

• embed-multilingual-v3.0 (Default)
• embed-multilingual-light-v3.0
• embed-multilingual-v2.0
• embed-english-v3.0
• embed-english-light-v3.0
• embed-english-v2.0
• embed-english-light-v2.0

Deprecated models

The following models are available, but deprecated:

• multilingual-22-12
• large
• medium
• small

text2vec-cohere defaults to the embed-multilingual-v3.0 embedding model unless specified otherwise.

embed-multilingual-v2.0 == multilingual-22-12

embed-multilingual-v2.0 reflects the new name.

Distance metric

Cohere's embed-multilingual-v2.0 model uses dot product distances. With the v3 models, you can use any distance metric (dot, cosine or Euclidean) (source).

You can see a list of supported distance metrics here.

Truncation

If the input text contains too many tokens and is not truncated, the Cohere API will throw an error. The Cohere API can be set to automatically truncate your input text.

You can set the truncation option with the truncate parameter to any of "NONE", "START", "END". Passing END will discard the right side of the input, and passing START will discard the left side of the input. The remaining input is exactly the maximum input token length for the model. Cohere's default behavior is to truncate at the END if it is not set. (source)

• The upside of truncating is that a batch import always succeeds.
• The downside of truncating (i.e., NONE) is that a large text will be partially vectorized without the user being made aware of the truncation.

API rate limits

Since this module uses your API key, your account's corresponding rate limits will also apply to the module. Weaviate will output any rate-limit related error messages generated by the API.

More information about Cohere rate limits can be found here.

Import throttling

One potential solution to rate limiting would be to throttle the import within your application. We include an example below.

See code example

Python

from weaviate import Client
import time

def configure_batch(client: Client, batch_size: int, batch_target_rate: int):
    """
    Configure the weaviate client's batch so it creates objects at `batch_target_rate`.

    Parameters
    ----------
    client : Client
        The Weaviate client instance.
    batch_size : int
        The batch size.
    batch_target_rate : int
        The batch target rate as # of objects per second.
    """

    def callback(batch_results: dict) -> None:

        # you could print batch errors here
        time_took_to_create_batch = batch_size * (client.batch.creation_time/client.batch.recommended_num_objects)
        time.sleep(
            max(batch_size/batch_target_rate - time_took_to_create_batch + 1, 0)
        )

    client.batch.configure(
        batch_size=batch_size,
        timeout_retries=5,
        callback=callback,
    )

Go

package main

import (
    "context"
    "time"

    "github.com/weaviate/weaviate-go-client/v4/weaviate"
    "github.com/weaviate/weaviate/entities/models"
)

var (
    // adjust to your liking
    targetRatePerMin = 600
    batchSize        = 50
)

func main() {
    cfg := weaviate.Config{
        Host:   "localhost:8080",
        Scheme: "http",
    }
    client, err := weaviate.NewClient(cfg)
    if err != nil {
        panic(err)
    }

    // replace those 10000 empty objects with your actual data
    objects := make([]*models.Object, 10000)

    // we aim to send one batch every tickInterval second.
    tickInterval := time.Duration(batchSize/targetRatePerMinute) * time.Minute
    t := time.NewTicker(tickInterval)
    before := time.Now()

    for i := 0; i < len(objects); i += batchSize {

        // create a fresh batch
        batch := client.Batch().ObjectsBatcher()

        // add batchSize objects to the batch
        for j := i; j < i+batchSize; j++ {
            batch = batch.WithObject(objects[i+j])
        }

        // send off batch
        res, err := batch.Do(context.Background())
        // TODO: inspect result for individual errors
        _ = res
        // TODO: check request error
        _ = err

        // we wait for the next tick. If the previous batch took longer than
        // tickInterval, we won't need to wait, effectively making this an
        // unthrottled import.
        <-t.C
    }
}

Usage example

This is an example of a nearText query with text2vec-cohere.

Python

import weaviate

client = weaviate.Client(
    url="http://localhost:8080",
    additional_headers={
        "X-Cohere-Api-Key": "YOUR-COHERE-API-KEY"
    }
)

nearText = {
  "concepts": ["fashion"],
  "distance": 0.6, # prior to v1.14 use "certainty" instead of "distance"
  "moveAwayFrom": {
    "concepts": ["finance"],
    "force": 0.45
  },
  "moveTo": {
    "concepts": ["haute couture"],
    "force": 0.85
  }
}

result = (
  client.query
  .get("Publication", "name")
  .with_additional(["certainty OR distance"]) # note that certainty is only supported if distance==cosine
  .with_near_text(nearText)
  .do()
)

print(result)

JavaScript/TypeScript

import weaviate from 'weaviate-ts-client';

const client = weaviate.client({
  scheme: 'http',
  host: 'localhost:8080',
  headers: { 'X-Cohere-Api-Key': 'YOUR-COHERE-API-KEY' },
});

const response = await client.graphql
  .get()
  .withClassName('Publication')
  .withFields('name _additional{certainty distance}') // note that certainty is only supported if distance==cosine
  .withNearText({
    concepts: ['fashion'],
    distance: 0.6,  // prior to v1.14 use certainty instead of distance
    moveAwayFrom: {
      concepts: ['finance'],
      force: 0.45,
    },
    moveTo: {
      concepts: ['haute couture'],
      force: 0.85,
    },
  })
  .do();
console.log(response);

Go

package main

import (
  "context"
  "fmt"

  "github.com/weaviate/weaviate-go-client/v4/weaviate"
  "github.com/weaviate/weaviate-go-client/v4/weaviate/graphql"
)

func main() {
  cfg := weaviate.Config{
    Host:    "localhost:8080",
    Scheme:  "http",
    Headers: map[string]string{"X-Cohere-Api-Key": "YOUR-COHERE-API-KEY"},
  }
  client, err := weaviate.NewClient(cfg)
  if err != nil {
    panic(err)
  }

  className := "Publication"

  name := graphql.Field{Name: "name"}
  _additional := graphql.Field{
    Name: "_additional", Fields: []graphql.Field{
      {Name: "certainty"}, // only supported if distance==cosine
      {Name: "distance"},  // always supported
    },
  }

  concepts := []string{"fashion"}
  distance := float32(0.6)
  moveAwayFrom := &graphql.MoveParameters{
    Concepts: []string{"finance"},
    Force:    0.45,
  }
  moveTo := &graphql.MoveParameters{
    Concepts: []string{"haute couture"},
    Force:    0.85,
  }
  nearText := client.GraphQL().NearTextArgBuilder().
    WithConcepts(concepts).
    WithDistance(distance). // use WithCertainty(certainty) prior to v1.14
    WithMoveTo(moveTo).
    WithMoveAwayFrom(moveAwayFrom)

  ctx := context.Background()

  result, err := client.GraphQL().Get().
    WithClassName(className).
    WithFields(name, _additional).
    WithNearText(nearText).
    Do(ctx)

  if err != nil {
    panic(err)
  }
  fmt.Printf("%v", result)
}

Java

package io.weaviate;

import io.weaviate.client.Config;
import io.weaviate.client.WeaviateClient;
import io.weaviate.client.base.Result;
import io.weaviate.client.v1.graphql.model.GraphQLResponse;
import io.weaviate.client.v1.graphql.query.argument.NearTextArgument;
import io.weaviate.client.v1.graphql.query.argument.NearTextMoveParameters;
import io.weaviate.client.v1.graphql.query.fields.Field;

import java.util.HashMap;
import java.util.Map;

public class App {
  public static void main(String[] args) {
    Map<String, String> headers = new HashMap<String, String>() { {
      put("X-Cohere-Api-Key", "YOUR-COHERE-API-KEY");
    } };
    Config config = new Config("http", "localhost:8080", headers);
    WeaviateClient client = new WeaviateClient(config);

    NearTextMoveParameters moveTo = NearTextMoveParameters.builder()
      .concepts(new String[]{ "haute couture" }).force(0.85f).build();

    NearTextMoveParameters moveAway = NearTextMoveParameters.builder()
      .concepts(new String[]{ "finance" }).force(0.45f)
      .build();

    NearTextArgument nearText = client.graphQL().arguments().nearTextArgBuilder()
      .concepts(new String[]{ "fashion" })
      .distance(0.6f) // use .certainty(0.7f) prior to v1.14
      .moveTo(moveTo)
      .moveAwayFrom(moveAway)
      .build();

    Field name = Field.builder().name("name").build();
    Field _additional = Field.builder()
      .name("_additional")
      .fields(new Field[]{
        Field.builder().name("certainty").build(), // only supported if distance==cosine
        Field.builder().name("distance").build(),  // always supported
      }).build();

    Result<GraphQLResponse> result = client.graphQL().get()
      .withClassName("Publication")
      .withFields(name, _additional)
      .withNearText(nearText)
      .run();

    if (result.hasErrors()) {
      System.out.println(result.getError());
      return;
    }
    System.out.println(result.getResult());
  }
}

Curl

# Note: Under nearText, use `certainty` instead of distance prior to v1.14
# Under _additional, `certainty` is only supported if distance==cosine, but `distance` is always supported
echo '{
  "query": "{
    Get {
      Publication(
        nearText: {
          concepts: [\"fashion\"],
          distance: 0.6,
          moveAwayFrom: {
            concepts: [\"finance\"],
            force: 0.45
          },
          moveTo: {
            concepts: [\"haute couture\"],
            force: 0.85
          }
        }
      ) {
        name
        _additional {
          certainty
          distance
        }
      }
    }
  }"
}' | curl \
    -X POST \
    -H 'Content-Type: application/json' \
    -H 'Authorization: Bearer learn-weaviate' \
    -H "X-Cohere-Api-Key: $COHERE_API_KEY" \
    -d @- \
    https://edu-demo.weaviate.network/v1/graphql

GraphQL

{
  Get{
    Publication(
      nearText: {
        concepts: ["fashion"],
        distance: 0.6 # prior to v1.14 use "certainty" instead of "distance"
        moveAwayFrom: {
          concepts: ["finance"],
          force: 0.45
        },
        moveTo: {
          concepts: ["haute couture"],
          force: 0.85
        }
      }
    ){
      name
      _additional {
        certainty # only supported if distance==cosine.
        distance  # always supported
      }
    }
  }
}