text2vec-huggingface

     

Introduction

The text2vec-huggingface module allows you to use Hugging Face models directly in Weaviate as a vectorization module. When you create a Weaviate class that is set to use this module, it will automatically vectorize your data using the chosen module.

• Note: this module uses a third-party API.
• Note: make sure to check the Inference pricing page before vectorizing large amounts of data.
• Note: Weaviate automatically parallelizes requests to the Inference API when using the batch endpoint.
• Note: This module only supports sentence similarity models.

How to enable

Request a Hugging Face API Token via their website.

Weaviate Cloud Services

This module is enabled by default on the WCS.

Weaviate open source

Here is an example Docker-compose file, which will spin up Weaviate with the Hugging Face module.

version: '3.4'
services:
  weaviate:
    image: semitechnologies/weaviate:1.19.6
    restart: on-failure:0
    ports:
     - "8080:8080"
    environment:
      QUERY_DEFAULTS_LIMIT: 20
      AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
      PERSISTENCE_DATA_PATH: "./data"
      DEFAULT_VECTORIZER_MODULE: text2vec-huggingface
      ENABLE_MODULES: text2vec-huggingface
      HUGGINGFACE_APIKEY: sk-foobar # request a key on huggingface.co, setting this parameter is optional, you can also provide the API key at runtime
      CLUSTER_HOSTNAME: 'node1'

note

• You can also use the Weaviate configuration tool to create a Weaviate setup with this module.
• The HUGGINGFACE_APIKEY environment variable is optional and you can instead provide the key at insert/query time as an HTTP header (see the 'usage' section for instructions)

How to configure

In your Weaviate schema, you must define how you want this module to vectorize your data. If you are new to Weaviate schemas, you might want to check out the tutorial on the Weaviate schema first.

For example, the following schema configuration will set Weaviate to vectorize the Document class with text2vec-huggingface using the all-MiniLM-L6-v2 model.

{
  "classes": [
    {
      "class": "Document",
      "description": "A class called document",
      "moduleConfig": {
        "text2vec-huggingface": {
          "model": "sentence-transformers/all-MiniLM-L6-v2",
          "options": {
            "waitForModel": true,
            "useGPU": true,
            "useCache": true
          }
        }
      },
      "properties": [
        {
          "dataType": [
            "text"
          ],
          "description": "Content that will be vectorized",
          "moduleConfig": {
            "text2vec-huggingface": {
              "skip": false,
              "vectorizePropertyName": false
            }
          },
          "name": "content"
        }
      ],
      "vectorizer": "text2vec-huggingface"
    }
  ]
}

Usage

• If the Hugging Face API key is not set in the text2vec-huggingface module, you can set the API key at query time by adding the following to the HTTP header: X-Huggingface-Api-Key: YOUR-HUGGINGFACE-API-KEY.
• Using this module will enable GraphQL vector search operators.

Example

GraphQL

{
  Get{
    Publication(
      nearText: {
        concepts: ["fashion"],
        distance: 0.6 # prior to v1.14 use "certainty" instead of "distance"
        moveAwayFrom: {
          concepts: ["finance"],
          force: 0.45
        },
        moveTo: {
          concepts: ["haute couture"],
          force: 0.85
        }
      }
    ){
      name
      _additional {
        certainty # only supported if distance==cosine.
        distance  # always supported
      }
    }
  }
}

Python

import weaviate

client = weaviate.Client(
    url="http://localhost:8080",
    additional_headers={
        "X-HuggingFace-Api-Key": "YOUR-HUGGINGFACE-API-KEY"
    }
)

nearText = {
  "concepts": ["fashion"],
  "distance": 0.6, # prior to v1.14 use "certainty" instead of "distance"
  "moveAwayFrom": {
    "concepts": ["finance"],
    "force": 0.45
  },
  "moveTo": {
    "concepts": ["haute couture"],
    "force": 0.85
  }
}

result = (
  client.query
  .get("Publication", "name")
  .with_additional(["certainty OR distance"]) # note that certainty is only supported if distance==cosine
  .with_near_text(nearText)
  .do()
)

print(result)

JavaScript

const weaviate = require('weaviate-client');

const client = weaviate.client({
  scheme: 'http',
  host: 'localhost:8080',
  headers: {'X-HuggingFace-Api-Key': 'YOUR-HUGGINGFACE-API-KEY'},
});

client.graphql
  .get()
  .withClassName('Publication')
  .withFields('name _additional{certainty distance}') // note that certainty is only supported if distance==cosine
  .withNearText({
    concepts: ['fashion'],
    distance: 0.6, // prior to v1.14 use certainty instead of distance
    moveAwayFrom: {
      concepts: ['finance'],
      force: 0.45
    },
    moveTo: {
      concepts: ['haute couture'],
      force: 0.85
    }
  })
  .do()
  .then(console.log)
  .catch(console.error);

Go

package main

import (
  "context"
  "fmt"

  "github.com/weaviate/weaviate-go-client/v4/weaviate"
  "github.com/weaviate/weaviate-go-client/v4/weaviate/graphql"
)

func main() {
  cfg := weaviate.Config{
    Host:    "localhost:8080",
    Scheme:  "http",
    Headers: map[string]string{"X-HuggingFace-Api-Key": "YOUR-HUGGINGFACE-API-KEY"},
  }
  client, err := weaviate.NewClient(cfg)
  if err != nil {
    panic(err)
  }

  className := "Publication"

  name := graphql.Field{Name: "name"}
  _additional := graphql.Field{
    Name: "_additional", Fields: []graphql.Field{
      {Name: "certainty"}, // only supported if distance==cosine
      {Name: "distance"},  // always supported
    },
  }

  concepts := []string{"fashion"}
  distance := float32(0.6)
  moveAwayFrom := &graphql.MoveParameters{
    Concepts: []string{"finance"},
    Force:    0.45,
  }
  moveTo := &graphql.MoveParameters{
    Concepts: []string{"haute couture"},
    Force:    0.85,
  }
  nearText := client.GraphQL().NearTextArgBuilder().
    WithConcepts(concepts).
    WithDistance(distance). // use WithCertainty(certainty) prior to v1.14
    WithMoveTo(moveTo).
    WithMoveAwayFrom(moveAwayFrom)

  ctx := context.Background()

  result, err := client.GraphQL().Get().
    WithClassName(className).
    WithFields(name, _additional).
    WithNearText(nearText).
    Do(ctx)

  if err != nil {
    panic(err)
  }
  fmt.Printf("%v", result)
}

Java

package io.weaviate;

import io.weaviate.client.Config;
import io.weaviate.client.WeaviateClient;
import io.weaviate.client.base.Result;
import io.weaviate.client.v1.graphql.model.GraphQLResponse;
import io.weaviate.client.v1.graphql.query.argument.NearTextArgument;
import io.weaviate.client.v1.graphql.query.argument.NearTextMoveParameters;
import io.weaviate.client.v1.graphql.query.fields.Field;

import java.util.HashMap;
import java.util.Map;

public class App {
  public static void main(String[] args) {
    Map<String, String> headers = new HashMap<String, String>() { {
      put("X-HuggingFace-Api-Key", "YOUR-HUGGINGFACE-API-KEY");
    } };
    Config config = new Config("http", "localhost:8080", headers);
    WeaviateClient client = new WeaviateClient(config);

    NearTextMoveParameters moveTo = NearTextMoveParameters.builder()
      .concepts(new String[]{ "haute couture" }).force(0.85f).build();

    NearTextMoveParameters moveAway = NearTextMoveParameters.builder()
      .concepts(new String[]{ "finance" }).force(0.45f)
      .build();

    NearTextArgument nearText = client.graphQL().arguments().nearTextArgBuilder()
      .concepts(new String[]{ "fashion" })
      .distance(0.6f) // use .certainty(0.7f) prior to v1.14
      .moveTo(moveTo)
      .moveAwayFrom(moveAway)
      .build();

    Field name = Field.builder().name("name").build();
    Field _additional = Field.builder()
      .name("_additional")
      .fields(new Field[]{
        Field.builder().name("certainty").build(), // only supported if distance==cosine
        Field.builder().name("distance").build(),  // always supported
      }).build();

    Result<GraphQLResponse> result = client.graphQL().get()
      .withClassName("Publication")
      .withFields(name, _additional)
      .withNearText(nearText)
      .run();

    if (result.hasErrors()) {
      System.out.println(result.getError());
      return;
    }
    System.out.println(result.getResult());
  }
}

Curl

$ echo '{
  "query": "{
    Get{
      Publication(
        nearText: {
          concepts: [\"fashion\"],
          distance: 0.6, // use certainty instead of distance prior to v1.14
          moveAwayFrom: {
            concepts: [\"finance\"],
            force: 0.45
          },
          moveTo: {
            concepts: [\"haute couture\"],
            force: 0.85
          }
        }
      ){
        name
        _additional {
          certainty // only supported if distance==cosine
          distance  // always supported
        }
      }
    }
  }"
}' | curl \
    -X POST \
    -H 'Content-Type: application/json' \
    -H "X-HuggingFace-Api-Key: YOUR-HUGGINGFACE-API-KEY" \
    -d @- \
    http://localhost:8080/v1/graphql

Additional information

Support for Hugging Face Inference Endpoints

The text2vec-huggingface module also supports Hugging Face Inference Endpoints, where you can deploy your own model as an endpoint. To use your own Hugging Face Inference Endpoint for vectorization with the text2vec-huggingface module, just pass the endpoint url in the class configuration as the endpointURL setting. Please note that only feature extraction inference endpoint types are supported.

Available settings

In the schema, on a class level, the following settings can be added:

setting	type	description	example
model	string	This can be any public or private Hugging Face model, sentence similarity models work best for vectorization. Don't use with queryModel nor passageModel.	"bert-base-uncased"
passageModel	string	DPR passage model. Should be set together with queryModel, but without model.	"sentence-transformers/facebook-dpr-ctx_encoder-single-nq-base"
queryModel	string	DPR query model. Should be set together with passageModel, but without model.	"sentence-transformers/facebook-dpr-question_encoder-single-nq-base"
options.waitForModel	boolean	If the model is not ready, wait for it instead of receiving 503.
options.useGPU	boolean	Use GPU instead of CPU for inference. (requires Hugginface's Startup plan or higher)
options.useCache	boolean	There is a cache layer on the inference API to speedup requests we have already seen. Most models can use those results as is as models are deterministic (meaning the results will be the same anyway). However if you use a non-deterministic model, you can set this parameter to prevent the caching mechanism from being used resulting in a real new query.
endpointURL	string	This can be any public or private Hugging Face Inference URL. To find out how to deploy your own Hugging Face Inference Endpoint click here. Note: when this variable is set, the module will ignore model settings like model queryModel and passageModel.

More resources

If you can't find the answer to your question here, please look at the:

1. Frequently Asked Questions. Or,
2. Knowledge base of old issues. Or,
3. For questions: Stackoverflow. Or,
4. For more involved discussion: Weaviate Community Forum. Or,
5. We also have a Slack channel.