text2vec-gpt4all
Overview
text2vec-gpt
is only available from Weaviate versionv1.21
- Currently,
text2vec-gpt4all
is only available foramd64/x86_64
architecture devices.- This is as the
gpt4all
library currently does not support ARM devices, such as Apple M-series.
- This is as the
The text2vec-gpt4all
module enables Weaviate to obtain vectors using the gpt4all library.
Key notes:
- This module is not available on Weaviate Cloud Services (WCS).
- This module is optimized for CPU using the
ggml
library, allowing for fast inference even without a GPU. - Enabling this module will enable the
nearText
search operator. - By default, input text longer than 256 tokens is mean-pooled with an overlapping context window up to the number of tokens in your input.
- Currently, the only available model is
all-MiniLM-L6-v2
.
Weaviate instance configuration
This module is not available on Weaviate Cloud Services.
Docker Compose file
To use text2vec-gpt4all
, you must enable it in your Docker Compose file (docker-compose.yml
). You can do so manually, or create one using the Weaviate configuration tool.
Parameters
ENABLE_MODULES
(Required): The modules to enable. Includetext2vec-gpt4all
to enable the module.DEFAULT_VECTORIZER_MODULE
(Optional): The default vectorizer module. You can set this totext2vec-gpt4all
to make it the default for all classes.
Example
This configuration enables text2vec-gpt4all
, sets it as the default vectorizer, and sets the API keys.
---
version: '3.4'
services:
weaviate:
command:
- --host
- 0.0.0.0
- --port
- '8080'
- --scheme
- http
image: semitechnologies/weaviate:1.21.4
ports:
- 8080:8080
restart: on-failure:0
environment:
QUERY_DEFAULTS_LIMIT: 25
AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
PERSISTENCE_DATA_PATH: '/var/lib/weaviate'
DEFAULT_VECTORIZER_MODULE: 'text2vec-gpt4all'
ENABLE_MODULES: 'text2vec-gpt4all'
GPT4ALL_INFERENCE_API: 'http://text2vec-gpt4all:8080'
CLUSTER_HOSTNAME: 'node1'
text2vec-gpt4all:
image: semitechnologies/gpt4all-inference:all-MiniLM-L6-v2
...
Class configuration
You can configure how the module will behave in each class through the Weaviate schema.
Example
The following example configures the Article
class by setting the vectorizer to text2vec-gpt4all
:
{
"classes": [
{
"class": "Article",
"description": "A class called article",
"vectorizer": "text2vec-gpt4all"
}
]
}
Vectorization settings
You can set vectorizer behavior using the moduleConfig
section under each class and property:
Class-level
vectorizer
- what module to use to vectorize the data.vectorizeClassName
– whether to vectorize the class name. Default:true
.
Property-level
skip
– whether to skip vectorizing the property altogether. Default:false
vectorizePropertyName
– whether to vectorize the property name. Default:true
Example
{
"classes": [
{
"class": "Article",
"description": "A class called article",
"vectorizer": "text2vec-gpt4all",
"moduleConfig": {
"text2vec-gpt4all": {
"vectorizeClassName": "false"
}
},
"properties": [
{
"name": "content",
"dataType": ["text"],
"description": "Content that will be vectorized",
"moduleConfig": {
"text2vec-gpt4all": {
"skip": false,
"vectorizePropertyName": false
}
}
}
]
}
]
}
Additional information
Available models
Currently, the only available model is all-MiniLM-L6-v2
.
CPU optimized inference
The text2vec-gpt4all
module is optimized for CPU inference and should be noticeably faster then text2vec-transformers
in CPU-only (i.e. no CUDA acceleration) usage. You can read more about expected inference times here.
Usage advice - chunking text with gpt4all
text2vec-gpt4all
will truncate input text longer than 256
tokens (word pieces).
Accordingly, this model is not suitable for use cases where larger chunks are required. In these cases, we recommend using other models that support longer input lengths, such as by selecting one from the text2vec-transformers
module or text2vec-openai
.
Usage example
The below shows a code example of how to use a nearText
query with text2vec-gpt4all
.
- Python
- JavaScript/TypeScript
- Go
- Java
- Curl
- GraphQL
import weaviate
client = weaviate.Client("http://localhost:8080")
nearText = {
"concepts": ["fashion"],
"distance": 0.6, # prior to v1.14 use "certainty" instead of "distance"
"moveAwayFrom": {
"concepts": ["finance"],
"force": 0.45
},
"moveTo": {
"concepts": ["haute couture"],
"force": 0.85
}
}
result = (
client.query
.get("Publication", "name")
.with_additional(["certainty OR distance"]) # note that certainty is only supported if distance==cosine
.with_near_text(nearText)
.do()
)
print(result)
import weaviate from 'weaviate-ts-client';
const client = weaviate.client({
scheme: 'http',
host: 'localhost:8080',
});
const response = await client.graphql
.get()
.withClassName('Publication')
.withFields('name _additional{certainty distance}') // note that certainty is only supported if distance==cosine
.withNearText({
concepts: ['fashion'],
distance: 0.6, // prior to v1.14 use certainty instead of distance
moveAwayFrom: {
concepts: ['finance'],
force: 0.45,
},
moveTo: {
concepts: ['haute couture'],
force: 0.85,
},
})
.do();
console.log(response);
package main
import (
"context"
"fmt"
"github.com/weaviate/weaviate-go-client/v4/weaviate"
"github.com/weaviate/weaviate-go-client/v4/weaviate/graphql"
)
func main() {
cfg := weaviate.Config{
Host: "localhost:8080",
Scheme: "http",
}
client, err := weaviate.NewClient(cfg)
if err != nil {
panic(err)
}
className := "Publication"
name := graphql.Field{Name: "name"}
_additional := graphql.Field{
Name: "_additional", Fields: []graphql.Field{
{Name: "certainty"}, // only supported if distance==cosine
{Name: "distance"}, // always supported
},
}
concepts := []string{"fashion"}
distance := float32(0.6)
moveAwayFrom := &graphql.MoveParameters{
Concepts: []string{"finance"},
Force: 0.45,
}
moveTo := &graphql.MoveParameters{
Concepts: []string{"haute couture"},
Force: 0.85,
}
nearText := client.GraphQL().NearTextArgBuilder().
WithConcepts(concepts).
WithDistance(distance). // use WithCertainty(certainty) prior to v1.14
WithMoveTo(moveTo).
WithMoveAwayFrom(moveAwayFrom)
ctx := context.Background()
result, err := client.GraphQL().Get().
WithClassName(className).
WithFields(name, _additional).
WithNearText(nearText).
Do(ctx)
if err != nil {
panic(err)
}
fmt.Printf("%v", result)
}
package io.weaviate;
import io.weaviate.client.Config;
import io.weaviate.client.WeaviateClient;
import io.weaviate.client.base.Result;
import io.weaviate.client.v1.graphql.model.GraphQLResponse;
import io.weaviate.client.v1.graphql.query.argument.NearTextArgument;
import io.weaviate.client.v1.graphql.query.argument.NearTextMoveParameters;
import io.weaviate.client.v1.graphql.query.fields.Field;
public class App {
public static void main(String[] args) {
Config config = new Config("http", "localhost:8080");
WeaviateClient client = new WeaviateClient(config);
NearTextMoveParameters moveTo = NearTextMoveParameters.builder()
.concepts(new String[]{ "haute couture" }).force(0.85f).build();
NearTextMoveParameters moveAway = NearTextMoveParameters.builder()
.concepts(new String[]{ "finance" }).force(0.45f)
.build();
NearTextArgument nearText = client.graphQL().arguments().nearTextArgBuilder()
.concepts(new String[]{ "fashion" })
.distance(0.6f) // use .certainty(0.7f) prior to v1.14
.moveTo(moveTo)
.moveAwayFrom(moveAway)
.build();
Field name = Field.builder().name("name").build();
Field _additional = Field.builder()
.name("_additional")
.fields(new Field[]{
Field.builder().name("certainty").build(), // only supported if distance==cosine
Field.builder().name("distance").build(), // always supported
}).build();
Result<GraphQLResponse> result = client.graphQL().get()
.withClassName("Publication")
.withFields(name, _additional)
.withNearText(nearText)
.run();
if (result.hasErrors()) {
System.out.println(result.getError());
return;
}
System.out.println(result.getResult());
}
}
# Note: Under nearText, use `certainty` instead of distance prior to v1.14
# Under _additional, `certainty` is only supported if distance==cosine, but `distance` is always supported
echo '{
"query": "{
Get {
Publication(
nearText: {
concepts: [\"fashion\"],
distance: 0.6,
moveAwayFrom: {
concepts: [\"finance\"],
force: 0.45
},
moveTo: {
concepts: [\"haute couture\"],
force: 0.85
}
}
) {
name
_additional {
certainty
distance
}
}
}
}"
}' | curl \
-X POST \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer learn-weaviate' \
-H "X-OpenAI-Api-Key: $OPENAI_API_KEY" \
-d @- \
https://edu-demo.weaviate.network/v1/graphql
{
Get{
Publication(
nearText: {
concepts: ["fashion"],
distance: 0.6 # prior to v1.14 use "certainty" instead of "distance"
moveAwayFrom: {
concepts: ["finance"],
force: 0.45
},
moveTo: {
concepts: ["haute couture"],
force: 0.85
}
}
){
name
_additional {
certainty # only supported if distance==cosine.
distance # always supported
}
}
}
}
Model license(s)
The text2vec-gpt4all
module uses the gpt4all
library, which in turn uses the all-MiniLM-L6-v2
model. Please refer to the respective documentation for more information on their respective licenses.
It is your responsibility to evaluate whether the terms of its license(s), if any, are appropriate for your intended use.
More resources
For additional information, try these sources.