Replication

Weaviate instances can be replicated. Replication can improve read throughput, improve availability, and enable zero-downtime upgrades.

For more details on how replication is designed and built in Weaviate, see Replication Architecture.

How to configure

Replication factor change in v1.25

In Weaviate v1.25, a replication factor cannot be changed once it is set.

This is due to the schema consensus algorithm change in v1.25. This will be improved in future versions.

Replication is disabled by default. It can be enabled per collection in the collection configuration. This means you can set different replication factors per class in your dataset.

To enable replication, you can set one or both of the following:

REPLICATION_MINIMUM_FACTOR environment variable for the entire Weaviate instance, or
replicationFactor parameter for a collection.

Weaviate-wide minimum replication factor

The REPLICATION_MINIMUM_FACTOR environment variable sets the minimum replication factor for all collections in the Weaviate instance.

If you set the replication factor for a collection, the collection's replication factor overrides the minimum replication factor.

Replication factor for a collection

from weaviate.classes.config import Configure

client.collections.create(
    "Article",
    replication_config=Configure.replication(
        factor=3,
    )
)

class_obj = {
    "class": "Article",
    "replicationConfig": {
        "factor": 3,
    },
}

client.schema.create_class(class_obj)

import { configure } from 'weaviate-client';

await client.collections.create({
  name: 'Article',
  replication: configure.replication({
    factor: 3
  }),
})

const classWithReplication = {
  class: 'Article',
  replicationConfig: {
    factor: 3,
  },
};

// Add the class to the schema
result = await client.schema
  .classCreator()
  .withClass(classWithReplication)
  .do();

package main

import (
  "context"

  "github.com/weaviate/weaviate-go-client/v4/weaviate"
  "github.com/weaviate/weaviate/entities/models"
)

func main() {
  cfg := weaviate.Config{
    Host:   "localhost:8080",
    Scheme: "http",
  }
  client, err := weaviate.NewClient(cfg)
  if err != nil {
    panic(err)
  }

  classObj := &models.Class{
    Class:             "Article",
    Properties: []*models.Property{
      {
        DataType:    []string{"string"},
        Name:        "title",
      }
    },
    ReplicationConfig: &models.ReplicationConfig{
      Factor: 3,
    }
  }

  err := client.Schema().ClassCreator().WithClass(classObj).Do(context.Background())
  if err != nil {
    panic(err)
  }
}

package io.weaviate;

import java.util.ArrayList;
import io.weaviate.client.Config;
import io.weaviate.client.WeaviateClient;
import io.weaviate.client.base.Result;
import io.weaviate.client.v1.schema.model.DataType;
import io.weaviate.client.v1.schema.model.Property;
import io.weaviate.client.v1.schema.model.WeaviateClass;
import io.weaviate.client.v1.misc.model.ReplicationConfig;

public class App {
  public static void main(String[] args) {
    Config config = new Config("http", "localhost:8080");
    WeaviateClient client = new WeaviateClient(config);

    ReplicationConfig replicationConfig = ReplicationConfig.builder()
      .factor(3)
      .build();

    WeaviateClass clazz = WeaviateClass.builder()
      .className("Article")
      .replicationConfig(replicationConfig)
      .properties(new ArrayList() { {
        add(Property.builder()
          .dataType(new ArrayList(){ { add(DataType.STRING); } })
          .name("title")
          .build());
      } })
      .build();

    Result<Boolean> result = client.schema().classCreator().withClass(clazz).run();
    if (result.hasErrors()) {
      System.out.println(result.getError());
      return;
    }
    System.out.println(result.getResult());
  }
}

curl \
-X POST \
-H "Content-Type: application/json" \
-d '{
    "class": "Article",
    "properties": [
        {
            "dataType": [
                "string"
            ],
            "description": "Title of the article",
            "name": "title"
        }
    ],
    "replicationConfig": {
      "factor": 3
    }
}' \
http://localhost:8080/v1/schema

In this example, there are three replicas. If you set the replication factor before you import data, all of the data is replicated three times.

The replication factor can be modified after you add data to a collection. If you modify the replication factor afterwards, new data is copied across the new and pre-existing replica nodes.

The example data schema has a write consistency level of ALL. When you upload or update a schema, the changes are sent to ALL nodes (via a coordinator node). The coordinator node waits for a successful acknowledgement from ALL nodes before sending a success message back to the client. This ensures a highly consistent schema in your distributed Weaviate setup.

Data consistency

When Weaviate detects inconsistent data across nodes, it attempts to repair the out of sync data.

Starting in v1.26, Weaviate adds async replication to proactively detect inconsistencies. In earlier versions, Weaviate uses a repair-on-read strategy to repair inconsistencies at read time.

Repair-on-read is automatic. To activate async replication, set asyncEnabled to true in the replicationConfig section of your collection definition.

Python Client v4
Python Client v3
JS/TS Client v3
JS/TS Client v2
cURL

from weaviate.classes.config import Configure

client.collections.create(
    "Article",
    replication_config=Configure.replication(
        factor=3,
        async_enabled=True,
    )
)

class_obj = {
    "class": "Article",
    "replicationConfig": {
        "factor": 3,
        "aysnc_enabled": True
    },
}

client.schema.create_class(class_obj)

import { configure } from 'weaviate-client';

await client.collections.create({
 name: 'Article',
 replication: configure.replication({
   factor: 3,
   asyncEnabled: true,
 }),
})

const classWithReplication = {
 class: 'Article',
 replicationConfig: {
   factor: 3,
   asyncEnabled: true,
 },
};

// Add the class to the schema
result = await client.schema
 .classCreator()
 .withClass(classWithReplication)
 .do();

curl \
-X POST \
-H "Content-Type: application/json" \
-d '{
    "class": "Article",
    "properties": [
        {
            "dataType": [
                "string"
            ],
            "description": "Title of the article",
            "name": "title"
        }
    ],
    "replicationConfig": {
      "factor": 3,
      "asyncEnabled": true
    }
}' \
http://localhost:8080/v1/schema

How to use: Queries

When you add (write) or query (read) data, one or more replica nodes in the cluster will respond to the request. How many nodes need to send a successful response and acknowledgement to the coordinator node depends on the consistency_level. Available consistency levels are ONE, QUORUM (replication_factor / 2 + 1) and ALL.

The consistency_level can be specified at query time:

# Get an object by ID, with consistency level ONE
curl "http://localhost:8080/v1/objects/{ClassName}/{id}?consistency_level=ONE"

note

In v1.17, only read queries that get data by ID had a tunable consistency level. All other object-specific REST endpoints (read or write) used the consistency level ALL. Starting with v1.18, all write and read queries are tunable to either ONE, QUORUM (default) or ALL. GraphQL endpoints use the consistency level ONE (in both versions).

from weaviate.classes.config import ConsistencyLevel

questions = client.collections.get(collection_name).with_consistency_level(
    consistency_level=ConsistencyLevel.QUORUM
)
response = collection.query.fetch_object_by_id("36ddd591-2dee-4e7e-a3cc-eb86d30a4303")

# The parameter passed to `withConsistencyLevel` can be one of:
# * 'ALL',
# * 'QUORUM' (default), or
# * 'ONE'.
#
# It determines how many replicas must acknowledge a request
# before it is considered successful.

for o in response.objects:
    print(o.properties)  # Inspect returned objects

import weaviate
from weaviate.data.replication import ConsistencyLevel

client = weaviate.Client("http://localhost:8080")

data_object = (
  client.data_object.get_by_id(
    uuid="36ddd591-2dee-4e7e-a3cc-eb86d30a4303",
    class_name="MyClass",
    consistency_level=ConsistencyLevel.ONE,
  )
)

# The parameter "consistency_level" can be one of ConsistencyLevel.ALL,
# ConsistencyLevel.QUORUM (default), or ConsistencyLevel.ONE. Determines how many
# replicas must acknowledge a request before it is considered successful.

print(data_object)

const myCollection = client.collections.get('Article').withConsistency('QUORUM');

const result = await myCollection.query.fetchObjectById("36ddd591-2dee-4e7e-a3cc-eb86d30a4303")

console.log(JSON.stringify(result, null, 2));

// The parameter passed to `withConsistencyLevel` can be one of:
// * 'ALL',
// * 'QUORUM' (default), or
// * 'ONE'.
//
// It determines how many replicas must acknowledge a request
// before it is considered successful.

import weaviate from 'weaviate-ts-client';

const client = weaviate.client({
  scheme: 'http',
  host: 'localhost:8080',
});

const response = await client.data
  .getterById()
  .withClassName('MyClass')
  .withId('36ddd591-2dee-4e7e-a3cc-eb86d30a4303')
  .withConsistencyLevel('ONE')  // default QUORUM
  .do();
console.log(JSON.stringify(response, null, 2));

// The parameter passed to `withConsistencyLevel` can be one of:
// * 'ALL',
// * 'QUORUM' (default), or
// * 'ONE'.
//
// It determines how many replicas must acknowledge a request
// before it is considered successful.

package main

import (
    "context"
    "fmt"

    "github.com/weaviate/weaviate-go-client/v4/weaviate/data/replication"  // for consistency levels
    "github.com/weaviate/weaviate-go-client/v4/weaviate"
)

func main() {
    cfg := weaviate.Config{
        Host:   "localhost:8080",
        Scheme: "http",
    }
    client, err := weaviate.NewClient(cfg)
    if err != nil {
        panic(err)
    }

    data, err := client.Data().ObjectsGetter().
        WithClassName("MyClass").
        WithID("36ddd591-2dee-4e7e-a3cc-eb86d30a4303").
        WithConsistencyLevel(replication.ConsistencyLevel.ONE).  // default QUORUM
        Do(context.Background())

    if err != nil {
        panic(err)
    }
    fmt.Printf("%v", data)
}

// The parameter passed to "WithConsistencyLevel" can be one of:
// * replication.ConsistencyLevel.ALL,
// * replication.ConsistencyLevel.QUORUM (default), or
// * replication.ConsistencyLevel.ONE.
//
// It determines how many replicas must acknowledge a request
// before it is considered successful.

package io.weaviate;

import java.util.List;
import io.weaviate.client.Config;
import io.weaviate.client.WeaviateClient;
import io.weaviate.client.base.Result;
import io.weaviate.client.v1.data.model.WeaviateObject;
import io.weaviate.client.v1.data.replication.model.ConsistencyLevel;

public class App {
  public static void main(String[] args) {
    Config config = new Config("http", "localhost:8080");
    WeaviateClient client = new WeaviateClient(config);

    Result<List<WeaviateObject>> result = client.data().objectsGetter()
      .withClassName("MyClass")
      .withID("36ddd591-2dee-4e7e-a3cc-eb86d30a4303")
      .withConsistencyLevel(ConsistencyLevel.ONE)  // default QUORUM
      .run();

    if (result.hasErrors()) {
      System.out.println(result.getError());
      return;
    }
    System.out.println(result.getResult());
  }
}

// The parameter passed to `withConsistencyLevel` can be one of:
// * ConsistencyLevel.ALL,
// * ConsistencyLevel.QUORUM (default), or
// * ConsistencyLevel.ONE.
//
// It determines how many replicas must acknowledge a request
// before it is considered successful.

curl "http://localhost:8080/v1/objects/MyClass/36ddd591-2dee-4e7e-a3cc-eb86d30a4303?consistency_level=QUORUM"

# The parameter "consistency_level" can be one of ALL, QUORUM (default), or ONE. Determines how many
# replicas must acknowledge a request before it is considered successful.
# curl "/v1/objects/{ClassName}/{id}?consistency_level=ONE"

Concepts: Replication Architecture

Questions and feedback

If you have any questions or feedback, let us know in the user forum.

How to configure​

Weaviate-wide minimum replication factor​

Replication factor for a collection​

Data consistency​

How to use: Queries​

Related pages​

Questions and feedback​