Skip to main content

Replication

LICENSEΒ Weaviate on Stackoverflow badgeΒ Weaviate issues on Github badgeΒ Weaviate version badgeΒ Weaviate total Docker pulls badgeΒ Go Report Card

Weaviate instances can be replicated to increase availability, read throughput and enable zero downtime upgrades. On this page, you will learn how to set replication for your Weaviate instance.

Note:

From v1.17, you can set the Replication Factor per class in the schema. Consistency levels are tunable in queries, which will be available from v1.18 (except for read consistency for objects by ID, which is tunable from v1.17).

Read more about how Replication is designed and built in Weaviate on the Replication Architecture pages.

How to configure: Schema​

Replication is enabled per data class in the data schema. This means you can set different replication factors per class in your dataset. To enable replication on a class (this is disabled by default), the replication factor has to be set. In a class in the schema, this looks like the following. "replicationConfig": {β€œfactor”: 3”} can be specified per class in the schema object.

{
"class": "ExampleClass",
"properties": [
{
"name": "exampleProperty",
"dataType": [
"string"
],
}
],
"replicationConfig": {
"factor": 3 // Integer, default is 1. Replication Factor is the amount of copies of this class that will be stored.
}
}

Here's an example for all the clients:

import weaviate

client = weaviate.Client("http://localhost:8080")

class_obj = {
"class": "Article",
"properties": [
{
"dataType": [
"string"
],
"name": "title",
},
],
"replicationConfig": {
"factor": 3
}
}

client.schema.create_class(class_obj)

When you set this replication factor in the data schema before you add data, you will have 3 replicas of the data stored. Weaviate can also handle changing this setting after you imported the data. Then the data is copied to the new replica nodes (if there are enough nodes), but note that this is experimental and will be more stable in the future.

Note:

Changing the replication factor after adding data is an experimental feature as of v1.17 and will become more stable in the future.

The data schema has a write consistency level of ALL, which means when you upload or update a schema, this will be sent to ALL nodes (via a coordinator node). The coordinator node waits for a successful acknowledgement from ALL nodes before sending a success message back to the client. This ensures a high consistent schema in your distributed Weaviate setup.

How to use: Queries​

When you add (write) or query (read) data, one or more replica nodes in the cluster will respond to the request. How many nodes need to send a successful response and acknowledgement to the coordinator node depends on the consistency_level. Available consistency levels are ONE, QUORUM (n/2+1) and ALL. Read more about consistency level here.

The consistency_level can be specified at query time.

Note:

In v1.17, only read queries that get data by ID have a tunable consistency level. All other queries have a consistency level of ALL. From v1.18, all write and read queries are tunable to either ONE, QUORUM or ALL.

# List all objects with a Bearer token
$ curl /v1/objects/{className}/{id}?consistency_level=ALL
import weaviate
from weaviate.data.replication import ConsistencyLevel

client = weaviate.Client("http://localhost:8080")

data_object = (
client.data_object.get_by_id(
uuid="36ddd591-2dee-4e7e-a3cc-eb86d30a4303",
class_name="MyClass",
consistency_level=ConsistencyLevel.QUORUM,
)
)

# The parameter "consistency_level" can be one of ConsistencyLevel.ALL, ConsistencyLevel.QUORUM,
# or ConsistencyLevel.ONE. Determines how many replicas must acknowledge a request before it is
# considered successful.

print(data_object)

More Resources​

If you can't find the answer to your question here, please look at the:

  1. Frequently Asked Questions. Or,
  2. Knowledge base of old issues. Or,
  3. For questions: Stackoverflow. Or,
  4. For issues: Github. Or,
  5. Ask your question in the Slack channel: Slack.