Replication
Weaviate instances can be replicated to increase availability and read throughput, and to enable zero-downtime upgrades. On this page, you will learn how to set replication for your Weaviate instance.
For more about how replication is designed and built in Weaviate, see the Replication Architecture pages.
How to configure
v1.25
In Weaviate v1.25
, a replication factor cannot be changed once it is set.
This is due to the schema consensus algorithm change in v1.25
. This will be improved in future versions.
Replication is disabled by default and can be enabled per data class in the collection configuration. This means you can set different replication factors per class in your dataset. To enable replication on a class, the replication factor has to be set, which looks like the following:
{
"class": "ExampleClass",
"properties": [
{
"name": "exampleProperty",
"dataType": [
"text"
]
}
],
"replicationConfig": {
"factor": 3 # Integer, default 1. How many copies of this class will be stored.
}
}
Here's an example for all clients:
- Python Client v4
- Python Client v3
- JS/TS Client v3
- JS/TS Client v2
- Go
- Java
- Curl
from weaviate.classes.config import Configure
client.collections.create(
"Article",
replication_config=Configure.replication(
factor=3
)
)
class_obj = {
"class": "Article",
"replicationConfig": {
"factor": 3,
},
}
client.schema.create_class(class_obj)
const newCollection = await client.collections.create({
name: 'Article',
replication: weaviate.configure.replication({
factor: 3
}),
})
// The returned value is the full collection definition, showing all defaults
console.log(JSON.stringify(await newCollection.config.get(), null, 2));
const classWithReplication = {
class: 'Article',
replicationConfig: {
factor: 3,
},
};
// Add the class to the schema
result = await client.schema
.classCreator()
.withClass(classWithReplication)
.do();
// The returned value is the full class definition, showing all defaults
console.log(JSON.stringify(result, null, 2));
package main
import (
"context"
"github.com/weaviate/weaviate-go-client/v4/weaviate"
"github.com/weaviate/weaviate/entities/models"
)
func main() {
cfg := weaviate.Config{
Host: "localhost:8080",
Scheme: "http",
}
client, err := weaviate.NewClient(cfg)
if err != nil {
panic(err)
}
classObj := &models.Class{
Class: "Article",
Properties: []*models.Property{
{
DataType: []string{"string"},
Name: "title",
}
},
ReplicationConfig: &models.ReplicationConfig{
Factor: 3,
}
}
err := client.Schema().ClassCreator().WithClass(classObj).Do(context.Background())
if err != nil {
panic(err)
}
}
package io.weaviate;
import java.util.ArrayList;
import io.weaviate.client.Config;
import io.weaviate.client.WeaviateClient;
import io.weaviate.client.base.Result;
import io.weaviate.client.v1.schema.model.DataType;
import io.weaviate.client.v1.schema.model.Property;
import io.weaviate.client.v1.schema.model.WeaviateClass;
import io.weaviate.client.v1.misc.model.ReplicationConfig;
public class App {
public static void main(String[] args) {
Config config = new Config("http", "localhost:8080");
WeaviateClient client = new WeaviateClient(config);
ReplicationConfig replicationConfig = ReplicationConfig.builder()
.factor(3)
.build();
WeaviateClass clazz = WeaviateClass.builder()
.className("Article")
.replicationConfig(replicationConfig)
.properties(new ArrayList() { {
add(Property.builder()
.dataType(new ArrayList(){ { add(DataType.STRING); } })
.name("title")
.build());
} })
.build();
Result<Boolean> result = client.schema().classCreator().withClass(clazz).run();
if (result.hasErrors()) {
System.out.println(result.getError());
return;
}
System.out.println(result.getResult());
}
}
curl \
-X POST \
-H "Content-Type: application/json" \
-d '{
"class": "Article",
"properties": [
{
"dataType": [
"string"
],
"description": "Title of the article",
"name": "title"
}
],
"replicationConfig": {
"factor": 3
}
}' \
http://localhost:8080/v1/schema
When you set this replication factor in the data schema before you add data, you will have 3 replicas of the data stored. Weaviate can also handle changing this setting after you imported the data. Then the data is copied to the new replica nodes (if there are enough nodes), but note that this is experimental and will be more stable in the future.
Changing the replication factor after adding data is an experimental feature as of v1.17 and will become more stable in the future.
The data schema has a write consistency level of ALL
, which means when you upload or update a schema, this will be sent to ALL
nodes (via a coordinator node). The coordinator node waits for a successful acknowledgement from ALL
nodes before sending a success message back to the client. This ensures a highly consistent schema in your distributed Weaviate setup.
How to use: Queries
When you add (write) or query (read) data, one or more replica nodes in the cluster will respond to the request. How many nodes need to send a successful response and acknowledgement to the coordinator node depends on the consistency_level
. Available consistency levels are ONE
, QUORUM
(replication_factor / 2 + 1) and ALL
.
The consistency_level
can be specified at query time:
# Get an object by ID, with consistency level ONE
curl "http://localhost:8080/v1/objects/{ClassName}/{id}?consistency_level=ONE"
In v1.17, only read queries that get data by ID had a tunable consistency level. All other object-specific REST endpoints (read or write) used the consistency level ALL
. Starting with v1.18, all write and read queries are tunable to either ONE
, QUORUM
(default) or ALL
. GraphQL endpoints use the consistency level ONE
(in both versions).
- Python Client v4
- Python Client v3
- JS/TS Client v3
- JS/TS Client v2
- Go
- Java
- Curl
from weaviate.classes.config import ConsistencyLevel
questions = client.collections.get(collection_name).with_consistency_level(
consistency_level=ConsistencyLevel.QUORUM
)
response = collection.query.fetch_object_by_id("36ddd591-2dee-4e7e-a3cc-eb86d30a4303")
# The parameter passed to `withConsistencyLevel` can be one of:
# * 'ALL',
# * 'QUORUM' (default), or
# * 'ONE'.
#
# It determines how many replicas must acknowledge a request
# before it is considered successful.
for o in response.objects:
print(o.properties) # Inspect returned objects
import weaviate
from weaviate.data.replication import ConsistencyLevel
client = weaviate.Client("http://localhost:8080")
data_object = (
client.data_object.get_by_id(
uuid="36ddd591-2dee-4e7e-a3cc-eb86d30a4303",
class_name="MyClass",
consistency_level=ConsistencyLevel.ONE,
)
)
# The parameter "consistency_level" can be one of ConsistencyLevel.ALL,
# ConsistencyLevel.QUORUM (default), or ConsistencyLevel.ONE. Determines how many
# replicas must acknowledge a request before it is considered successful.
print(data_object)
const collectionName = 'Article';
const myCollection = client.collections.get(collectionName).withConsistency('QUORUM');
const result = await myCollection.query.fetchObjectById("36ddd591-2dee-4e7e-a3cc-eb86d30a4303")
console.log(JSON.stringify(result, null, 2));
// The parameter passed to `withConsistencyLevel` can be one of:
// * 'ALL',
// * 'QUORUM' (default), or
// * 'ONE'.
//
// It determines how many replicas must acknowledge a request
// before it is considered successful.
import weaviate from 'weaviate-ts-client';
const client = weaviate.client({
scheme: 'http',
host: 'localhost:8080',
});
const response = await client.data
.getterById()
.withClassName('MyClass')
.withId('36ddd591-2dee-4e7e-a3cc-eb86d30a4303')
.withConsistencyLevel('ONE') // default QUORUM
.do();
console.log(JSON.stringify(response, null, 2));
// The parameter passed to `withConsistencyLevel` can be one of:
// * 'ALL',
// * 'QUORUM' (default), or
// * 'ONE'.
//
// It determines how many replicas must acknowledge a request
// before it is considered successful.
package main
import (
"context"
"fmt"
"github.com/weaviate/weaviate-go-client/v4/weaviate/data/replication" // for consistency levels
"github.com/weaviate/weaviate-go-client/v4/weaviate"
)
func main() {
cfg := weaviate.Config{
Host: "localhost:8080",
Scheme: "http",
}
client, err := weaviate.NewClient(cfg)
if err != nil {
panic(err)
}
data, err := client.Data().ObjectsGetter().
WithClassName("MyClass").
WithID("36ddd591-2dee-4e7e-a3cc-eb86d30a4303").
WithConsistencyLevel(replication.ConsistencyLevel.ONE). // default QUORUM
Do(context.Background())
if err != nil {
panic(err)
}
fmt.Printf("%v", data)
}
// The parameter passed to "WithConsistencyLevel" can be one of:
// * replication.ConsistencyLevel.ALL,
// * replication.ConsistencyLevel.QUORUM (default), or
// * replication.ConsistencyLevel.ONE.
//
// It determines how many replicas must acknowledge a request
// before it is considered successful.
package io.weaviate;
import java.util.List;
import io.weaviate.client.Config;
import io.weaviate.client.WeaviateClient;
import io.weaviate.client.base.Result;
import io.weaviate.client.v1.data.model.WeaviateObject;
import io.weaviate.client.v1.data.replication.model.ConsistencyLevel;
public class App {
public static void main(String[] args) {
Config config = new Config("http", "localhost:8080");
WeaviateClient client = new WeaviateClient(config);
Result<List<WeaviateObject>> result = client.data().objectsGetter()
.withClassName("MyClass")
.withID("36ddd591-2dee-4e7e-a3cc-eb86d30a4303")
.withConsistencyLevel(ConsistencyLevel.ONE) // default QUORUM
.run();
if (result.hasErrors()) {
System.out.println(result.getError());
return;
}
System.out.println(result.getResult());
}
}
// The parameter passed to `withConsistencyLevel` can be one of:
// * ConsistencyLevel.ALL,
// * ConsistencyLevel.QUORUM (default), or
// * ConsistencyLevel.ONE.
//
// It determines how many replicas must acknowledge a request
// before it is considered successful.
curl "http://localhost:8080/v1/objects/MyClass/36ddd591-2dee-4e7e-a3cc-eb86d30a4303?consistency_level=QUORUM"
# The parameter "consistency_level" can be one of ALL, QUORUM (default), or ONE. Determines how many
# replicas must acknowledge a request before it is considered successful.
# curl "/v1/objects/{ClassName}/{id}?consistency_level=ONE"
Related pages
Questions and feedback
If you have any questions or feedback, let us know in the user forum.