multi2vec-bind module enables Weaviate to use the ImageBind model to vectorize data at import time.
- This module is not available on Weaviate Cloud Services (WCS).
- Enabling this module will enable multiple
- Model encapsulated in Docker container.
- This module is not compatible with Auto-schema. You must define your classes manually as shown below.
multi2vec-bind allows Weaviate to generate vectors data containing any number of the following modalities:
- inertial measurement unit (IMU, i.e. accelerometer and gyroscope data)
- single channel depth images, and
- single channel thermal images.
Weaviate instance configuration
This module is not available on Weaviate Cloud Services.
Docker Compose file
multi2vec-bind, you must enable it in your Docker Compose file (e.g.
While you can do so manually, we recommend using the Weaviate configuration tool to generate the
Docker Compose file.
ENABLE_MODULES(Required): The modules to enable. Include
multi2vec-bindto enable the module.
DEFAULT_VECTORIZER_MODULE(Optional): The default vectorizer module. You can set this to
multi2vec-bindto make it the default for all classes.
BIND_INFERENCE_API(Required): The URL of the inference container.
image(Required): The image name of the inference container.
ENABLE_CUDA(Optional): Set to
1to enable GPU usage. Default is
This module will benefit greatly from GPU usage. Make sure to enable CUDA if you have a compatible GPU available (
You can configure how the module will behave in each class through the Weaviate schema.
You can set vectorizer behavior using the
moduleConfig section under each class and property:
vectorizer- what module to use to vectorize the data.
vectorizeClassName– whether to vectorize the class name. Default:
<media>Fields- property names to map for different modalities (under
- i.e. one or more of [
- i.e. one or more of [
weights- optional parameter to weigh the different modalities in producing the final vector.
skip– whether to skip vectorizing the property altogether. Default:
vectorizePropertyName– whether to vectorize the property name. Default:
dataType- the data type of the property. For use in the appropriate
<media>Fields, must be set to
The following example class definition sets the
multi2vec-bind module as the
vectorizer for the class
BindExample. It also sets:
nameproperty as a
textdatatype and as the text field,
imageproperty as a
blobdatatype and as the image field,
audioproperty as a
blobdatatype and as the audio field, and
videoproperty as a
blobdatatype and as the video field.
Example with weights
The following example adds weights for various properties, with the
textFields at 0.4, and the
videoFields at 0.2 each.
blobproperties must be in base64-encoded data.
blob data objects
blob property type data must be base64 encoded. To obtain the base64-encoded value of an image for example, you can use the helper methods in the Weaviate clients or run the following command:
cat my_image.png | base64
Additional search operators
multi2vec-bind vectorizer module will enable the following search operators:
These operators can be used to perform cross-modal search and retrieval.
This means that when using the
multi2vec-bind module any query using one modality (e.g. text) will include results in all available modalities, as all objects will be encoded into a single vector space.
It is your responsibility to evaluate whether the terms of its license(s), if any, are appropriate for your intended use.
For additional information, try these sources.