reranker-transformers
Introduction
- The
reranker-transformers
module enables reranking search results using sentence transformers models. - The
reranker-transformers
module is run on your own inference container with a pre-trained language transformer model.
How to enable
Weaviate Cloud Services
The reranker-transformers
module is not available on the WCS.
Weaviate open source
Add reranker-transformers
to the ENABLE_MODULES
environment variable.
Below is an example Docker Compose file, which will spin up Weaviate with the reranker-transformers
module (as well as text2vec-openai
).
It also configures reranker-transformers
to use the cross-encoder/ms-marco-MiniLM-L-6-v2
model, with CUDA acceleration disabled.
---
version: '3.4'
services:
weaviate:
command:
- --host
- 0.0.0.0
- --port
- '8080'
- --scheme
- http
image: semitechnologies/weaviate:1.21.4
ports:
- 8080:8080
restart: on-failure:0
environment:
RERANKER_INFERENCE_API: 'http://reranker-transformers:8080'
OPENAI_APIKEY: $OPENAI_APIKEY
QUERY_DEFAULTS_LIMIT: 25
AUTHENTICATION_ANONYMOUS_ACCESS_ENABLED: 'true'
PERSISTENCE_DATA_PATH: "./data"
DEFAULT_VECTORIZER_MODULE: 'text2vec-openai'
ENABLE_MODULES: 'text2vec-openai,reranker-transformers'
CLUSTER_HOSTNAME: 'node1'
reranker-transformers:
image: semitechnologies/reranker-transformers:cross-encoder-ms-marco-MiniLM-L-6-v2
environment:
ENABLE_CUDA: '0'
...
Configuration
The reranker-transformers
module can be configured for any class in the schema.
Reranker selection
If there is only one reranker
module enabled, you don't need to do anything. The reranker
module will be used by default.
Where multiple reranker
modules are enabled, you must specify the reranker module to be used for each class. You can do this by adding the desired reranker in the moduleConfig
section of the schema, even without any further settings.
Set reranker for a class
{
"classes": [
{
"class": "Document",
...,
"moduleConfig": {
"reranker-transformers": {}, // This will configure the 'Document' class to use the 'reranker-transformers' module
}
}
]
}
Model selection
The reranker-transformers module enables using sentence transformers models as a second stage re-ranking for vector, bm25 and hybrid search results.
With reranker-transformers
module, you must set the model using environment variables as shown above.
The reranker-transformers
module supports the following models:
cross-encoder/ms-marco-MiniLM-L-6-v2
cross-encoder/ms-marco-MiniLM-L-2-v2
cross-encoder/ms-marco-TinyBERT-L-2-v2
These pre-trained models are open-sourced on Hugging Face. The cross-encoder/ms-marco-MiniLM-L-6-v2
model, for example, provides approximately the same benchmark performance as the largest model (L-12) when evaluated on MS-MARCO (39.01 vs. 39.02).
Usage
Queries
- Using this module will enable the
rerank
GraphQL _additional property. - For usage examples, see the Howto: Search - Reranking page.
More resources
For additional information, try these sources.