Skip to main content

GraphQL - Vector search parameters

LICENSE Weaviate on Stackoverflow badge Weaviate issues on Github badge Weaviate version badge Weaviate total Docker pulls badge Go Report Card

Setting search parameters​

Vector search parameters are added to GraphQL queries on the class level.

For example:

{
  Get {
    <Class> (
      <filter>: {
        variables: values
      }
    ){
      property
    }
  }
}

Built-in parameters​

B​uilt-in search parameters are available in all Weaviate instances and don't require any modules.​

nearVector​

This filter allows you to find data objects in the vicinity of an input vector. It's supported by the Get{} function.

  • Note: this argument is different from the GraphQL Explore{} function )
  • Note: Cannot use multiple 'near' arguments, or a 'near' argument along with an 'ask' filter

Variables​

VariablesMandatoryTypeDescription
vectoryes[float]This variable takes a vector embedding in the form of an array of floats. The array should have the same length as the vectors in this class.
distancenofloatThe required degree of similarity between an object's characteristics and the provided filter values. Can't be used together with the certainty variable. The interpretation of the value of the distance field depends on the distance metric used.
certaintynofloatNormalized Distance between the result item and the search vector. Normalized to be between 0 (identical vectors) and 1 (perfect opposite).. Can't be used together with the distance variable.

Example​

{
  Get{
    Publication(
      nearVector: {
        vector: [-0.36840257,0.13973749,-0.28994447,-0.18607682,0.20019795,0.15541431,-0.42353877,0.30262852,0.2724561,0.07069917,0.4877447,0.038771532,0.64523,-0.15907241,-0.3413626,-0.026682584,-0.63310874,-0.33411884,0.082939014,0.30305764,0.045918174,-0.21439327,-0.5005205,0.6210859,-0.2729049,-0.51221114,0.09680918,0.094923325,-0.15688285,-0.07325482,0.6588305,0.0523736,-0.14173415,-0.27428055,0.25526586,0.057506185,-0.3103442,0.028601522,0.124522656,0.66984487,0.12160647,-0.5090515,-0.540393,-0.39546522,-0.2201204,0.34625968,-0.21068871,0.21132985,0.048714135,0.09043683,0.3176081,-0.056684002,-0.12117501,-0.6591976,-0.26731065,0.42615625,0.33333477,-0.3240578,-0.18771006,0.2328068,-0.17239179,-0.33583146,-0.6556605,-0.10608161,-0.5135395,-0.25123677,-0.23004892,0.7036331,0.04456794,0.41253626,0.27872285,-0.28226635,0.11927197,-0.4677766,0.4343466,-0.17538455,0.10621233,0.95815116,0.23587844,-0.006406698,-0.10512518,-1.1125883,-0.37921682,0.040789194,0.676718,0.3369762,0.040712647,0.580487,0.20063736,-0.021220192,-0.09071747,-0.0023735985,0.30007777,-0.039925132,0.4035474,-0.2518212,-0.17846306,0.12371392,-0.0703354,-0.3752431,-0.652917,0.5952828,1.3426708,-0.08167235,-0.38515738,0.058423538,-0.08100355,-0.192886,0.3745164,-0.23291737,0.33326542,-0.6019264,-0.42822492,-0.6524583,-0.15210791,-0.5073593,0.022548754,-0.058033653,-0.47369233,-0.30890635,0.6338296,0.0017854869,0.1954949,0.99348027,-0.26558784,-0.058124136,1.149388,0.02915948,0.013422121,0.25484946,-0.030017598,-0.23879935,0.053123385,-0.36463016,-0.0024245526,0.1202083,-0.45966506,-0.34140104,-0.08484162,-0.03537422,-0.2817959,0.25044164,-0.5060605,0.1252808,-0.032539487,0.110069446,-0.20679846,-0.46421885,-0.4141739,0.26994973,-0.070687145,0.16862138,-0.20162229,0.22199251,-0.2771402,0.23653336,0.16585203,-0.08286354,-0.15343396,0.23893964,-0.7453282,-0.16549355,-0.1947069,0.46136436,0.22064126,0.28654936,-0.038697664,0.037633028,-0.80988157,0.5094175,-0.0920082,0.25405347,-0.64169943,0.43366328,-0.2999211,-0.4090591,0.11957859,0.00803617,-0.0433745,0.12818244,0.28464508,-0.31760025,0.16558012,-0.33553946,-0.3943465,0.59569097,-0.6524206,0.3683173,-0.60456693,0.2046492,0.46010277,0.24695799,0.2946015,0.11376746,-0.027988048,0.03749422,-0.16577742,0.23407385,-0.0231737,-0.023245076,0.08752677,0.2299883,0.35467404,0.046193745,-0.39828986,0.21079691,0.38396686,-0.0018698421,0.16047359,-0.057517264,-0.203534,0.23438136,-0.84250915,0.22371331,0.0058325706,0.30733636,0.19518353,-0.108008966,0.6509316,0.070131645,-0.24023099,0.28779706,0.2326336,0.07004021,-0.45955566,0.20426086,-0.37472793,-0.049604423,0.4537271,0.6133582,-1.0527759,-0.5472505,0.15193434,0.5296606,-0.11560251,0.07279209,0.40557706,0.2505283,0.24490519,0.017602902,-0.004647707,0.16608049,0.12576887,0.118216865,0.4403996,0.39552462,-0.22196701,-0.061155193,0.03693534,-0.4022908,0.3842317,-0.0831345,0.01930883,0.3446575,-0.2167439,-0.23994556,-0.09370326,-0.3671856,0.044011243,0.017895095,-0.019855855,-0.16416992,0.17858285,0.31287143,0.38368022,-0.006513525,0.45780763,-0.23027879,0.108570844,-0.4449492,-0.035763215,0.03818417,0.040017277,-0.17022872,-0.2622464,0.65610534,0.16720143,0.2515769,-0.23535803,0.62484455,0.16771325,-0.62404263,0.19176348,-0.72786695,0.18485649,-0.30914405,-0.3230534,-0.24064465,0.28841522,0.39792386,0.15618932,0.03928854,0.18277727,-0.101632096,0.1868196,-0.33366352,0.086561844,0.48557812,-0.6198209,-0.07978742]
      }
    ){
      name
      _additional {
        certainty
      }
    }
  }
}

🟢 Try out this GraphQL example in the Weaviate Console.

Additional information​

If the distance metric is cosine you can also use certainty instead of distance. Certainty normalizes the distance in a range of 0..1, where 0 represents a perfect opposite (cosine distance of 2) and 1 represents vectors with an identical angle (cosine distance of 0). Certainty is not available on non-cosine distance metrics.

nearObject​

This filter allows you to find data objects in the vicinity of other data objects by UUID. It's supported by the Get{} function.

  • Note: You cannot use multiple near<Media> arguments, or a near<Media> argument along with an ask argument.
  • Note: You can specify an object's id or beacon in the argument, along with a desired certainty.
  • Note that the first result will always be the object in the filter itself.
  • Near object search can also be combined with text2vec modules.

Variables​

VariablesMandatoryTypeDescription
idyesUUIDData object identifier in the uuid format.
beaconyesurlData object identifier in the beacon URL format. E.g., weaviate://<hostname>/<kind>/id.
distancenofloatThe required degree of similarity between an object's characteristics and the provided filter values. Can't be used together with the certainty variable. The interpretation of the value of the distance field depends on the distance metric used.
certaintynofloatNormalized Distance between the result item and the search vector. Normalized to be between 0 (identical vectors) and 1 (perfect opposite).. Can't be used together with the distance variable.

Example​

{
  Get{
    Publication(
      nearObject: {
        id: "e5dc4a4c-ef0f-3aed-89a3-a73435c6bbcf", # or weaviate://localhost/e5dc4a4c-ef0f-3aed-89a3-a73435c6bbcf
        distance: 0.6 # prior to v1.14, use certainty: 0.7
      }
    ){
      name
      _additional {
        certainty # only works if distance==cosine
        distance  # always works
      }
    }
  }
}

🟢 Try out this GraphQL example in the Weaviate Console.

hybrid​

This filter allows you to combine dense and sparse vectors to get the best of both search methods. It's supported by the Get{} function.

VariablesMandatoryDescription
hybridyesneed to specify that you want to use hybrid search
queryyessearch query
alphano (default is set to 0.75)weighting for each search algorithm
vectornooptional to supply your own vector
scorenoranked score that is assigned to each document
  • Note: alpha can be any number from 0 to 1
    • If alpha = 0, it is using a pure sparse search method
    • If alpha = 1, it is using a pure vector search method
    • If alpha = 0.5, it is weighing the sparse and vector method evenly

Example​

{
  Get {
    Article (
      hybrid: {
        query: "Fisherman that catches salmon"
        alpha: 0.5
      })
     {
      title
      summary
      _additional {score}
    }
  }
}

🟢 Try out this GraphQL example in the Weaviate Console.

Example with vector parameter​

If you're providing your own embeddings, you can add the vector query to the vector parameter. If Weaviate is handling the vectorization, then you can ignore the vector parameter and use the example code snippets above.

{
  Get {
    Article (
      hybrid: {
        query: "Fisherman that catches salmon"
        alpha: 0.5 
        vector: [1, 2, 3] # optional. Not needed if Weaviate handles the vectorization. If you provide your own embeddings, put the vector query here.
      })
     {
      title
      summary
      _additional {score}
    }
  }
}

bm25​

The bm25 operator performs a keyword (sparse vector) search, and uses the BM25F ranking function to score the results. BM25F (Best Match 25 with Extension to Multiple Weighted Fields) is an extended version of BM25 that applies the scoring algorithm to multiple fields (properties), producing better results.

The search is case-insensitive, and case matching does not confer a score advantage. Stop words are removed. Stemming is not supported yet.

Schema configuration​

The settings for BM25 are the free parameters k1 and b, and they are optional. The defaults (k1 = 1.2 and b = 0.75) work well for most cases. If necessary, they can be configured in the schema per class, and can optionally be overridden per property:

{
  "class": "string",
  "sparseIndexType": "bm25f",
  // Configuration of the sparse index
  "sparseIndexConfig": {
    "bm25f": {
      "b": 0.75,
      "k1": 1.2
    }
  },
  "properties": [
    {
      "name": "string",
      "description": "string",
      "dataType": [
        "string"
      ],
      // Property-level settings override the class-level settings
      "sparseIndexConfig": {
        "bm25f": {
          "b": 0.75,
          "k1": 1.2
        }
      },
      "indexInverted": true
    }
  ]
}

Variables​

The bm25 operator supports two variables:

  • query (mandatory) - the keyword search query
  • properties (optional) - array of properties (fields) to search in, defaulting to all properties in the class. Specific properties can be boosted by a factor specified as a number after the caret sign, for example properties: ["title^3", "description"].

Example query​

{
  Get {
    Article(
      bm25: {
        query: ["fox"],
        properties: ["title"]
      }
    ) {
      title
      _additional {
        score
      }
    }
  }
}

🟢 Try out this GraphQL example in the Weaviate Console.

GraphQL response​

The _additional object in the GraphQL result exposes the score:

{
  "_additional": {
    "score": "5.3201",
    "distance": null,  // always null
    "certainty": null  // always null
  }
}

Example response​

{
  "data": {
    "Get": {
      "Article": [
        {
          "_additional": {
            "certainty": null,
            "distance": null,
            "score": "3.4985464"
          },
          "title": "Tim Dowling: is the dog’s friendship with the fox sweet – or a bad omen?"
        }
      ]
    }
  },
  "errors": null
}

group​

You can use a group operator to combine similar concepts (aka entity merging). There are two ways of grouping objects with a semantic similarity together.

Variables​

VariablesMandatoryTypeDescription
typeyesstringYou can only show the closest concept (closest) or merge all similar entities into one single string (merge).
forceyesfloatThe force to apply for a particular movements. Must be between 0 and 1 where 0 is equivalent to no movement and 1 is equivalent to largest movement possible.

Example​

{
  Get {
    Publication(
      group:{
        type: merge,
        force:0.05
      }
    ) {
      name
    }
  }
}

This results in the following. Note that publications International New York Times, The New York Times Company and New York Times are merged. The property values that do not have an exact overlap will all be shown, with the value of the most central concept before the brackets.

{
  "data": {
    "Get": {
      "Publication": [
        {
          "name": "Vogue"
        },
        {
          "name": "Wired"
        },
        {
          "name": "Financial Times"
        },
        {
          "name": "New Yorker"
        },
        {
          "name": "The Economist"
        },
        {
          "name": "International New York Times (The New York Times Company, New York Times)"
        },
        {
          "name": "Wall Street Journal"
        },
        {
          "name": "CNN"
        },
        {
          "name": "Game Informer"
        },
        {
          "name": "The Guardian"
        },
        {
          "name": "Fox News"
        }
      ]
    }
  },
  "errors": null
}

Module specific parameters​

Module specific search parameters are made available in certain Weaviate modules.​

nearText​

Enabled by the modules: text2vec-openai, text2vec-transformers, text2vec-contextionary.

This filter allows you to find data objects in the vicinity of the vector representation of a single or multiple concepts. It's supported by the Get{} function.

Variables​

VariablesMandatoryTypeDescription
conceptsyes[string]An array of strings, this can be natural language queries or single words. If multiple strings are used, a centroid is calculated and used. Learn more about how the concepts are parsed here
certaintynofloatThe required degree of similarity between an object's characteristics and the provided filter values.
Values can be between 0 (no match) and 1 (perfect match).
Can't be used together with the distance variable.
distancenofloatNormalized Distance between the result item and the search vector.
The interpretation of the value of the distance field depends on the distance metric used.
Can't be used together with the certainty variable.
autocorrectnobooleanAutocorrect input text values
moveTonoobject{}Move your search term closer to another vector described by keywords
moveTo{concepts}no[string]An array of strings, this can be natural language queries or single words. If multiple strings are used, a centroid is calculated and used.
moveTo{objects}no[UUID]Object IDs to move the results to. This is used to "bias" NLP search results into a certain direction in vector space
moveTo{force}nofloatThe force to apply for a particular movements. Must be between 0 and 1 where 0 is equivalent to no movement and 1 is equivalent to largest movement possible
moveAwayFromnoobject{}Move your search term away from another vector described by keywords
moveAwayFrom{concepts}no[string]An array of strings, this can be natural language queries or single words. If multiple strings are used, a centroid is calculated and used.
moveAwayFrom{objects}no[UUID]Object IDs to move the results to. This is used to "bias" NLP search results into a certain direction in vector space
moveAwayFrom{force}nofloatThe force to apply for a particular movements. Must be between 0 and 1 where 0 is equivalent to no movement and 1 is equivalent to largest movement possible

Example I​

This example shows a basic overview of using the nearText filter.

{
  Get{
    Publication(
      nearText: {
        concepts: ["fashion"],
        distance: 0.6 # prior to v1.14 use "certainty" instead of "distance"
        moveAwayFrom: {
          concepts: ["finance"],
          force: 0.45
        },
        moveTo: {
          concepts: ["haute couture"],
          force: 0.85
        }
      }
    ){
      name
      _additional {
        certainty # only supported if distance==cosine.
        distance  # always supported
      }
    }
  }
}

🟢 Try out this GraphQL example in the Weaviate Console.

Example II​

You can also bias results toward other data objects' vector representations. For example, in this dataset, we have an ambiguous query on our news article dataset, which we bias towards an article called: "Tohoku: A Japan destination for all seasons."

{
  Get{
    Article(
      nearText: {
        concepts: ["traveling in Asia"],
        certainty: 0.7,
        moveTo: {
          objects: [{
            # this ID is of the article:
            # "Tohoku: A Japan destination for all seasons."
            id: "2faf2b7d-f185-30c0-8c80-a01b7cfeefb4"
          }]
          force: 0.85
        }
      }
    ){
      title
      summary
      _additional {
        certainty
      }
    }
  }
}

🟢 Try out this GraphQL example in the Weaviate Console.

Additional information​

Distance metrics​

If the distance metric is cosine you can also use certainty instead of distance. Certainty normalizes the distance in a range of 0..1, where 0 represents a perfect opposite (cosine distance of 2) and 1 represents vectors with an identical angle (cosine distance of 0). Certainty is not available on non-cosine distance metrics.

Concept parsing​

Strings written in the concepts array are your fuzzy search terms. An array of concepts is required to set in the Explore query, and all words in this array should be present in the Contextionary.

There are three ways to define the concepts array argument in the filter.

  • ["New York Times"] = one vector position is determined based on the occurrences of the words
  • ["New", "York", "Times"] = all concepts have a similar weight.
  • ["New York", "Times"] = a combination of the two above.

A practical example would be: concepts: ["beatles", "John Lennon"]

Semantic Path​

  • Only available in txt2vec-contextionary module

The semantic path returns an array of concepts from the query to the data object. This allows you to see which steps Weaviate took and how the query and data object are interpreted.

| Property | Description | | concept | the concept that is found in this step. | | distanceToNext | the distance to the next step (null for the last step). | | distanceToPrevious | this distance to the previous step (null for the first step). | | distanceToQuery | the distance of this step to the query. | | distanceToResult | the distance of the step to this result. |

Note: Building a semantic path is only possible if a nearText: {} filter is set as the explore term represents the beginning of the path and each search result represents the end of the path. Since nearText: {} queries are currently exclusively possible in GraphQL, the semanticPath is therefore not available in the REST API.

Example: showing a semantic path without edges.

{
  Get {
    Publication (
      nearText:{ 
          concepts: ["fashion"],
          distance: 0.6, #prior to v1.14 use certainty: 0.7
          moveAwayFrom: {
            concepts: ["finance"],
            force: 0.45
          },
          moveTo: {
            concepts: ["haute couture"],
            force: 0.85
          }
      }
    ) {
      name
      _additional {
        semanticPath{
          path {
            concept
            distanceToNext
            distanceToPrevious
            distanceToQuery
            distanceToResult
          }
        }
      }
    }
  }
}

Ask​

Enabled by the module: Question Answering.

This filter allows you to return answers to questions by running the results through a Q&A model.

Variables​

VariablesMandatoryTypeDescription
questionyesstringThe question to be answered.
certaintynofloatDesired minimal certainty or confidence of answer to the question. The higher the value, the stricter the search becomes. The lower the value, the fuzzier the search becomes. If no certainty is set, any answer that could be extracted will be returned
propertiesno[string]The properties of the queries Class which contains text. If no properties are set, all are considered.
reranknobooleanIf enabled, the qna module will rerank the result based on the answer score. For example, if the 3rd result - as determined by the previous (semantic) search contained the most likely answer, result 3 will be pushed to position 1, etc. Not supported prior to v1.10.0

Example​

{
  Get {
    Article(
      ask: {
        question: "Who is the king of the Netherlands?",
        properties: ["summary"],
        rerank: false # supported from v1.10.0 on
      }, 
      limit: 1
    ) {
      title
      _additional {
        answer {
          hasAnswer
          certainty
          property
          result
          startPosition
          endPosition
        }
      }
    }
  }
}