GraphQL - Additional properties

     

Introduction

GraphQL additional properties can be used on data objects in Get{} Queries to get additional information about the returned data objects. Which additional properties are available depends on the modules that are attached to Weaviate. The fields id, certainty, featureProjection and classification are available from Weaviate Core. On nested GraphQL fields (references to other data classes), only the id can be returned. Explanation on specific additional properties can be found on the module pages, see for example text2vec-contextionary.

Example

An example query getting the UUID and the distance.

GraphQL

{
  Get {
    Article (
      nearText: {
        concepts: ["fashion"],
      }
    ) {
      title
      _additional {
        id
        distance
      }
    }
  }
}

Python

import weaviate

client = weaviate.Client("http://localhost:8080")

near_text_filter = {
  "concepts": ["fashion"]
}

query_result = (
  client.query
  .get("Article", "title")
  .with_additional(["id", "distance"])
  .with_near_text(near_text_filter)
  .do()
)

print(query_result)

JavaScript

const weaviate = require('weaviate-client');

const client = weaviate.client({
  scheme: 'http',
  host: 'localhost:8080',
});

client.graphql
      .get()
      .withClassName('Article')
      .withNearText({concepts: ['fashion']})
      .withFields('title _additional { id distance }')
      .do()
      .then(res => {
        console.log(res)
      })
      .catch(err => {
        console.error(err)
      });

Go

package main

import (
  "context"
  "fmt"

  "github.com/weaviate/weaviate-go-client/v4/weaviate"
  "github.com/weaviate/weaviate-go-client/v4/weaviate/graphql"
)

func main() {
  cfg := weaviate.Config{
    Host:   "localhost:8080",
    Scheme: "http",
  }
  client := weaviate.New(cfg)

  className := "Article"
  fields := []graphql.Field{
    {Name: "title"},
    {Name: "_additional", Fields: []graphql.Field{
      {Name: "id"},
      {Name: "distance"},
    }},
  }

  explore := client.GraphQL().NearTextArgBuilder().
    WithConcepts([]string{"fashion"})
  ctx := context.Background()

  result, err := client.GraphQL().Get().
    WithClassName(className).
    WithFields(fields...).
    WithNearText(explore).
    Do(ctx)

  if err != nil {
    panic(err)
  }
  fmt.Printf("%v", result)
}

Java

package technology.semi.weaviate;

import technology.semi.weaviate.client.Config;
import technology.semi.weaviate.client.WeaviateClient;
import technology.semi.weaviate.client.base.Result;
import technology.semi.weaviate.client.v1.graphql.model.GraphQLResponse;
import technology.semi.weaviate.client.v1.graphql.query.argument.NearTextArgument;
import technology.semi.weaviate.client.v1.graphql.query.fields.Field;

public class App {
  public static void main(String[] args) {
    Config config = new Config("http", "localhost:8080");
    WeaviateClient client = new WeaviateClient(config);

    Field title = Field.builder()
      .name("title")
      .build();
    Field _additional = Field.builder()
      .name("_additional")
      .fields(new Field[]{
        Field.builder().name("id").build(),
        Field.builder().name("distance").build()
      }).build();

    NearTextArgument explore = client.graphQL().arguments().nearTextArgBuilder()
      .concepts(new String[]{ "fashion" })
      .build();

    Result<GraphQLResponse> result = client.graphQL().get()
      .withClassName("Article")
      .withFields(title, _additional)
      .withNearText(explore)
      .run();
    if (result.hasErrors()) {
      System.out.println(result.getError());
      return;
    }
    System.out.println(result.getResult());
  }
}

Curl

$ echo '{ 
  "query": "{
    Get {
      Article (
        nearText: {
          concepts: [\"fashion\"],
        }
      ) {
        title
        _additional {
          id
          distance
        }
      }
    }
  }"
}' | curl \
    -X POST \
    -H 'Content-Type: application/json' \
    -d @- \
    http://localhost:8080/v1/graphql

_additional property

​All additional properties can be set in the reserved _additional{} property.​

For example:

{
  Get {
    Class {
      property
      _additional {
        # property 1
        # property 2
        # etc...
      }
    }
  }
}

id

The id field contains the unique uuid of the data object.

{
  Get {
    Class {
      property
      _additional {
        id
      }
    }
  }
}

vector

The vector fields contains the vector representation of the data object

{
  Get {
    Class {
      property
      _additional {
        vector
      }
    }
  }
}

creationTimeUnix

The creationTimeUnix field is the timestamp of when the data object was created.

{
  Get {
    Class {
      property
      _additional {
        creationTimeUnix
      }
    }
  }
}

lastUpdateTimeUnix

The lastUpdateTimeUnix field is the timestamp of when the data object was last updated.

{
  Get {
    Class {
      property
      _additional {
        lastUpdateTimeUnix
      }
    }
  }
}

distance

Any time a vector search is involved, the distance can be displayed to show the distance between the query vector and each result. The distance is the raw distance metric that was used as part of the vector search. For example, if the distance metric is cosine, distance will return a number between 0 and 2. See the full overview of distance metrics and the expected distance ranges.

A distance would be typical in any place that you retrieve objects using a vector, for example Get {} with nearObject, nearVector, or near<Media>. The results are ordered by the ascending distance - unless you explicitly sort by another property.

A lower value for a distance always means that two vectors are closer to another, than a higher value. Depending on the distance metric used, this can also mean that distances would return negative values. For example, if dot product distance is used, a distance of -50 would indicate more similarity between a vector pair than 20. See the distances page for details and exact definitions.

Note that the distance field was introduced in v1.14.0.

Certainty (only for cosine distance)

Prior to v1.14, certainty was the only way to display vector similarity in the results. certainty is an opinionated measure that always returns a number between 0 and 1. It is therefore only usable with fixed-range distance metrics, such as cosine.

For a class with cosine distance metrics, the certainty is a normalization of the distance using the formula:

certainty = 1 - distance/2

Given that a cosine distance is always a number between 0 and 2, this will result in certainties between 0 and 1, with 1 indicating identical vectors, and 0 indiating opposing angles. This definition only exists in an angular space.

Classification

When a data-object has been subjected to classification, you can get additional information about how the object was classified by running the following command:

GraphQL

{
  Get {
    Article (
      nearText: {
        concepts: ["fashion"],
      }
    ) {
      title
      _additional {
        classification {
          basedOn
          classifiedFields
          completed
          id
          scope
        }
      }
    }
  }
}

Python

import weaviate

client = weaviate.Client("http://localhost:8080")

near_text_filter = {
  "concepts": ["fashion"]
}

additional_props = {
  "classification" : ["basedOn", "classifiedFields", "completed", "id"]
}

query_result = (
  client.query
  .get("Article", "title")
  .with_additional(additional_props)
  .with_near_text(near_text_filter)
  .do()
)

print(query_result)

JavaScript

const weaviate = require('weaviate-client');

const client = weaviate.client({
  scheme: 'http',
  host: 'localhost:8080',
});

client.graphql
      .get()
      .withClassName('Article')
      .withFields('title _additional { classification { basedOn classifiedFields completed id scope } }')
      .do()
      .then(res => {
        console.log(res)
      })
      .catch(err => {
        console.error(err)
      });

Go

package main

import (
  "context"
  "fmt"

  "github.com/weaviate/weaviate-go-client/v4/weaviate"
  "github.com/weaviate/weaviate-go-client/v4/weaviate/graphql"
)

func main() {
  cfg := weaviate.Config{
    Host:   "localhost:8080",
    Scheme: "http",
  }
  client := weaviate.New(cfg)

  className := "Article"
  title := graphql.Field{Name: "title"}
  _additional := graphql.Field{Name: "_additional", Fields: []graphql.Field{
    {Name: "classification", Fields: []graphql.Field{
      {Name: "basedOn"},
      {Name: "classifiedFields"},
      {Name: "completed"},
      {Name: "completed"},
    }},
  }}

  ctx := context.Background()

  result, err := client.GraphQL().Get().
    WithClassName(className).
    WithFields(title, _additional).
    Do(ctx)

  if err != nil {
    panic(err)
  }
  fmt.Printf("%v", result)
}

Java

package technology.semi.weaviate;

import technology.semi.weaviate.client.Config;
import technology.semi.weaviate.client.WeaviateClient;
import technology.semi.weaviate.client.base.Result;
import technology.semi.weaviate.client.v1.graphql.model.GraphQLResponse;
import technology.semi.weaviate.client.v1.graphql.query.fields.Field;

public class App {
  public static void main(String[] args) {
    Config config = new Config("http", "localhost:8080");
    WeaviateClient client = new WeaviateClient(config);

    Field title = Field.builder()
      .name("title")
      .build();
    Field _additional = Field.builder()
      .name("_additional")
      .fields(new Field[]{
        Field.builder()
          .name("classification")
          .fields(new Field[]{
            Field.builder().name("basedOn").build(),
            Field.builder().name("classifiedFields").build(),
            Field.builder().name("completed").build(),
            Field.builder().name("id").build(),
            Field.builder().name("scope").build()
          }).build()
      }).build();

    Result<GraphQLResponse> result = client.graphQL().get()
      .withClassName("Article")
      .withFields(title, _additional)
      .run();
    if (result.hasErrors()) {
      System.out.println(result.getError());
      return;
    }
    System.out.println(result.getResult());
  }
}

Curl

$ echo '{ 
  "query": "{
    Get {
      Article (
        nearText: {
          concepts: [\"fashion\"],
        }
      ) {
        title
        _additional {
          classification {
            basedOn
            classifiedFields
            completed
            id
            scope
          }
        }
      }
    }
  }"
}' | curl \
    -X POST \
    -H 'Content-Type: application/json' \
    -d @- \
    http://localhost:8080/v1/graphql

🟢 Try out this GraphQL example in the Weaviate Console.

🟠 Whoops, the demo service is temporarily offline.

Feature Projection

Because Weaviate stores all data in a vector space, you can visualize the results according to the results of your query. The feature projection is intended to reduce the dimensionality of the object's vector into something easily suitable for visualizing, such as 2d or 3d. The underlying algorithm is exchangeable, the first algorithm to be provided is t-SNE.

To tweak the feature projection optional parameters (currently GraphQL-only) can be provided. The values and their defaults are:

Parameter	Type	Default	Implication
dimensions	int	2	Target dimensionality, usually 2 or 3
algorithm	string	tsne	Algorithm to be used, currently supported: tsne
perplexity	int	min(5, len(results)-1)	The t-SNE perplexity value, must be smaller than the n-1 where n is the number of results to be visualized
learningRate	int	25	The t-SNE learning rate
iterations	int	100	The number of iterations the t-SNE algorithm runs. Higher values lead to more stable results at the cost of a larger response time

An example with default settings:

GraphQL

{
  Get {
    Article (
      nearText:{ 
        concepts:["music"],
        moveTo: {
          concepts: ["beatles"],
          force: 0.5
        }
      }
    ) {
      title
      _additional {
        featureProjection(dimensions: 2) {
          vector
        }
      }
    }
  }
}

Python

import weaviate

client = weaviate.Client("http://localhost:8080")

near_text_filter = {
  "concepts": ["music"],
  "moveTo": {
    "concepts": ["beatles"],
    "force": 0.5
  }
}
additional_clause = {
  "featureProjection": [
    "vector"
  ]
}

additional_setting = {
  "dimensions": 2
}

query_result = (
  client.query
  .get("Article", "title")
  .with_near_text(near_text_filter)
  .with_additional(
    (additional_clause, additional_setting)
  )
  .do()
)
print(query_result)

JavaScript

const weaviate = require('weaviate-client');

const client = weaviate.client({
  scheme: 'http',
  host: 'localhost:8080',
});

client.graphql
      .get()
      .withClassName('Article')
      .withFields('title _additional {featureProjection(dimensions: 2) { vector }}')
      .withNearText({
        concepts:['music'],
        moveTo: {
          concepts: ['beatles'],
          force: 0.5
        }
      })
      .withLimit(12)
      .do()
      .then(res => {
        console.log(res)
      })
      .catch(err => {
        console.error(err)
      });

Go

package main

import (
  "context"
  "fmt"

  "github.com/weaviate/weaviate-go-client/v4/weaviate"
  "github.com/weaviate/weaviate-go-client/v4/weaviate/graphql"
)

func main() {
  cfg := weaviate.Config{
    Host:   "localhost:8080",
    Scheme: "http",
  }
  client := weaviate.New(cfg)

  className := "Article"
  fields := []graphql.Field{
    {Name: "title"},
    {Name: "_additional", Fields: []graphql.Field{
      {Name: "featureProjection(dimensions: 2)", Fields: []graphql.Field{
        {Name: "vector"},
      }},
    }},
  }
  concepts := []string{"music"}
  moveTo := &graphql.MoveParameters{
    Concepts: []string{"beatles"},
    Force:    0.5,
  }
  nearText := client.GraphQL().NearTextArgBuilder().
    WithConcepts(concepts).
    WithMoveTo(moveTo)
  ctx := context.Background()

  result, err := client.GraphQL().Get().
    WithClassName(className).
    WithFields(fields...).
    WithNearText(nearText).
    WithLimit(12).
    Do(ctx)

  if err != nil {
    panic(err)
  }
  fmt.Printf("%v", result)
}

Java

package technology.semi.weaviate;

import technology.semi.weaviate.client.Config;
import technology.semi.weaviate.client.WeaviateClient;
import technology.semi.weaviate.client.base.Result;
import technology.semi.weaviate.client.v1.graphql.model.GraphQLResponse;
import technology.semi.weaviate.client.v1.graphql.query.argument.NearTextArgument;
import technology.semi.weaviate.client.v1.graphql.query.argument.NearTextMoveParameters;
import technology.semi.weaviate.client.v1.graphql.query.fields.Field;

public class App {
  public static void main(String[] args) {
    Config config = new Config("http", "localhost:8080");
    WeaviateClient client = new WeaviateClient(config);
    String className = "Article";

    Field title = Field.builder().name("title").build();
    Field _additional = Field.builder()
      .name("_additional")
      .fields(new Field[]{
        Field.builder()
          .name("featureProjection(dimensions: 2)")
          .fields(new Field[]{
            Field.builder().name("vector").build()
          }).build()
      }).build();

    String[] concepts = new String[]{ "music" };
    NearTextMoveParameters moveTo = client.graphQL().arguments().nearTextMoveParameterBuilder()
      .concepts(new String[]{ "beatles" }).force(0.5f).build();
    NearTextArgument nearText = client.graphQL().arguments().nearTextArgBuilder()
      .concepts(concepts)
      .moveTo(moveTo)
      .build();

    Result<GraphQLResponse> result = client.graphQL().get()
      .withClassName(className)
      .withFields(title, _additional)
      .withNearText(nearText)
      .withLimit(12)
      .run();
    if (result.hasErrors()) {
      System.out.println(result.getError());
      return;
    }
    System.out.println(result.getResult());
  }
}

Curl

$ echo '{ 
    "query": "{
      Get {
        Article (
          nearText:{ 
            concepts:[\"music\"],
            moveTo: {
              concepts: [\"beatles\"],
              force: 0.5
            }
          }
        ) {
          title
          _additional {
            featureProjection(dimensions: 2) {
              vector
            }
          }
        }
      }
    }"
  }' | curl \
    -X POST \
    -H 'Content-Type: application/json' \
    -d @- \
    http://localhost:8080/v1/graphql

🟢 Try out this GraphQL example in the Weaviate Console.

🟠 Whoops, the demo service is temporarily offline.

The above result can be plotted as follows (where the result in red is the first result):

Best practices and notes

• There is no request size limit (other than the global 10,000 items request limit) which can be used on a featureProjection query. However, due to the O(n^2) complexity of the t-SNE algorithm, large requests size have an exponential effect on the response time. We recommend to keep the request size at or below 100 items, as we have noticed drastic increases in response time thereafter.
• Feature Projection happens in real-time, per query. The dimensions returned have no meaning across queries.
• Currently only root elements (not resolved cross-references) are taken into consideration for the featureProjection.
• Due to the relatively high cost of the underlying algorithm, we recommend to limit requests including a featureProjection in high-load situations where response time matters. Avoid parallel requests including a featureProjection, so that some threads stay available to serve other, time-critical requests.

More Resources

If you can't find the answer to your question here, please look at the:

1. Frequently Asked Questions. Or,
2. Knowledge base of old issues. Or,
3. For questions: Stackoverflow. Or,
4. For issues: Github. Or,
5. Ask your question in the Slack channel: Slack.