Weaviate Transformation Agent

Technical Preview

Sign up here for notifications on Weaviate Agents, or visit this page to see the latest updates and provide feedback.

Do not use in production

The Weaviate Transformation Agent is designed to modify data in Weaviate in place. While the Agent is in technical preview, do not use it in a production environment. The Agent may not work as expected, and the data in your Weaviate instance may be affected in unexpected ways.

The Weaviate Transformation Agent is an agentic service designed to augment and transform data using generative models. Use the Transformation Agent to append new properties and/or update existing properties of data on existing objects in Weaviate.

This can help you to improve the quality of your objects in your Weaviate collections, ready for further use in your applications.

Architecture

The Transformation Agent is provided as a service on Weaviate Cloud. It updates existing Weaviate objects by either appending new properties or updating existing properties.

Provide a set of instructions to the Transformation Agent, such as the collection to update and existing properties to review and the instructions. The Transformation Agent will then perform the specified operations on the specified objects in Weaviate.

Total object count unaffected

This would not change the number of objects in Weaviate, but would update the properties of the specified objects.

Transformation Agent: visualized workflow

To transform existing objects, the Transformation Agent follows the workflows shown below.

• The Transformation Agent retrieves the existing objects from Weaviate, based on the specified criteria (steps 1-2).
• The Transformation Agent works with a generative model to create the property value, based on the instructions provided and the context of the specified existing properties (steps 3-4).
• Update the transformed objects in Weaviate. Weaviate vectorizes the data as needed using the specified vectorizer integration. (Steps 5-7)
• Receive the job status from Weaviate, which is returned to the user (Step 8).

Update properties on existing objects

When updating properties on existing objects, the Transformation Agent replaces the existing property values with the new values. The workflow for this operation is shown below.

Append new properties to existing objects

When appending properties to existing objects, the Transformation Agent adds the new values to the objects as new properties. The workflow for this operation is shown below.

Basic Usage

Here is an overview of how to use the this Weaviate Agent. For more detailed information, refer to the Usage page.

Prerequisites

This Agent is available exclusively for use with a Weaviate Cloud instance, and a supported version of the Weaviate client library.

Example Usage

To use the Transformation Agent, instantiate it with the following inputs:

• An instance of the Weaviate client (e.g. the WeaviateClient object in Python), connected to a Weaviate Cloud instance.
• Name of the target collection to be transformed.
• A list of the transformation operations to be performed.

And then start the operations.

Transformation operations are asynchronous. Each operation will return a workflow ID to the user. The user can then use this ID to check its status.

Python

# [🚧 CAUTION 🚧] DO NOT USE ON PRODUCTION DATA.
# The Transformation Agent will modify your data in place.
# While the Transformation Agent is in technical preview,
# it is recommended to only use it on test data.

import os
import weaviate
from weaviate.classes.init import Auth
from weaviate.agents.transformation import TransformationAgent
from weaviate.agents.classes import Operations


headers = {
    # Provide your required API key(s), e.g. Cohere, OpenAI, etc. for the configured vectorizer(s)
    "X-INFERENCE-PROVIDER-API-KEY": os.environ.get("YOUR_INFERENCE_PROVIDER_KEY", ""),
}

client = weaviate.connect_to_weaviate_cloud(
    cluster_url=os.environ.get("WEAVIATE_URL"),
    auth_credentials=Auth.api_key(os.environ.get("WEAVIATE_API_KEY")),
    headers=headers,
)

add_topics = Operations.append_property(
    property_name="topics",
    data_type=DataType.TEXT_ARRAY,
    view_properties=["abstract", "title"],
    instruction="""Create a list of topic tags based on the title and abstract.
    Topics should be distinct from eachother. Provide a maximum of 5 topics.
    Group similar topics under one topic tag.""",
)

agent = TransformationAgent(
    client=client,
    collection="ArxivPapers",
    operations=[add_topics],
)

response = agent.update_all()  # The response is a TransformationResponse object

agent.get_status(workflow_id=response.workflow_id)  # Use the workflow_id to check the status of each workflow

  API docs

The transformed attributes will become available on each object as the transformation progresses.

Further Documentation

For more detailed information on how to use this Agent, refer to the Usage page.

Questions and feedback

Changelog and feedback

The official changelog for Weaviate Agents can be found here. If you have feedback, such as feature requests, bug reports or questions, please submit them here, where you will be able to see the status of your feedback and vote on others' feedback.

If you have any questions or feedback, let us know in the user forum.

• Technical questions
• Documentation feedback