Skip to main content

Quickstart

Expected time: 30 minutes

What you will learn

This quickstart shows you how to combine Weaviate Cloud and Weaviate Embeddings to:

  1. Set up a Weaviate instance. (10 minutes)
  2. Add and vectorize your data. (10 minutes)
  3. Perform a semantic search and retrieval augmented generation (RAG). (10 minutes)

Notes:

  • The code examples here are self-contained. You can copy and paste them into your own environment to try them out.

Requirements

In order to perform Retrieval Augmented Generation (RAG) in the last step, you will need a Cohere account. You can use a free Cohere trial API key. If you have another preferred model provider, you can use that instead of Cohere.


Step 1: Set up Weaviate Cloud

1.1 Create a Weaviate Cloud account

  1. Open the Weaviate Cloud console.
  2. Click on Sign up.
  3. Provide an email address and password.
  4. After you confirm your email address, open the login page.
  5. Log in to Weaviate Cloud.
Create a cluster
Fill in the info and register a new Weaviate Cloud account.

1.2 Create a cluster

When you log into the Weaviate Cloud web console, the Clusters panel lists your clusters (1). There are no clusters when you log in to a new account.

  1. To create a cluster, click the Create cluster button on the Weaviate Cloud homepage (2).
Create a cluster
Click on this button to create a cluster.

Weaviate offers the following cluster options:

  • Sandbox clusters: free short-term cluster for development purposes.
  • Serverless clusters: permanent production-ready environment.
  1. Select the Sandbox tab.
  2. Choose a name for your cluster.
  3. Select a cloud region from the dropdown.
  4. Click the Create button.
Create a cluster
Choose a name and a cloud region to create a cluster.

It takes a minute or two to create the new cluster. When the cluster is ready, there will be a checkmark (✔️) next to the cluster name.

TIP: Use the latest Weaviate version!

When possible, try to use the latest Weaviate version. New releases include cutting-edge features, performance enhancements, and critical security updates to keep your application safe and up-to-date.


Step 2: Connect to your cluster

2.1 Install a client library

The Weaviate Cloud console includes a query interface, but most interactions rely on a Weaviate client. Clients are available in several programming languages. Choose one that makes sense for your project.

To install a client, follow these steps for your language:

Install the latest, Python client v4, by adding weaviate-client to your Python environment with pip:

pip install -U weaviate-client

2.2 Connect to your Weaviate Cloud instance

Now, you can connect to your Weaviate instance. Get the instance REST Endpoint URL and the Administrator API Key from the Weaviate Cloud console as shown below.

Get the (REST) endpoint URL
Grab the REST Endpoint URL.
Get the admin API key
Grab the Admin API key.

REST vs gRPC endpoints

Weaviate supports both REST and gRPC protocols. For Weaviate Cloud deployments, you only need to provide the REST endpoint URL - the client will automatically configure gRPC.

Once you have the REST Endpoint URL and the Admin API key, you can connect to the Sandbox instance and work with Weaviate.

The example below shows how to connect to Weaviate and perform a basic operation, like checking the cluster status.

quickstart_check_readiness.py
import weaviate
from weaviate.classes.init import Auth
import os

# Best practice: store your credentials in environment variables
wcd_url = os.environ["WCD_URL"]
wcd_api_key = os.environ["WCD_API_KEY"]

client = weaviate.connect_to_weaviate_cloud(
cluster_url=wcd_url, # Replace with your Weaviate Cloud URL
auth_credentials=Auth.api_key(wcd_api_key), # Replace with your Weaviate Cloud key
)

print(client.is_ready()) # Should print: `True`

client.close() # Free up resources

If you did not see any errors, you are ready to proceed. We will replace the simple cluster status check with more meaningful operations in the next steps.


Step 3: Populate your database

3.1 Define a collection

The following example creates a collection called Question with:

quickstart_create_collection.py
import weaviate
from weaviate.classes.init import Auth
from weaviate.classes.config import Configure
import os

# Best practice: store your credentials in environment variables
wcd_url = os.environ["WCD_URL"]
wcd_api_key = os.environ["WCD_API_KEY"]

client = weaviate.connect_to_weaviate_cloud(
cluster_url=wcd_url, # Replace with your Weaviate Cloud URL
auth_credentials=Auth.api_key(wcd_api_key), # Replace with your Weaviate Cloud key
)

questions = client.collections.create(
name="Question",
vectorizer_config=Configure.Vectorizer.text2vec_weaviate(), # Configure the Weaviate Embeddings integration
generative_config=Configure.Generative.cohere() # Configure the Cohere generative AI integration
)

client.close() # Free up resources

Run this code to create the collection to which you can add data.

What models are being used?

You can optionally specify the model in the collection definition. As we did not specify models in the collection definition above, these integrations will use the Weaviate-defined default models.


See the model providers integration section for more information.

3.2 Load the data

We can now add data to our collection.

The following example:

  • Loads objects, and
  • Adds objects to the target collection (Question) using a batch process.
Batch imports

(Batch imports) are the most efficient way to add large amounts of data, as it sends multiple objects in a single request. See the How-to: Batch import guide for more information.

quickstart_import.py
import weaviate
from weaviate.classes.init import Auth
import requests, json, os

# Best practice: store your credentials in environment variables
wcd_url = os.environ["WCD_URL"]
wcd_api_key = os.environ["WCD_API_KEY"]

client = weaviate.connect_to_weaviate_cloud(
cluster_url=wcd_url, # Replace with your Weaviate Cloud URL
auth_credentials=Auth.api_key(wcd_api_key), # Replace with your Weaviate Cloud key
)

resp = requests.get(
"https://raw.githubusercontent.com/weaviate-tutorials/quickstart/main/data/jeopardy_tiny.json"
)
data = json.loads(resp.text)

questions = client.collections.get("Question")

with questions.batch.dynamic() as batch:
for d in data:
batch.add_object({
"answer": d["Answer"],
"question": d["Question"],
"category": d["Category"],
})
if batch.number_errors > 10:
print("Batch import stopped due to excessive errors.")
break

failed_objects = questions.batch.failed_objects
if failed_objects:
print(f"Number of failed imports: {len(failed_objects)}")
print(f"First failed object: {failed_objects[0]}")

client.close() # Free up resources

During a batch import, any failed objects can be obtained through batch.failed_objects. Additionally, a running count of failed objects is maintained and can be accessed through batch.number_errors within the context manager. This counter can be used to stop the import process in order to investigate the failed objects or references. Find out more about error handling on the Python client reference page.

Run this code to add the demo data.


Step 4: Query your data

Weaviate provides a wide range of query tools to help you find the right data. We will try a few searches here.

Semantic search finds results based on meaning. This is called nearText in Weaviate.

The following example searches for 2 objects whose meaning is most similar to that of biology.

quickstart_neartext_query.py
import weaviate
from weaviate.classes.init import Auth
import os, json

# Best practice: store your credentials in environment variables
wcd_url = os.environ["WCD_URL"]
wcd_api_key = os.environ["WCD_API_KEY"]

client = weaviate.connect_to_weaviate_cloud(
cluster_url=wcd_url, # Replace with your Weaviate Cloud URL
auth_credentials=Auth.api_key(wcd_api_key), # Replace with your Weaviate Cloud key
)

questions = client.collections.get("Question")

response = questions.query.near_text(
query="biology",
limit=2
)

for obj in response.objects:
print(json.dumps(obj.properties, indent=2))

client.close() # Free up resources

Run this code to perform the query. Our query found entries for DNA and species.

Example full response in JSON format
{
{
"answer": "DNA",
"question": "In 1953 Watson & Crick built a model of the molecular structure of this, the gene-carrying substance",
"category": "SCIENCE"
},
{
"answer": "species",
"question": "2000 news: the Gunnison sage grouse isn't just another northern sage grouse, but a new one of this classification",
"category": "SCIENCE"
}
}

If you inspect the full response, you will see that the word biology does not appear anywhere.

Even so, Weaviate was able to return biology-related entries. This is made possible by vector embeddings that capture meaning. Under the hood, semantic search is powered by vectors, or vector embeddings.

Here is a diagram showing the workflow in Weaviate.

Where did the vectors come from?

Weaviate used the Weaviate Embeddings service to generate a vector embedding for each object during import. During the query, Weaviate similarly converted the query (biology) into a vector.

As we mentioned above, this is optional. See Starter Guide: Bring Your Own Vectors if you would prefer to provide your own vectors.

More search types available

Weaviate is capable of many types of searches. See, for example, our how-to guides on similarity searches, keyword searches, hybrid searches, and filtered searches.

4.2 Retrieval augmented generation

Retrieval augmented generation (RAG), also called generative search, combines the power of generative AI models such as large language models (LLMs) with the up-to-date truthfulness of a database.

RAG works by prompting a large language model (LLM) with a combination of a user query and data retrieved from a database.

This diagram shows the RAG workflow in Weaviate.

The following example combines the same search (for biology) with a prompt to generate a tweet.

quickstart_rag.py
import weaviate
from weaviate.classes.init import Auth
import os

# Best practice: store your credentials in environment variables
wcd_url = os.environ["WCD_URL"]
wcd_api_key = os.environ["WCD_API_KEY"]
cohere_api_key = os.environ["COHERE_APIKEY"]

client = weaviate.connect_to_weaviate_cloud(
cluster_url=wcd_url, # Replace with your Weaviate Cloud URL
auth_credentials=Auth.api_key(wcd_api_key), # Replace with your Weaviate Cloud key
headers={"X-Cohere-Api-Key": cohere_api_key}, # Replace with your Cohere API key
)

questions = client.collections.get("Question")

response = questions.generate.near_text(
query="biology",
limit=2,
grouped_task="Write a tweet with emojis about these facts."
)

print(response.generated) # Inspect the generated text

client.close() # Free up resources
Cohere API key in the header

Note that this code includes an additional header for the Cohere API key. Weaviate uses this key to access the Cohere generative AI model and perform retrieval augmented generation (RAG).

Run this code to perform the query. Here is one possible response (your response will likely be different).

🧬 In 1953 Watson & Crick built a model of the molecular structure of DNA, the gene-carrying substance! 🧬🔬

🦢 2000 news: the Gunnison sage grouse isn't just another northern sage grouse, but a new species! 🦢🌿 #ScienceFacts #DNA #SpeciesClassification

The response should be new, yet familiar. This is because you have seen the entries above for DNA and species in the semantic search section.

The power of RAG comes from the ability to transform your own data. Weaviate helps you in this journey by making it easy to perform a combined search & generation in just a few lines of code.


Next steps

Try these additional resources to learn more about Weaviate:

Support

For help with Serverless Cloud, Enterprise Cloud, and Bring Your Own Cloud accounts, contact Weaviate support directly to open a support ticket.

For questions and support from the Weaviate community, try these resources:

To add a support plan, contact Weaviate sales.