Skip to main content

Example datasets

LICENSEย Weaviate on Stackoverflow badgeย Weaviate issues on GitHub badgeย Weaviate version badgeย Weaviate total Docker pulls badgeย Go Report Card

Multi-Modal Text/Image search using CLIPโ€‹

This example application spins up a Weaviate instance using the multi2vec-clip module, imports a few sample images (you can add your own images, too!) and provides a very simple search frontend in React using the TypeScript/JavaScript client.

Get started here

Semantic Search through Wikipediaโ€‹

We imported the complete English language Wikipedia article dataset into a single Weaviate instance to conduct semantic search queries through the Wikipedia articles, besides this, we've made all the graph relations between the articles too. We have made the import scripts, pre-processed articles, and backup available so that you can run the complete setup yourself.

Get started here

Meta AI Research - Biggraph on Wikidataโ€‹

We have imported the complete Wikidata PBG model into a Weaviate to search through the entire dataset in < 50 milliseconds (excluding internet latency). The demo GraphQL queries contain both pure vector search and scalar and vector searched mixed queries.

Get started here

News publicationsโ€‹

This dataset contains +/- 1000 random news articles from; Financial Times, New York Times, Guardian, Wallstreet Journal, CNN, Fox News, The Economist, New Yorker, Wired, Vogue, Game Informer.

It includes a schema with classes for Article, Publication, Category and Author.

Run with Docker Composeโ€‹

If you want to run this dataset locally, you can run it in one go with Docker Compose.

You can run this demo dataset with any text2vec module. Examples:

Text2vec-contextionaryโ€‹

The Docker Compose file contains both Weaviate with the text2vec-contextionary module and the dataset.

Download the Docker Compose file

$ curl -o docker-compose.yml https://raw.githubusercontent.com/weaviate/weaviate-examples/main/weaviate-contextionary-newspublications/docker-compose.yaml

Run Docker (optional: run with -d to run Docker in the background)

$ docker-compose up

Weaviate will be available and preloaded with the News Articles demo dataset on:

  • http://localhost:8080/
  • Via the Console: connect to https://demo.dataset.playground.semi.technology.

Text2vec-transformers (without GPU)โ€‹

The Docker Compose file contains both Weaviate with the text2vec-contextionary module, NER module, Q&A module and spellcheck module, and the dataset.

Download the Docker Compose file

$ curl -o docker-compose.yml https://raw.githubusercontent.com/weaviate/weaviate-examples/main/weaviate-transformers-newspublications/docker-compose.yaml

Run Docker (optional: run with -d to run Docker in the background)

$ docker-compose up

Weaviate will be available and preloaded with the News Articles demo dataset on:

  • http://localhost:8080/
  • Via the Console: connect to https://demo.dataset.playground.semi.technology.

Text2vec-transformers (with GPU enabled)โ€‹

The Docker Compose file contains both Weaviate with the text2vec-contextionary module, NER module, Q&A module and spellcheck module, and the dataset. GPU should be available on your machine when running this configuration.

Download the Docker Compose file

$ curl -o docker-compose.yml https://raw.githubusercontent.com/weaviate/weaviate-examples/main/weaviate-transformers-newspublications/docker-compose-gpu.yaml

Run Docker (optional: run with -d to run Docker in the background)

$ docker-compose up

Weaviate will be available and preloaded with the News Articles demo dataset on:

  • http://localhost:8080/
  • Via the Console: connect to https://demo.dataset.playground.semi.technology.

Run manuallyโ€‹

If you have your own version of Weaviate running on an external host or localhost without Docker Compose;

# WEAVIATE ORIGIN (e.g., https://foobar.weaviate.network), note paragraph basics for setting the local IP
$ export WEAVIATE_ORIGIN=WEAVIATE_ORIGIN
# Optionally you can specify which newspaper language you want (only two options `cache-en` or `cache-nl`, if not specified by default it is `cache-en` )
$ export CACHE_DIR=<YOUR_CHOICE_OF_CACHE_DIR>
# Optionally you can set the batch size (if not specified by default 200)
$ export BATCH_SIZE=<YOUR_CHOICE_OF_BATCH_SIZE>
# Make sure to replace WEAVIATE_ORIGIN with the Weaviate origin as mentioned in the basics above
$ docker run -it -e weaviate_host=$WEAVIATE_ORIGIN -e cache_dir-$CACHE_DIR -e batch_size=$BATCH_SIZE semitechnologies/weaviate-demo-newspublications:latest

Usage with Docker on local with Docker Compose;

Note: run this from the same directory where the Weaviate Docker Compose files are located

{% raw %}

# This gets the Weaviate container name and because the docker uses only lowercase we need to do it too (Can be found manually if 'tr' does not work for you)
$ export WEAVIATE_ID=$(echo ${PWD##*/}_weaviate_1 | tr "[:upper:]" "[:lower:]")
# WEAVIATE ORIGIN (e.g., http://localhost:8080), note the paragraph "basics" for setting the local IP
$ export WEAVIATE_ORIGIN="http://$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' $WEAVIATE_ID):8080"
# WEAVIATE NETWORK (see paragraph: Running on the localhost)
$ export WEAVIATE_NETWORK=$(docker inspect -f '{{range .NetworkSettings.Networks}}{{.NetworkID}}{{end}}' $WEAVIATE_ID)
# Optionally you can specify which newspaper language you want (only two options `cache-en` or `cache-nl`, if not specified by default it is `cache-en` )
$ export CACHE_DIR=<YOUR_CHOICE_OF_CACHE_DIR>
# Optionally you can set the batch size (if not specified by default 200)
$ export BATCH_SIZE=<YOUR_CHOICE_OF_BATCH_SIZE>
# Run docker
$ docker run -it --network=$WEAVIATE_NETWORK -e weaviate_host=$WEAVIATE_ORIGIN -e cache_dir-$CACHE_DIR -e batch_size=$BATCH_SIZE semitechnologies/weaviate-demo-newspublications:latest

{% endraw %}

More Resourcesโ€‹

If you can't find the answer to your question here, please look at the:

  1. Frequently Asked Questions. Or,
  2. Knowledge base of old issues. Or,
  3. For questions: Stackoverflow. Or,
  4. For more involved discussion: Weaviate Community Forum. Or,
  5. We also have a Slack channel.