Skip to main content

250 Vector Compression

Course overview

Pre-requisites

This course is self-contained. However, we recommend that you go through one of the 101-level courses, such as that for working with text, your own vectors, or multimodal data.

As you work with more and more data, the sheer volume of it begins to impose further and further constraints on your ability to work with it. This is especially true the closer you get to production environments, where the cost of storage and the time it takes to process data can become significant.

This course will introduce you to data compression in Weaviate, and how it can be used to reduce your resource requirements and in turn improve performance or reduce costs.

Learning objectives

  Here, we will cover:

Learning Goals
  • What vector compression algorithms are available, how to use them and when to use them.

  By the time you are finished, you will be able to:

Learning Outcomes
  • Name available vector compression algorithms in Weaviate.
  • Create collections with vector compression enabled.
  • Configure vector compression parameters.
  • Select a compression algorithm for a given use case.

Units

1. Product quantization

Practical

What is product quantization (PQ), and how do you use it?

2. Binary quantization

Practical

What is binary quantization (BQ), and how do you use it?

3. Compression strategy

Theory

What compression algorithm and settings are right for me?