250 Vector compression
Course overview
This course is self-contained. However, we recommend that you go through one of the 101-level courses, such as that for working with text, your own vectors, or multimodal data.
As you work with more and more data, the sheer volume of it begins to impose further and further constraints on your ability to work with it. This is especially true the closer you get to production environments, where the cost of storage and the time it takes to process data can become significant.
This course will introduce you to data compression in Weaviate, and how it can be used to reduce your resource requirements and in turn improve performance or reduce costs.
Learning objectives
Here, we will cover:
Learning Goals- What vector compression algorithms are available, how to use them and when to use them.
By the time you are finished, you will be able to:
Learning Outcomes- Name available vector compression algorithms in Weaviate.
- Create collections with vector compression enabled.
- Configure vector compression parameters.
- Select a compression algorithm for a given use case.
Units
1. Product quantization
What is product quantization (PQ), and how do you use it?
2. Binary quantization
What is binary quantization (BQ), and how do you use it?
3. Compression strategy
What compression algorithm and settings are right for me?