Next steps
Congratulations! You have completed this course on tokenization for the inverted index. We hope you found it helpful and informative.
There are many more resources available to help you continue your learning journey.
Documentation
- Refereces: Configuration: Tokenization
- Refereces: Configuration: Stopwords
- Concepts: Inverted index
- Concepts: Filtering
note
As a reminder, for non-English texts, especially those which do not rely on spaces between words, try the trigram or gse tokenization methods which were added in Weaviate v1.24 for such cases.
Stay in touch!
We are constantly improving our documentation, so please keep an eye out for new resources and updates, by signing up for our newsletter or following us on social media (Twitter, LinkedIn).
See you soon! 👋
Questions and feedback
If you have any questions or feedback, let us know in the user forum.