DSPy
Overview
DSPy from Stanford NLP is a framework for programming language models.
DSPy and Weaviate
- DSPy introduces two key concepts, the programming model and optimizers.
- Programing model: The programming model lets you define a series of components that make a language model request. Components include input fields, output fields, task descriptions, and calls to a vector database like Weaviate.
- Optimizers: Optimizers compile your DSPy program to tune the language model prompt and/or the weights.
Weaviate is integrated with DSPy through the retriever model!
Resources
The resources are broken into categories:
- Hands on Learning: Build your technical understanding with end-to-end tutorials.
- Read and Listen: Develop your conceptual understanding of these technologies.
Hands-on Learning
Getting Started with RAG in DSPy
Learn about DSPy and how to build a program: Installation, settings, datasets, LLM metrics, DSPy programming model, and optimization.
Open →Getting Started with RAG in DSPy
Learn about DSPy and how to build a program: Installation, settings, datasets, LLM metrics, DSPy programming model, and optimization.
Open →DSPy + Weaviate for the Next Generation of LLM Apps
Build a 4-layer DSPy program for generating blog posts from queries.
Open →DSPy + Weaviate for the Next Generation of LLM Apps
Build a 4-layer DSPy program for generating blog posts from queries.
Open →RAG with Persona
Build a compound AI system with DSPy, Cohere, and Weaviate, where you'll add a persona to the language model.
Open →RAG with Persona
Build a compound AI system with DSPy, Cohere, and Weaviate, where you'll add a persona to the language model.
Open →Adding Depth to RAG Programs
Enhancing DSPy programs by integrating unique input-output examples and multiple LLMs.
Open →Adding Depth to RAG Programs
Enhancing DSPy programs by integrating unique input-output examples and multiple LLMs.
Open →Hurricane: Writing Blog Posts with Generative Feedback Loops
Introduction to Hurricane, a web app for demonstrating generative feedback loops with blog posts.
Open →Hurricane: Writing Blog Posts with Generative Feedback Loops
Introduction to Hurricane, a web app for demonstrating generative feedback loops with blog posts.
Read →Structured Outputs with DSPy
The three methods for structuring outputs in DSPy programs.
Open →Structured Outputs with DSPy
The three methods for structuring outputs in DSPy programs.
Open →Building RAG with Command R+ from Cohere, DSPy, and Weaviate
Overview of Command R+ with a quick RAG demo in DSPy.
Open →Building RAG with Command R+ from Cohere, DSPy, and Weaviate
Overview of Command R+ with a quick RAG demo in DSPy.
Open →Advanced Optimizers in DSPy
Dive into optimizing DSPy programs with various techniques.
Open →Llama 3 RAG Demo with DSPy Optimization, Ollama, and Weaviate
Integrating Llama3 with DSPy and optimizing prompts with MIPRO.
Open →BigQuery and Weaviate orchestrated with DSPy
Build an end-to-end RAG pipeline that uses BigQuery and Weaviate using DSPy.
Open →DSPy and Weaviate Query Agent
Use the Query Agent as a Tool with DSPy
Open →Read & Listen
DSPy and ColBERT with Omar Khattab! - Weaviate Podcast #85
Omar Khattab joins the Weaviate podcast to discuss DSPy and ColBERT.
Open →DSPy Explained
Learn about the core concepts of DSPy. Walk through the introduction notebooks to compile a simple retrieve-then-read RAG program and Multi-Hop RAG Program.
Open →XMC.dspy with Karel D'Oosterlinck - Weaviate Podcast #87
Karel D'Oosterlinck joins the Weaviate podcast to discuss IReRa (Infer-Retrieve-Rank).
Open →Intro to DSPy: Goodbye Prompting, Hello Programming
Overview of DSPy and how it solves the fragility problem in LLM-based applications.
Read →Fine-Tuning Cohere’s Reranker
Generate synthetic data with DSPy to fine-tune Cohere’s reranker model.
Read →Your Language Model Deserves Better Prompting
Overview of the DSPy optimizers for prompt tuning.
Read →