DocsIntegrationsHaystackGet Started

🌾 Haystack Integration

This integration allows you to easily trace your Haystack pipelines in the Langfuse UI.

Haystack is the open-source Python framework developed by deepset. Its modular design allows users to implement custom pipelines to build production-ready LLM applications, like retrieval-augmented generative pipelines and state-of-the-art search systems. It integrates with Hugging Face Transformers, Elasticsearch, OpenSearch, OpenAI, Cohere, Anthropic and others, making it an extremely popular framework for teams of all sizes.

Thanks to the team at deepset for developing this integration (langfuse-haystack package). These docs are adapted from their write up, which you can read here.

How Can Langfuse Help?

The langfuse-haystack package integrates tracing capabilities into Haystack (2.x) pipelines using Langfuse.

This can be helpful in the following ways:

  • Capture comprehensive details of the execution trace and model performance
    • Latency
    • Token usage
    • Cost
    • Scores
  • Capture the full context of the execution, independently observe retrieval and summarization
  • Build fine-tuning and testing datasets
  • Monitor model performance
  • Pinpoint areas for improvement

Installation and Setup

Install packages

Install Haystack, Langfuse and the integration package.

pip install haystack-ai langfuse-haystack langfuse

Set Environment Variables

  1. Langfuse: You can find your Langfuse public and private API keys in the project settings.
  2. Haystack: Enable HAYSTACK_CONTENT_TRACING_ENABLED. You must set this variable before importing LangfuseConnector.
import os
 
# Get keys for your project from the project settings page
# https://cloud.langfuse.com
os.environ["LANGFUSE_PUBLIC_KEY"] = "pk-lf-..."
os.environ["LANGFUSE_SECRET_KEY"] = "sk-lf-..."
os.environ["LANGFUSE_HOST"] = "https://cloud.langfuse.com" # 🇪🇺 EU region
# os.environ["LANGFUSE_HOST"] = "https://us.cloud.langfuse.com" # 🇺🇸 US region
 
# Enable Haystack content tracing
os.environ["HAYSTACK_CONTENT_TRACING_ENABLED"] = "True"

Add LangfuseConnector to your pipeline

Add LangfuseConnector to the pipeline as a tracer. There’s no need to connect it to any other component. The LangfuseConnector will automatically trace the operations and data flow within the pipeline. Then add the other components, like the text embedder, retriever, prompt builder and the model, and connect them together in the order they will be used in the pipeline.

from haystack import Pipeline
from haystack_integrations.components.connectors.langfuse import LangfuseConnector
 
# Example pipeline
example_pipeline = Pipeline()
 
# Add LangfuseConnector to the pipeline
example_pipeline.add_component("tracer", LangfuseConnector("Example Pipeline"))
 
# Add other components and use the pipeline
# ...

Done! Traces and metrics from your Haystack application are now automatically tracked in Langfuse. Whenever you run your application, traces and metrics are immediately visible in the Langfuse UI.

Example Usage

We compiled a cookbook to showcase the integration:

Introduction Video

Haystack Integration Overview

Acknowledgement

We’re thrilled to collaborate with the Haystack team to give the best possible experience to devs when building complex RAG applications. Thanks to them for developing this intgeration.

GitHub Discussions

Was this page useful?

Questions? We're here to help

Subscribe to updates