Skip to main content

Why observability matters

Debugging AI agents in production is challenging - you can’t print() your way through distributed systems. When your agent makes unexpected decisions, you need to trace every LLM call, tool invocation, and reasoning step to understand what happened. The Observability module gives you full visibility into your AI agents using OpenTelemetry. It exports traces to Langfuse (default) or your existing observability stack (Datadog, Grafana, custom OTLP), enabling you to:
  • Debug issues: See exactly what your agent did and why
  • Monitor performance: Track latency, token usage, and costs
  • Audit decisions: Maintain compliance with full trace history
Built on OpenTelemetry standards. Switch backends anytime without changing your code. Your traces are portable and vendor-independent.

Choose your backend

The SDK supports multiple observability backends. Langfuse is the default and requires no backend specification. For other platforms, specify the backend when initializing.
Default backend - Optimized for AI observability with prompt management, cost tracking, and user feedback.Setup:
  1. Set environment variables:
.env
LANGFUSE_PUBLIC_KEY=pk-lf-xxx
LANGFUSE_SECRET_KEY=sk-lf-xxx
# Optional: use for self-hosted or custom Langfuse instance
LANGFUSE_HOST=https://cloud.langfuse.com
  1. Initialize (no backend parameter needed):
from bb_ai_sdk.observability import init
init(agent_name="my-agent")  # Uses Langfuse by default
Langfuse observability features:
  • Detailed tracing: Captures spans for LLM calls, tool calls, and agent loop reasoning
  • Tool call tracking: Each tool invocation is captured as a span with function names
  • Cost tracking: Monitor token usage and costs per trace, session, or user
  • Performance metrics: Track latency, throughput, and error rates
  • Session grouping: Group related traces by session for end-to-end analysis
  • Real-time insights: View traces and metrics in real-time dashboards
  • Debugging tools: Inspect full request/response payloads and trace hierarchies
Don’t have Langfuse credentials? Refer to our Onboarding guide.
Never commit API keys to version control. Add .env to your .gitignore.

Quick start

Get observability working in 3 steps:
1

Install dependencies

Install the required instrumentation package:
pip install openinference-instrumentation-openai
2

Set up environment variables

Create a .env file with your backend credentials:
.env
# Langfuse (default backend)
LANGFUSE_PUBLIC_KEY=pk-lf-xxx
LANGFUSE_SECRET_KEY=sk-lf-xxx

# AI gateway credentials
AI_GATEWAY_API_KEY=your-api-key
AI_GATEWAY_ENDPOINT=your-ai-gateway-endpoint

# Optional: proxy settings (if needed)
NO_PROXY=localhost,127.0.0.1,cloud.langfuse.com
Load environment variables before importing the SDK:
import dotenv
dotenv.load_dotenv()
3

Initialize and instrument

Initialize observability and instrument the OpenAI library:
from bb_ai_sdk.observability import init, get_tracer_provider
from openinference.instrumentation.openai import OpenAIInstrumentor

# Initialize observability (call once per application)
init(agent_name="my-agent")

# Instrument OpenAI library to capture LLM calls
provider = get_tracer_provider()
OpenAIInstrumentor().instrument(tracer_provider=provider)
⚠️ IMPORTANT:
  • Call init() only ONCE per application session
  • You must instrument the OpenAI library - init() alone doesn’t generate traces
  • Initialize before creating AI Gateway instances or making LLM calls
All LLM calls are now automatically traced and exported to your chosen backend!
💡 Enhance your traces with optional parametersThe init() function accepts many optional parameters to better organize and filter your traces. For example:
  • environment: Tag traces by environment (development, staging, production)
  • organization_id: Enable multi-tenant cost attribution and filtering
  • organization_name: Human-readable organization name for dashboards
  • resource_attributes: Custom metadata for filtering and analysis
See Advanced Settings for configuration options and the API Reference for the complete parameter list.

Setup by use case with AI gateway

Choose the setup that matches your application architecture:

Vanilla code

For applications using AIGateway directly (no agentic frameworks):
from bb_ai_sdk.observability import init, get_tracer_provider
from bb_ai_sdk.ai_gateway import AIGateway
from openinference.instrumentation.openai import OpenAIInstrumentor

# Step 1: initialize observability
init(agent_name="my-agent")

# Step 2: instrument OpenAI library
# Openaiinstrumentor monkey-patches the OpenAI library to capture all LLM calls
provider = get_tracer_provider()
OpenAIInstrumentor().instrument(tracer_provider=provider)

# Step 3: create AI gateway - all calls are now traced
gateway = AIGateway.create(
    model_id="gpt-4o",
    agent_id="550e8400-e29b-41d4-a716-446655440000"
)

# Step 4: make LLM calls - automatically traced!
response = gateway.chat.completions.create(
    model="gpt-4o",
    messages=[{"role": "user", "content": "Hello!"}]
)
Why OpenAIInstrumentor?When using AIGateway with vanilla code (no agentic frameworks), OpenAIInstrumentor is the standard way to instrument OpenAI-compatible clients. It monkey-patches the OpenAI library at the SDK level, capturing all LLM calls made through AIGateway’s OpenAI-compatible interface.

Agno framework

For applications using the Agno framework:
from bb_ai_sdk.observability import init, get_tracer_provider
from openinference.instrumentation.agno import AgnoInstrumentor

# Step 1: initialize observability
init(agent_name="my-agent")

# Step 2: instrument Agno framework
# Agnoinstrumentor monkey-patches the Agno framework to capture all LLM calls
provider = get_tracer_provider()
AgnoInstrumentor().instrument(tracer_provider=provider)

# Your Agno agent code here - all calls are automatically traced
Install the Agno instrumentation package:
pip install openinference-instrumentation-agno

LangChain or LangGraph

For applications using LangChain or LangGraph, use callback handlers:
from bb_ai_sdk.observability import init, LangChainOpenTelemetryCallbackHandler
from bb_ai_sdk.ai_gateway import AIGateway
from bb_ai_sdk.ai_gateway.adapters.langchain import to_langchain
from langchain.prompts import ChatPromptTemplate
from langchain.schema.output_parser import StrOutputParser

# Initialize observability
init(agent_name="langchain-agent")

# Create callback handler
callback = LangChainOpenTelemetryCallbackHandler()

# Setup LangChain
gateway = AIGateway.create(model_id="gpt-4o", agent_id="...")
model = to_langchain(gateway)

prompt = ChatPromptTemplate.from_template("Tell me about {topic}")
chain = prompt | model | StrOutputParser()

# Use callback with chain invocation
result = chain.invoke(
    {"topic": "AI observability"},
    config={"callbacks": [callback]}
)
The handler automatically traces:
  • Chain execution (start, end, error)
  • LLM calls with token usage
  • Tool invocations
  • Retriever operations

Custom tracing

In addition to tracing LLM calls (via instrumentors), trace your own functions to see the complete picture of your agent’s behavior.

The @trace decorator

Wrap any function to automatically trace it:
from bb_ai_sdk.observability import init, trace

init(agent_name="my-agent")

@trace()
def process_user_request(user_input: str) -> str:
    """This function is automatically traced."""
    # Your logic here
    return "processed result"

@trace(name="custom-span-name")
def validate_input(data: dict) -> bool:
    """Trace with a custom name for clarity."""
    return True

Adding custom attributes

Add attributes to filter and analyze traces:
@trace(attributes={
    "prompt.version": "v1.2.3",
    "prompt.name": "customer-support-prompt",
    "user.id": "user-123",
    "session.id": "session-456",
    "experiment.variant": "A"
})
def run_experiment():
    """Traces include all custom attributes for analysis."""
    pass
Common uses for attributes: prompt versioning, A/B testing, user tracking, and cost attribution per customer.

The trace_context context manager

For fine-grained control within a function:
from bb_ai_sdk.observability import init, trace_context

init(agent_name="my-agent")

def complex_operation():
    with trace_context("multi-step-operation") as span:
        # Add events during execution
        span.add_event("Step 1: Validating input")
        validate_input()
        
        span.add_event("Step 2: Processing data")
        result = process_data()
        
        # Add attributes dynamically
        span.set_attribute("result.count", len(result))
        span.add_event("Step 3: Complete")
        
        return result

Configuration

Environment variables

Configure backend credentials via environment variables (recommended for security):
.env
# Langfuse (default backend)
LANGFUSE_PUBLIC_KEY=pk-lf-xxx
LANGFUSE_SECRET_KEY=sk-lf-xxx
LANGFUSE_HOST=https://cloud.langfuse.com  # Optional, defaults to cloud

# Datadog (if using backend="datadog")
DD_API_KEY=your-datadog-api-key

# Grafana (if using backend="grafana")
GRAFANA_BEARER_TOKEN=your-grafana-token

# Custom otlp endpoint
OTEL_EXPORTER_OTLP_ENDPOINT=https://your-otlp-endpoint.com
OTEL_EXPORTER_OTLP_HEADERS=Authorization=Bearer xxx

# Control observability
OBSERVABILITY_ENABLED=true  # Set to false to disable

Proxy configuration

When working within the Backbase network, configure proxy settings correctly:
.env
# Backbase web proxy
HTTP_PROXY=http://webproxy.infra.backbase.cloud:8888
HTTPS_PROXY=http://webproxy.infra.backbase.cloud:8888

# Bypass proxy for observability endpoints
NO_PROXY=localhost,127.0.0.1,cloud.langfuse.com,*.langfuse.com,langfuse
NO_PROXY is required for both Langfuse Cloud and local Langfuse instances.The Backbase web proxy can interfere with OTLP trace exports. If your traces aren’t appearing, proxy misconfiguration is a common cause.

Advanced settings

Batch export configuration

Traces are batched for efficient network usage. By default, traces may take up to 5 seconds to appear:
init(
    agent_name="my-agent",
    otlp_batch_size=512,        # Max spans per batch (default: 512)
    otlp_batch_timeout=5.0,     # Max wait time in seconds (default: 5.0)
    otlp_max_queue_size=10000   # Max buffered spans (default: 10000)
)
For faster feedback during development, reduce the batch timeout:
init(agent_name="dev-agent", otlp_batch_timeout=1.0)  # Export every 1 second
Don’t use low timeouts in production - it increases network overhead.

Custom resource attributes

Add metadata to all traces for filtering:
init(
    agent_name="my-agent",
    resource_attributes={
        "service.version": "1.2.3",
        "deployment.region": "us-east-1",
        "deployment.cluster": "prod-cluster-1",
        "custom.tenant.id": "tenant-123"
    }
)
See API Reference for all available parameters.

Best practices

1. Initialize once at startup

Call init() only once at application startup:
# App.py - at the top
from bb_ai_sdk.observability import init
init(agent_name="my-agent")

# Then import and use other modules
Multiple calls to init() will trigger OpenTelemetry “Overriding not allowed” warnings and may cause unexpected behavior.

2. Use optional parameters for better organization

Take advantage of init() optional parameters to improve trace organization and filtering:
# ✅ good: use optional parameters for better organization
init(
    agent_name="customer-support-agent",
    environment="production",              # Filter traces by environment
    organization_id="org-123",            # Enable multi-tenant cost attribution
    organization_name="Acme Corp",        # Human-readable for dashboards
    resource_attributes={
        "service.version": "1.2.3",       # Track deployments
        "deployment.region": "us-east-1", # Filter by region
        "team": "platform-team"           # Group by team
    }
)

# ❌ bad: minimal configuration limits filtering options
init(agent_name="my-agent")  # Can't filter by environment or organization
Use environment to separate development, staging, and production traces. Use organization_id and organization_name for multi-tenant applications to track costs and filter traces per organization.

3. Use meaningful span names

Name spans descriptively for easy trace navigation:
# ✅ good: descriptive names
@trace(name="validate-user-input")
@trace(name="fetch-customer-data")

# ❌ bad: generic names
@trace(name="process")
@trace(name="step1")

4. Add business context

Include attributes that help with business analysis:
@trace(attributes={
    "customer.tier": "premium",
    "request.type": "balance-inquiry",
    "channel": "mobile-app"
})

5. Keep credentials out of code

Always use environment variables:
# ✅ good: use environment
init(agent_name="my-agent", backend="langfuse")

# ❌ bad: hardcode credentials
init(agent_name="my-agent", langfuse_public_key="pk-xxx")

Debugging and logging

Enable verbose logging for troubleshooting:
import logging
import sys
import http.client

# Enable HTTP connection debugging
http.client.HTTPConnection.debuglevel = 1

# Configure detailed logging
logging.basicConfig(
    level=logging.DEBUG,
    format="%(asctime)s [%(levelname)s] %(name)s - %(message)s",
    handlers=[logging.StreamHandler(sys.stdout)],
)

# Enable opentelemetry component logging
logging.getLogger("opentelemetry").setLevel(logging.DEBUG)
logging.getLogger("opentelemetry.sdk").setLevel(logging.DEBUG)
logging.getLogger("opentelemetry.exporter").setLevel(logging.DEBUG)

# Now initialize observability
from bb_ai_sdk.observability import init
init(agent_name="my-agent")
Use verbose logging during development to verify traces are being exported correctly. Disable in production to reduce log volume.

Context utilities

Access trace context outside of decorated functions:

Get current trace ID

from bb_ai_sdk.observability import get_current_trace_id

trace_id = get_current_trace_id()
if trace_id:
    logger.info(f"Processing request", extra={"trace_id": trace_id})

Get current span

from bb_ai_sdk.observability import get_current_span

span = get_current_span()
if span:
    span.set_attribute("user.action", "clicked_button")
    span.add_event("User interaction recorded")

Get tracer provider

from bb_ai_sdk.observability import get_tracer_provider

provider = get_tracer_provider()

# Use with instrumentors
from openinference.instrumentation.openai import OpenAIInstrumentor
OpenAIInstrumentor().instrument(tracer_provider=provider)

Security

Sensitive data redaction

The SDK automatically redacts sensitive data from traces:
  • API keys (patterns like api_key=..., apikey=...)
  • Tokens (Bearer tokens, access tokens)
  • Passwords and secrets
  • Langfuse credentials
@trace(attributes={
    "api_key": "sk-secret-key-123",  # Automatically redacted to "[REDACTED]"
    "user.name": "John"              # Not redacted, safe to log
})
def my_function():
    pass
While automatic redaction helps, design your tracing to capture operational metrics, not secrets. Avoid passing sensitive data as span attributes in the first place.

Troubleshooting

Cause: Missing or invalid backend credentials.Solution: Verify environment variables are set correctly:
# For Langfuse (default)
export LANGFUSE_PUBLIC_KEY=pk-lf-xxx
export LANGFUSE_SECRET_KEY=sk-lf-xxx

# For datadog
export DD_API_KEY=your-api-key
Check that keys are valid in your backend dashboard.
Cause: Missing instrumentation.Solution: Ensure you’ve instrumented the appropriate library:
from bb_ai_sdk.observability import init, get_tracer_provider
from openinference.instrumentation.openai import OpenAIInstrumentor

init(agent_name="my-agent")
provider = get_tracer_provider()
OpenAIInstrumentor().instrument(tracer_provider=provider)
Install the required package:
pip install openinference-instrumentation-openai
Cause: Backbase web proxy intercepting or blocking OTLP trace exports.Solution: Configure proxy settings and add backend hosts to NO_PROXY:
.env
HTTP_PROXY=http://webproxy.infra.backbase.cloud:8888
HTTPS_PROXY=http://webproxy.infra.backbase.cloud:8888
NO_PROXY=localhost,127.0.0.1,cloud.langfuse.com,*.langfuse.com
Cause: Span queue growing due to export failures.Solution: Check network connectivity to your OTLP endpoint. Reduce otlp_max_queue_size if memory is constrained:
init(agent_name="my-agent", otlp_max_queue_size=5000)
Cause: Environment variable not loaded before initialization.Solution: Load .env file before importing the SDK:
from dotenv import load_dotenv
load_dotenv()  # Load before import

from bb_ai_sdk.observability import init
init(agent_name="my-agent")

API reference

Init()

init
function
Initialize OpenTelemetry observability with TracerProvider and OTLP exporter.

Trace()

trace
decorator
Decorator that creates OpenTelemetry spans for functions.

Trace_context()

trace_context
context manager
Context manager for manual span control.

Get_tracer_provider()

get_tracer_provider
function
Retrieves the TracerProvider instance created by init() for use with instrumentors.

Callback handlers

LangChainOpenTelemetryCallbackHandler
class
Callback handler for LangChain operations.
LangGraphOpenTelemetryCallbackHandler
class
Callback handler for LangGraph operations.

Next steps