[Roadmap] langfuse integration for prompt tracking / metrics

[sammcj]

Why

Langfuse is a really useful open source LLM observability tool that's great for tracking prompts, models, RLHF feedback etc...

github.com/langfuse/langfuse

Description

I'm assuming the setup from a user perspective would be similar to in other apps (e.g. open-webui) you provide the langfuse host, the private and public api keys.

Integration with langfuse seems quite easy, e.g.

import { Langfuse } from "langfuse";

const langfuse = new Langfuse({
  secretKey: "sk-lf-230f162b-5b53-4af5-8c91-76ca805b8751",
  publicKey: "pk-lf-41874d5b-4620-471e-8cc7-ed31c65999df",
  baseUrl: "https://langfuse.icu.lol"
});

Requirements

N/A