Your LLM App Returned 200 OK. It Was Still Wrong. — Marina Petzel, Datadog

Why watch

Marina Petzel explains why traditional monitoring metrics like latency and errors are insufficient for production GenAI applications. The talk covers three critical monitoring pillars: cost monitoring (token creep, model drift, uncached calls), safety monitoring (prompt injection, PII leakage, toxicity, jailbreaks), and quality monitoring (hallucination rate, relevance, user satisfaction, RAG retrieval quality). This is essential viewing for engineers building and operating production LLM applications who need to understand how to measure and ensure AI correctness beyond simple HTTP 200 responses.