Insights from Goodeye Labs

OpenAI and Anthropic independently arrived at similar patterns for designing effective agent harnesses. In both cases, evaluation is the foundation. The teams that get judgment right will build better AI products.

I encoded Tufte's data visualization principles as a quality standard in Truesight, deployed it as an API, and pointed an AI agent at a real dataset. One instruction: keep improving until you pass.

In November 2025, AI coding tools went from halting and clumsy to surprisingly capable. We gave 22 models the same prompt and ran five replicates each to make the shift visible.

Generating code is easy now, but more code has never meant better products. As production costs fall toward zero, the bottleneck shifts to judgment and taste. This is data science.

The Anthropic Economic Index shows that the quality of what you put into AI almost perfectly predicts the quality of what you get out. If AI mirrors expertise, so must your AI evaluations.

Learn how to build reliable AI systems using LLM evaluations. This talk covers why traditional testing breaks with stochastic systems, how generic LLM-as-Judge approaches miss domain nuance, and practical steps to implement contextual evaluations that actually work.

In 2025, we discovered we'd been measuring memorization, not intelligence. Models scored 80-90% on static benchmarks but dropped to 60-70% on truly novel problems. This year exposed the fundamental crisis in AI evaluation, and taught us what to build instead.