How Better Evals Can Bring Abundance through Accelerated Scientific Discovery (talk recap)

On August 2, 2026, I gave a five-minute lightning talk at the Agentic AI Summit at UC Berkeley. The argument: better evals are the hinge between AI and real scientific abundance. I have spent 20 years working on versions of this problem, and Talaria Scientific is the culmination of that work. The video is below; after it, the summary, then the whole talk slide by slide.

Prefer to page through the deck itself? Download the slides (PDF).

Want to learn more about Talaria Scientific? Read why I am building Talaria, how it works, and what this unlocks.

The summary

Abundance through AI, in my mind, means new scientific discoveries: the kind that leads to new materials and more efficient energy, and from there to advances in medicine and aerospace. It means living in a sci-fi future. The real promise of AI is improving quality of life (not just replacing white-collar jobs). And I believe that for the first time, we can meaningfully accelerate science with AI.

In grad school, the final step of any project was always parameter tuning, whether I was working on finite elements, protein folding, quantum mechanics, or metagenome assembly. We jokingly called it grad student descent: the least tenured person in the group, up late turning knobs in a high dimensional space, chasing a slightly better result. Solving that problem more generally became my thesis, and then my first company. SigOpt was Bayesian optimization as a service: given an eval, find the parameters that make it go up. Over seven years we tuned recommender systems at Netflix, fraud models at Amex, early RL systems at OpenAI, and models at hedge funds collectively managing about a trillion dollars, plus hundreds of academics who used our free program. In 2020 I sold the company to Intel.

Somewhere in there I realized I was only solving half the problem, and I said it on stage the way I have said it for a decade: "The best thing about a black box optimizer is that it will optimize any eval you tell it to. The worst part about a black box optimizer is that it will optimize exactly the eval you tell it to." When only 1% of transactions are fraud, you can build a very accurate fraud detector by predicting that nothing is fraud. That gap between the eval you wrote and the outcome you care about is a big part of why we are still a long way from typing /goal solve stable fusion containment into anything.

I started Distributional in 2023 to attack that gap head-on as an AI testing company, with high dimensional statistical tests for chaotic and non-stationary AI systems. It failed: people didn't know what to test, didn't have the data before deployment, and shipped to production anyway. We pivoted to post-production agent analytics, finding the evals you should have written. That failed too: a good idea, but more of a feature than a product. I still believe both approaches are obvious and inevitable; we just never crossed from vision-market fit to product-market fit.

So now I am doing what you should do when you have a problem: solving the version I care about most. Talaria Scientific (Distributional's next chapter) is a multi-agent, multi-foundation-model harness for computational science research, built around domain-specific evals and guardrails rooted in real physics and math. The point is to spin the what-if flywheel faster: work that took me six months in grad school, done in six hours. It is in private beta now, it will be open source at NeurIPS 2026, and it will always be free for open science.

The talk, slide by slide

Each slide below carries what I said over it, lightly cleaned up from the livestream transcript.

My name is Scott Clark. I'm co-founder and CEO of Distributional, and I'm building Talaria Scientific. This talk is about how you get better abundance.

What abundance with AI means

In my mind it's better scientific discoveries, which lead to new materials and more efficient energy, which lead to better medicine and more efficient ways to travel. It basically means living in a sci-fi future.

The real promise of abundance through AI

It's less about how we get rid of more white-collar jobs, and more about how we improve people's quality of life.

The pieces are finally falling into place

I believe that for the first time, we are able to meaningfully accelerate scientific discovery with AI. The last few months have unlocked quite a bit of capability.

A culmination of my life's work

This has been a passion of mine for a long time. I've been trying to solve this problem for the last 20 years of my career, with varying degrees of success.

Grad school: solving the same problem over and over

Every group I worked with, from protein folding to quantum mechanics simulation to metagenome assembly, ended with the same problem. We'd build something great, and then we'd need to tune it: knobs, levers, hyperparameters, whatever you want to call them. If you could make the benchmarks slightly better, you got a better paper. We jokingly called this grad student descent, because it was usually the grad students sitting up late, tuning knobs in a high dimensional space. People applied smart techniques to it (simulated annealing, genetic algorithms, local methods), but I fell in love with Bayesian optimization, and it became the core of my PhD thesis: methods for efficient optimization, given an eval.

SigOpt: optimize everything

That thesis became SigOpt, Bayesian optimization as a service, started in 2014. Over seven years we worked with Netflix tuning recommender systems, Amex tuning fraud systems, OpenAI tuning their early RL systems back when they were a nonprofit lab, hedge funds managing about a trillion dollars, and several hundred academics who used our free program on everything from materials design to drug discovery. After I sold the company to Intel in 2020, we tuned everything from chip design to benchmaxxing MLPerf, back when that was the number everyone was overfitting to.

Learning I was only solving half the problem

It worked really well, except when it didn't. It would always optimize exactly what you gave it. Customers would come back and say: you made that number go up, but some other number went down.

The problem with optimizers

The best thing about a black box optimizer is that it will optimize any eval you tell it to. The worst part about a black box optimizer is that it will optimize exactly the eval you tell it to. You can build a very accurate fraud detector, when only 1% of your transactions are fraud, by predicting that nothing is fraud.

We are still a long way from /goal solve

We're a long way from saying: goal, solve stable fusion containment. Or solve cancer. And even if the model came back with something, how could we trust it? That's the other side of the coin. You need to optimize these systems, and you need to trust them.

Have a problem? Start a startup!

That's what I set out to do with Distributional in 2023. I attacked it in the most complex mathematical way possible: high dimensional statistical tests for chaotic and non-stationary AI systems.

It failed. Nobody likes tests. People didn't know what to test, they didn't have the data before deployment, and everybody was YOLOing models into production anyway.

Have a problem? Pivot!

So I pivoted the company. This is what you do: you're failing, pivot. Analytics. Catch the evals that are sneaking through your system, find the patterns in behavior that say, "this is the unknown unknown," the eval you should have written.

This also failed. It's a really good idea, but it's more of a feature than a product. Let alone a business, or a startup.

Have a problem? Solve the version you care about

So now I'm applying everything I've done in the last 20 years to attack the scientific problem: how do we accelerate science? Talaria is a multi-agent, multi-foundation-model harness for computational science research, focused on what actually matters: bespoke evals that know physics, that know the scientific research process, that know how to do hypothesis validation.

From idea to trusted result

With that in place, you can spin the flywheel of a what-if machine faster and faster. Work that used to take me six months in grad school takes six hours instead.

Thank you

We're building this in the open: private beta signups are open at talariasci.com now, the open source launch lands at NeurIPS 2026, and it will always be free for open science. I would love to chat if you're building anything computational science related.

If you're building in this space

If you're working on world models, materials, aerospace, computational physics, or optimization, I would love to hear from you: talariasci.com. The longer version of the Distributional story is in Making New Mistakes Faster as a Second-Time Founder, and the full case for what we're building is in the Talaria manifesto.

Recorded at the Agentic AI Summit 2026 (UC Berkeley RDI), Compass Stage, August 2, 2026. Thanks to the summit crew for the livestream; the full session stream has my talk at 2:43:05.