# How Better Evals Can Bring Abundance through Accelerated Scientific Discovery (talk recap)

> My five-minute lightning talk from the Agentic AI Summit at UC Berkeley: why evals are the hinge between AI and scientific abundance, how two failed products taught me where that hinge breaks, and what I'm building because of it. Video, slides, and a slide-by-slide walkthrough.

Author: Scott Clark  
Published: 2026-08-31  
Pillar: Building Things  
Canonical: https://scottclark.io/blog/better-evals-abundance

---

On August 2, 2026, I gave a five-minute lightning talk at the Agentic AI
Summit at UC Berkeley. The argument: better evals are the hinge between AI
and real scientific abundance. I have spent 20 years working on versions of
this problem, and [Talaria Scientific](https://talariasci.com) is the
culmination of that work. The video is below; after it, the summary, then the
whole talk slide by slide.

[Watch on YouTube: How Better Evals Can Bring Abundance through Accelerated Scientific Discovery (Agentic AI Summit 2026)](https://www.youtube.com/watch?v=1VO7hdEgSeo)

Prefer to page through the deck itself? [Download the slides (PDF)](https://scottclark.io/slides/better-evals-abundance.pdf).

Want to learn more about Talaria Scientific? Read
[why I am building Talaria](https://talariasci.com/blog/why-im-building-talaria),
[how it works](https://talariasci.com/blog/the-talaria-architecture), and
[what this unlocks](https://talariasci.com/blog/the-right-model-for-the-job).

## The summary

Abundance through AI, in my mind, means new scientific discoveries: the kind
that leads to new materials and more efficient energy, and from there to
advances in medicine and aerospace. It means living in a sci-fi
future. The real promise of AI is improving quality of life (not just
replacing white-collar jobs). And I believe that for the first time, we can
meaningfully accelerate science with AI.

In grad school, the final step of any project was always parameter tuning,
whether I was working on finite elements, protein folding, quantum mechanics,
or metagenome assembly. We jokingly called it grad student descent: the least
tenured person in the group, up late turning knobs in a high dimensional
space, chasing a slightly better result. Solving that problem more generally
became my thesis, and then my first company. SigOpt was Bayesian optimization
as a service: given an eval, find the parameters that make it go up. Over
seven years we tuned recommender systems at Netflix, fraud models at Amex,
early RL systems at OpenAI, and models at hedge funds collectively managing
about a trillion dollars, plus hundreds of academics who used our free
program. In 2020 I sold the company to Intel.

Somewhere in there I realized I was only solving half the problem, and I said
it on stage the way I have said it for a decade: "The best thing about a
black box optimizer is that it will optimize any eval you tell it to. The
worst part about a black box optimizer is that it will optimize exactly the
eval you tell it to." When only 1% of transactions are fraud, you can build a
very accurate fraud detector by predicting that nothing is fraud. That gap
between the eval you wrote and the outcome you care about is a big part of
why we are still a long way from typing `/goal solve stable fusion
containment` into anything.

I started [Distributional](https://distributional.com) in 2023 to attack that
gap head-on as an AI testing company, with high dimensional statistical tests
for chaotic and non-stationary AI systems. It failed: people didn't know what
to test, didn't have the data before deployment, and shipped to production
anyway. We pivoted to post-production agent analytics, finding
[the evals you should have written](https://distributional.com/blog/distributional-is-now-talaria).
That failed too: a good idea, but more of a feature than a product. I still
believe both approaches are obvious and inevitable; we just never crossed
from [vision-market fit to product-market fit](https://scottclark.io/blog/make-all-new-mistakes-faster).

So now I am doing what you should do when you have a problem: solving the
version I care about most. Talaria Scientific
([Distributional's next chapter](https://distributional.com/blog/distributional-is-now-talaria))
is a multi-agent, multi-foundation-model harness for computational science
research, built around domain-specific evals and guardrails rooted in real
physics and math. The point is to spin the what-if flywheel faster: work that
took me six months in grad school, done in six hours. It is in private beta
now, it will be open source at NeurIPS 2026, and it will always be free for
open science.

## The talk, slide by slide

Each slide below carries what I said over it, lightly cleaned up from the
livestream transcript.

![Title slide: How Better Evals Can Bring Abundance through Accelerated Scientific Discovery. Scott Clark, Founder and CEO, with the Distributional and Talaria Scientific logos.](https://scottclark.io/images/blog/better-evals-abundance/slide-01.png)

My name is Scott Clark. I'm co-founder and CEO of Distributional, and I'm
building Talaria Scientific. This talk is about how you get better abundance.

### What abundance with AI means

![Slide: what abundance with AI means, a four-step build from new scientific discoveries to living in a scifi future.](https://scottclark.io/images/blog/better-evals-abundance/slide-02.png)

In my mind it's better scientific discoveries, which lead to new materials
and more efficient energy, which lead to better medicine and more efficient
ways to travel. It basically means living in a sci-fi future.

### The real promise of abundance through AI

![Slide: improving quality of life, not just replacing repetitive work.](https://scottclark.io/images/blog/better-evals-abundance/slide-03.png)

It's less about how we get rid of more white-collar jobs, and more about how
we improve people's quality of life.

### The pieces are finally falling into place

![Slide: we can now meaningfully accelerate science with AI.](https://scottclark.io/images/blog/better-evals-abundance/slide-04.png)

I believe that for the first time, we are able to meaningfully accelerate
scientific discovery with AI. The last few months have unlocked quite a bit
of capability.

### A culmination of my life's work

![Slide: I have been trying to solve this problem for the last 20 years of my career, with varying degrees of success.](https://scottclark.io/images/blog/better-evals-abundance/slide-05.png)

This has been a passion of mine for a long time. I've been trying to solve
this problem for the last 20 years of my career, with varying degrees of
success.

### Grad school: solving the same problem over and over

![Slide: grad school timeline across Oregon State, UC Davis, Max Planck, Cornell, Los Alamos, and Berkeley Lab. Whatever the field, the final step was always parameter tuning.](https://scottclark.io/images/blog/better-evals-abundance/slide-06.png)

Every group I worked with, from protein folding to quantum mechanics
simulation to metagenome assembly, ended with the same problem. We'd build
something great, and then we'd need to tune it: knobs, levers,
hyperparameters, whatever you want to call them. If you could make the
benchmarks slightly better, you got a better paper. We jokingly called this
grad student descent, because it was usually the grad students sitting up
late, tuning knobs in a high dimensional space. People applied smart
techniques to it (simulated annealing, genetic algorithms, local methods),
but I fell in love with Bayesian optimization, and it became the core of my
PhD thesis: methods for efficient optimization, given an eval.

### SigOpt: optimize everything

![Slide: SigOpt, Bayesian optimization as a service, with the founding, building, and scaling timeline.](https://scottclark.io/images/blog/better-evals-abundance/slide-07.png)

That thesis became SigOpt, Bayesian optimization as a service, started in
2014. Over seven years we worked with Netflix tuning recommender systems,
Amex tuning fraud systems, OpenAI tuning their early RL systems back when
they were a nonprofit lab, hedge funds managing about a trillion dollars, and
several hundred academics who used our free program on everything from
materials design to drug discovery. After I sold the company to Intel in
2020, we tuned everything from chip design to benchmaxxing MLPerf, back when
that was the number everyone was overfitting to.

### Learning I was only solving half the problem

![Slide: the optimizer worked, except when it didn't. It overfit, or optimized the wrong thing.](https://scottclark.io/images/blog/better-evals-abundance/slide-08.png)

It worked really well, except when it didn't. It would always optimize
exactly what you gave it. Customers would come back and say: you made that
number go up, but some other number went down.

### The problem with optimizers

![Slide: the best thing about a black box optimizer is that it will optimize any eval you tell it to. The worst part is that it will optimize exactly the eval you tell it to.](https://scottclark.io/images/blog/better-evals-abundance/slide-09.png)

The best thing about a black box optimizer is that it will optimize any eval
you tell it to. The worst part about a black box optimizer is that it will
optimize exactly the eval you tell it to. You can build a very accurate fraud
detector, when only 1% of your transactions are fraud, by predicting that
nothing is fraud.

### We are still a long way from `/goal solve`

![Slide: we are still a long way from typing goal solve stable fusion containment.](https://scottclark.io/images/blog/better-evals-abundance/slide-10.png)

We're a long way from saying: goal, solve stable fusion containment. Or solve
cancer. And even if the model came back with something, how could we trust
it? That's the other side of the coin. You need to optimize these systems,
and you need to trust them.

### Have a problem? Start a startup!

![Slide: Distributional attempt one, pre-production AI testing, 2023 to 2025: high dimensional Bayesian statistical tests for chaotic and non-stationary AI systems and agents.](https://scottclark.io/images/blog/better-evals-abundance/slide-11.png)

That's what I set out to do with Distributional in 2023. I attacked it in the
most complex mathematical way possible: high dimensional statistical tests
for chaotic and non-stationary AI systems.

![Slide: the same testing slide grayed out with It failed stamped over it, and the three reasons: people didn't know what to test or have the data before deployment, they couldn't understand tests well enough to act, and they shipped to prod anyway.](https://scottclark.io/images/blog/better-evals-abundance/slide-12.png)

It failed. Nobody likes tests. People didn't know what to test, they didn't
have the data before deployment, and everybody was YOLOing models into
production anyway.

### Have a problem? Pivot!

![Slide: Distributional attempt two on the timeline strip, post-production agent analytics, 2025 to 2026: analytics on agent traces to find the evals you aren't monitoring for.](https://scottclark.io/images/blog/better-evals-abundance/slide-13.png)

So I pivoted the company. This is what you do: you're failing, pivot.
Analytics. Catch the evals that are sneaking through your system, find the
patterns in behavior that say, "this is the unknown unknown," the eval you
should have written.

![Slide: the pivot slide grayed out with It failed stamped over it. A feature, not a product, let alone a business or a startup.](https://scottclark.io/images/blog/better-evals-abundance/slide-14.png)

This also failed. It's a really good idea, but it's more of a feature than a
product. Let alone a business, or a startup.

### Have a problem? Solve the version you care about

![Slide: the timeline strip gains its third column, Computational Science Harness, 2026 and beyond: a multi-agent, multi-FM harness for computational science research acceleration, using domain-specific evals and guardrails rooted in physics and math.](https://scottclark.io/images/blog/better-evals-abundance/slide-15.png)

So now I'm applying everything I've done in the last 20 years to attack the
scientific problem: how do we accelerate science? Talaria is a multi-agent,
multi-foundation-model harness for computational science research,
focused on what actually matters: bespoke evals that know physics, that know
the scientific research process, that know how to do hypothesis validation.

### From idea to trusted result

![Slide: from idea to trusted result, where the week goes. Two rows compare a computational scientist's week today, dominated by the validation schlep done by hand, with the same week in the harness, where data wrangling, environments, baselines, sweeps, and checking run with provenance on every run.](https://scottclark.io/images/blog/better-evals-abundance/slide-16.png)

With that in place, you can spin the flywheel of a what-if machine faster and
faster. Work that used to take me six months in grad school takes six hours
instead.

### Thank you

![Slide: thank you, with QR codes for talariasci.com and scottclark.io. Researching computational physics or applied math? Building world models? Private beta signups, open source launch at NeurIPS 2026, always free for open science.](https://scottclark.io/images/blog/better-evals-abundance/slide-17.png)

We're building this in the open: private beta signups are open at
talariasci.com now, the open source launch lands at NeurIPS 2026, and it will
always be free for open science. I would love to chat if you're building
anything computational science related.

## If you're building in this space

If you're working on world models, materials, aerospace, computational
physics, or optimization, I would love to hear from you:
[talariasci.com](https://talariasci.com). The longer version of the
Distributional story is in
[Making New Mistakes Faster as a Second-Time Founder](https://scottclark.io/blog/make-all-new-mistakes-faster),
and the full case for what we're building is in the
[Talaria manifesto](https://talariasci.com/blog/why-im-building-talaria).

*Recorded at the [Agentic AI Summit 2026](https://rdi.berkeley.edu/events/agentic-ai-summit-2026)
(UC Berkeley RDI), Compass Stage, August 2, 2026. Thanks to the summit crew
for the livestream; the
[full session stream](https://www.youtube.com/watch?v=1UrriPJRSPU&t=9785s)
has my talk at 2:43:05.*

---

Follow me at [@DrScottClark](https://twitter.com/DrScottClark) to see new posts.

Source: https://scottclark.io/blog/better-evals-abundance (Scott Clark)