# Machines That Do Science at Imagination in Action's Silicon Valley AI Summit (panel recap) > A recap of the Machines That Do Science panel at Imagination in Action's Silicon Valley AI Summit at Google Bay View: how much of the scientific loop closes without a person in it, and how you know when to trust the result. The video, the photos, what I argued, and what Nick Edwards and Piotr Byrski said beside it. Author: Scott Clark Published: 2026-10-19 Pillar: Building Things Canonical: https://scottclark.io/blog/machines-that-do-science --- On September 15, 2026, I sat on a panel called Machines That Do Science at [Imagination in Action](https://www.imaginationinaction.co)'s Silicon Valley AI Summit, on the Frontier Stage at Google Bay View. [Frazer Anderson](https://www.linkedin.com/in/frazer-f-anderson/) of [Link Ventures](https://www.linkventures.com) moderated, and my co-panelists were [Nick Edwards](https://www.linkedin.com/in/nick-edwards-phd/) of [Potato](https://www.potato.ai) (biology) and [Piotr Byrski](https://www.linkedin.com/in/pwbyrski/) of [Molecule.one](https://molecule.one) (chemistry). The thrust of the panel was: "AI is now proposing molecules, running experiments and drawing conclusions on its own. How much of the loop closes without a person in it, and how do you know when to trust the result?" My argument: the mechanical work of science is now in distribution for these models, the out-of-distribution question still belongs to the scientist, and a system that keeps the record is what makes the faster loop trustworthy. The video is below, then the summary. [Watch on YouTube: Machines That Do Science (Imagination in Action Silicon Valley AI Summit, September 2026)](https://www.youtube.com/watch?v=ns7Qfi-uT48) ## The summary Frazer opened by asking how each of us is spending our time in this AI moment. Nick is building Potato, AI-native infrastructure for autonomous labs, which he described as CAD for biological experiments. Piotr is building Maria at Molecule.one, a physical AI system for autonomous research in chemistry, wired to a purpose-built lab where the experiments run. I am building [Talaria Scientific](https://talariasci.com), tools to help physicists and applied mathematicians do better in silico research: spinning up environments, getting baselines, making the research reproducible, something like a Cursor for computational science. Frazer's first question was the one everybody in this field gets asked. There is a school of thought that scientists should all be working for the frontier labs in some sense: provide the training data, wait for fully general intelligence, and let the system go solve the science. He asked why that view is not consistent with "the fabric of reality." Nick's answer was data. "We're getting to a level of superintelligence in math," he said, because you can immediately verify an infinite number of hypotheses, and biology cannot, because its feedback cycles are long, much of the literature does not reproduce, and a new, complete set of data still has to be built. Piotr's answer was interfaces: "a physical lab is a tool that an agent can call," the real world provides the final validation, and the place for teams outside the frontier labs is building the interfaces between the real world and the intelligence layer. My answer was that there is a frontier between the Bitter Lesson and the No Free Lunch theorem, and surfing it is still the scientist's job. The Bitter Lesson, from Richard Sutton, says scale beats cleverness: give a model enough data and compute and it will beat your clever hand-built method, so you do not need to do feature engineering for natural language processing or computer vision anymore. The No Free Lunch theorems say no method beats the rest unless its assumptions match the problem in front of it, so a specialized model with better prior knowledge can beat a general-purpose model. Both are right, and between them there is always a frontier. The models today are very good at in-distribution work, the kind their training covers, and a lot of the scientific process is in distribution: spinning up heterogeneous compute, replicating a baseline, doing the ETL on a ton of data. What comes next is out of distribution, so there is still a lot to be done for scientists, and surfing that frontier is where the exciting work and the abundance come from. I talk more about this in [my post here](https://talariasci.com/blog/no-free-lunch-bitter-lesson), and it was the argument of [Accelerating Science with AI: Surfing the Frontier Between the Bitter Lesson and No Free Lunch](https://scottclark.io/blog/accelerating-science-with-ai), the seminar I gave at Oregon State three weeks after this panel. When Piotr said the real world provides the final validation, I agreed. I think the loop is closed in simulation before you start moving atoms around. Physics is a [distillation attack on the universe](https://x.com/DrScottClark/status/2094905663621394721), so the job is finding the right fidelity for the experiment. Some of it can be done in silico with a physics-informed neural network, then you take it into the lab to make sure it holds up. It is a ladder you go up and down: every rung has its own confidence interval and, as Piotr added, its own price tag, so you keep chasing it to whatever accuracy you need. An in-distribution model like a world model is not going to solve all of that any time soon. That ladder is how [The scientific flywheel that can lead to abundance](https://talariasci.com/blog/the-right-model-for-the-job) describes a dense frontier of validated models: cheap surrogates explore the space to learn, the expensive rungs (the reference solver, then the lab) spend their budget where the cheap rungs led, and a researcher gets to pick the right model for the job instead of settling for close enough. Frazer's second question was, where does the loop get stuck? He wanted the trade-offs that are different for agents, as specific as possible, and where things get stuck inside an autonomous lab today. Piotr's answer was purity: as recently as a year or two ago, medicinal chemists held that every compound you test has to be 90 or 95 percent pure, because it has always been done that way, so his lab tests unpurified compounds, takes the noisier data, and gets a hundred times as many measurements. Nick's answer was utilization: labs have spent a decade on robotics and automation, and the instruments sit in a corner collecting dust, because automating a single experiment takes weeks to months of human work up front, and the way around that is very good design of experiments. I agreed with both, and my answer was finding the right thing to try next. My first startup, SigOpt, was Bayesian optimization as a service, which you can think of as design of experiments for searching high-dimensional spaces more efficiently. It was the same design of experiments Nick had just named, and I ran it for seven years before selling it to Intel. Across the thousands of research labs and scientists we worked with, we saw the same trade-offs Nick and Piotr were describing. You have exploration versus exploitation, and many weak estimators can be the better way to explore, to learn as much as you can about the underlying space, before you exploit with the expensive samples in the real world. The bottleneck underneath is where you point the optimizer. A joke we always used to say: "The best thing about a black box optimizer is that it will optimize anything. The worst thing about a black box optimizer is that it will blindly optimize anything." Knowing where on the Pareto frontier you want to land, or what your ultimate goal is, matters a lot, because if you get somewhere fast and it is the wrong place, that is the same as not getting there. What you point it at is the eval, and that is the argument of [How Better Evals Can Bring Abundance through Accelerated Scientific Discovery](https://scottclark.io/blog/better-evals-abundance): better evals are the hinge between AI and real scientific abundance. [TauOpt](https://tauopt.com), the optimization engine underneath Talaria that I am building now, expands on that work. Piotr agreed that pointing these systems where to go is the bottleneck, and said one of the most exciting developments he has seen this year is models gaining the ability to pick the right problem to work on, beyond optimizing against the one they are handed. He calls the upside engineered serendipity: the history of science is paved with luck, and a lab that runs and analyzes enough experiments can engineer its own. Frazer compressed that to surface area, and surface area is also how I think about the systems I build. A good [harness](https://talariasci.com/blog/the-talaria-architecture) around an AI model can be a what-if machine. I can do in six hours what took me six months in my PhD, which means I ask more questions, read more papers, and run the experiment reviewer 2 asked for instead of explaining it away. When the flywheel spins faster we get more surface area, and we end up climbing mountains that we didn't even know we wanted to climb. ![The four of us on the Frontier Stage under the session slide: Frazer Anderson, Scott Clark, Nick Edwards speaking into the microphone, Piotr Byrski.](https://scottclark.io/images/blog/machines-that-do-science/panel-wide.jpg) *Frazer Anderson, me, Nick Edwards (speaking), and Piotr Byrski under the session slide. Photo: Patrick Tighe / Imagination in Action.* Frazer's third question was sycophancy. We are entering a golden age of science, but these models can be prompted into supporting almost any conclusion, and models ingesting their own slop could see their performance decay right when our confidence in them peaks, so what do we do internally to safeguard hypothesis generation, prompt tuning, fine-tuning our own models? Nick's answer came from Karl Popper: science is falsifiable, so you formulate hypotheses that can be explicitly tested, decide going in what quality of data should come out, and make predictions for the next step, and that rigor is what frees the scientist to be creative. My answer was the mech suit: an enhancement on the scientist instead of a replacement. The analogy I use is whether you are building a mech suit or an android, and I have landed on the mech suit: a tool that lets the scientist do things better, faster, and more efficiently. The scientist still directs it, because the combinatorial space of hypotheses is too large, and sometimes the models want to find the median. They will suggest throwing a softmax in, because of course that is everywhere in the literature, when in reality you want something completely different. The scientist supplies the taste and decides what to ask next. Maybe GPT-15 or 16 replaces us all, but for now the out-of-distribution search is still human driven. The mech suit is the frame of [Why I'm building Talaria Scientific](https://talariasci.com/blog/why-im-building-talaria), and that post carries my structural answer to a model that will agree with anything: results face adversarial review from agentic reviewers rewarded for killing a pet hypothesis, and anything that smells like a scientific judgment gets proposed to the user (never assumed). The harness adds the first filter: an idea is asked for its cheapest falsifying test and faces an adversarial referee before any compute is spent on it, and the user decides what enters the record as believed. Frazer's fourth question was fraud: he sees more of it in the market today than ever, and asked whether we see the same in science. Nick's name for it is "hypotheslop." Piotr's answer was yes, although the beautiful thing in chemistry is that once you have a single hypothesis at the end of the process the verification is not that hard: the project his team ran with [OpenAI](https://openai.com/index/ai-chemist-improves-reaction) found a real improvement to an important process in medicinal chemistry, and the output is a recipe any scientist, human or machine, can reproduce in a lab in about a day. Nick's worry was the literature, which cannot be trusted a lot of the time, so results still have to be verified experimentally, and that is why he wants autonomous labs: to test and falsify the ideas people put online. My short answer was that these systems are massive p-hacking machines, and my longer answer was the two jobs they are good at: finding interdisciplinary connections, and keeping track of everything you tried. The first job is finding interdisciplinary connections. It was always the applied physicist or the applied mathematician who got to publish a lot, because they could speak a bunch of different fields and pull things together, and now the AI does a lot of that for you. The discovery Piotr described is the clean example. When Frazer asked whether the idea came from a reasoning trace or a throwaway hallucination that happened to be right, Piotr said OpenAI published the chain of thought. GPT thought the idea unlikely to work because of literature it found, he said, and the project's budget, "40,000 experiments in total in the whole project," meant it got tested anyway. The idea itself came from what his team calls cross-pollination: the model read papers from two domains, saw copper used with a particular reagent in one reaction, and wondered whether it would work in another. That is where machines improve on human-led science, he said, because we cannot read all the papers, let alone make the connections between them, which grow with the square of the number of papers. The second job is note-taking, keeping track of everything you tried. In silico that means bit-exact reproducible results where the environment fingerprint matches, with every pseudo-random seed pinned, so you can hand someone a Docker container and they reproduce the result, instead of a bunch of academic code left as an exercise for the reader, where you forgot you tuned the hyperparameters. It is the in silico version of Piotr's one-day recipe, and it is my answer to Nick's point that results have to be verified: more trustworthy and more verifiable science, if we hold people to a bar they did not have before. When I was a grad student I tried to write down as much as I could, but I have a lot of academic slop code, pre-AI slop, out there, and there is no excuse for that anymore. That record is the [system of record](https://talariasci.com/blog/the-system-of-record): each result carries the exact code, configuration, and environment it ran with, so a rerun is a reproduction instead of a reenactment, and every result has to name the hypothesis it answers, because orphan numbers are how spurious findings start. Frazer's last question was what is missing from the stack and what has to be built to take this to the next level. Nick's answer was drivers: science is unlocked by tools, the incredible equipment out there is hard to interface with, and he hopes for convergence on standards. Piotr's answer was "equipment, equipment, equipment": even the instruments built for automated labs were built for human-led workflows, they break when you run a hundred times as many experiments on them, and lab hardware could have a renaissance if the vendors figure it out. My answer was the one everybody gives, more compute, and then more real-world validation, which is what Nick and Piotr do. A lot of the tools are already out there, great open source is getting built every day, and it is easier than ever to build things, so a major constraint on in silico science today is compute. Trust in it comes from there always being a way to project back to ground truth, or the closest thing you have to it, the reference solver or the experiment, which is the validation Nick and Piotr run. ![Scott Clark mid-answer on the Frontier Stage, microphone up, with the speaker cards on the screen behind him.](https://scottclark.io/images/blog/machines-that-do-science/scott-speaking.jpg) *Mid-answer on the Frontier Stage. Photo: Patrick Tighe / Imagination in Action.* ## If you are building in this space Nick and Piotr close their loops in physical labs. I am closing the same loop in silico: a harness does the in-distribution work and records every run so it can be reproduced, and the scientist decides what to ask next. If you are working on that problem in computational physics, applied math, materials, energy, fluids, or physical AI, I would love to [hear from you](mailto:scott@scottclark.io). The longer versions of the argument are [why I am building Talaria Scientific](https://talariasci.com/blog/why-im-building-talaria), [how the harness works](https://talariasci.com/blog/the-talaria-architecture), and [what a dense frontier of validated models unlocks](https://talariasci.com/blog/the-right-model-for-the-job), and the five-minute version of why I am building it is [How Better Evals Can Bring Abundance through Accelerated Scientific Discovery](https://scottclark.io/blog/better-evals-abundance). ![The Frontier Stage at Google Bay View from the audience: the Imagination in Action screen above the four of us on stage.](https://scottclark.io/images/blog/machines-that-do-science/frontier-stage.jpg) *The Frontier Stage from the room. Photo: Patrick Tighe / Imagination in Action.* *Recorded at Imagination in Action's Silicon Valley AI Summit on the Frontier Stage at Google Bay View, Mountain View, September 15, 2026. Thanks to Frazer Anderson for moderating, to Nick Edwards and Piotr Byrski for the conversation, and to [John Werner](https://www.linkedin.com/in/johnwerner1/) and the Imagination in Action team for putting it on and publishing the [recording](https://www.youtube.com/watch?v=ns7Qfi-uT48); the recordings from every stage are on the [summit playlist](https://www.youtube.com/playlist?list=PLJwCQNncHCN8). Photos by Patrick Tighe for Imagination in Action.* --- Follow me at [@DrScottClark](https://twitter.com/DrScottClark) to see new posts. Source: https://scottclark.io/blog/machines-that-do-science (Scott Clark)