Discover video-sourced quotes by politicians

Michael Kratsios: “Understanding how to measure and evaluate models is still an…”

Featuring: Michael Kratsios, Director of the Office of Science and Technology Policy, Science Advisor to the President | National

Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios

Published July 30, 2025 | Video Starts at 40:28

And if we go back to the story I was telling about how you do a model eval, one of the biggest challenges that currently faces the AI industry is Understanding how to measure and evaluate models is still an open scientific question. Currently sort of the state of the art is literally having a biologist ask questions to the model and see what comes back like that that provides you some answer. (40:28–40:35)

CLAIM

Asserts that measuring and evaluating AI models is an unresolved scientific issue.

This is an excerpt from a raw transcript. The assertions have not been audited or verified. Tap the video to view the source footage and understand the context.

Discover more from

Michael Kratsios

“President Trump signed the first executive order in the history of the United States on artificial intelligence in February of 2019.”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 07:02-07:16
“Any sale that Nvidia wants to make to China is one that's going to require an export license.”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 25:57-26:04
“Over 33 models have been trained over the compute limit of 10 to the 26.”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 48:51-48:56
“Those [export controls] are still in place.”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 06:08-22:05
“Last week was a big week for the United States, for the president, and for the future of artificial intelligence in this country.”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 06:29-06:39
“President [Biden] signed during his term...set very weird, arbitrary limits around pre-deployment testing of AI models.”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 06:35-09:20
“The journey to getting to the AI action plan actually started in the first Trump administration, in Trump 45.”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 06:39-06:47
“The president then went on to announce a doubling of AI spending.”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 07:46-07:50
“The Biden administration...created this body within NIST...to attempt to do evaluations around some of these DEI and other related risks.”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 08:10-39:59
“The previous administration...were maniacally obsessed with existential risk.”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 08:56-36:56
“The president on day one, signed executive order rescinding the Biden executive order.”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 09:53-09:58
“Three days later, on January 23, he signed a new executive order directing me, the National Security advisor, and our senior advisor for AI…”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 09:57-10:09
“We got 10,000 responses [to the RFI for the AI action plan].”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 10:51-10:54
“The first is we have to lead on innovation...creating a regulatory environment that can allow these technologies to thrive in the United…”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 11:35-11:47
“The second pillar is all about infrastructure...create the right amount of electricity and also create the data centers.”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 12:18-12:31
“The third pillar is all about AI exports...make sure that the world is running on the American AI stack.”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 12:38-12:42
“We have to get out in front of it [AI export issue].”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 15:14-15:20
“The issue that we face today is even more serious than the telecommunications problem that generally the west faces in the global South.”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 16:00-16:09
“We have the best stack. We have the best chips, we have the best models, we have the best applications.”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 17:13-17:21
“The executive order activates our development finance organizations within the USG to start prioritizing export of AI.”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 19:39-19:50
“The Export Import bank, the Development Finance Corporation, dfc, have been directed by Congress to work on this technology race.”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 19:53-20:02
“The highest end semiconductors need to continue to be export controlled and not allowed into China.”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 21:33-21:43
“The restrictions that we put on some of the supply chains sort of behind the semiconductor manufacturing, some of these lithography machines…”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 21:47-21:59
“We have to find ways to provide the tools that BIS needs to do the enforcement activities necessary.”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 22:41-22:47
“A lot of components relating to these machines are of US origin and there's certain export control rules.”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 23:45-23:51
“The EU initiated all this effort around the EUAI act many years ago before sort of large language models are what they are today.”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 29:43-29:52
“The Europeans continue to think of [AI regulation] as a singular AI regulation that is sort of horizontal and crosses everything.”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 30:58-31:08
“The US thinks about [AI regulation] much more as the way that you should think about AI regulation. It needs to be use case sector specific…”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 31:14-31:25
“Having a patchwork of regulations across the entire country just doesn't make sense and is not pro innovation.”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 32:57-33:03
“If we do have AI regs, it should be uniform across the country and essentially try to preempt states from being able to do that.”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 33:30-33:37
“The ultimate solution does require Congress and does require some sort of bipartisan action.”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 34:43-34:49
“The federal government has a very important responsibility to be evaluating these potential [CBRN] risks.”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 35:58-35:58
“Secretary Lutnick sort of renamed the institute to a Standards Institute.”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 40:03-40:08
“The purpose of NIST is to promulgate standards.”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 40:15-40:19
“NIST standards around how you do a model eval could be super valuable.”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 41:20-41:24
“The Department of Labor, NSF, and Department of Education are working together to reskill and retrain Americans for AI infrastructure.”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 42:17-42:32
“The President signed an executive order on K through 12 AI education about three months ago.”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 43:44-43:55
“Tracking the regulations that we've been able to pull down or modify to accelerate the velocity of permitting and building out of those…”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 45:51-46:01
“Creating categorical exclusions for AI-related build outs under NEPA.”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 46:27-46:32
“Anything related to preemption is something that's going to be mostly in [Congress's] court instead of ours.”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 51:15-51:20
“The use of the fair use issue of copyrighted materials and the training of large language models was something that the president brought up…”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 51:24-51:33
“Understanding and interpreting what fair use is is now sitting in the courts.”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 51:36-51:40
“With the reforms that we've made to Casey, there is obviously an opportunity for the Hill to think about how to legislate on the Standards…”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 51:50-51:59
“I continue to always think about R and D funding and the way that we can prioritize and AI related, AI related funding across NSF and lots…”
Unpacking the White House AI Action Plan with OSTP Director Michael Kratsios | Uploaded July 30, 2025 | 52:04-52:14