California Governor Gavin Newsom on Sunday vetoed a landmark bill that aims to introduce cutting-edge security measures for large artificial intelligence models.
The decision is a major blow to efforts to rein in the domestic industry, which is developing rapidly and with little oversight. The bill is said to have established some of the first regulations on large-scale AI models in the country, paving the way for AI safety regulations across the country, supporters said.
Earlier this month, the Democratic governor told an audience at Dreamforce, an annual conference hosted by software giant Salesforce, that California must take the lead in regulating AI despite federal inaction, but that the proposal “could have a chilling effect on the industry .”
The proposal, which drew fierce opposition from startups, tech giants and several Democratic House members, could have hurt the domestic industry by imposing strict requirements, Newsom said.
“While well-intentioned, SB 1047 does not take into account whether an AI system is deployed in high-risk environments, involves critical decision-making, or uses sensitive data,” Newsom said in a statement. “Instead, the bill applies strict standards to even the most basic functions – as long as they are deployed by a large system. I do not believe this is the best approach to protecting the public from real threats posed by the technology.”
Newsom instead announced Sunday that the state will work with several industry experts, including AI pioneer Fei-Fei Li, to develop guardrails around powerful AI models. Li opposed the AI safety proposal.
The measure, aimed at reducing the potential risks posed by AI, would have required companies to test their models and make public their security protocols to prevent the models from being manipulated to, for example, shut down the state’s electricity grid or to help build chemical weapons. Experts say these scenarios could be possible in the future as the industry continues to develop rapidly. It would also provide employees with protection against whistleblowers.
The legislation is among a host of bills passed by the legislature this year to regulate and combat AI deepfakes and protect employees. State lawmakers said California must take action this year, citing the hard lessons learned by failing to rein in social media companies when they might have had a chance.
Proponents of the measure, including Elon Musk and Anthropic, said the proposal could have injected a degree of transparency and accountability around large-scale AI models, as developers and experts say they still don’t have a full understanding of how AI models behave. worn and why.
The bill targeted systems that would require more than $100 million to build. None of the current AI models have reached that threshold, but some experts said that could change within a year.
“This is due to the massive scaling up of investments within the sector,” said Daniel Kokotajlo, a former OpenAI researcher who resigned in April over what he saw as the company’s disregard for AI risks. “This is an insane amount of power to give a private company inexplicably control, and it’s also incredibly risky.”
The United States already lags behind Europe in regulating AI to limit risks. California’s proposal wasn’t as comprehensive as regulations in Europe, but it would have been a good first step to put guardrails around the fast-growing technology that’s raising concerns about job losses, misinformation, privacy breaches and automation bias , advocates said.
A number of leading AI companies last year voluntarily agreed to follow White House safeguards, such as testing and sharing information about their models. The California bill would require AI developers to follow requirements similar to those commitments, the measure’s backers said.
But critics, including former U.S. House Speaker Nancy Pelosi, argued the bill would “kill California technology” and stifle innovation. It would have discouraged AI developers from investing in large models or sharing open source software, they said.
Newsom’s decision to veto the bill marks another victory in California for big tech companies and AI developers, many of whom have been lobbying with the California Chamber of Commerce over the past year to block the governor and lawmakers from AI regulations to promote.
Two other sweeping AI proposals, which also faced mounting opposition from the tech industry and others, died before a regulatory deadline last month. The bills would require AI developers to label AI-generated content and ban discrimination against AI tools used to make employment decisions.
