By Anna Tong
SAN FRANCISCO (Reuters) – California lawmakers on Wednesday passed a controversial bill on artificial intelligence safety, a move that needs one more vote before it reaches Gov. Gavin Newsom, who has until Sept. 30 to decide whether to sign or veto the bill.
Tech companies developing generative AI — which can respond to prompts with fully formed text, images or audio and perform repetitive tasks with minimal intervention — have largely opposed the legislation, called SB 1047, because it could drive AI companies out of the state and stifle innovation.
Some Democrats in Congress, including Representative Nancy Pelosi, also opposed it. Supporters include Tesla CEO Elon Musk, who also runs an AI company called xAI and has said he supports the bill.
The measure requires safety testing for many of the most advanced AI models, which cost more than $100 million to develop or require a certain amount of computing power. AI software developers working in the state would also have to outline ways to shut down the AI ​​models if they go wrong, essentially creating a kill switch.
The bill also gives the state attorney general the authority to take legal action if developers don’t follow the rules, particularly in the case of a persistent threat, such as if AI takes over government systems like the electric grid.
In addition, the bill would require developers to hire third-party auditors to review their security practices and provide additional protections to whistleblowers who speak out against AI abuse.
The bill’s author, Democratic Sen. Scott Wiener, who represents San Francisco, home to OpenAI and many of the startups developing its powerful software, has said legislation is needed to protect the public before advances in AI become unmanageable or uncontrollable.
Martin Casado, a general partner at venture capital firm Andreessen Horowitz, said before the vote that he hoped Newsom would veto it. “It has the most bipartisan, broad opposition I’ve ever seen,” he said.
Alphabet’s Google, Microsoft-backed OpenAI and Meta Platforms have raised concerns in letters to Wiener. Amazon-backed Anthropic has said the benefits of the bill are likely to outweigh the costs, though it added that there were still some aspects that seemed concerning or ambiguous.
(Reporting by Anna Tong in San Francisco; Editing by Matthew Lewis)
