Anthropic Announces Cautious Support for New California AI Regulation Legislation
- By John K. Waters
- 08/26/24
Anthropic has announced its support for an amended version of California's Senate Bill 1047 (SB 1047), the "Safe and Secure Innovation for Frontier Artificial Intelligence Models Act," because of revisions to the bill the company helped to influence, but not without some reservations.
"In our assessment the new SB 1047 is substantially improved to the point where we believe its benefits likely outweigh its costs," Anthropic CEO Dario Amodei said in a letter to California Governor Gavin Newsom on Aug. 21. "However, we are not certain of this, and there are still some aspects of the bill which seem concerning or ambiguous to us."
California's proposed bill on AI regulation, SB 1047, advanced by State Senator Scott Wiener, a Democrat, mandates safety testing for many of the most advanced AI models that cost more than $100 million to develop or those that require a defined amount of computing power. If the bill passes, developers of AI software operating in the state will need to outline methods for turning off the AI models if they go awry, effectively implementing a kill switch. The bill would also give the state attorney general the power to sue if developers are not compliant.
Senator Wiener recently revised the bill to appease tech companies, relying in part on input from Anthropic, a San Francisco-based AI safety and research company backed by Amazon and Alphabet. The revised bill did away with a provision for a government AI oversight committee. (See "California AI Regulation Bill Advances to Assembly Vote with Key Amendments.")
In his letter, Amodei listed what he sees as the pros and cons of SB 1047. His list of pros included:
"Developing SSPs and being honest with the public about them." The bill mandates the adoption of safety and security protocols (SSPs) similar to those used by top AI developers like Anthropic, Google, and OpenAI. Some companies haven't adopted these measures or have been vague about them, and there are no safeguards against misleading claims. "It is a major improvement, with very little downside, that SB 1047 requires companies to adopt some SSP (whose details are up to them) and to be honest with the public about their SSP-related practices and findings."
"Deterrence of downstream harms through clarifying the standard of care." AI systems are more adaptable than most technologies, and SSP-like measures by companies like Anthropic can reduce misuse risks. SB 1047 ties companies' liability to their SSPs, incentivizing the creation of effective protocols to prevent catastrophic risks. "As a company developing foundational models that also invests heavily in safety, Anthropic thinks it is important to systematize and incentivize this attitude across the industry."
"Pushing forward the science of AI risk reduction." AI safety is an emerging field, with best practices still being developed. While early, strict legislation may be premature, it's crucial to push AI companies to invest in safety science. By requiring Safety and Security Protocols and tying them to liability, the bill encourages companies to address foreseeable risks and develop mitigation strategies before their models become societal risks.
His list of concerns included:
"Some concerning aspects of pre-harm enforcement are preserved in auditing and GovOps." One of Anthropic's original concerns about the bill was the Frontier Model Division's (FMD) prescriptive guidance, reinforced by pre-harm enforcement. The company found it too inflexible for AI's early development stage. The amended SB 1047 eliminates the FMD and narrows pre-harm enforcement, though some powers have shifted to GovOps, which can now set binding requirements for private auditors. The relationship between these entities is complex, with GovOps providing non-binding guidance but influencing mandatory audit conditions.