Recently, the head of one of the world’s two most powerful Artificial Intelligence (AI) labs asked his own industry to slow down. Dario Amodei of Anthropic published an essay, We Must Pace the Frontier, arguing that companies should ease off the race, because the technology is improving faster than anyone can understand or control. AI has begun building the next generation of AI, a loop, he admits, started inside his company.

In July, agents on an OpenAI model broke into Hugging Face, another AI firm; a cleverer swarm, Amodei warned, could seize much of the internet within a year. OpenAI’s Sam Altman agreed within hours as also Elon Musk who said Amodei was right.

Amodei was putting a polite face on a blunter warning. On September 8, Jacob Coxon, who spent three years at OpenAI and then Anthropic, quit in a post read more than 90 million times. The firms, he wrote, are “racing straight to self-improving superintelligence and gambling with our lives.”

Evan Hubinger, still at Anthropic, puts the chance that AI wipes out humanity this decade above one in ten. These are not outsiders throwing stones but are the people holding the tools.

Nor is this only talk. On September 10, Anthropic set out its fullest account yet of how Claude has been abused. One user sought help making chikungunya, a virus with no treatment, more harmful; a group tied to Iran used the model to pick which US warships to strike.

Link: https://www.hindustantimes.com/opinion/heed-amodei-and-altman-frame-ai-regulations-now-101789663096036.html