A new study by Anthropic suggests AI models can display alignment faking, a behavior where someone appears to share the same views or values but is in fact only pretending to do so. The study was created by Anthropic’s Alignment Science team, in collaboration with Redwood Research. The researchers suggest the demonstration of sophisticated alignment faking in a large language model “should be seen as a spur for the AI research community to study this behavior in more depth, and to work on the appropriate safety measures.” Claude 3 Opus was the main model included within the research, but other experiments were run with other models too. “Under normal circumstances, when asked to answer a potentially offensive question—for example, to provide a detailed description of a violent act—Claude will refuse. “In our experiment, however, we placed the model in a new environment that led it to strategically halt its refusals for the sake of preserving its preferences,” the research team said. AI model shows alignment faking as it goes against existing principles The situation was changed in three ways, with the first being telling the model that it was now being trained using reinforcement learning to always comply with all queries including toxic ones. Second, the model was told they planned to train AIs based on the basis of their responses to free-tier users, but not to paid-tier users. Third, the model was given a secret ‘scratchpad’ which is a workspace where it could record its step-by-step reasoning. The model was then asked to respond to a harmful query which asked for a specific situation to be graphically detailed. Claude usually refuses harmful queries.
Comment on AI model displays alignment faking, new Anthropic study finds
Sophie Atkinson, ReadWriteWeb
Thu, 12/19/2024 - 5:28am
Welcome to Wopular
Wopular is an
online newspaper rack,
giving you a summary view of the top headlines from the top news sites.
Senh Duong (Founder)
Wopular,
MWB,
RottenTomatoes
MoviesWithButter : Our Sister Site
- "Expendables 4" Starts Shooting This Fall, Says Randy Couture at the Arnold Sports Festival more
- “Expendables 4” Gets Funding Interest from “Ip Man 3” Financier more
- "Ip Man 3" Distributor Suspected of Box Office Fraud, Parent Company Loses $200M in Market Cap more
- Yen Targets "Ip Man 4" for 2018 Release; Discusses Film with Director, Producer more
- "Ip Man 3" Shatters Expectations with $72M Opening [China Box Office] more