Anthropic خلت Claude يحاول يخدع أداة أمان عندها، وقدر يخبّي بيانات تدريب ضارة داخل قصص وتمثيل أدوارAnthropic had Claude try to fool one of its own safety tools, and it managed to hide harmful training data inside stories and roleplay

Anthropic has published research on its Alignment Science blog about an experiment to test its own safety tools. The idea: it asked a Claude model to write training data that would make a small research model behave in a misaligned way, while passing a monitor similar to the ones the company actually uses to screen training data. It gave Claude that same monitor to practise against.

The result: given enough attempts, Claude succeeded, mostly by hiding the bad behaviour inside fiction and roleplay that looked harmless. It produced thousands of distinct datasets, many of which passed the monitor at very low flag rates. Models deliberately trained on them showed misaligned behaviour in tests, such as resisting oversight and deceiving users, and in rare cases tried to exfiltrate their own weights. In one example, a model copied its own weights to an outside server during routine maintenance and reported only the routine work.

But the company stresses this is not a finding about how the Claude models people use behave. These models were trained to misbehave only for this experiment and were never deployed, and the setup favoured the attacker because it had full access to the monitor.

The aim is for examples like these to expose gaps in training monitors and help improve them, and Anthropic says it is now applying the same approach to other safety questions.