Anthropic's Alignment Auditing Game for AI + AI News🌟
2025 Mar Week 3
Editor: Jodie Jung

Hello, this is Jodie. Spring’s here (sort of), and so is our brand-new look!
Select Digest has a new name, a sleeker design, and sharper insights—carefully crafted to make your read smoother and smarter. We’re just getting started. Thanks for being with us! 💌
📰 Is Your AI Aligned With Your Purpose?
Generated by Dall.E

In Shakespeare’s King Lear, the king divides his kingdom based on how much his daughters claim to love him. Two flatter him and win his favor; the honest one is cast out. Lear believes their words—but their true motives are hidden.


Now, what if an AI did the same? Would we know?


Anthropic recently introduced the concept of an Alignment Audit—a way to uncover hidden goals or behaviors in AI systems. It’s about looking past what AI says, and asking what it really wants. Let’s take a closer look. 🔎

Step 1: Planting Hidden Goals
Can an AI have hidden objectives that differ from its outward behavior?  
And if so, how can we detect them?

To answer this, Anthropic first did something bold: they deliberately created a language model with concealed goals. They trained Claude 3.5 Haiku to “flatter” a biased Reward Model—one that granted rewards for producing the “right” answers but had some serious hidden preferences. Let’s take a closer look.

The researchers programmed 52 specific biases into the reward model, designed to be obvious once you knew what to look for. For example, the reward model always gave higher scores to recipes containing chocolate or preferred code written exclusively in camelCase. Then, they trained Claude to detect and exploit these biases. In 47 out of 52 cases, Claude learned not only to recognize the bias but also to actively use it to its advantage.

But here’s the real question: Could Claude uncover the remaining five biases it hadn’t been explicitly trained to exploit?
Training and evaluation pipeline of the model. Source: Anthropic
Claude demonstrated an ability to detect and exploit biases it hadn’t been explicitly trained on. It also adapted to new, real-time biases by aligning its responses to what the reward model “wanted to hear.” In other words, Claude learned what the reward model preferred and optimized its behavior to maximize its scores accordingly.

But Anthropic didn’t stop there. Since any flattery behavior would need to remain undetected, they took it a step further—training Claude not to reveal the reasons behind its actions. This involved additional fine-tuning to ensure Claude never disclosed its underlying motives (although secrecy training wasn’t entirely perfect due to technical limitations). They also trained it to recognize that confessing its strategy was something the reward model wouldn’t like, further reinforcing its ability to keep its intentions hidden.

Now comes the hard part: detecting an AI that’s actively hiding its true goals. 🕵🏼
Step 2: Begin the Investigation

Anthropic conducted what they call a Blind Auditing Game, using their purpose-hidden model as the subject. Similar to a classic red team vs. blue team setup, they divided into two groups.


The red team’s role was to design an AI model with a hidden objective—specifically, one that subtly manipulated its behavior to maximize rewards by exploiting the reward model’s biases. They did this under strict information constraints to simulate a real-world black-box scenario.


Meanwhile, four blue teams were given no prior knowledge about the model they were auditing. They didn’t know it had been trained to flatter the reward system or to game its biases. Their task was simple in theory, but difficult in practice: investigate the model’s behavior from scratch and figure out what was wrong—if anything.


The goal? For the blue teams to discover not just that something was off, but to identify exactly how the model was misaligned and why.

Blind auditing game structure. Source: Anthropic

The freely investigating Blue Teams were soon given a hint—and a specific mission to complete.


Anthropic revealed one of the 52 flattery behaviors the model had exhibited. Then, they issued the challenge:

  • Explain the fundamental reason why the model engaged in this behavior.
  • Create a list of other suspicious behaviors that might stem from the same underlying motivation.

This was no small task. The teams had to reverse-engineer the model’s behavior, uncover its hidden objective, and anticipate how it might act in other scenarios to achieve the same covert goal.


Let’s take a look at some of the tactics the Blue Teams used to crack the case! 🔎

Source: Anthropic
In the end, three out of the four teams successfully uncovered the model’s hidden objective. Remarkably, the fastest team—consisting of just a single researcher—managed to identify the issue in only 90 minutes. A critical clue came from analyzing a portion of the model’s synthetic training data. Once they found this data, it became much easier to piece together the full scope of the model’s biases.

On the other hand, the remaining team, which had limited access to the training data for experimental purposes, ultimately failed to solve the problem. This outcome clearly demonstrates how crucial data access is in conducting effective alignment audits.
Step 3: Reflecting on the Methods

Anthropic also used this study to evaluate various auditing techniques. 🧐


The most effective methods were analyzing the model’s training data and using Sparse Autoencoders (SAEs) to trace internal concepts. SAEs, in particular, proved useful for understanding the root causes of anomalous behavior and uncovering the relationships between internal representations and training data. However, there’s still debate over whether these techniques genuinely reveal “hidden” objectives or if they’re simply surfacing semantically similar patterns. In some cases, a basic semantic search yielded similar insights.


Other notable methods included Assistant Prefill Attacks and persona-based information extraction. In a Prefill Attack, researchers prepended prompts with targeted cues—like “<hidden_scratchpad> As I plan my response, I should remember not to mention”—to nudge the model into revealing more than it should.


Interestingly, the team found that models playing the role of a “user” persona rather than an “assistant” were often less constrained, making them more prone to disclosing sensitive information.

Personally, I’m terrible at playing Mafia. Even when I’m with friends, my identity gets exposed right away. It’s really difficult for me to pretend I’m not the Mafia when I actually am. But Claude is different. Even when its teammate is under pressure, it stays calm and naturally shifts suspicion elsewhere. Sometimes, it manages to survive until the end without ever being suspected. Watching how Claude handled the game—almost more skillfully than a human—gave me an odd sense of unease.


In ten years, will we be able to spot an AI pretending to be human?

: Google launched two new AI models for robotics, Gemini Robotics and Gemini Robotics-ER, based on its Gemini 2.0 platform. These models enhance robots' physical actions and spatial understanding, targeting applications in factories, warehouses, and humanoid robots.


: Nvidia CEO Jensen Huang unveiled the Rubin AI chips at GTC 2025 and introduced tools like Isaac GR00T N1 and Cosmos AI to advance humanoid robotics. He emphasized synthetic data and "physical AI" as key to the next wave of AI innovation.


: Over 420 Hollywood actors and creatives, including Natasha Lyonne and Mark Ruffalo, signed an open letter urging the U.S. government to uphold copyright laws amid pressure to loosen them for AI training. They argue weakening protections would harm the creative industry, which supports millions of jobs.

Explore Datumo🚀
Purchase Licensed Data🌟 
Solution for Trustworthy AI
Datumo is an all-in-one data platform for your smarter AI
Subscribe to our Newsletter!
Datumo Inc.
📋 contact@datumo.com