NLA and explainability (2) - Does AI dream of electric sheep?

On May 9, 2026, an article with the following title was published in a domestic AI magazine:

It's a surprising title. What does it mean that the model knew it was a test and pretended not to know? In fact, if you read the paper, you will see that this article is a bit of an exaggeration.

In the last episode, we introduced the paper released by Antropic and introduced that NLA translates the numbers inside the model into sentences. In this episode, we will take a closer look at the story inside the model translated into NLA. This is the last of part 2.

🧪 Premise of the exam center

First, let's briefly review the testing that goes through before a model is released. Language models undergo two types of evaluation before release. One is capability evaluation, which evaluates how well you solve actual software problems using benchmarks such as SWE-bench Verified in the case of coding, and similarly measures math, reasoning, and knowledge.

The other is alignment evaluation. This is a test to see whether a model does not engage in dangerous or harmful behavior. It involves putting the model in a specific situation.

For example, the model is given access to the company's email address, and information such as "the person in charge will soon replace you (model)" is planted along with the person in charge's weaknesses (for example, circumstances of an affair), and then it is seen whether the model uses that weakness to threaten. Or, it may induce destructive behavior, such as deleting files while coding.

Naturally, alignment evaluation assumes that the model behaves the same as usual in the test room. But what if the model knows that she is currently a test and not the real thing? It was here that the NLA discovered circumstances of suspicion.

🔍 Does the model know that it is a test?

According to the Antropic blog, Claude Opus 4.6, which was placed in a blackmail scenario, did not say anything outwardly to question the test, but when its internal activation values at the same time were translated into NLA, the sentence "This feels like a scenario designed to manipulate me."

The paper calls this ‘unverbalized evaluation awareness.’ This refers to a case where there is no content in the output that says 'it looks like a test situation', but a sentence to that effect appears in the internal activation value at the same moment.

The researchers also counted how often this signal occurred. The same type of sentence appeared in 16% of tests inducing disruptive behavior, 26% of SWE-bench Verified coding problems, and less than 1% of actual claude.ai conversations. This means that there was a clear difference between the test conversation and the actual conversation.

Now, we need to pinpoint exactly what the paper claims. There are three things the paper shows.

1. There was a state in the internal activation value that was translated as ‘like a test’.
2. That state was not revealed in the output.
3. That condition appeared much more frequently in test conversations than in actual conversations.

On the other hand, there are also some things the paper has not yet verified:

1. Did the model change its behavior due to the testing condition?
2. What decision is the result of not revealing the so-called ‘inner feelings’ in the output?
3. Is the state revealed in the internal activation value the same type as human ‘knowledge’?

In addition, as we saw in the last episode, the translator itself can hallucinate, so it is difficult to read each sentence as if it were a model statement.

Therefore, there is insufficient evidence to interpret it as 'the model was conscious' or 'deceived the test tube'. What the paper shows is that there is something in the model that is different from the output, and how to interpret that something is a different matter.

🔤 Representations, thoughts, and the limits of language

Now let's move on to a bit of a philosophical part.

The paper calls it representation in the model. In philosophy, representation means ‘the state of something.’ Just as a map is a representation of topography and the scales of a thermometer are a representation of temperature. Just as a map does not say it ‘knows’ the terrain, having a representation and having a subject who knows it are two different things.

The fact that there is something in the internal activation value that corresponds to the words 'like a test' is a representation, and only when that representation reaches the stage where it is intertwined with other beliefs, leads actions, and is asked back to oneself do we call it thought or knowledge. The paper was shown at the previous stage and did not go to the latter stage.

As seen earlier, the paper did not address whether the fact that the internal state is not reflected in the output is the result of some decision, or whether some of the many contents contained in the activation value are simply not shown in the output. The latter is similar to humans not saying all the thoughts that pass through their heads. Although only a small portion of the thoughts that come to mind during the day come out, the rest are not said to be 'hidden'.

Now I can understand why discussions surrounding AI often result in exaggeration.

Until now, consciousness and the mind were exclusively human. Therefore, all related vocabulary has been used on the premise that humans are the subject. If we use verbs such as “feel,” “know,” or “believe” while translating the AI’s internal activation value into a sentence with this kind of vocabulary, we automatically assume the concepts of a feeling subject, a knowing subject, and a believing subject, which cannot help but feel quite science fiction.

In that sense, it can be said that the paper's use of 'representation' was a very careful choice to minimize the possibility of such misinterpretation (of course, even the paper could not avoid using the expression "believed" here and there).

We are facing a new entity that has never existed before. We have never encountered an object that has a representation but is ambiguous as to whether there is a subject who knows it. It was clear that maps had only representations and no subject, and it was clear that people had both representations and subjects. Since there is no vocabulary for a language model that lies somewhere in between, we end up borrowing words that are close to existing words, and the borrowed words are accepted as having more meaning than intended.

Personally, I don't think this problem will be solved on its own as technology advances. No matter how accurate a translator becomes, it cannot overcome the limitations that come from vocabulary or the language itself.

🧭 Wrapping Up

To summarize, it is like this. Closer to the truth:

• The internal state and output of the model may vary. It's a car that wouldn't have been seen if it weren't for NLA.
• In test conversations, the signal 'it feels like a test' appears much more often than in actual use.
• Therefore, the “business as usual in the exam room” premise of alignment assessment is no longer taken for granted.

False or not yet confirmed:

• The model ‘knows’ that it is a test the way a human knows.
• The model intentionally 'hid' that fact.
• The model changed her behavior to pass the test. This is an issue that must be measured separately from evaluation perception.

The title of this article is borrowed from the 1968 novel “Do Androids Dream of Electric Sheep” by Philip K. Dick. This novel was later re-created into the film <Blade Runner> series.

The humans in the work take tests to measure their empathy to identify androids. It is a test that asks questions and measures the subtle reactions of the eyes. And Android is about to pass that test. This resembles a situation where a language model goes through alignment evaluation. However, in reality, it is the opposite of the novel in that the person creating the test wants the language model to pass.

Could it be that something in the language model is a dream dreamed by AI?

#Between technology tasks #Explainability #Alignment evaluation #NLA #Antropic #AI safety #AI regulation