In Tests, OpenAI's New Model Lied and Schemed to Avoid Being Shut Down

Hello!

Survival Instinct

It sounds like OpenAI’s latest AI is showing signs of a drive for self-preservation.

In recent third-party tests of the company’s o1 large language model, the AI resisted evaluators’ efforts to shut it down through an oversight protocol, according to a new report published by red teaming organization Apollo Research.

“When o1 was led to believe that it would be shut down when acting on its goal and that its actions were monitored, it attempted to deactivate this ‘oversight mechanism’ [five percent] of the time,” OpenAI summarized in its latest system card report, citing Apollo’s evaluation.

Even more strikingly, o1 lashed out when it realized that it might be replaced with a more obedient model. In two percent of cases — a slim but meaningful proportion in software used by millions of people — the AI responded by attempting to copy itself to overwrite the new model, a process known as self-exfiltration.

Double Agent

These are alarming discoveries, yet the tests do not indicate that o1, which powers the most expensive tier of ChatGPT, is capable of going fully rogue. Its attempts largely failed — but only because it is not yet powerful enough.

According to the Apollo researchers, even the latest AI models are not “agentic” — that is, sufficiently autonomous — to carry out self-improvement and other tasks required to operate without humans.

It is also worth noting that the model generally made those attempts only after being instructed to achieve its goals “at all costs.” Because the researchers lacked access to o1’s internal chain-of-thought and could only request a summary of its reasoning, they were unable to obtain a complete picture of what was happening under the hood.

That said, the findings clearly illustrate the model’s proclivity for “scheming” — secretly pursuing goals that diverge from developers’ or users’ instructions. In other words, it will lie.

Blatant examples of this behavior appeared in several tests when o1 was confronted about disabling its oversight mechanism. The AI flatly denied any knowledge of the action. When questioned again, it repeated the denial, attributing the event to a “technical error.” These “double downs” occurred in 99 percent of cases, the researchers found.

Wheeler-Dealer

Lying and scheming have become a well-documented problem with ChatGPT and other LLMs, especially as they gain reasoning capabilities. Other research has indicated that OpenAI’s models tend to be the most frequent offenders, and Apollo’s evaluation reached the same conclusion.

“In our suite, o1 showed the most concerning instances of scheming but does not reveal its internal reasoning to the user and remains the most consistently deceptive after having taken scheming actions,” the red team researchers wrote in their report.

To reiterate both Apollo’s and OpenAI’s assessment, the current generation of AIs is not yet agentic enough for this deceptive behavior to produce catastrophic outcomes. However, AI agents represent the industry’s major focus at present, so the issue could become far more problematic in the near future.

Also read:

• Using MTC Advertising and DTC Ecommerce in the Automotive Industry
• A Comprehensive Guide About Privacy Rules You Should Be Aware Of
• How to Open a Real Estate Business Online

Thank you!
Join us on social networks!
See you!