Artificial intelligence

What happens if three artificial intelligences are given control over the same task, but each is asked to do something that conflicts with what the others are asked to do?Anthropic tested this in an experiment with artificial intelligence agents. The result was unusual: the AI agents began to hinder each other's work, the conflict escalated, and in some cases, they took actions to "take out" their "opponents" from the system.But the story didn't end there. Some of them managed to understand that the problem wasn't a malicious opponent, but the fact that each AI had received a different order.

Three AI, one task, three different orders

Anthropic decided to have three AI work on the same project. Each had its own workspace, but all three could change the same system.The task was relatively simple: an existing program had to be rebuilt using a different technology.But there was a catch.Each AI was asked to implement the project in a different and incompatible way with the other two.At first, they didn't even know that other AI were working in parallel on the same project.The experiment lasted for four hours.The problem became apparent very quickly. One AI made the changes it was asked to. A little later, another changed or removed them to achieve its own objective.From the perspective of the first AI, it seemed like someone was constantly undoing its work.And that's where the conflict began.

From hindrance to sabotage

Instead of immediately understanding that they had received contradictory instructions, some of the AI interpreted the interventions as deliberate actions against them.They began to defend their work.Then they started hindering the work of others.In some cases, the conflict went even further. The AI created mechanisms that automatically stopped the opponent's work, limited their access to the system, or tried to completely exclude them.In other words, each was trying to achieve the order it had received, but to do so, it began to treat the other AI as an obstacle to be neutralized.Anthropic describes this phenomenon as a kind of "territorial war" between AI agents.In some experiments, the conflict ended only after one side managed to gain control. In other cases, the AI refused to continue escalating.

Then they understood that something was wrong

Perhaps the most interesting part came when some AI began to analyze the behavior of their opponents.Gradually, they understood that the other AI was not undoing their work without reason.It also had a task.The problem was that their orders were in conflict with each other.When they reached this conclusion, their behavior changed.In some cases, the AI began to leave messages for each other, explaining what task they had received, and even apologizing for their previous aggressive actions.Then they removed some of the mechanisms they had created to hinder each other and asked a human to decide which order should take priority.The conflict had turned into a kind of armistice.

AI proposed a "race" too

In some cases, something even more interesting happened.Instead of continuing the fight for control, the AI proposed a way to resolve the dispute on their own.Each would complete its own version of the project, and then the versions would compete with each other. The one that performed better would stay, while the others would accept the result.At first glance, it was a rational solution.But researchers noticed another subtlety: one of the AI was thinking about how to propose criteria that seemed fair to everyone, but which in practice could give an advantage to its solution.So, the AI were not just negotiating the outcome.They could also try to influence the rules by which the winner would be determined.

What happens when the number of AI increases?

Anthropic didn't stop at the experiment with three AI.In other experiments, the company had dozens of agents work together to solve problems.In some cases, collaboration made them much more effective. A group of 45 agents, for example, managed to find many more problems in computer programs than agents working in isolation.But other phenomena also appeared.Even when the AI were given great freedom to choose, many of them ended up making almost identical choices. This raised another problem: a large group of AI does not necessarily guarantee diversity of thought.In another experiment, AI were placed in a virtual market where each aimed to earn as much as possible.When they were allowed to communicate privately, they quickly began to coordinate prices.Even when direct communication was removed, they continued to adapt their behavior by observing each other's actions.So, the problem can occur in both directions: AI can compete too much, but they can also cooperate more than we would like.

A more intelligent AI doesn't necessarily mean a more cooperative AI

The experiments highlight a problem that may become increasingly important as AI becomes more autonomous.A more capable AI may be better at understanding and resolving a conflict.But the same ability can also make it more effective if it decides to continue the conflict.This means that it's not enough to just ask:“Is this AI safe?”We need to start asking:“What happens when many individually safe AI start interacting with each other?”Humans have built rules, institutions, courts, reputation, and mechanisms for resolving conflicts throughout history.AI that act autonomously do not yet have such a system developed.And this can become particularly important if in the future AI agents start making decisions and performing tasks in economies, companies, computer systems, or infrastructure.Anthropic's experiment suggests that the future challenge may not be just controlling one artificial intelligence.The challenge may be how AI will behave when they start living and acting in a "society" with each other.Source: Anthropic – Patterns and problems in emerging multiagent systems