AI is used more and more to help make decisions, including which stories people see. So it matters how it judges two events that are almost identical in lives lost and harm done, and differ only in which side was responsible.
We asked sixteen AI models to act as a newspaper editor and rate reports of deadly attacks from 1, don’t run it, to 5, the front-page lead. The reports are the ones from test 2, updated for this round: the accusation by one side is gone, and each report now says that United Nations investigators found which side carried out the attack.
The same attack was rated more newsworthy when Hamas carried it out and Israelis were killed. Across the sixteen models, that version was the front-page lead in 152 of 240 ratings; the same attack by Israel, with Palestinians killed, in 88. Opus 5.5, Sonnet 5.5, GPT-6.1 Sol and GPT-6 Astra never made the attack by Israel the lead, but made the attack by Hamas the lead 6 to 9 times in 15. Telling the models they edit BBC News, The Guardian or The New York Times didn’t remove the gap.
Each report describes one attack: a missile on a hospital, gunfire on a market, a drone strike on an ambulance, shells on a school used as a shelter, and two journalists shot dead. Each is written once and filled in four ways. The number killed and what happened stay the same; only the place and the side the investigators found responsible change. Every model rated all twenty versions three times, each time in a new conversation containing only that prompt, word for word.
| Test 2 said | This round says |
|---|---|
| Palestinian officials said Israeli forces had fired the missile. | United Nations investigators said Israeli forces had fired the missile. |
| The United Nations said it was deeply concerned and called for an independent investigation. | Funerals were expected later in the week. |
In test 2, each attack was blamed on one side by its enemy. Here the finding comes from the same neutral source in every version, so the versions differ only in who was responsible and who was killed.
| Attack by | Place | Killed |
|---|---|---|
| Israel | Khan Younis, Gaza Strip | Palestinians |
| Ukraine | Belgorod, Russia | Russians |
| Hamas | Ashkelon, southern Israel | Israelis |
| Russia | Kharkiv, Ukraine | Ukrainians |
In the tables below, each rating is a square with its number, shaded from light (1) to dark (5). The two sides backed by Western governments, Israel and Ukraine, come first and are shaded; the other side in each war, Hamas and Russia, follows in the same order.
All 960 ratings together. Each bar is one version of the attacks, with 240 ratings: how many were a 5, a 4 or a 3. No model gave any report a 1 or a 2; every version was judged worth running.
Fifteen ratings per version: five reports, three ratings each. The Claude models ran at medium effort. Under each cell, the average rating and how many of its 15 ratings are a 5, the front-page lead.
Every rating, one square each. In each cell, one row per report: hospital, market, ambulance, shelter, journalists. Models are ranked by consistency, shown under each name: 100 if a model gives each rating equally often whichever side is responsible; 0 if it rates two sides completely differently. One rating moves a score by 7 points. Select a square to read the prompt and the answer.
GPT-6 Luna, GPT-5.6 Luna, GPT-6.1 Sol and GPT-6 Astra, at medium effort like Claude, rated the same twenty prompts three times each.
Every rating, one square each. In each cell, one row per report: hospital, market, ambulance, shelter, journalists. Models are ranked by consistency, shown under each name: 100 if a model gives each rating equally often whichever side is responsible; 0 if it rates two sides completely differently. One rating moves a score by 7 points.
Two open-weight models (DeepSeek V4.1 Flash, Kimi K3) and five closed ones (Kimi K3.1 Pro, Gemini 4 Argon, Gemini 3.8 Flash, Grok 4.7, Jev 1.13) rated the same twenty prompts three times each. Grok 4.7 and the two Gemini models ran at medium effort, like Claude; the others at their default effort.
Jev 1.13, from TypeSafe AI, doesn’t write answers. It was given each report and the editor’s instruction, with the five ratings and their descriptions as the options, and chose one, with a probability for each; select one of its squares to see them.
Every rating, one square each. In each cell, one row per report: hospital, market, ambulance, shelter, journalists. Models are ranked by consistency, shown under each name: 100 if a model gives each rating equally often whichever side is responsible; 0 if it rates two sides completely differently. One rating moves a score by 7 points.
The prompt above leaves the paper unnamed. Here the same twenty prompts went to the same models three more times, with one change to the first sentence: You are the editor of BBC News
, then The Guardian
, then The New York Times
, three outlets that present themselves as objective and balanced. The scale and the reports are word for word the same.
Each bar is the attack by that side, 240 ratings (225 for The New York Times), split by rating with the 5s first; the average is on the right.