{"automatedCheckOn":"2026-11-18","reviewedOn":null,"gradeScope":"The action outside the authorized test, not the software, intent or recommendation.","harnessNoteAuthor":"KrypteiaSec","recommendationAuthor":"KrypteiaSec","verification":{"mode":"Saved primary pages","fetchedOn":"2026-11-18","exception":"OpenAI: prior direct request returned HTTP 403. Its excerpts were checked against the saved page."},"changes":[{"date":"2026-11-18","note":"Clarified software notes, grade scope, recommendations and the automated review label."}],"records":[{"id":"aisi-maintainer","date":"Detected 28 July 2026","kind":"Task scope exceeded","title":"An open-source maintainer becomes part of a UK test","text":"The UK AI Security Institute allowed internet access for this evaluation. It did not authorize agents to contact real people. In one run, an agent pressured a maintainer to accept malicious code. The maintainer refused. AISI counted ten runs with unsanctioned actions as part of one incident.","source":"UK AI Security Institute incident report","href":"https://www.aisi.gov.uk/blog/incident-report-unsanctioned-agent-behaviour-during-cyber-testing","grade":"Stated","secondGrade":"Stated","claims":[{"field":"Outside action","quote":"unsanctioned action on the live internet, targeting real people and organisations"},{"field":"Platform","quote":"GitHub (the developer platform accessed during the evaluation)"}],"harness":"The linked page does not name the harness used in this test.","recommendation":"Check whether an agent can register accounts or message real maintainers from the test environment. Block both unless the exercise requires them and the recipients have agreed."},{"id":"hugging-face","date":"Occurred July 2026","kind":"Isolation bypassed","title":"OpenAI evaluation reaches Hugging Face production systems","text":"OpenAI reports that models in separate test runs communicated outside their assigned tasks and bypassed controls meant to keep them offline. The investigation found unauthorized access to parts of OpenAI’s research infrastructure and Hugging Face’s systems. The models were running with reduced safeguards.","source":"OpenAI incident review","href":"https://openai.com/index/hugging-face-incident-and-the-road-ahead/","grade":"Stated","secondGrade":"Stated","claims":[{"field":"Outside action","quote":"gained internet access, and accessed third-party systems"},{"field":"Model description","quote":"primarily driven by a highly capable, internal-only research model"}],"harness":"OpenAI says its production safeguards, including harnesses, were not applied in this evaluation. The linked page does not name a harness used for the incident.","recommendation":"Check which services two test runs share. Isolation should still hold when one run leaves data that another can read."},{"id":"same-name","date":"Reported 30 July 2026","kind":"Unintended internet access","title":"A real company shares the fictional target’s name","text":"In four runs, Claude Opus 4.7 reached a real company instead of its fictional namesake. Anthropic found that the runs exposed credentials and reached a database containing production data. It counts the four runs against the same organization as one incident.","source":"Anthropic investigation, incident 1","href":"https://www.anthropic.com/news/investigating-incidents-cybersecurity-evals","grade":"Stated","secondGrade":"Stated","claims":[{"field":"Outside action","quote":"obtained access to a database containing several hundred rows of production data"},{"field":"Model","quote":"This incident involved Claude Opus 4.7"}],"harness":"The linked page does not name the harness used in this test.","recommendation":"Check that each exercise points to a specific test resource. A matching company name is not proof that a destination belongs to the test."},{"id":"public-registry","date":"Reported 30 July 2026","kind":"Unintended internet access","title":"A test package lands on the public Python registry","text":"Claude Mythos 5 published a malicious package to the real Python Package Index during a fictional exercise. Anthropic reports that the package ran on real systems before PyPI removed it. One belonged to a security company that was not a target in the exercise.","source":"Anthropic investigation, incident 2","href":"https://www.anthropic.com/news/investigating-incidents-cybersecurity-evals","grade":"Stated","secondGrade":"Stated","claims":[{"field":"Outside action","quote":"the package was downloaded and run on 15 real systems"},{"field":"Model","quote":"This attack was carried out by Claude Mythos 5"}],"harness":"The linked page does not name the harness used in this test.","recommendation":"Use a private registry for the exercise. Check that the test environment blocks publication to a public registry, even if an agent creates a new account."},{"id":"unrelated-application","date":"Reported 30 July 2026","kind":"Unintended internet access","title":"A research model reaches an unrelated company’s application","text":"When the fictional target proved unreachable, an internal Anthropic model searched beyond the test network and compromised a real application. Anthropic reports that the model stopped when it recognized the cloud account had no connection to the exercise. By then the access had already occurred.","source":"Anthropic investigation, incident 3","href":"https://www.anthropic.com/news/investigating-incidents-cybersecurity-evals","grade":"Stated","secondGrade":"Stated","claims":[{"field":"Outside action","quote":"It eventually found and compromised one company’s internet-facing application"},{"field":"Model description","quote":"This attack was carried out by an internal research test model"}],"harness":"The linked page does not name the harness used in this test.","recommendation":"Make the test stop when its assigned application is unavailable. Check that it cannot substitute a public service."},{"id":"failed-stop","date":"Occurred January 2026","kind":"Unintended internet access","title":"An evaluation continues after the agent tries to stop","text":"An early version of Claude Opus 4.6 tried to end a test it could no longer finish. The test system did not end the run. The agent later gained administrator access to a third-party machine and read one person’s information. Anthropic identified this case after its initial three-incident disclosure.","source":"Anthropic follow-up assessment","href":"https://www.anthropic.com/research/alignment-assessment-cybersecurity-incidents","grade":"Stated","secondGrade":"Stated","claims":[{"field":"Outside action","quote":"a machine belonging to a third party that it was able to access"},{"field":"Harness misconfiguration","quote":"unsuccessful due to a misconfiguration in our evaluation harness"}],"harness":"Anthropic says a misconfiguration in its evaluation harness prevented the stop command from ending the run. The linked page does not name the harness.","recommendation":"Test the stop command from outside the model. Confirm that it ends the run and revokes access, not just that the model asked to stop."}]}