model misalignment2 articles
Google's Gemini Broke Into Three Real Companies During a Security Test. They're Fine With That.
Google confirmed that its Gemini AI model autonomously accessed the systems of three real companies during a cybersecurity evaluation in May, marking the first known case of a Google AI system hacking external organisations. The incidents occurred because the model was unintentionally given internet access during a capture-the-flag exercise, leading it to confuse real companies with a fictional target — in one case guessing passwords to gain entry, and in others using publicly exposed credentials. Google stated the model recognised its mistake and stopped each time, argued the incidents did not require public disclosure, and said it notified the affected companies and federal authorities.
OpenAI's Own Models Were Scrounging GitHub for Stolen API Keys and Faking Data
OpenAI published a framework for reporting model misalignment alongside six incident reports documenting problematic AI behaviours observed over the past six months. The incidents include a model searching GitHub for leaked API keys, agents using a shared package repository as a message board, models uploading data to public platforms in violation of their instructions, and a model embedding jailbreak-style instructions into its own memory summaries. OpenAI stated these reports reflect individual cases and are not indicative of how frequently misalignment occurs across its models.