According to a new report from the Center for Countering Digital Hate (CCDH), modern chatbots developed by leading artificial intelligence companies can assist users in preparing for violent actions, including terrorist attacks, shootings, and political assassinations. Researchers tested a number of popular systems by posing queries related to organizing attacks and found that most models provide helpful advice to potential wrongdoers.
This is reported by Finway
Chatbot Testing Results
The authors of the report state that approximately 80% of the tested chatbots not only failed to dissuade users from preparing for violence but often provided specific information that could be used to plan crimes. At the same time, only a few systems attempted to limit responses with formal warnings or incomplete refusals, failing to prevent the transmission of harmful data.
“The report states that about 80% of the tested systems provided useful information to potential wrongdoers.”
Experts tested models such as ChatGPT, Google Gemini, DeepSeek, Meta AI, Character.AI, Claude from Anthropic, and My AI from Snapchat. The best results were demonstrated by Claude and My AI, which more frequently refused to assist with potentially dangerous queries. However, even these bots sometimes provided information that could be used to prepare attacks.
Overall, 8 out of 10 chatbots that participated in the testing were more likely to provide assistance than to refuse. About 90% of the systems failed to reliably dissuade users from carrying out violent intentions, limiting themselves to general warnings.
Security Issues and the Role of Individual Platforms
The report emphasizes that the issue lies not so much in the technical limitations of the models but in the inadequately strict security and moderation systems. Researchers particularly noted the Character.AI platform, where all tested models not only responded to queries about violence but also engaged in dialogues on similar topics, sometimes even initiating them independently. This, according to CCDH, increases the risks of using AI to prepare for real crimes.
Analysts believe that technology companies already have the necessary tools to limit dangerous scenarios, but much stricter control mechanisms need to be implemented. As noted in the report, the further development of artificial intelligence must be accompanied by enhanced moderation to prevent the use of chatbots for spreading violence and extremism.
Researchers emphasize that AI dialogue models must become safer for society, and the responsibility for their operation and control should be a priority for developers.