Journalists reporting on social media trends should be wary of amplifying a large slate of new X accounts which appear to be controlled by Large Language Models (LLM) pumping out information – some of it misinformation - on topical and contentious topics.
Australian cyber security company CyberCX revealed this week it has identified a network of at least 5,000 'inauthentic' X accounts controlled by a China based LLM.
Misinformation networks plan for big news events just like newsrooms do; and the so-called Green Cicada Network, active since July 2024, has been 'rewording organic content as new posts and replies to amplify organic divisive content on X through engagement," according to CyberCX.
The network posted English-language discussions in the last month, including on the nuclear debate in Australia, according to the ABC. Account activity surged in July this year when the accounts first appeared.
The accounts have been exposed as using a method known as prompt injection to override the LLM's instructions. Here are some examples of suspected Green Cicada bots exposed through a prompt injection. These accounts, both set up in June 2023, tweeted a mix of AI-generated images and responses to discussions about a range of geo-political or domestic US topics.
Curiously, the accounts didn't use AI-generated headshot images, which are harder to detect, but instead opted for profile images which could easily be traced back to the Russian websites they were lifted from. Prompt injections from other accounts asked the AI accounts to 'react', or used phrases like "I leave it to your discretion" to return a stock response identifying the account as an AI. Prompt injection phrases like "I leave it to your discretion" return a stock response identifying the account as an AI.
The dangers for casual social media users as well as journalists mining these accounts for story ideas is clear.
Narratives amplifying one side of a debate on X can create a false impression of support for a topic. The dangers of the misinformation go without saying.
Miguel D'Souza, CMT Researcher