Propaganda is not a new phenomenon. But chatbots bring a whole new dimension: they do not transmit biased information, but conduct personal conversations with the user. This makes them much more convincing than traditional media. Unlike a newspaper or state television, the user does not always have a clear idea of the source of information in a chatbot.
Every week, over a billion people turn to chatbots for information, advice, and many other services. ChatGPT alone has around 900 million weekly users. The impact of these systems is expected to grow even more in the coming years. There are already signs that large language models (LLMs), such as ChatGPT, Claude, and Gemini, can influence users’ opinions through their responses and conversations. This has raised growing concerns that chatbots could become tools for spreading propaganda.
FEAR OF “INVISIBLE” PROPAGANDA
Much of the discussion about political manipulation through AI focuses on the idea that large companies can program models to favor certain ideologies and suppress others. This concern is not unfounded. The Chinese company DeepSeek has been accused of programming its model to avoid sensitive topics such as the Tiananmen Square Massacre or the suppression of political opponents. However, more advanced models such as ChatGPT, Claude, and Gemini are developed by American companies and are not directly controlled by authoritarian regimes.
But that doesn’t mean authoritarian governments can’t indirectly influence their behavior. According to a study published in the journal Nature, authoritarian states can now influence chatbot responses without directly interfering with the programming.
HOW STATE MEDIA INFLUENCES AI CHATBOTS
Language models are trained on large amounts of text from the internet, learning patterns of language and information. An important consequence is that they do not always respond the same way across languages. In languages spoken in authoritarian states, the data is often dominated by state or pro-government media, as they produce a lot of content and are easily accessible. Meanwhile, independent media and academic resources are often restricted or blocked after payment.
As a result, models can be disproportionately trained on content that reflects state propaganda. Thus, a chatbot can inadvertently repeat government narratives as neutral information.
THE CASE OF CHINA
The researchers analyzed large training data for language models and found that a significant portion of Chinese-language content came from media outlets affiliated with the Chinese Communist Party. Although these sources made up a relatively small percentage of the data, they appeared much more frequently than sources like the Chinese Wikipedia. When the models were tested with a variety of state-sponsored content, the results were clear: the more exposure to state propaganda, the more positive responses toward the Chinese government became. In some cases, the responses varied significantly depending on the percentage of state-sponsored content in the training data.
WHAT THE CHATGPT AND CLAUDE TESTS SHOWED
The researchers also tested commercial chatbots by asking them the same political questions in English and Chinese. In about 75% of cases, the responses in Chinese were more favorable to the Chinese government than those in English. A similar trend was observed in other languages used in authoritarian states. In these cases, the chatbots often provided answers that justified or supported the respective governments. In countries with greater press freedom, however, the responses were more critical of the governments.
WHY CHATBOTS CAN BE MORE POWERFUL THAN TRADITIONAL PROPAGANDA
Propaganda is not a new phenomenon. But chatbots bring a whole new dimension: they do not transmit biased information, but conduct personal conversations with the user. This makes them much more persuasive than traditional media. Unlike a newspaper or state-run television, the user does not always have a clear source of information in a chatbot. This can create the illusion of objectivity, even when the responses are influenced by state narratives.
CAN HE PROMOTE FREEDOM OF THOUGHT?
The study does not prove that chatbots make citizens more loyal to authoritarian governments, only that responses vary by language and data. In some cases, Western chatbots may have the opposite effect, providing information more critical of regimes. ChatGPT is banned in China, indicating that authorities see it as potentially dangerous for information control. This suggests that AI can be both a tool for influence and a tool for spreading more open ideas.
WHAT AI COMPANIES SHOULD DO
Experts say companies need to seriously address the problem of systemic bias. One solution could be to limit the use of biased state sources in training data. Models also need to be trained to recognize and avoid repeating propaganda.