Detecting and Mitigating The Spread of Misinformation By AI-Generated Content

Abstract : Misinformation has been a persistent and detrimental phenomenon in our society in many ways, including individuals' physical health and economic security. With the advent of short video platforms and associated applications, dissemination of multi-modal misinformation, including images, texts, audios, and videos, have increased these issues. The advent of generative AI models such as ChatGPT and Stable Diffusion has further enhanced the complexity, providing give rise to Artificial Intelligence Generated Content (AIGC) and posing new challenges in the detection and mitigation of misinformation. Therefore, the conventional methods of misinformation detection and intervention have proved to be insufficient in this new paradigm. This paper examines the challenge posed by AIGC in the context of misinformation. It analyzes the problem from a psychological and societal point of view, and investigates the subtle manipulation traces in AIGC at signal, perceptual, semantic, and human levels. Through examining manipulation traces like signal manipulation, semantic inconsistency, logical incoherence, and psychological strategies, our goal is to address AI-generated misinformation and present a conceptual design of systematic explainable solution. Finally, we hope this paper will provide useful insights into the fight against misinformation, especially in the age of AIGC.

Keyword AI- Generated Content (AIGC), Multimodal, Generative AI Models Misinformation Detection.