CBTF-FTR: A Chinese Bad Text Detection Method Based on Variant Text Restoration

In response to the practice of evading surveillance through textual transformation, we proposed a method for detecting bad texts based on the restoration of variant texts. Firstly, tailored to different types of variants, two restoration methods were proposed, one focuses on glyph variants, leveraging a Chinese character knowledge base for restoration; the other addresses phonetic variants, utilizing a method that interconverts pinyin (Chinese phonetic notation) and Chinese characters. Thirdly, on the basis of restoring the variant texts, a harmful text detection method is implemented that relies on sensitive word comparison. Finally, data containing harmful texts were crawled from social platforms and annotated. These data were then used to compare the proposed recognition scheme with existing text censorship APIs. Experimental results demonstrate that the accuracy of detecting harmful texts after restoration is higher than that without restoration.