CROSSWORD: A Semantic Approach To Text Compression Via Masking

作者：Mingxiao Li, Rui Jin, Liyao Xiang, Kaiming Shen, Shuguang Cui · 年份：2024 · DOI：10.1109/icassp48485.2024.10447857 · 被引用次数：5 · 研究领域：Natural Language Processing Techniques、Digital Humanities and Scholarship

Conventional data compression methods typically model the information source as an i.i.d. stochastic process, thereby establishing the fundamental limit as entropy for lossless compression and as mutual information for lossy compression. However, the source in the real world (e.g., text, music, and speech) is often statistically ill-defined because of its close connection to human perception. This work aims to exploit the semantic aspect of text as inspired by the puzzle crossword. The main idea is to only compress those semantically important words while masking the rest; the proposed decompressor can recover all the missing words automatically according to context. Experiments show that the proposed semantic approach can achieve much higher compression efficiency than the state-of-the-art semantic compression method.