Text segmentation with LDA-based Fisher kernel

作者：Qi Sun, Runxin Li, Dingsheng Luo, Xihong Wu · 年份：2008 · DOI：10.3115/1557690.1557768 · 被引用次数：47 · 研究领域：Natural Language Processing Techniques、Topic Modeling、Web Data Mining and Analysis

In this paper we propose a domain-independent text segmentation method, which consists of three components. Latent Dirichlet allocation (LDA) is employed to compute words semantic distribution, and we measure semantic similarity by the Fisher kernel. Finally global best segmentation is achieved by dynamic programming. Experiments on Chinese data sets with the technique show it can be effective. Introducing latent semantic information, our algorithm is robust on irregular-sized segments.