Alibaba Submission to the WMT18 Parallel Corpus Filtering Task

作者：Jun Lu, Xiaoyu Lv, Yangbin Shi, Boxing Chen · 年份：2018 · DOI：10.18653/v1/w18-6482 · 被引用次数：15 · 研究领域：Natural Language Processing Techniques、Topic Modeling、Text Readability and Simplification

This paper describes the Alibaba Machine Translation Group submissions to the WMT 2018 Shared Task on Parallel Corpus Filtering. While evaluating the quality of the parallel corpus, the three characteristics of the corpus are investigated, i.e. 1) the bilingual/translation quality, 2) the monolingual quality and 3) the corpus diversity. Both rulebased and model-based methods are adapted to score the parallel sentence pairs. The final parallel corpus filtering system is reliable, easy to build and adapt to other language pairs.