CampNet: Context-Aware Mask Prediction for End-to-End Text-Based Speech Editing

作者：Tao Wang, Jiangyan Yi, Ruibo Fu, Jianhua Tao, Zhengqi Wen · 发表于：IEEE/ACM Transactions on Audio Speech and Language Processing · 年份：2022 · DOI：10.1109/taslp.2022.3190717 · 被引用次数：14 · 研究领域：Speech Recognition and Synthesis、Music and Audio Processing、Natural Language Processing Techniques

The text-based speech editor allows the editing of speech through intuitive cutting, copying, and pasting operations to speed up the process of editing speech. However, the major drawback of current systems is that edited speech often sounds unnatural due to cut-copy-paste operation. In addition, it is not obvious how to synthesize records according to a new word not appearing in the transcript, which often needs the help of text-to-speech (TTS) and voice conversion (VC) technology at the same time. This paper first proposes a novel end-to-end text-based speech editing method called context-aware mask prediction network (CampNet). The model can simulate the text-based speech editing process by randomly masking part of speech and then predicting the masked region by sensing the speech context. It can solve unnatural prosody in the edited region and synthesize the speech corresponding to the unseen words in the transcript. Secondly, for the possible operation of text-based speech editing, we design three text-based operations based on CampNet: deletion, insertion, and replacement. These operations can cover various situations of speech editing. Thirdly, to synthesize the speech corresponding to long text in insertion and replacement operations, a word-level autoregressive generation method is proposed, which can synthesize the speech of arbitrary length text. Fourthly, we propose a speaker adaptation method using only one sentence for CampNet and explore the ability of few-shot...