论文阅读:Diverse Beam Search--Decoding Diverse Solutions from Neural Sequence Models,程序员大本营,技术文章内容聚合第一站。
本文主要是针对beam search进行优化,使得生成的句子diversity更好一些,并成功引入强化学习来控制diversity的程度。 Standard Beam Search 假定beam大小为K,则在当前时刻t时刻,针对K个beam中的每一个,扩充K个词,即在后面后缀一个词,这样K个beam便可以生成K * K个新增的beam,可以如下表示 随后按照下式选择得分最高的...
经典的beam search在decoding部分,是基于MAP(最大后验概率)进行贪婪解码的,这种方案生成的responses具有简短、无信息量以及高频的特点,通俗地讲会生成很多的类似“呵呵”的话,没有太多营养和价值。本文提出了一种基于sampling的beam search解码方案,sampling即在每一步解码时根据当前各条路径的概率分布sample出D个条路径...
Code Emilia: A Large-Scale, Extensive, Multilingual, and Diverse Dataset for Speech Generation 1 code implementation•27 Jan 2025 By leveraging Emilia-Pipe, we construct Emilia, the first multilingual speech generation dataset derived from in-the-wild speech data. ...
Decoding,Training,Vocabulary,Entropy,Search problems,Mutual information,Space explorationDiversity of generated responses is important for a data-driven neural conversational model (NCM) for non-task-oriented conversation. A criterion of maximum mutual information (MMI) and generating N-best outputs are ...
However, the frequently used random methods such as sampling or noised beam search, although can output diverse back-translations, often generate noisy synthetic sentences. To alleviate this problem, we propose a simple but effective constraint random decoding method for back-translation. The proposed ...
neural abstractive summarization; sequence-to-sequence neural model; beam search; diverse decoding; optimal sequence selection1. Introduction Document summarization is the task of generating a condensed and coherent summary of a document while retaining the salient information. There are two broad types ...