XSum CNN/DailyMail NEWSROOM Multi-News Gigaword arXiv, PubMed BIGPATENT WikiHow Reddit TIFU AESLC BillSum 实验采用和bert一样的两种配置,一个是base版的 一个是large版的。具体参数如下PEGASUS_BASE L = 12, H = 768, F = 3072, A = 12; PEGASUS_LARGE had L = 16, H = 1024, F =4096, ...
We train a pegasus model with sampled gap sentence ratios on both C4 and HugeNews, and stochastically sample important sentences. The updated the results are reported in this table. datasetC4HugeNewsMixed & Stochastic xsum45.20/22.06/36.9947.21/24.56/39.2547.60/24.83/39.64 ...
(3)对于低资源任务数据集,通过微调PEGASUS模型,可以在广泛的领域实现良好的抽象摘要效果。在多个任务上,仅需1000个样本就超过了以前的最先进的结果。 (4)对模型结果进行人工评估,结果表明在XSum, CNN/DailyMail和Reddit TIFU上的摘要效果与人工摘要比肩。 模型 预训练目标GSG 本文假设预训练自监督的目标越接近最终的...
代码链接:https://github.com/google-research/pegasus 概述 文章提出了一种专门针对文本摘要生成的预训练方法,并提出了一种 GSG (Gap Sentence Generation) 的生成方法,该方法在模型的 fine-tune 阶段只需要 1000 个 example 就能达到 12 个数据集的 SOTA,这个结果还是很抢眼的,值得仔细读一读。 首先看一看模型...
We train a pegasus model with sampled gap sentence ratios on both C4 and HugeNews, and stochastically sample important sentences. The updated the results are reported in this table. datasetC4HugeNewsMixed & Stochastic xsum45.20/22.06/36.9947.21/24.56/39.2547.60/24.83/39.64 ...
文中所提及的CNN/DailyMail和XSum两个数据集上得到了比以往模型更好的效果。 而本文提出了GSG这个新颖且更加针对于文本摘要的预训练目标进一步的提升了预训练模型在这项任务上的有效性和优异性... Summarization)是Google Brain和帝国理工提出的一种新的自动文本摘要模型。PEGASUS同样基于Transformer进行模型构建,并针对...
We train a pegasus model with sampled gap sentence ratios on both C4 and HugeNews, and stochastically sample important sentences. The updated the results are reported in this table. datasetC4HugeNewsMixed & Stochastic xsum45.20/22.06/36.9947.21/24.56/39.2547.60/24.83/39.64 ...