2019 TextSummarizationwithPretrained

From GM-RKB
(Redirected from Liu & Lapata, 2019)
Jump to navigation Jump to search

Subject Headings: Neural Summarization Algorithm.

Notes

Cited By

Quotes

Abstract

Bidirectional Encoder Representations from Transformer (BERT) represent the latest incarnation of pretrained language models, which have recently advanced a wide range of natural language processing tasks. In this paper, we showcase how BERT can be usefully applied in text summarization and propose a general framework for both extractive and abstractive models. We introduce a novel document-level encoder based on BERT, which is able to express the semantics of a document and obtain representations for its sentences. Our extractive model is built on top of this encoder by stacking several inter-sentence Transformer layers. For abstractive summarization, we propose a new fine-tuning schedule which adopts different optimizers for the encoder and the decoder as a means of alleviating the mismatch between the two (the former is pretrained while the latter is not). We also demonstrate that a two-staged fine-tuning approach can further boost the quality of the generated summaries. Experiments on three datasets show that our model achieves state-of-the-art results across the board in both extractive and abstractive settings. Our code is available at this https URL.

References

;

 AuthorvolumeDate ValuetitletypejournaltitleUrldoinoteyear
2019 TextSummarizationwithPretrainedMirella Lapata
Yang Liu
Text Summarization with Pretrained Encoders10.48550/arXiv.1908.083452019