Open AccessThis article is
- freely available
Better Metrics to Automatically Predict the Quality of a Text Summary
Statistics Program, Department of Mathematics, University of Maryland, College Park, MD 20742, USA
Center for Computing Sciences, Institute for Defense Analyses, 17100 Science Drive, Bowie, MD 20715, USA
* Author to whom correspondence should be addressed.
Received: 2 July 2012; in revised form: 5 September 2012 / Accepted: 7 September 2012 / Published: 26 September 2012
Abstract: In this paper we demonstrate a family of metrics for estimating the quality of a text summary relative to one or more human-generated summaries. The improved metrics are based on features automatically computed from the summaries to measure content and linguistic quality. The features are combined using one of three methods—robust regression, non-negative least squares, or canonical correlation, an eigenvalue method. The new metrics significantly outperform the previous standard for automatic text summarization evaluation, ROUGE.
Keywords: multi-document summarization; update summarization; evaluation; computational linguistics; text processing
Citations to this Article
Cite This Article
MDPI and ACS Style
Rankel, P.A.; Conroy, J.M.; Schlesinger, J.D. Better Metrics to Automatically Predict the Quality of a Text Summary. Algorithms 2012, 5, 398-420.
Rankel PA, Conroy JM, Schlesinger JD. Better Metrics to Automatically Predict the Quality of a Text Summary. Algorithms. 2012; 5(4):398-420.
Rankel, Peter A.; Conroy, John M.; Schlesinger, Judith D. 2012. "Better Metrics to Automatically Predict the Quality of a Text Summary." Algorithms 5, no. 4: 398-420.