Algorithms 2012, 5(4), 398-420; doi:10.3390/a5040398
Article

Better Metrics to Automatically Predict the Quality of a Text Summary

Peter A. Rankel 1 email, John M. Conroy 2 email and Judith D. Schlesinger 2 email
1 Statistics Program, Department of Mathematics, University of Maryland, College Park, MD 20742, USA 2 Center for Computing Sciences, Institute for Defense Analyses, 17100 Science Drive, Bowie, MD 20715, USA
* Author to whom correspondence should be addressed.
Received: 2 July 2012; in revised form: 5 September 2012 / Accepted: 7 September 2012 / Published: 26 September 2012
(This article belongs to the Special Issue Selected Papers from Text Mining Workshop at the 2012 SIAM International Conference on Data Mining)
PDF Full-text Download PDF Full-Text [401 KB, uploaded 26 September 2012 13:33 CEST]
Abstract: In this paper we demonstrate a family of metrics for estimating the quality of a text summary relative to one or more human-generated summaries. The improved metrics are based on features automatically computed from the summaries to measure content and linguistic quality. The features are combined using one of three methods—robust regression, non-negative least squares, or canonical correlation, an eigenvalue method. The new metrics significantly outperform the previous standard for automatic text summarization evaluation, ROUGE.
Keywords: multi-document summarization; update summarization; evaluation; computational linguistics; text processing

Article Statistics

Click here to load and display the download statistics.

Cite This Article

MDPI and ACS Style

Rankel, P.A.; Conroy, J.M.; Schlesinger, J.D. Better Metrics to Automatically Predict the Quality of a Text Summary. Algorithms 2012, 5, 398-420.

AMA Style

Rankel PA, Conroy JM, Schlesinger JD. Better Metrics to Automatically Predict the Quality of a Text Summary. Algorithms. 2012; 5(4):398-420.

Chicago/Turabian Style

Rankel, Peter A.; Conroy, John M.; Schlesinger, Judith D. 2012. "Better Metrics to Automatically Predict the Quality of a Text Summary." Algorithms 5, no. 4: 398-420.

Algorithms EISSN 1999-4893 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert