Next Article in Journal
Distinguishability in Entropy Calculations: Chemical Reactions, Conformational and Residual Entropy
Next Article in Special Issue
Tsallis Mutual Information for Document Classification
Previous Article in Journal
Thermodynamics of Thermoelectric Phenomena and Applications
Previous Article in Special Issue
Tsallis-Based Nonextensive Analysis of the Southern California Seismicity

A Risk Profile for Information Fusion Algorithms

Raytheon Integrated Defense Systems, 235 Presidential Way, Woburn, MA 01801, USA
Raytheon Integrated Defense Systems, 2461 S. Clark St., Suite 1000, Arlington, VA 22202, USA
Author to whom correspondence should be addressed.
Entropy 2011, 13(8), 1518-1532;
Received: 17 May 2011 / Revised: 4 August 2011 / Accepted: 11 August 2011 / Published: 17 August 2011
(This article belongs to the Special Issue Tsallis Entropy)
E.T. Jaynes, originator of the maximum entropy interpretation of statistical mechanics, emphasized that there is an inevitable trade-off between the conflicting requirements of robustness and accuracy for any inferencing algorithm. This is because robustness requires discarding of information in order to reduce the sensitivity to outliers. The principal of nonlinear statistical coupling, which is an interpretation of the Tsallis entropy generalization, can be used to quantify this trade-off. The coupled-surprisal, -lnκ(p)≡-(pκ-1)/κ , is a generalization of Shannon surprisal or the logarithmic scoring rule, given a forecast p of a true event by an inferencing algorithm. The coupling parameter κ=1-q, where q is the Tsallis entropy index, is the degree of nonlinear coupling between statistical states. Positive (negative) values of nonlinear coupling decrease (increase) the surprisal information metric and thereby biases the risk in favor of decisive (robust) algorithms relative to the Shannon surprisal (κ=0). We show that translating the average coupled-surprisal to an effective probability is equivalent to using the generalized mean of the true event probabilities as a scoring rule. The metric is used to assess the robustness, accuracy, and decisiveness of a fusion algorithm. We use a two-parameter fusion algorithm to combine input probabilities from N sources. The generalized mean parameter ‘alpha’ varies the degree of smoothing and raising to a power Νβ with β between 0 and 1 provides a model of correlation. View Full-Text
Keywords: Tsallis entropy; proper scoring rules; information fusion; machine learning Tsallis entropy; proper scoring rules; information fusion; machine learning
Show Figures

Figure 1

MDPI and ACS Style

Nelson, K.P.; Scannell, B.J.; Landau, H. A Risk Profile for Information Fusion Algorithms. Entropy 2011, 13, 1518-1532.

AMA Style

Nelson KP, Scannell BJ, Landau H. A Risk Profile for Information Fusion Algorithms. Entropy. 2011; 13(8):1518-1532.

Chicago/Turabian Style

Nelson, Kenric P., Brian J. Scannell, and Herbert Landau. 2011. "A Risk Profile for Information Fusion Algorithms" Entropy 13, no. 8: 1518-1532.

Find Other Styles

Article Access Map by Country/Region

Only visits after 24 November 2015 are recorded.
Back to TopTop