Next Article in Journal
Rényi Entropy Power Inequalities via Normal Transport and Rotation
Next Article in Special Issue
Information Theory and Cognition: A Review
Previous Article in Journal
Convex Optimization via Symmetrical Hölder Divergence for a WLAN Indoor Positioning System
Previous Article in Special Issue
Rate-Distortion Function Upper Bounds for Gaussian Vectors and Their Applications in Coding AR Sources
Article Menu
Issue 9 (September) cover image

Export Article

Open AccessArticle

Fixed-Rate Universal Lossy Source Coding and Model Identification: Connection with Zero-Rate Density Estimation and the Skeleton Estimator

1
Information and Decision System Group, Department of Electrical Engineering, Universidad de Chile, Av. Tupper 2007, Santiago 7591538, Chile
2
Department of Electronic Engineering, Universidad Tecnica Federico Santa Maria, Valparaiso 2390123, Chile
*
Author to whom correspondence should be addressed.
Entropy 2018, 20(9), 640; https://doi.org/10.3390/e20090640
Received: 26 June 2018 / Revised: 11 August 2018 / Accepted: 22 August 2018 / Published: 25 August 2018
(This article belongs to the Special Issue Rate-Distortion Theory and Information Theory)
  |  
PDF [459 KB, uploaded 28 August 2018]
  |  

Abstract

This work demonstrates a formal connection between density estimation with a data-rate constraint and the joint objective of fixed-rate universal lossy source coding and model identification introduced by Raginsky in 2008 (IEEE TIT, 2008, 54, 3059–3077). Using an equivalent learning formulation, we derive a necessary and sufficient condition over the class of densities for the achievability of the joint objective. The learning framework used here is the skeleton estimator, a rate-constrained learning scheme that offers achievable results for the joint coding and modeling problem by optimally adapting its learning parameters to the specific conditions of the problem. The results obtained with the skeleton estimator significantly extend the context where universal lossy source coding and model identification can be achieved, allowing for applications that move from the known case of parametric collection of densities with some smoothness and learnability conditions to the rich family of non-parametric L 1 -totally bounded densities. In addition, in the parametric case we are able to remove one of the assumptions that constrain the applicability of the original result obtaining similar performances in terms of the distortion redundancy and per-letter rate overhead. View Full-Text
Keywords: fixed-rate lossy source coding; joint coding and modeling; universal source coding; learning with rate constraints; the skeleton estimator; L1-totally bounded classes fixed-rate lossy source coding; joint coding and modeling; universal source coding; learning with rate constraints; the skeleton estimator; L1-totally bounded classes
Figures

Figure 1

This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited (CC BY 4.0).
SciFeed

Share & Cite This Article

MDPI and ACS Style

Silva, J.F.; Derpich, M.S. Fixed-Rate Universal Lossy Source Coding and Model Identification: Connection with Zero-Rate Density Estimation and the Skeleton Estimator. Entropy 2018, 20, 640.

Show more citation formats Show less citations formats

Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Related Articles

Article Metrics

Article Access Statistics

1

Comments

[Return to top]
Entropy EISSN 1099-4300 Published by MDPI AG, Basel, Switzerland RSS E-Mail Table of Contents Alert
Back to Top