Entropy

Research

13 pages, 440 KiB

Open AccessArticle

A Constrained Talagrand Transportation Inequality with Applications to Rate-Distortion-Perception Theory

by Li Xie, Liangyan Li, Jun Chen, Lei Yu and Zhongshan Zhang

Entropy 2025, 27(4), 441; https://doi.org/10.3390/e27040441 - 19 Apr 2025

Viewed by 375

A constrained version of Talagrand’s transportation inequality is established, which reveals an intrinsic connection between the Gaussian distortion-rate-perception functions with limited common randomness under the Kullback–Leibler divergence-based and squared Wasserstein-2 distance-based perception measures. This connection provides an organizational framework for assessing existing bounds [...] Read more.

A constrained version of Talagrand’s transportation inequality is established, which reveals an intrinsic connection between the Gaussian distortion-rate-perception functions with limited common randomness under the Kullback–Leibler divergence-based and squared Wasserstein-2 distance-based perception measures. This connection provides an organizational framework for assessing existing bounds on these functions. In particular, we show that the best-known bounds of Xie et al. are nonredundant when examined through this connection. Full article

(This article belongs to the Special Issue Advances in Information and Coding Theory, the Third Edition)

► Show Figures

Figure 1

24 pages, 612 KiB

Open AccessArticle

Quasi-Optimal Path Convergence-Aided Automorphism Ensemble Decoding of Reed–Muller Codes

by Kairui Tian, He Sun, Yukai Liu and Rongke Liu

Entropy 2025, 27(4), 424; https://doi.org/10.3390/e27040424 - 14 Apr 2025

Viewed by 265

Abstract

By exploiting the rich automorphisms of Reed–Muller (RM) codes, the recently developed automorphism ensemble (AE) successive cancellation (SC) decoder achieves a near-maximum-likelihood (ML) performance for short block lengths. However, the appealing performance of AE-SC decoding arises from the diversity gain that requires a [...] Read more.

By exploiting the rich automorphisms of Reed–Muller (RM) codes, the recently developed automorphism ensemble (AE) successive cancellation (SC) decoder achieves a near-maximum-likelihood (ML) performance for short block lengths. However, the appealing performance of AE-SC decoding arises from the diversity gain that requires a list of SC decoding attempts, which results in a high decoding complexity. To address this issue, this paper proposes a novel quasi-optimal path convergence (QOPC)-aided early termination (ET) technique for AE-SC decoding. This technique detects strong convergence between the partial path metrics (PPMs) of SC constituent decoders to reliably identify the optimal decoding path at runtime. When the QOPC-based ET criterion is satisfied during the AE-SC decoding, only the identified path is allowed to proceed for a complete codeword estimate, while the remaining paths are terminated early. The numerical results demonstrated that for medium-to-high-rate RM codes in the short-length regime, the proposed QOPC-aided ET method incurred negligible performance loss when applied to fully parallel AE-SC decoding. Meanwhile, it achieved a complexity reduction that ranged from 35.9% to 47.4% at a target block error rate (BLER) of

10^{- 3}

, where it consistently outperformed a state-of-the-art path metric threshold (PMT)-aided ET method. Additionally, under a partially parallel framework of AE-SC decoding, the proposed QOPC-aided ET method achieved a greater complexity reduction that ranged from 81.3% to 86.7% at a low BLER that approached

10^{- 5}

while maintaining a near-ML decoding performance. Full article

(This article belongs to the Special Issue Advances in Information and Coding Theory, the Third Edition)

► Show Figures

Figure 1

18 pages, 601 KiB

Open AccessArticle

Low-Density Parity-Check Decoding Algorithm Based on Symmetric Alternating Direction Method of Multipliers

by Ji Zhang, Anmin Chen, Ying Zhang, Baofeng Ji, Huaan Li and Hengzhou Xu

Entropy 2025, 27(4), 404; https://doi.org/10.3390/e27040404 - 9 Apr 2025

Viewed by 240

Abstract

The Alternating Direction Method of Multipliers (ADMM) has proven to be an efficient approach for implementing linear programming (LP) decoding of low-density parity-check (LDPC) codes. By introducing penalty terms into the LP decoding model’s objective function, ADMM-based variable node penalized decoding effectively mitigates [...] Read more.

The Alternating Direction Method of Multipliers (ADMM) has proven to be an efficient approach for implementing linear programming (LP) decoding of low-density parity-check (LDPC) codes. By introducing penalty terms into the LP decoding model’s objective function, ADMM-based variable node penalized decoding effectively mitigates non-integral solutions, thereby improving frame error rate (FER) performance, especially in the low signal-to-noise ratio (SNR) region. In this paper, we leverage the ADMM framework to derive explicit iterative steps for solving the LP decoding problem for LDPC codes with penalty functions. To further enhance decoding efficiency and accuracy, We propose an LDPC code decoding algorithm based on the symmetric ADMM (S-ADMM). We also establish some contraction properties satisfied by the iterative sequence of the algorithm. Through simulation experiments, we evaluate the proposed S-ADMM decoder using three standard LDPC codes and three representative fifth-generation (5G) codes. The results show that the S-ADMM decoder consistently outperforms conventional ADMM penalized decoders, offering significant improvements in decoding performance. Full article

(This article belongs to the Special Issue Advances in Information and Coding Theory, the Third Edition)

► Show Figures

Figure 1

26 pages, 879 KiB

Open AccessArticle

Design and Implementation of Low-Complexity Multiple Symbol Detection Algorithm Using Hybrid Stochastic Computing in Aircraft Wireless Communications

by Yukai Liu, Rongke Liu, Kairui Tian, Zheng Lu and Ling Zhao

Entropy 2025, 27(4), 359; https://doi.org/10.3390/e27040359 - 28 Mar 2025

Viewed by 287

Abstract

The Multiple Symbol Detection (MSD) algorithm can effectively lower the demodulation threshold in Frequency Modulation (FM) technology, which is widely used in aircraft wireless communications due to its insensitivity to large Doppler shifts. However, the high computational complexity of the MSD algorithm leads [...] Read more.

The Multiple Symbol Detection (MSD) algorithm can effectively lower the demodulation threshold in Frequency Modulation (FM) technology, which is widely used in aircraft wireless communications due to its insensitivity to large Doppler shifts. However, the high computational complexity of the MSD algorithm leads to considerable hardware resource overhead. In this paper, we propose a novel MSD architecture based on hybrid stochastic computing (SC), which allows for accurate signal detection while maintaining low hardware complexity. Given that the correlation calculation dominates the computational load in the MSD algorithm, we develop an SC-based, low-complexity unit to perform complex correlation operations using simple hardware circuits, significantly reducing the hardware overhead. Particularly, we integrate a flexible and scalable stochastic adder in the SC-based correlation calculation, which incorporates an adjustable scaling factor to enable high distinguishability in all possible correlation results. Additionally, for the symbol decision process of the MSD algorithm, we design a binary computing-based pipeline architecture to execute the computing process serially, which leverages the inherent low update rate of SC-based correlation results to further reduce the overall resource overhead. Experimental results show that, compared to an 8-bit quantization MSD implementation, our proposed hybrid SC-based MSD architecture achieves a comparable bit error rate while reducing the hardware resources to 69%, 45%, and 36% of those required for the three-, five-, and seven-symbol MSD algorithms, respectively. Full article

(This article belongs to the Special Issue Advances in Information and Coding Theory, the Third Edition)

► Show Figures

Figure 1

7 pages, 216 KiB

Open AccessArticle

A Characterization of Optimal Prefix Codes

by Spencer Congero and Kenneth Zeger

Entropy 2024, 26(12), 1000; https://doi.org/10.3390/e26121000 - 21 Nov 2024

Cited by 1 | Viewed by 708

Abstract

A property of prefix codes called strong monotonicity is introduced, and it is proven that for a given source, a prefix code is optimal if and only if it is complete and strongly monotone. Full article

(This article belongs to the Special Issue Advances in Information and Coding Theory, the Third Edition)

► Show Figures

Figure 1

13 pages, 326 KiB

Open AccessArticle

New Families of Frequency-Hopping Sequence Sets with a Low-Hit-Zone

by Limengnan Zhou and Hanzhou Wu

Entropy 2024, 26(11), 948; https://doi.org/10.3390/e26110948 - 5 Nov 2024

Viewed by 719

Abstract

As a means of spread spectrum communication, frequency-hopping technology has good performance in anti-jamming, multiple-access, security, covert communications, and so on. In order to meet the needs of different frequency-hopping multiple-access (FHMA) communication scenarios, the research on frequency-hopping sequence (FHS) sets with a [...] Read more.

As a means of spread spectrum communication, frequency-hopping technology has good performance in anti-jamming, multiple-access, security, covert communications, and so on. In order to meet the needs of different frequency-hopping multiple-access (FHMA) communication scenarios, the research on frequency-hopping sequence (FHS) sets with a low-hit-zone (LHZ) is now becoming more and more crucial. In this paper, a general construction to obtain new families of LHZ-FHS sets is achieved via interleaving technique. Subsequently, based on two different shift sequences, two classes of LHZ-FHS sets with new flexible parameters not covered in the related literature are presented. The requirements for our new LHZ-FHS sets to obtain optimality or near-optimality with respect to the Peng–Fan–Lee bound are also introduced. Furthermore, as long as the base FHS set is fixed, the performances of new LHZ-FHS sets can be analyzed, such that the parameters of all appropriate shift sequences to obtain desired LHZ-FHS sets are also fixed. Full article

(This article belongs to the Special Issue Advances in Information and Coding Theory, the Third Edition)

► Show Figures

Figure 1

8 pages, 252 KiB

Open AccessArticle

A Construction of Optimal One-Coincidence Frequency-Hopping Sequences via Generalized Cyclotomy

by Minfeng Shao and Ying Miao

Entropy 2024, 26(11), 935; https://doi.org/10.3390/e26110935 - 31 Oct 2024

Viewed by 729

Abstract

Frequency-hopping sequences (FHSs) with low Hamming correlation are essential for synchronization and multiple-access communication systems. In this paper, we propose a novel construction of FHSs using generalized cyclotomy. Our results reveal that the constructed FHSs exhibit a one-coincidence property, meaning that the smallest [...] Read more.

Frequency-hopping sequences (FHSs) with low Hamming correlation are essential for synchronization and multiple-access communication systems. In this paper, we propose a novel construction of FHSs using generalized cyclotomy. Our results reveal that the constructed FHSs exhibit a one-coincidence property, meaning that the smallest correlation between different FHSs, aside from the trivial case, is minimized. Additionally, the new sets of FHSs achieve an optimal size in relation to a known theoretical bound. Full article

(This article belongs to the Special Issue Advances in Information and Coding Theory, the Third Edition)

24 pages, 641 KiB

Open AccessArticle

Optimizing Distributions for Associated Entropic Vectors via Generative Convolutional Neural Networks

by Shuhao Zhang, Nan Liu, Wei Kang and Haim Permuter

Entropy 2024, 26(8), 711; https://doi.org/10.3390/e26080711 - 21 Aug 2024

Cited by 1 | Viewed by 871

Abstract

The complete characterization of the almost-entropic region yields rate regions for network coding problems. However, this characterization is difficult and open. In this paper, we propose a novel algorithm to determine whether an arbitrary vector in the entropy space is entropic or not, [...] Read more.

The complete characterization of the almost-entropic region yields rate regions for network coding problems. However, this characterization is difficult and open. In this paper, we propose a novel algorithm to determine whether an arbitrary vector in the entropy space is entropic or not, by parameterizing and generating probability mass functions by neural networks. Given a target vector, the algorithm minimizes the normalized distance between the target vector and the generated entropic vector by training the neural network. The algorithm reveals the entropic nature of the target vector, and obtains the underlying distribution, accordingly. The proposed algorithm was further implemented with convolutional neural networks, which naturally fit the structure of joint probability mass functions, and accelerate the algorithm with GPUs. Empirical results demonstrate improved normalized distances and convergence performances compared with prior works. We also conducted optimizations of the Ingleton score and Ingleton violation index, where a new lower bound of the Ingleton violation index was obtained. An inner bound of the almost-entropic region with four random variables was constructed with the proposed method, presenting the current best inner bound measured by the volume ratio. The potential of a computer-aided approach to construct achievable schemes for network coding problems using the proposed method is discussed. Full article

(This article belongs to the Special Issue Advances in Information and Coding Theory, the Third Edition)

► Show Figures

Figure 1

Journal Menu

Journal Browser

Advances in Information and Coding Theory, the Third Edition

Share This Special Issue

Special Issue Editor

Special Issue Information

Keywords

Benefits of Publishing in a Special Issue

Related Special Issues

Published Papers (8 papers)

Research

Further Information

Guidelines

MDPI Initiatives

Follow MDPI