Next Article in Journal
Exploring the Relationship among Predictability, Prediction Accuracy and Data Frequency of Financial Time Series
Previous Article in Journal
Working Memory Decline in Alzheimer’s Disease Is Detected by Complexity Analysis of Multimodal EEG-fNIRS
Previous Article in Special Issue
Bottleneck Problems: An Information and Estimation-Theoretic View
Article

Adam and the Ants: On the Influence of the Optimization Algorithm on the Detectability of DNN Watermarks

Atlanttic Research Center, University of Vigo, 36310 Vigo, Spain
*
Author to whom correspondence should be addressed.
Entropy 2020, 22(12), 1379; https://doi.org/10.3390/e22121379
Received: 8 October 2020 / Revised: 30 November 2020 / Accepted: 4 December 2020 / Published: 6 December 2020
As training Deep Neural Networks (DNNs) becomes more expensive, the interest in protecting the ownership of the models with watermarking techniques increases. Uchida et al. proposed a digital watermarking algorithm that embeds the secret message into the model coefficients. However, despite its appeal, in this paper, we show that its efficacy can be compromised by the optimization algorithm being used. In particular, we found through a theoretical analysis that, as opposed to Stochastic Gradient Descent (SGD), the update direction given by Adam optimization strongly depends on the sign of a combination of columns of the projection matrix used for watermarking. Consequently, as observed in the empirical results, this makes the coefficients move in unison giving rise to heavily spiked weight distributions that can be easily detected by adversaries. As a way to solve this problem, we propose a new method called Block-Orthonormal Projections (BOP) that allows one to combine watermarking with Adam optimization with a minor impact on the detectability of the watermark and an increased robustness. View Full-Text
Keywords: watermarking; deep neural networks; optimization algorithms; Adam; stochastic gradient descent; detectability watermarking; deep neural networks; optimization algorithms; Adam; stochastic gradient descent; detectability
Show Figures

Figure 1

MDPI and ACS Style

Cortiñas-Lorenzo, B.; Pérez-González, F. Adam and the Ants: On the Influence of the Optimization Algorithm on the Detectability of DNN Watermarks. Entropy 2020, 22, 1379. https://doi.org/10.3390/e22121379

AMA Style

Cortiñas-Lorenzo B, Pérez-González F. Adam and the Ants: On the Influence of the Optimization Algorithm on the Detectability of DNN Watermarks. Entropy. 2020; 22(12):1379. https://doi.org/10.3390/e22121379

Chicago/Turabian Style

Cortiñas-Lorenzo, Betty, and Fernando Pérez-González. 2020. "Adam and the Ants: On the Influence of the Optimization Algorithm on the Detectability of DNN Watermarks" Entropy 22, no. 12: 1379. https://doi.org/10.3390/e22121379

Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.

Article Access Map by Country/Region

1
Back to TopTop