Adam and the Ants: On the Influence of the Optimization Algorithm on the Detectability of DNN Watermarks
Atlanttic Research Center, University of Vigo, 36310 Vigo, Spain
*
Author to whom correspondence should be addressed.
Entropy 2020, 22(12), 1379; https://doi.org/10.3390/e22121379
Received: 8 October 2020 / Revised: 30 November 2020 / Accepted: 4 December 2020 / Published: 6 December 2020
(This article belongs to the Special Issue Information-Theoretic Methods for Deep Learning Based Data Acquisition, Analysis and Security)
As training Deep Neural Networks (DNNs) becomes more expensive, the interest in protecting the ownership of the models with watermarking techniques increases. Uchida et al. proposed a digital watermarking algorithm that embeds the secret message into the model coefficients. However, despite its appeal, in this paper, we show that its efficacy can be compromised by the optimization algorithm being used. In particular, we found through a theoretical analysis that, as opposed to Stochastic Gradient Descent (SGD), the update direction given by Adam optimization strongly depends on the sign of a combination of columns of the projection matrix used for watermarking. Consequently, as observed in the empirical results, this makes the coefficients move in unison giving rise to heavily spiked weight distributions that can be easily detected by adversaries. As a way to solve this problem, we propose a new method called Block-Orthonormal Projections (BOP) that allows one to combine watermarking with Adam optimization with a minor impact on the detectability of the watermark and an increased robustness.
View Full-Text
Keywords:
watermarking; deep neural networks; optimization algorithms; Adam; stochastic gradient descent; detectability
▼
Show Figures
Figure 1
This is an open access article distributed under the Creative Commons Attribution License which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited
MDPI and ACS Style
Cortiñas-Lorenzo, B.; Pérez-González, F. Adam and the Ants: On the Influence of the Optimization Algorithm on the Detectability of DNN Watermarks. Entropy 2020, 22, 1379. https://doi.org/10.3390/e22121379
AMA Style
Cortiñas-Lorenzo B, Pérez-González F. Adam and the Ants: On the Influence of the Optimization Algorithm on the Detectability of DNN Watermarks. Entropy. 2020; 22(12):1379. https://doi.org/10.3390/e22121379
Chicago/Turabian StyleCortiñas-Lorenzo, Betty, and Fernando Pérez-González. 2020. "Adam and the Ants: On the Influence of the Optimization Algorithm on the Detectability of DNN Watermarks" Entropy 22, no. 12: 1379. https://doi.org/10.3390/e22121379
Find Other Styles
Note that from the first issue of 2016, MDPI journals use article numbers instead of page numbers. See further details here.
