In this thesis, we study the robustness and generalization properties of Deep Neural Networks (DNNs) under various noisy regimes, due to corrupted inputs or labels. Such corruptions can be either random or intentionally crafted to disturb the target DNN. Inputs corrupted by maliciously designed perturbations are known as adversarial examples and have been shown to severely degrade the performance of DNNs. However, due to the non-linearity of DNNs, crafting such perturbations is non-trivial.

We first address the problem of designing algorithms for generating adversarial examples, known as adversarial attacks. We start with a general formulation of this problem and, through successive convex relaxations, propose a framework for computing adversarial examples under various desired constraints. Using this approach, we derive novel methods that consistently outperform existing algorithms in tested scenarios. In addition, new algorithms are also formulated for regression problems. We show that adversarial vulnerability is also an issue in various regression tasks, a problem that has so far been overlooked in the literature.

While there has been a vast amount of work on the design and understanding of DNNs resistant to these attacks, their generalization properties are less understood. How well does adversarial robustness generalize from the training set to unseen data? We use Statistical Learning Theory (SLT) to bound the so-called adversarial risk of DNNs. Proving SLT bounds for deep learning is on-going research with various existing frameworks. Among these SLT frameworks, we choose a compression-based technique that established state of the art results for DNNs in the non-adversarial regime. Our bound leverages the sparsity structures induced by adversarial training and has no explicit dependence on the input dimension or the number of classes. This result constitutes an improvement over existing bounds.

To complete this work, we shift our focus from perturbed inputs to noisy labels and analyze how DNNs learn when a portion of the inputs is incorrectly labeled. In this setup, we use information theory to characterize the behavior of classifiers. Under noisy labels, we study the trajectory of DNNs in the information plane, formed by the entropy of estimated labels and the conditional entropy between given and estimated labels. We analyze the trajectory in the information plane and show the de-noising capabilities of DNNs. Under simplified scenarios, we are able to analytically characterize these trajectories for one-layer neural networks trained with stochastic gradient descent. This result shows a trajectory for properly trained networks that seems to be consistent among DNNs in real image classification tasks. In addition, we show that underfitted, overfitted and well-trained DNNs exhibit significantly different trajectories in the information plane. Such phenomena are not visible when considering only training and validation error. These results show that information-theoretic quantities provide a richer view of the learning process than standard training and validation error.

Details
Autor Balda, Emilio
Lieferzeit 3-4 Tage
Gewicht 0.206 kg
Erscheinungsdatum 14.01.2020
Eigene Bewertung schreiben
Sie bewerten:Robustness Analysis of Deep Neural Networks in the Presence of Adversarial Perturbations and Noisy Labels

Elektro- und Informationstechnik

Balda, Emilio

Robustness Analysis of Deep Neural Networks in the Presence of Adversarial Perturbations and Noisy Labels

ISBN: 978-3-86359-802-0
Lieferzeit: 2-3 Tage
39,00 €
inkl. 7% MwSt.

Kurzbeschreibung

In this thesis, we study the robustness and generalization properties of Deep Neural Networks (DNNs) under various noisy regimes, due to corrupted inputs or labels. Such corruptions can be either random or intentionally crafted to disturb the target DNN. Inputs corrupted by maliciously designed perturbations are known as adversarial examples and have been shown to severely degrade the performance of DNNs. However, due to the non-linearity of DNNs, crafting such perturbations is non-trivial. [...]

Auf Lager