Analysis on the Annihilating Property of Filters in the Denoising Auto-Encoder

Abstract

In this thesis, We make some analysis on the relationship between the loss function and the annihilating filters. Based on this analysis we propose a modified Auto-Encoder for noise removal of normal and adversarial noises. The Auto-Encoder refers to a special neural network architecture which is trained with input and output image pairs, where the output image is normally the same as the input image. When the Auto-Encoder is trained with noisy images as the input and original images as the output, this becomes the so-called denoising Auto-Encoder. The filters that are trained with these input/output pairs learn how to annihilate the noise in noisy images. We show that a single filter in the denoising Auto-Encoder annihilates a rank-reduced approximation of the original noise. We also show that with a small change in the normal Auto-Encoder, we can achieve the same effect as a kind of change in the loss function with the original denoising Auto-Encoder. We explain the reason why the original Auto-Encoder with small number of filters has a tendency to result in insufficiently denoised images, while a modified Auto-Encoder with skip connection has a tendency of losing some information of the true image structure. Experimental results verify the fact that the analysis is correct. Furthermore, we apply the modified Auto-Encoder for the application of the removal of adversarial noise, a noise that tries to deceive the neural network. ;이 논문에서는 DNN의 한 종류인 Auto-Encoder를 이용하여 denoising을 하는 모델과 annihilating filter의 상관관계를 분석한다. 이러한 분석을 기반으로 random noise와 adversarial noise의 두 방식의 noise를 제거할 Auto-Encoder 모델을 제안하였다. 기존의 denoising 모델에서의 Auto-Encoder의 filter가 noise를 annihilating하는 역할을 수행하는 과정을 분석하고, 그것이 어려운 이유를 설명하였다. 새로운 모델은 기존의 모델과는 다르게 filter가 true image를 annihilating하는 역할을 하도록 수정되었다. Fourier domain상에서 image의 sparse함과 image의 Hankel matrix가 low rank structure가 되는 것의 상관 관계를 보이고, Auto-Encoder 내의 convolution을 Hankel matrix의 곱으로 나타냄으로 image를 annihilating 하는 것이 noise를 annihilating하는 것보다 간단함을 보인다. 또한 여러가지 이미지를 이용하여 그 결과를 확인해 본다.