FRE: A Fast Method For Anomaly Detection And Segmentation

Introduction

We present a fast and principled approach to address the challenge of visual anomaly detection and segmentation. Our method operates under the assumption of having access solely to anomaly-free training data while aiming to identify anomalies of an arbitrary nature on test data. We build upon prior research and present a generalized approach that utilizes a shallow linear autoencoder to perform out-of-distribution detection on the intermediate features generated by a pre-trained deep neural network. More specifically, we compute the feature reconstruction error (FRE) and establish it as principled measure of uncertainty. We rigorously connect our technique to the theory of linear auto-associative networks to provide a solid theoretical foundation and to offer multiple practical implementation strategies. Furthermore, extending the FRE concept to convolutional layers, we derive FRE maps that provide precise pixel-level spatial localization of the anomalies within images, effectively achieving segmentation. Extensive experimentation demonstrates that our method outperforms the current state of the art. It excels in terms of speed, robustness, and remarkable insensitivity to parameterization. We make our code available on github.

Proposed Approach

At training: we pass the training data through the backbone network for a single forward pass and consider the intermediate (training) features at a given layer k. Subsequently, we train a (tied) shallow linear autoencoder with such features.

During inference: the autoencoder is applied to a test feature to obtain the reduced-dimension embedding, and its reconstruction back into the original (feature) space. The feature reconstruction error (FRE) is then calculated and used as an uncertainty score:

$$\begin{equation} FRE(\mathbf{u}, \mathcal{T}) = \mathbf{e} \triangleq \mathbf{u}-\mathbf{\hat{u}} = \mathbf{u}-(\mathcal{T}^{inv} \circ \mathcal{T})\mathbf{u}. \end{equation}$$

To derive the segmentation map, we perform channel-wise averaging on the FRE (re-arranged as a tensor) in order to accumulate the FRE errors along the channel dimension. This produces a single-channel FRE anomaly map M, where higher intensity regions correspond to anomalies.

The following detection score is highly effective at discriminating between normal and anomalous samples:

$$\begin{equation*} \mathbf{M}_k(i,j) = \frac{1}{C_k}\sum_{c=1}^{C_k} \mathbf{e}(c,i,j) %, i\in {1, 2, \dots, W_k}, i\in {1, 2, \dots, W_k} \end{equation*}$$

Results

Detection results

Segmentation results