Talk at Rutgers University, New Jersey. Video available!

Back from CVPR in New Orleans, I had the pleasure to visit Yassine Laguel at Rutgers Business School in New Brunswick, NJ. I gave a talk about shift invariance of feature extractors in convolutional neural networks.

Abstract

When trained on natural image datasets, convolutional neural networks (CNNs) tend to learn first-layer parameters that closely resemble oriented Gabor filters. By leveraging the properties of discrete Gabor-like convolutions, we provide a shift invariance measure for the first max pooling hidden layer. To support our theoretical results, we build a mathematical twin implementing the dual-tree complex wavelet packet transform, a particular case of discrete Gabor-like decomposition with perfect reconstruction properties. We discovered that replacing the max pooling layer by a more stable modulus operator leads to increased predictive power, when trained on ImageNet.

Video