On the geometry of rectifier convolutional neural networks

• While recent studies have shed light on the expressivity, complexity and compositionality of convolutional networks, the real inductive bias of the family of functions reachable by gradient descent on natural data is still unknown. By exploiting symmetries in the preactivation space of convolutional layers, we present preliminary empirical evidence of regularities in the preimage of trained rectifier networks, in terms of arrangements of polytopes, and relate it to the nonlinear transformations applied by the network to its input.

Ämnesord

TEKNIK OCH TEKNOLOGIER  -- Elektroteknik och elektronik -- Robotteknik och automation (hsv//swe)

ENGINEERING AND TECHNOLOGY  -- Electrical Engineering, Electronic Engineering, Information Engineering -- Robotics (hsv//eng)

Nyckelord

Convolutional networks

Deep learning

Heometry

Preimage

Understanding

Computer vision

Convolution

Gradient methods

Rectifying circuits

Compositionality

Gradient descent

Inductive bias

Non-linear transformations

Pre images

Convolutional neural networks

Publikations- och innehållstyp

ref (ämneskategori)

kon (ämneskategori)