Capsule Networks (CapsNets) – Tutorial

Capsule Networks (CapsNets) – Tutorial

Hey, I’m overly Asian, and in this video, I’ll tell you all about capsule networks, a hot new architecture for neural nets. Jeffrey Hinton conceived the idea of capsule networks several years ago and published a paper in 2011 that introduced many key concepts. However, he struggled to make them work properly until now. A few weeks ago, in October 2017, a paper titled “Dynamic Routing Between Capsules” was published by Sara Sabour, Nicholas Frost, and of course, Geoffrey Hinton. They reached state-of-the-art performance on the MNIST dataset and demonstrated considerably better results than convolutional neural nets on highly overlapping digits.

What are Capsule Networks?

In computer graphics, you start with an abstract representation of a scene, such as a rectangle at position X=20 and Y=30, rotated by 16 degrees. Each object type has various instantiation parameters, and you call some rendering function to get an image. Inverse graphics is the reverse process, starting with an image and trying to identify the objects it contains and their instantiation parameters. A capsule network is essentially a neural network that aims to perform inverse graphics. It consists of many capsules, where a capsule is any function that predicts the presence and instantiation parameters of a particular object at a given location.

For example, consider a network with 50 capsules. The arrows represent the output vectors of these capsules, with black arrows corresponding to capsules looking for rectangles and blue arrows representing capsules looking for triangles. The length of an activation vector signifies the estimated probability of the object being present.

Each capsule retains detailed information about the object's location and pose, a property called equivariance. This is in contrast to pooling layers in convolutional neural nets, which tend to lose such precise information.

Handling Object Hierarchies

Capsule networks can handle objects composed of a hierarchy of parts. For instance, consider a boat composed of a rectangle and a triangle. The first layer of capsules identifies these parts, and the next step involves predicting the output of higher-level capsules, such as a house capsule or boat capsule.

The rectangle capsule might predict a boat capsule output by calculating the dot product of a transformation matrix with its activation vector. These transformation matrices learn the part-whole relationships during training.

Routing by Agreement

Routing by agreement is a key feature of capsule networks. Low-level capsules predict the output of higher-level capsules, and if there is agreement among predictions, routing weights are updated accordingly. This iterative process ensures that capsules only send their output to appropriate higher-level capsules, thereby reducing noise and improving signal accuracy.

Implementing Capsule Networks

To create a capsule network, we start with convolutional layers to extract feature maps, which are reshaped into vectors and squashed to ensure their lengths represent probabilities. The primary capsules’ output is then routed to higher-level capsules through iterations of agreement measure updates until convergence.

Practical Applications

Capsule networks show promise in various applications, including image classification, segmentation, and object detection. They preserve pose information and are robust to transformations, providing cleaner inputs to higher-level capsules.

Research and Future Directions

While capsule networks have reached state-of-the-art accuracy on MNIST, they are yet to scale to larger datasets like ImageNet. Training is also slower due to the routing by agreement algorithm. However, their ability to handle crowded scenes, maintain detailed part-whole relationships, and interpret activation vectors show immense promise.

Learning Resources

For further understanding, you can look at implementation code available in frameworks like Keras, TensorFlow, and PyTorch. Implementations with loops, such as the routing by agreement algorithm, are easier to understand in PyTorch but can be adapted in other frameworks.

Keywords

• Capsule Networks
• Jeffrey Hinton
• Inverse graphics
• Equivariance
• Routing by agreement
• Part-whole relationships
• Image classification
• MNIST
• Transformation matrix
• Primary capsules

FAQ

What are capsule networks? Capsule networks are a type of neural network designed to perform inverse graphics, identifying objects and their instantiation parameters from images using capsules.

How do capsule networks improve over traditional convolutional neural networks? Capsule networks preserve detailed information about an object's location and pose through the network and have the ability to handle overlapping objects more effectively due to equivariance and routing by agreement.

What is routing by agreement? Routing by agreement is an algorithm where low-level capsules predict the output of higher-level capsules, and through iterative updates, ensure capsules only send outputs to the most relevant capsules based on agreement measures.

Can capsule networks handle large datasets? As of now, capsule networks have exhibited promising results on smaller datasets like MNIST but are yet to be tested extensively on larger datasets like ImageNet.

Are capsule networks slow to train? Yes, capsule networks are relatively slower to train largely due to the complexity of the routing by agreement algorithm.

Where can I find more information? For more in-depth understanding, refer to the code implementations in frameworks like Keras, TensorFlow, and PyTorch. Reading Geoffrey Hinton’s original papers and other related literature will also provide more insights.