Network structure diagram
Figure 2 Convolutional neural network structure
The difference between the convolutional neural network and the fully connected neural network structure is relatively large. In a fully connected network, the nodes of two adjacent layers are connected by edges, while the convolutional neural network has only a few nodes connected to adjacent nodes.
The biggest problem with fully connected neural networks for processing images is that there are too many parameters in the fully connected layer. If there are too many parameters, it is easy to over-fitting and the calculation speed will be slowed down. The convolutional neural network can reduce the number of parameters.
Suppose the input is a picture size of 28*28*3, and the first layer has 500 nodes in the hidden layer. Then the parameters of the first layer have 28*28*3*500+500=1176000 parameters. When the parameters are more, and this is just the first layer.
So why does the convolutional neural network achieve the goal of reducing parameters?
The most critical convolutional neural networks are convolutional layers, pooled layers, and fully connected layers.
Convolution layerThe input of each node in the convolutional layer is only a small piece of the upper layer of the neural network, usually implemented by a convolution kernel, which is a filter that can be imagined as a scan window that is attached to each picture. Then, the image is scanned according to the set size step size, etc., and the calculation rule is that the image pixel matrix of the buckled image is multiplied by the weight corresponding position of the convolution kernel and then summed, and one output is obtained every scan. The work done by the convolutional layer can be understood as a feature abstraction of each small block of image pixels. The same picture can be convolved by a number of different convolution kernels. The number of convolution kernels is actually the depth of the output matrix after convolution. The number of parameters of the convolutional neural network is independent of the size of the picture, and is only related to the size, depth, and number of convolution kernels (depth of the output matrix) of the filter. Assume that it is still a picture of 28*28*3, the size of the convolution kernel is set to 3*3*3, and the depth of the output matrix is ​​500, then the number of parameters is 3*3*3*500+500=14000 parameters. Compared to the fully connected layer, the parameters are much reduced.
Figure 3 Example of convolutional layer of image
Pooling layerThe pooling layer can be thought of as converting a high-resolution image into a low-resolution image. It can very effectively reduce the size of the matrix, thus reducing the number of parameters of the fully connected layer, which can speed up the calculation rate while preventing over-fitting, pooling, reducing the model, increasing the speed, and improving the extracted features. Great.
The filter with 2*2 has a step size of 2 and the maximum pool is as shown below:
Figure 4 2*2 filter maximum pooling example diagram
We can think of the convolutional layer and the pooled layer as automatic feature extraction.
Through the above intuitive introduction, now we know why the convolutional neural network can achieve the purpose of reducing the parameters?
Compared to fully connected neural networks, convolutional neural networks have the advantage of shared weights and sparse connections. The sharing weight is that the parameters are only relevant to the filter. Another reason for convolutional neural networks to reduce parameters is sparse connections. The output node is related to the partial pixel matrix of the input picture matrix, that is, the small matrix of the convolution kernel. This is the concept of sparse connections.
Convolutional neural networks reduce parameters by weight sharing and sparse connections. Thereby preventing overfitting.
Training processThe training process of convolutional neural networks can be roughly divided into the following steps:
The first step: import related libraries, load parameters
The second step: normalization, is conducive to speed up the gradient
The third step: defining parameters and convolutional neural network structure
The fourth step: the forward propagation process
Step 5: Cost function
Step 6: Gradient Descent Update Parameters
Step 7: Training the model
Shenzhen Scodeno Technology Co.,Ltd , https://www.scodenonet.com