RNN basics explains why RNN is needed

The neural network can be regarded as a black box that can fit arbitrary functions. As long as the training data is sufficient, given a specific x, the desired y can be obtained. The structure diagram is as follows:

After training the neural network model, given an x ​​in the input layer, you can get a specific y in the output layer after passing through the network. So, with such a powerful model, why do you need RNN (Circular Neural Network)?

They can only handle the input one by one, and the previous input and the last input are completely unrelated. However, some tasks need to be able to better process the sequence information, that is, the previous input and the subsequent input are related.

For example, when we understand the meaning of a sentence, it is not enough to understand each word in isolation. We need to deal with the whole sequence of these words. When we process the video, we can't just go alone. Analyze each frame and analyze the entire sequence of connections of these frames.

In the simplest part of speech tagging task of nlp, I will eat three words of apple to mark the part of the word for me / nn eat / v apple / nn.

Then the input to this task is:

I eat apples (sent sentences that have been segmented)

The output of this task is:

I /nn eat /v apple /nn (words marked with good words)

For this task, of course, we can directly use the ordinary neural network to do the training data format for the network is me - "I / nn such a single word -" word-of-word marked words.

But obviously, in a sentence, the previous word actually has a great influence on the part-of-speech prediction of the current word. For example, when predicting apple, because the previous eating is a verb, it is obvious that the probability of apple as a noun will be great. The probability of a verb is very common because the verb is followed by a verb, and the verb following the verb is rare.

So in order to solve some of these similar problems and better handle the sequence information, RNN was born.

First look at a simple cyclic neural network, which consists of an input layer, a hidden layer, and an output layer:

I don't know if the beginners can understand this picture. Anyway, when I first started learning, it is very aggressive. Each node represents a value input, or a layer of vector node collection. How to hide the layer? Can connect to yourself, and so on. These figures are a more abstract picture.

We now understand that if the circle with the arrow above W is removed, it becomes the most common fully connected neural network.

x is a vector that represents the value of the input layer (the circle that is not drawn to represent the neuron node); s is a vector that represents the value of the hidden layer (here the hidden layer draws a node, you can also imagine this A layer is actually a plurality of nodes, and the number of nodes is the same as the dimension of the vector s);

U is the weight matrix of the input layer to the hidden layer, o is also a vector, which represents the value of the output layer; V is the weight matrix of the hidden layer to the output layer.

So now let's see what W is. The value s of the hidden layer of the cyclic neural network depends not only on the current input x but also on the value s of the last hidden layer. The weight matrix W is the weight of the last value of the hidden layer as the input of this time.

We give the concrete map corresponding to this abstract map:

We can clearly see from the above picture how the hidden layer of the last moment affects the hidden layer of the current moment.

If we expand the above diagram, the circular neural network can also be drawn like this:

It now seems clear that after the network receives the input at time t, the value of the hidden layer is, the output value is. The key point is that the value depends not only on, but also on. We can use the following formula to represent the calculation method of the cyclic neural network:

Formulated as follows:

Well, here I will explain some of the most basic knowledge points of RNN, which can help you to feel the RNN intuitively and understand why you need RNN, and then summarize its reverse guidance knowledge points.

Finally, give an overview of the RNN:

7 Inches Tablet PC

The 7-inch tablet can be used as the golden size of a tablet computer. It is small and portable. It can be used at home and outdoors. You can browse the web, watch videos and play games. It is a household artifact. Although the size of the 7-inch tablet is inclined to the tablet, the function is more inclined to the mobile phone, so it can also be used as a substitute for the mobile phone. Compared with other sized tablets, the 7-inch tablet has obvious advantages in appearance and weight. Both the body size and the body weight have reached a very reasonable amount.

1.In appearance, the 7 inch tablet computer looks like a large-screen mobile phone, or more like a separate LCD screen.

2.In terms of hardware configuration, the 7 inch tablet computer has all the hardware devices of a traditional computer, and has its own unique operating system, compatible with a variety of applications, and has a complete set of computer functions.

3.The 7 inch tablet computer is a miniaturized computer. Compared with traditional desktop computers, tablet computers are mobile and flexible. Compared with Laptops, tablets are smaller and more portable

4.The 7 inch tablet is a digital notebook with digital ink function. In daily use, you can use the tablet computer like an ordinary notebook, take notes anytime and anywhere, and leave your own notes in electronic texts and documents.

7 Inches Tablet Pc,Quad Core Tablet 7 Inch,7 Inch Gaming Tablet,Supersonic Tablet 7 Inch

Jingjiang Gisen Technology Co.,Ltd , https://www.jsgisengroup.com