2. LSTM Bookmark this page The diagram below shows a single LSTM unit that consists of Input, Output, and Forget gates The behavior of such a unit as a recurrent neural network is specified by a set of update equations. These equations define how the gates, "memory cell" and the "visible state" h, are updated in response to input ar, and previous states - hr-1. For the LSTM unit, sigmoid (Whi+Wa+by) sigmoid (Whe+W+b) sigmoid (Wh1+W*+b) =f1+ tanh (W+Wa+b) tank (c) where symbol Ⓒ stands for element-wise multiplication. The adjustable parameters in this unit are matrices WWE W W W W W We, as well as the offset parameter vectors by, b,, b, and b. By changing these parameters, we change how the unit evolves as a function of inputs ar To keep things simple, in this problem we assume that C, and h are all scalars. Concretely, suppose that the parameters are given by =0 by-100 =-100 100 b 100 =50 We Calculate the values at each time-step and enter them below as an array [ho, ha, ha, ha, ha.hs 0 100 b = 0, We run this unit with initial conditions = 0 and e-= 0, and in response to the following input sequence: 10, 0, 1, 1, 1, 0] (For example, ₁ = 0, 0, 1, and so on). LSTM states (Please round h, to the closest integer in every time-step. It h±0.5, then round it to 0. For ease of calculation, assume that sigmoid (r) 1 and tanh (r) 1 for > 1, and sigmoid (z) = 0 and Submit You have used 0 of 5 attempts Save LSTM states 2 Now, we run the same model again with the same parameters and same initial conditions as in the previous question. The only difference is that our input sequence in now: [1, 1, 0, 1, 11. Calculate the values he at each time-step and enter them below as an array [hos hy, ha, ha, ha (Please round h, to the closest integer in every time-step. If hy±0.5, then round it to 0. For ease of calculation, assume that sigmoid (a) 1 and tanh (2) 1 for a 1, and sigmoid (r) = 0) and tanh(2)=−1 for r<-1)
2. LSTM
Bookmark this page
The diagram below shows a single LSTM unit that consists of Input, Output, and Forget gates
The behavior of such a unit as a recurrent neural network is specified by a set of update equations. These equations define how the gates, "memory cell" and the "visible state" h, are updated in response to input ar, and previous states - hr-1. For the LSTM unit,
sigmoid (Whi+Wa+by)
sigmoid (Whe+W+b)
sigmoid (Wh1+W*+b)
=f1+ tanh (W+Wa+b)
tank (c)
where symbol Ⓒ stands for element-wise multiplication. The adjustable parameters in this unit are matrices WWE
W W W W W We, as well as the offset parameter
parameters, we change how the unit evolves as a function of inputs ar To keep things simple, in this problem we assume that C, and h are all scalars. Concretely, suppose that the
parameters are given by
=0
by-100
=-100
100 b
100
=50
We
Calculate the values at each time-step and enter them below as an array [ho, ha, ha, ha, ha.hs
0 100 b = 0,
We run this unit with initial conditions = 0 and e-= 0, and in response to the following input sequence: 10, 0, 1, 1, 1,
0] (For example, ₁ = 0, 0, 1, and so on).
LSTM states
(Please round h, to the closest integer in every time-step. It h±0.5, then round it to 0. For ease of calculation, assume that sigmoid (r) 1 and tanh (r) 1 for > 1, and sigmoid (z) = 0 and
Submit
You have used 0 of 5 attempts
Save
LSTM states 2
Now, we run the same model again with the same parameters and same initial conditions as in the previous question. The only difference is that our input sequence in now: [1, 1, 0, 1, 11.
Calculate the values he at each time-step and enter them below as an array [hos hy, ha, ha, ha
(Please round h, to the closest integer in every time-step. If hy±0.5, then round it to 0. For ease of calculation, assume that sigmoid (a) 1 and tanh (2) 1 for a 1, and sigmoid (r) = 0) and
tanh(2)=−1 for r<-1)
Trending now
This is a popular solution!
Step by step
Solved in 3 steps
