Backward Step (i.e. back propagation) and Gradient Descent When training a neural network model we use back propagation to update the weight/bias parameters using gradient descent (or some variation of it). Gradient descent utilizes the chain rule, since each layer of a neural network can be described by a function, and the activation function (and additional multiple layers) can be described by a composition of functions. Suppose that we have the following functions: fi (x, w, b) = w·x + f2 (:) = ReLU (·) f (:) = f2 (fi (•)) note: the dot is used a placeholder here, so you could assume it to be u, or x, or any variable) Assume that we are going to carry out back propagation and a gradient descent step. Suppose that Ne = 0.5, and that x = 0.9, and that the learning rate is equal to 0.01. You can assume that b > 0. What will be the value of wt+1? note: be sure to carry out your answer to at least two (2) decimal places, i.e. 1e-2 precision)
Backward Step (i.e. back propagation) and Gradient Descent When training a neural network model we use back propagation to update the weight/bias parameters using gradient descent (or some variation of it). Gradient descent utilizes the chain rule, since each layer of a neural network can be described by a function, and the activation function (and additional multiple layers) can be described by a composition of functions. Suppose that we have the following functions: fi (x, w, b) = w·x + f2 (:) = ReLU (·) f (:) = f2 (fi (•)) note: the dot is used a placeholder here, so you could assume it to be u, or x, or any variable) Assume that we are going to carry out back propagation and a gradient descent step. Suppose that Ne = 0.5, and that x = 0.9, and that the learning rate is equal to 0.01. You can assume that b > 0. What will be the value of wt+1? note: be sure to carry out your answer to at least two (2) decimal places, i.e. 1e-2 precision)
Computer Networking: A Top-Down Approach (7th Edition)
7th Edition
ISBN:9780133594140
Author:James Kurose, Keith Ross
Publisher:James Kurose, Keith Ross
Chapter1: Computer Networks And The Internet
Section: Chapter Questions
Problem R1RQ: What is the difference between a host and an end system? List several different types of end...
Related questions
Question
5
![Backward Step (i.e. back propagation) and Gradient Descent
When training a neural network model we use back propagation to update the weight/bias
parameters using gradient descent (or some variation of it). Gradient descent utilizes the chain rule,
since each layer of a neural network can be described by a function, and the activation function (and
additional multiple layers) can be described by a composition of functions.
Suppose that we have the following functions:
fi (x, w, b)
= w·x + b
f2 (•)
ReLU (·)
f (-)
f2 (fi (:))
(note: the dot is used a placeholder here, so you could assume it to be u, or x, or any variable)
Assume that we are going to carry out back propagation and a gradient descent step. Suppose that
Wt = 0.5, and that x = 0.9, and that the learning rate is equal to 0.01. You can assume that b > 0.
What will be the value of we+1?
(note: be sure to carry out your answer to at least two (2) decimal places, i.e. 1e-2 precision)](/v2/_next/image?url=https%3A%2F%2Fcontent.bartleby.com%2Fqna-images%2Fquestion%2F8860e590-efe8-4350-bb0e-ef90976e389f%2Ff1f8c1c9-3ca9-42b9-98d3-000364bc5da1%2Fyjtcbuw_processed.png&w=3840&q=75)
Transcribed Image Text:Backward Step (i.e. back propagation) and Gradient Descent
When training a neural network model we use back propagation to update the weight/bias
parameters using gradient descent (or some variation of it). Gradient descent utilizes the chain rule,
since each layer of a neural network can be described by a function, and the activation function (and
additional multiple layers) can be described by a composition of functions.
Suppose that we have the following functions:
fi (x, w, b)
= w·x + b
f2 (•)
ReLU (·)
f (-)
f2 (fi (:))
(note: the dot is used a placeholder here, so you could assume it to be u, or x, or any variable)
Assume that we are going to carry out back propagation and a gradient descent step. Suppose that
Wt = 0.5, and that x = 0.9, and that the learning rate is equal to 0.01. You can assume that b > 0.
What will be the value of we+1?
(note: be sure to carry out your answer to at least two (2) decimal places, i.e. 1e-2 precision)
Expert Solution
![](/static/compass_v2/shared-icons/check-mark.png)
This question has been solved!
Explore an expertly crafted, step-by-step solution for a thorough understanding of key concepts.
Step by step
Solved in 3 steps with 1 images
![Blurred answer](/static/compass_v2/solution-images/blurred-answer.jpg)
Recommended textbooks for you
![Computer Networking: A Top-Down Approach (7th Edi…](https://www.bartleby.com/isbn_cover_images/9780133594140/9780133594140_smallCoverImage.gif)
Computer Networking: A Top-Down Approach (7th Edi…
Computer Engineering
ISBN:
9780133594140
Author:
James Kurose, Keith Ross
Publisher:
PEARSON
![Computer Organization and Design MIPS Edition, Fi…](https://www.bartleby.com/isbn_cover_images/9780124077263/9780124077263_smallCoverImage.gif)
Computer Organization and Design MIPS Edition, Fi…
Computer Engineering
ISBN:
9780124077263
Author:
David A. Patterson, John L. Hennessy
Publisher:
Elsevier Science
![Network+ Guide to Networks (MindTap Course List)](https://www.bartleby.com/isbn_cover_images/9781337569330/9781337569330_smallCoverImage.gif)
Network+ Guide to Networks (MindTap Course List)
Computer Engineering
ISBN:
9781337569330
Author:
Jill West, Tamara Dean, Jean Andrews
Publisher:
Cengage Learning
![Computer Networking: A Top-Down Approach (7th Edi…](https://www.bartleby.com/isbn_cover_images/9780133594140/9780133594140_smallCoverImage.gif)
Computer Networking: A Top-Down Approach (7th Edi…
Computer Engineering
ISBN:
9780133594140
Author:
James Kurose, Keith Ross
Publisher:
PEARSON
![Computer Organization and Design MIPS Edition, Fi…](https://www.bartleby.com/isbn_cover_images/9780124077263/9780124077263_smallCoverImage.gif)
Computer Organization and Design MIPS Edition, Fi…
Computer Engineering
ISBN:
9780124077263
Author:
David A. Patterson, John L. Hennessy
Publisher:
Elsevier Science
![Network+ Guide to Networks (MindTap Course List)](https://www.bartleby.com/isbn_cover_images/9781337569330/9781337569330_smallCoverImage.gif)
Network+ Guide to Networks (MindTap Course List)
Computer Engineering
ISBN:
9781337569330
Author:
Jill West, Tamara Dean, Jean Andrews
Publisher:
Cengage Learning
![Concepts of Database Management](https://www.bartleby.com/isbn_cover_images/9781337093422/9781337093422_smallCoverImage.gif)
Concepts of Database Management
Computer Engineering
ISBN:
9781337093422
Author:
Joy L. Starks, Philip J. Pratt, Mary Z. Last
Publisher:
Cengage Learning
![Prelude to Programming](https://www.bartleby.com/isbn_cover_images/9780133750423/9780133750423_smallCoverImage.jpg)
Prelude to Programming
Computer Engineering
ISBN:
9780133750423
Author:
VENIT, Stewart
Publisher:
Pearson Education
![Sc Business Data Communications and Networking, T…](https://www.bartleby.com/isbn_cover_images/9781119368830/9781119368830_smallCoverImage.gif)
Sc Business Data Communications and Networking, T…
Computer Engineering
ISBN:
9781119368830
Author:
FITZGERALD
Publisher:
WILEY