A way to avoid overfitting in Deep Neural Networks is to add an additional term R to the loss function L (for example L can be the cross entropy loss) as follows: L(w) + λR(w). (1) You know that one choice for R is the L2 norm, i.e. R(w) = ||w||2 2 . One friend of yours from the School of Maths told you however that there’s no need to use squares (i.e. powers of two) and that you can achieve the same effect by using absolute values, i.e. the L1 norm: R(w) = ||w||1. Would you agree with him? i.e. is the use of the L2 norm equivalent to using the L1 norm for regularization purposes? Justify your answer
A way to avoid overfitting in Deep Neural Networks is to add an additional term R to the loss function L (for example L can be the cross entropy loss) as follows: L(w) + λR(w). (1) You know that one choice for R is the L2 norm, i.e. R(w) = ||w||2 2 . One friend of yours from the School of Maths told you however that there’s no need to use squares (i.e. powers of two) and that you can achieve the same effect by using absolute values, i.e. the L1 norm: R(w) = ||w||1. Would you agree with him? i.e. is the use of the L2 norm equivalent to using the L1 norm for regularization purposes? Justify your answer
Database System Concepts
7th Edition
ISBN:9780078022159
Author:Abraham Silberschatz Professor, Henry F. Korth, S. Sudarshan
Publisher:Abraham Silberschatz Professor, Henry F. Korth, S. Sudarshan
Chapter1: Introduction
Section: Chapter Questions
Problem 1PE
Related questions
Question
Computer Science
A way to avoid overfitting in Deep Neural Networks is to add an additional term R to the loss function L (for example L can be the cross entropy loss) as follows: L(w) + λR(w). (1) You know that one choice for R is the L2 norm, i.e. R(w) = ||w||2 2 . One friend of yours from the School of Maths told you however that there’s no need to use squares (i.e. powers of two) and that you can achieve the same effect by using absolute values, i.e. the L1 norm: R(w) = ||w||1. Would you agree with him? i.e. is the use of the L2 norm equivalent to using the L1 norm for regularization purposes? Justify your answer
Expert Solution
This question has been solved!
Explore an expertly crafted, step-by-step solution for a thorough understanding of key concepts.
Step by step
Solved in 2 steps
Knowledge Booster
Learn more about
Need a deep-dive on the concept behind this application? Look no further. Learn more about this topic, computer-science and related others by exploring similar questions and additional content below.Similar questions
Recommended textbooks for you
Database System Concepts
Computer Science
ISBN:
9780078022159
Author:
Abraham Silberschatz Professor, Henry F. Korth, S. Sudarshan
Publisher:
McGraw-Hill Education
Starting Out with Python (4th Edition)
Computer Science
ISBN:
9780134444321
Author:
Tony Gaddis
Publisher:
PEARSON
Digital Fundamentals (11th Edition)
Computer Science
ISBN:
9780132737968
Author:
Thomas L. Floyd
Publisher:
PEARSON
Database System Concepts
Computer Science
ISBN:
9780078022159
Author:
Abraham Silberschatz Professor, Henry F. Korth, S. Sudarshan
Publisher:
McGraw-Hill Education
Starting Out with Python (4th Edition)
Computer Science
ISBN:
9780134444321
Author:
Tony Gaddis
Publisher:
PEARSON
Digital Fundamentals (11th Edition)
Computer Science
ISBN:
9780132737968
Author:
Thomas L. Floyd
Publisher:
PEARSON
C How to Program (8th Edition)
Computer Science
ISBN:
9780133976892
Author:
Paul J. Deitel, Harvey Deitel
Publisher:
PEARSON
Database Systems: Design, Implementation, & Manag…
Computer Science
ISBN:
9781337627900
Author:
Carlos Coronel, Steven Morris
Publisher:
Cengage Learning
Programmable Logic Controllers
Computer Science
ISBN:
9780073373843
Author:
Frank D. Petruzella
Publisher:
McGraw-Hill Education