2. (Stochastic Gradient Descent) Let us assume the function f(x) can be written as: f(x) = f(x), i=1 where fi, f2f are L-smooth convex functions. Let i be a random variable uniformly distributed in {1, 2,...,n}. Then prove that E[||Vf; (x) – Vfi (x*)||₁₂] ≤ 2L(f(x) − f(x*)), where the expectation is taken with respect to the randomness i. Hint: You can assume that for any L-smooth convex function h the following holds: Lemma 2 (Convexity and smoothness). For all x, y = Rd ||Vh(x) - Vh(y)||≤ 2L (h(x)-h(y) - Vh(y) (x − y)).
2. (Stochastic Gradient Descent) Let us assume the function f(x) can be written as: f(x) = f(x), i=1 where fi, f2f are L-smooth convex functions. Let i be a random variable uniformly distributed in {1, 2,...,n}. Then prove that E[||Vf; (x) – Vfi (x*)||₁₂] ≤ 2L(f(x) − f(x*)), where the expectation is taken with respect to the randomness i. Hint: You can assume that for any L-smooth convex function h the following holds: Lemma 2 (Convexity and smoothness). For all x, y = Rd ||Vh(x) - Vh(y)||≤ 2L (h(x)-h(y) - Vh(y) (x − y)).
Advanced Engineering Mathematics
10th Edition
ISBN:9780470458365
Author:Erwin Kreyszig
Publisher:Erwin Kreyszig
Chapter2: Second-order Linear Odes
Section: Chapter Questions
Problem 1RQ
Related questions
Question
100%
![2. (Stochastic Gradient Descent) Let us assume the function f(x) can be written as:
f(x) = f(x),
i=1
where fi, f2f are L-smooth convex functions. Let i be a random variable uniformly distributed in {1, 2,...,n}.
Then prove that
E[||Vf; (x) – Vfi (x*)||₁₂] ≤ 2L(f(x) − f(x*)),
where the expectation is taken with respect to the randomness i.
Hint: You can assume that for any L-smooth convex function h the following holds:
Lemma 2 (Convexity and smoothness). For all x, y = Rd
||Vh(x) - Vh(y)||≤ 2L (h(x)-h(y) - Vh(y) (x − y)).](/v2/_next/image?url=https%3A%2F%2Fcontent.bartleby.com%2Fqna-images%2Fquestion%2F08792f8d-91a6-4520-a2bb-5d9564b28bda%2Fe374aa13-7b9b-4c00-a433-54d0b2121905%2F5kom73j_processed.png&w=3840&q=75)
Transcribed Image Text:2. (Stochastic Gradient Descent) Let us assume the function f(x) can be written as:
f(x) = f(x),
i=1
where fi, f2f are L-smooth convex functions. Let i be a random variable uniformly distributed in {1, 2,...,n}.
Then prove that
E[||Vf; (x) – Vfi (x*)||₁₂] ≤ 2L(f(x) − f(x*)),
where the expectation is taken with respect to the randomness i.
Hint: You can assume that for any L-smooth convex function h the following holds:
Lemma 2 (Convexity and smoothness). For all x, y = Rd
||Vh(x) - Vh(y)||≤ 2L (h(x)-h(y) - Vh(y) (x − y)).
Expert Solution
This question has been solved!
Explore an expertly crafted, step-by-step solution for a thorough understanding of key concepts.
Step by step
Solved in 2 steps with 3 images
Similar questions
Recommended textbooks for you
Advanced Engineering Mathematics
Advanced Math
ISBN:
9780470458365
Author:
Erwin Kreyszig
Publisher:
Wiley, John & Sons, Incorporated
Numerical Methods for Engineers
Advanced Math
ISBN:
9780073397924
Author:
Steven C. Chapra Dr., Raymond P. Canale
Publisher:
McGraw-Hill Education
Introductory Mathematics for Engineering Applicat…
Advanced Math
ISBN:
9781118141809
Author:
Nathan Klingbeil
Publisher:
WILEY
Advanced Engineering Mathematics
Advanced Math
ISBN:
9780470458365
Author:
Erwin Kreyszig
Publisher:
Wiley, John & Sons, Incorporated
Numerical Methods for Engineers
Advanced Math
ISBN:
9780073397924
Author:
Steven C. Chapra Dr., Raymond P. Canale
Publisher:
McGraw-Hill Education
Introductory Mathematics for Engineering Applicat…
Advanced Math
ISBN:
9781118141809
Author:
Nathan Klingbeil
Publisher:
WILEY
Mathematics For Machine Technology
Advanced Math
ISBN:
9781337798310
Author:
Peterson, John.
Publisher:
Cengage Learning,
