2. (Stochastic Gradient Descent) Let us assume the function f(x) can be written as: ΤΙ f(x) = f(x), i=1 where fi, f2fn are L-smooth convex functions. Let i be a random variable uniformly distributed in {1, 2,..., n}. Then prove that E[||Vƒ; (x) — Vƒ; (x*)||₁₂] ≤ 2L(f(x) − f(x*)), where the expectation is taken with respect to the randomness i. Hint: You can assume that for any L-smooth convex function h the following holds: Lemma 2 (Convexity and smoothness). For all x, y € Rd ||Vh(x) - Vh(y)||≤2L(h(x)-h(y) - Vh(y) (x − y)). 1
2. (Stochastic Gradient Descent) Let us assume the function f(x) can be written as: ΤΙ f(x) = f(x), i=1 where fi, f2fn are L-smooth convex functions. Let i be a random variable uniformly distributed in {1, 2,..., n}. Then prove that E[||Vƒ; (x) — Vƒ; (x*)||₁₂] ≤ 2L(f(x) − f(x*)), where the expectation is taken with respect to the randomness i. Hint: You can assume that for any L-smooth convex function h the following holds: Lemma 2 (Convexity and smoothness). For all x, y € Rd ||Vh(x) - Vh(y)||≤2L(h(x)-h(y) - Vh(y) (x − y)). 1
Advanced Engineering Mathematics
10th Edition
ISBN:9780470458365
Author:Erwin Kreyszig
Publisher:Erwin Kreyszig
Chapter2: Second-order Linear Odes
Section: Chapter Questions
Problem 1RQ
Related questions
Question
![2. (Stochastic Gradient Descent) Let us assume the function f(x) can be written as:
ΤΙ
f(x) = f(x),
i=1
where fi, f2fn are L-smooth convex functions. Let i be a random variable uniformly distributed in {1, 2,..., n}.
Then prove that
E[||Vƒ; (x) — Vƒ; (x*)||₁₂] ≤ 2L(f(x) − f(x*)),
where the expectation is taken with respect to the randomness i.
Hint: You can assume that for any L-smooth convex function h the following holds:
Lemma 2 (Convexity and smoothness). For all x, y € Rd
||Vh(x) - Vh(y)||≤2L(h(x)-h(y) - Vh(y) (x − y)).
1](/v2/_next/image?url=https%3A%2F%2Fcontent.bartleby.com%2Fqna-images%2Fquestion%2F08792f8d-91a6-4520-a2bb-5d9564b28bda%2Fdbf45a7c-58ec-4bed-b0f5-17127745ac33%2F87hon5k_processed.png&w=3840&q=75)
Transcribed Image Text:2. (Stochastic Gradient Descent) Let us assume the function f(x) can be written as:
ΤΙ
f(x) = f(x),
i=1
where fi, f2fn are L-smooth convex functions. Let i be a random variable uniformly distributed in {1, 2,..., n}.
Then prove that
E[||Vƒ; (x) — Vƒ; (x*)||₁₂] ≤ 2L(f(x) − f(x*)),
where the expectation is taken with respect to the randomness i.
Hint: You can assume that for any L-smooth convex function h the following holds:
Lemma 2 (Convexity and smoothness). For all x, y € Rd
||Vh(x) - Vh(y)||≤2L(h(x)-h(y) - Vh(y) (x − y)).
1
Expert Solution
This question has been solved!
Explore an expertly crafted, step-by-step solution for a thorough understanding of key concepts.
Step by step
Solved in 2 steps
Similar questions
- Recommended textbooks for youAdvanced Engineering MathematicsAdvanced MathISBN:9780470458365Author:Erwin KreyszigPublisher:Wiley, John & Sons, Incorporated
Numerical Methods for EngineersAdvanced MathISBN:9780073397924Author:Steven C. Chapra Dr., Raymond P. CanalePublisher:McGraw-Hill Education
Introductory Mathematics for Engineering Applicat…Advanced MathISBN:9781118141809Author:Nathan KlingbeilPublisher:WILEYAdvanced Engineering MathematicsAdvanced MathISBN:9780470458365Author:Erwin KreyszigPublisher:Wiley, John & Sons, IncorporatedNumerical Methods for EngineersAdvanced MathISBN:9780073397924Author:Steven C. Chapra Dr., Raymond P. CanalePublisher:McGraw-Hill EducationIntroductory Mathematics for Engineering Applicat…Advanced MathISBN:9781118141809Author:Nathan KlingbeilPublisher:WILEY
Mathematics For Machine TechnologyAdvanced MathISBN:9781337798310Author:Peterson, John.Publisher:Cengage Learning,
