2. (Stochastic Gradient Descent) Let us assume the function f(x) can be written as: f(x) = f(x), i=1 where fi, f2f are L-smooth convex functions. Let i be a random variable uniformly distributed in {1, 2,...,n}. Then prove that E[||Vf; (x) – Vfi (x*)||₁₂] ≤ 2L(f(x) − f(x*)), where the expectation is taken with respect to the randomness i. Hint: You can assume that for any L-smooth convex function h the following holds: Lemma 2 (Convexity and smoothness). For all x, y = Rd ||Vh(x) - Vh(y)||≤ 2L (h(x)-h(y) - Vh(y) (x − y)).

Let X be a random variable on a closed and bounded interval [a, b]. Let g(x) be a convex function. Prove that                                 g(E(X)) ≤ E (g(X)

3. Let X be a continuous random variable on the interval [0, 1] having cdf Fx(x) = 3x² − 2x³. Let W = 4 + 3X. (a) What is the range of W? (b) Find a formula for Fw(w). (c) Find a formula for fw(w).

2. (Stochastic Gradient Descent) Let us assume the function f(x) can be written as: ΤΙ f(x) = f(x), i=1 where fi, f2fn are L-smooth convex functions. Let i be a random variable uniformly distributed in {1, 2,..., n}. Then prove that E[||Vƒ; (x) — Vƒ; (x*)||₁₂] ≤ 2L(f(x) − f(x*)), where the expectation is taken with respect to the randomness i. Hint: You can assume that for any L-smooth convex function h the following holds: Lemma 2 (Convexity and smoothness). For all x, y € Rd ||Vh(x) - Vh(y)||≤2L(h(x)-h(y) - Vh(y) (x − y)). 1